Write a new library instrumentation end-to-end. Use when the user ask to add a new APM integration or a library instrumentation.
Write a new APM end-to-end integration for dd-trace-java, based on library instrumentations, following all project conventions.
Before writing any code, read all three files in full:
docs/how_instrumentations_work.md — full reference (types, methods, advice, helpers, context stores, decorators)docs/add_new_instrumentation.md — step-by-step walkthroughdocs/how_to_test.md — test types and how to run themThese files are the single source of truth. Reference them while implementing.
After reading the docs, sync this skill with them:
Compare the content of the three docs against the rules encoded in Steps 2–11 of this skill file. Look for:
For every discrepancy found, edit this file (.claude/skills/apm-integrations/SKILL.md) to correct it using the
Edit tool before continuing. Keep changes targeted: fix what diverged, add what is missing, remove what is wrong.
Do not touch content that already matches the docs.
If the user has not already provided all of the following, ask before proceeding:
okhttp-3.0)Tracing, Profiling, AppSec, Iast, CiVisibility, Usm, ContextTrackingSearch dd-java-agent/instrumentation/ for a structurally similar integration:
Read the reference integration's InstrumenterModule, Advice, Decorator, and test files to understand the established
pattern before writing new code. Use it as a template.
dd-java-agent/instrumentation/$framework/$framework-$minVersion/src/main/java/ — instrumentation codesrc/test/groovy/ — Spock testsbuild.gradle with:
compileOnly dependencies for the target frameworktestImplementation dependencies for testsmuzzle { pass { } } directives (see Step 9)settings.gradle.kts in alphabetical orderConventions to enforce:
@AutoService(InstrumenterModule.class) annotation — required for auto-discoveryInstrumenterModule.* subclass (never the bare abstract class)Instrumenter interface possible:
ForSingleType > ForKnownTypes > ForTypeHierarchyclassLoaderMatcher() if a sentinel class identifies the framework on the classpathhelperClassNames():
Foo$Bar), anonymous classes (Foo$1), and enum synthetic classescontextStore() entries if context stores are needed (key class → value class)Do not extract one-shot method return values into static constants.
Methods like triggerClasses(), contextStore(), classLoaderMatcher(), and methodAdvice()
are called once by AgentInstaller / the framework wiring. Extracting their return value
into a private static final constant provides no performance benefit and needlessly bloats
the constant pool of the instrumentation class.
❌ private static final String[] TRIGGER_CLASSES = new String[]{"com.example.Foo"};
public String[] triggerClasses() { return TRIGGER_CLASSES; }
✅ public String[] triggerClasses() { return new String[]{"com.example.Foo"}; }
HttpClientDecorator, DatabaseClientDecorator, ServerDecorator, MessagingClientDecorator, etc.public static final DECORATE instanceUTF8BytesString constants for the component name and operation namespanType(), component(), spanKind() as appropriatestatic@Advice.OnMethodEnter(suppress = Throwable.class)@Advice.OnMethodExit(onThrowable = Throwable.class, suppress = Throwable.class)
suppress when hooking a constructor@Advice.Local("...") for values shared between enter and exit (span, scope)@Advice.This — the receiver object@Advice.Argument(N) — a method argument by index@Advice.Return — the return value (exit only)@Advice.Thrown — the thrown exception (exit only)@Advice.Enter — the return value of the enter method (exit only)CallDepthThreadLocalMap to guard against recursive instrumentation of the same methodEnter method:
AgentSpan span = startSpan(DECORATE.operationName(), ...)DECORATE.afterStart(span) + set domain-specific tagsAgentScope scope = activateSpan(span) — return or store via @Advice.LocalExit method:
4. DECORATE.onError(span, throwable) — only if throwable is non-null
5. DECORATE.beforeFinish(span)
6. span.finish()
7. scope.close()
InstrumentationContext.get() outside of Advice codeinline=false in production code (only for debugging; must be removed before committing)java.util.logging.*, java.nio.*, or javax.management.* in bootstrap instrumentationsFor context propagation to and from upstream services, like HTTP headers,
implement AgentPropagation.Setter / AgentPropagation.Getter adapters that wrap the framework's specific header API.
Place them in the helpers package, declare them in helperClassNames().
Cover all mandatory test types:
InstrumentationSpecificationsrc/test/groovy/TEST_WRITER.waitForTraces(N) for assertionsrunUnderTrace("root") { ... } for synchronous codeFor tests that need a separate JVM, suffix the test class with ForkedTest and run via the forkedTest task.
In build.gradle, add muzzle blocks:
muzzle {
pass {
group = "com.example"
module = "framework"
versions = "[$minVersion,)"
assertInverse = true // ensures versions below $minVersion fail muzzle
}
}Use the latestDepTestLibrary helper in build.gradle to pin the latest available version. Run with:
./gradlew :dd-java-agent:instrumentation:$framework-$version:latestDepTestAdd a smoke test in dd-smoke-tests/ only if the framework warrants a full end-to-end demo-app test.
Run these commands in order and fix any failures before proceeding:
./gradlew :dd-java-agent:instrumentation:$framework-$version:muzzle
./gradlew :dd-java-agent:instrumentation:$framework-$version:test
./gradlew :dd-java-agent:instrumentation:$framework-$version:latestDepTest
./gradlew spotlessCheckIf muzzle fails: check for missing helper class names in helperClassNames().
If tests fail: verify span lifecycle order (start → activate → error → finish → close), helper registration,
and contextStore() map entries match actual usage.
If spotlessCheck fails: run ./gradlew spotlessApply to auto-format, then re-check.
Output this checklist and confirm each item is satisfied:
settings.gradle.kts entry added in alphabetical orderbuild.gradle has compileOnly deps and muzzle directives with assertInverse = true@AutoService(InstrumenterModule.class) annotation present on the module classhelperClassNames() lists ALL referenced helpers (including inner, anonymous, and enum synthetic classes)static with @Advice.OnMethodEnter / @Advice.OnMethodExit annotationssuppress = Throwable.class on enter/exit (unless the hooked method is a constructor)triggerClasses(), contextStore(), etc.)inline=false left in production codejava.util.logging.* / java.nio.* / javax.management.* in bootstrap pathlatestDepTest passesspotlessCheck passesAfter the instrumentation is complete (or abandoned), review the full session and improve this skill for future use.
Collect lessons from four sources:
For each lesson identified, edit this file (.claude/skills/apm-integrations/SKILL.md) using the Edit tool:
Keep each change minimal and targeted. Do not rewrite sections that worked correctly. After editing, confirm to the user which improvements were made to the skill.
4ee2247
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.