Kafka Connect Datadog APM instrumentation

Kafka Connect
1

Hi,

I have enabled APM Datadog instrumentation (via bytecode manipulation) and I observer the following:

  • I see traces in Datadog related to the connect cluster itself (e.g. using the REST API to like connectors or listing the plugins)
  • But I do not see S3 sink connector traces (aka events being consumed from the source topic and pushed to S3). E.g. I would expect to see the Kafka Consumer and the AWS SDK being instrumented as they are fully supported by the Datadog APM instrumentation (Java Compatibility Requirements).

I suspect the fact that connectors seem to be running on separate class loaders might be causing this behaviour.

Has anyone done this before?

From the docs (Tracing Java Applications):

" Automatic instrumentation for Java uses the `java-agent` instrumentation capabilities [provided by the JVM](https://docs.oracle.com/javase/8/docs/api/java/lang/instrument/package-summary.html). When a `java-agent` is registered, it can modify class files at load time.

**Note:** Classes loaded with remote ClassLoader are not instrumented automatically."

Thanks