Source connector: retrieve the current task.count?

Kafka Connect
1

I am writing a source connector. I want to programatically retrieve the current task.count, to add to the message header.
This is used by a matching sink connector; which calculates the pipeline throughput based on tumestamps and the source connector task count. The current task count is a worker configuration - is it possible to access this? I prefer not to use the REST api or do anything to parse logfiles. I also need to retrieve the bootstrap.servers; so that the message header has provenance metadata about the (first) kafka cluster that the message was produced to,
Is worker config accessible to a source connector?

2

answering my own question:
this.produceTaskName = MDC.get(“connector.context”);

sends the task name that did the produce. This is enough for me to work with; as I can count distinct task names in the sink.
“produce.task.name=[halo-windows1-uploader|task-0]”

5

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.