Kafka Consumer not consuming data immediately they're producded

Kafka Connect Self-Managed Connectors
1

My consumers are not pushing data to destination DB immediately it is produced.
I have set the sink connector to “fetch.max.wait.ms=10”.

Please what configuration setting do I need to have it consume the data the minute it is produced to the topic.

2

Kafka producer send call is never immediate, as producers batch network requests themselves. Therefore, you cannot expect consumers to be, either. Consumers will block while polling and wait for the next batch of records as soon as they are available (after durably written to the broker). 10 milliseconds may not be a reasonable time frame for all of this to occur.

I notice you posted this question in connectors forum, so what connector are you using? Please share its config, as well as your producer code.

3

I am using JDBC connector
“connector.class”: “io.confluent.connect.jdbc.JdbcSourceConnector”,
“connection.url”: “jdbc:oracle:thin:@xx.xx.xx.xx:1521:xx”,
“connection.user”:“username”,
“connection.password”:“password”,
“numeric.mapping”:“best_fit”,
“mode”:“timestamp”,
“poll.interval.ms”:“10”,
“database.history.kafka.bootstrap.servers” : “kafka1:9092”,
“database.history.kafka.topic”: “schema-changes.xxx”,
“database.schema”: “xxx”,
“errors.log.enable”: “true”,
“snapshot.lock.timeout.ms”:“5000”,
“include.schema.changes”: “true”,
“snapshot.mode”:“initial”,
“validate.non.null”:“false”,
“table.whitelist”:“TableName”,
“topic.prefix”:“test-”,
“decimal.handling.mode”: “double”,
“transforms”: “Cast,createKey”,
“transforms.createKey.fields”: “ID”,
“transforms.createKey.type”: “org.apache.kafka.connect.transforms.ValueToKey”,
“transforms.Cast.type”: “org.apache.kafka.connect.transforms.Cast$Value”,
“transforms.Cast.spec”: “ID”

4

Thanks. And producer code?