I downloaded the Confluent’s Self-Hosted connector kafka-connect-jdbc Version 10.8.0 I have kafka version kafka_2.13-3.8.1. I placed the kafka-connect-jdbc as a plugin and started kafka and connector. I tried with mode = buld and incrementing. It works fine. But when I tried with mode = timestamp, I get an error while starting task
[2025-02-04 18:05:14,439] INFO [jdbc_source_conn_2|task-0] Database connection established. (io.confluent.connect.jdbc.util.CachedConnectionProvider:64) [2025-02-04 18:05:14,443] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,443] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,443] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,443] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,443] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,444] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,444] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,444] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,444] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,444] INFO [jdbc_source_conn_2|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122) [2025-02-04 18:05:14,444] INFO [jdbc_source_conn_2|task-0] Current Result is null. Executing query. (io.confluent.connect.jdbc.source.TimestampIncrementingTab
The POST request is below
{
"name":"jdbc_source_conn_2",
"config":{
"connector.class":"io.confluent.connect.jdbc.JdbcSourceConnector",
"connection.url":"jdbc:oracle:thin:@xxx:xxx:xxx",
"connection.user":"xxx",
"connection.password":"xxx",
"mode":"timestamp",
"topic.prefix":"mode-timestamp-",
"timestamp.column.name":"UPDATED_TS",
"table.whitelist":"TEST_CUSTOMERS"
}
}
desc TEST_CUSTOMERS
Name Null? Type
---------- -------- -------------
ID NOT NULL NUMBER(38)
FIRST_NAME VARCHAR2(255)
LAST_NAME VARCHAR2(255)
CREATED_TS NOT NULL TIMESTAMP(6)
UPDATED_TS NOT NULL TIMESTAMP(6)
The ojdbc driver version available with the plugin is ojdbc8-19.7.0.0.jar
Could you please help me resolve the issue
Based on some internal discussion I am seeing about this logging, it’s not problematic. It is confusing though that it says JDBC driver version 2 is newer than required version 4.0 since 2 is < 4.0, but that’s just buggy logging since in your case the version is 4.2, i.e., it logs the minor version. The logging has already been fixed in this pull request and changed from INFO to DEBUG.
Thank you. Should I wait for new version or should I go back to any versions?
AIUI there’s no need to wait or downgrade. It’s misleading logging but is benign.
Thank you. But when I tried to create a record or update an existing record, the record is not pushed to kafka stream. I assumed that due to the jdbc driver version error, the updates were not pushed to kafka stream. If this is just a logging issue, then could you please help me how to push records to kafka stream on incrementing timestamp columns.
Hard to say what the issue might be. Is there any helpful info elsewhere in the log?
If nothing helpful turns up, then I would turn on trace logging to debug this. E.g., see what gets logged for this line. This should give you the timestamp filter query that the connector runs to determine rows to land in Kafka. With trace logging enabled, hopefully you’d be able to see if something looks off with either the query or the data in the DB.
I enabled the trace logging, and got below logs
[2025-02-10 17:00:03,959] TRACE [source-bulk-conn-7|task-0] Set the committed offset: 2025-02-04 12:15:59.310495 (io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier:183)
[2025-02-10 17:00:08,961] INFO [source-bulk-conn-7|task-0] Database connection established. (io.confluent.connect.jdbc.util.CachedConnectionProvider:64)
[2025-02-10 17:00:08,961] DEBUG [source-bulk-conn-7|task-0] Creating PreparedStatement (io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier:131)
[2025-02-10 17:00:08,962] TRACE [source-bulk-conn-7|task-0] TimestampTableQuerier{table=“xxx”.“TEST_CUSTOMERS”, query=‘null’, topicPrefix=‘mode-timestamp-’, timestampColumns=[UPDATED_TS]} prepared SQL query: SELECT * FROM “xxx”.“TEST_CUSTOMERS” WHERE “xxx”.“TEST_CUSTOMERS”.“UPDATED_TS” > ? AND “xxx”.“TEST_CUSTOMERS”.“UPDATED_TS” < ? ORDER BY “xxx”.“TEST_CUSTOMERS”.“UPDATED_TS” ASC (io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier:160)
[2025-02-10 17:00:08,964] TRACE [source-bulk-conn-7|task-0] Statement to execute: oracle.jdbc.driver.OraclePreparedStatementWrapper@3f253fc6 (io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier:219)
[2025-02-10 17:00:08,966] INFO [source-bulk-conn-7|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122)
[2025-02-10 17:00:08,966] INFO [source-bulk-conn-7|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122)
[2025-02-10 17:00:08,966] INFO [source-bulk-conn-7|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122)
[2025-02-10 17:00:08,966] INFO [source-bulk-conn-7|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122)
[2025-02-10 17:00:08,966] INFO [source-bulk-conn-7|task-0] JDBC driver version 2 is newer than required version 4.0 (io.confluent.connect.jdbc.util.JdbcDriverInfo:122)
[2025-02-10 17:00:08,966] INFO [source-bulk-conn-7|task-0] Current Result is null. Executing query. (io.confluent.connect.jdbc.source.TimestampIncrementingTableQuerier:174)
Can you tell what the begin and end timestamps are? The prepared statement only shows ? but maybe you can find this line:
log.debug("Executing prepared statement with timestamp value = {} end time = {}",
DateTimeUtils.formatTimestamp(beginTime, timeZone),
DateTimeUtils.formatTimestamp(endTime, timeZone)
);
Alternatively, if you have access to the DB query log, see what query hits Oracle.
I enabled tracing for TimestampIncrementingCriteria and figured out that begin and end timestamps were in UTC timezone and the db timestamp column was in UTC+05:30 timezone. So I added the property “db.timezone”:“Asia/Kolkata” to the POST request to connector and then it worked fine. Thanks for the help.
1 Like