Ksql stream from a Kafka topic

Hi guys,

I’m using Confluent Platform 7.0.1. I need some hint to process in KSQLDB a data stream from a Kafka topic. I have an application which sends data to a Kafka topic named MyData. I need to intercept the variation of the value of a variable (lat) and write this result on another Kafka topic.
I created the topic and I defined a valid JSON schema.
In the KSQLDB editor I create a stream data table:

CREATE TABLE MyData_stream (id VARCHAR PRIMARY KEY) WITH (kafka_topic='MyData', value_format='JSON');

when I try to print topics with


it works…If I try to print data from my topic in the KSQLDB editor with:


it doesn’t show anything (no data, no message).
The second step is to create a raw data stream:

CREATE STREAM MyData_raw (id INT KEY, simulation_id INT, timestamp BIGINT, lat DOUBLE, long DOUBLE,.....) WITH (KAFKA_TOPIC='MyData', KEY_FORMAT='KAFKA', VALUE_FORMAT='JSON');

but when I try the query:


I have any result.
So any suggestion? There some error?
Thanks for your support.

Hi @GiuseppeR . What result/error do you get?

Did you set auto.offset.reset to "earliest" before submitting the query?

1 Like

Hi guys,

thank you for your replies.
With my server admin we have established KSQLdb doesn’t work correctly so the stream is not fed from the Kafka topic.
The task described it is not in KSQLdb…so I wrote a python consumer and It works pretty good.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.