Ksql stream from a Kafka topic

[GiuseppeR]

Hi guys,

I’m using Confluent Platform 7.0.1. I need some hint to process in KSQLDB a data stream from a Kafka topic. I have an application which sends data to a Kafka topic named MyData. I need to intercept the variation of the value of a variable (lat) and write this result on another Kafka topic.
I created the topic and I defined a valid JSON schema.
In the KSQLDB editor I create a stream data table:

CREATE TABLE MyData_stream (id VARCHAR PRIMARY KEY) WITH (kafka_topic='MyData', value_format='JSON');

when I try to print topics with

SHOW TOPIC

it works…If I try to print data from my topic in the KSQLDB editor with:

PRINT 'MyData' FROM BEGINNING;

it doesn’t show anything (no data, no message).
The second step is to create a raw data stream:

CREATE STREAM MyData_raw (id INT KEY, simulation_id INT, timestamp BIGINT, lat DOUBLE, long DOUBLE,.....) WITH (KAFKA_TOPIC='MyData', KEY_FORMAT='KAFKA', VALUE_FORMAT='JSON');

but when I try the query:

SELECT * FROM MyData_raw EMIT CHANGES;

I have any result.
So any suggestion? There some error?
Thanks for your support.

[vnadkarni]

@GiuseppeR

I have any result.
So any suggestion? There some error?

Hi @GiuseppeR . What result/error do you get?

[mjsax]

@GiuseppeR

but when I try the query:

SELECT * FROM MyData_raw EMIT CHANGES;

I have any result.

Did you set auto.offset.reset to "earliest" before submitting the query?

[GiuseppeR]

Hi guys,

thank you for your replies.
With my server admin we have established KSQLdb doesn’t work correctly so the stream is not fed from the Kafka topic.
The task described it is not in KSQLdb…so I wrote a python consumer and It works pretty good.
Thanks.