I am trying to create a stream over kafka topic. When I am doing describe against stream it is showing just column names & I am unable to fetch data either. Showing empty table with just column index names. Please help regarding this.
ROWKEY and ROWTIME as implicit columns and thus are excluded from DESCRIBE. There is also DESCRIBE EXTENDED that provides more information though: DESCRIBE - ksqlDB Documentation
It would help if you post the commands you issued, and the output you get, and describe in more detail what output you expect. Note that DESCRIBE show metadata and it does not show records. If you want to get actual data, you need to issue a query.
Showing empty table with just column index names.
Not sure what you mean by this.
1 Like
Case Study :
To segregate data records a topic to two different topics according to the value of one of columns.
Example:
topic name - fruits
Data in topic - JSON
{“fruit”:“apple”, “color”:“red”}
{“fruit”:“mango”, “color”:'“yellow”}
and so on.
I want to filter these records on color i.e records from topic fruits should with color red should enter into redfruits topic and with yellow color in yellowfruits topic.
But when I am creating stream with this topic and value format JSON. Stream is created but no data is shown after throwing select query with emit changes.
I have done auto offset reset to earliest still no effect .
In kafka logs I’m frequently seeing this log message after throwing select query with emit changes:
[2021-06-16 05:24:23,755] INFO stream-client [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5] State transition from CREATED to REBALANCING (org.apache.kafka.streams.KafkaStreams:281)
[2021-06-16 05:24:23,755] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-1] Starting (org.apache.kafka.streams.processor.internals.StreamThread:516)
[2021-06-16 05:24:23,755] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-2] Starting (org.apache.kafka.streams.processor.internals.StreamThread:516)
[2021-06-16 05:24:23,757] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-1] State transition from CREATED to STARTING (org.apache.kafka.streams.processor.internals.StreamThread:225)
[2021-06-16 05:24:23,757] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-2] State transition from CREATED to STARTING (org.apache.kafka.streams.processor.internals.StreamThread:225)
[2021-06-16 05:24:23,757] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-3] Starting (org.apache.kafka.streams.processor.internals.StreamThread:516)
[2021-06-16 05:24:23,757] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-3] State transition from CREATED to STARTING (org.apache.kafka.streams.processor.internals.StreamThread:225)
[2021-06-16 05:24:23,757] INFO Streaming query 'select * from jsondata emit changes;' (io.confluent.ksql.rest.server.resources.streaming.StreamedQueryResource:314)
[2021-06-16 05:24:23,757] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-4] Starting (org.apache.kafka.streams.processor.internals.StreamThread:516)
[2021-06-16 05:24:23,757] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-4] State transition from CREATED to STARTING (org.apache.kafka.streams.processor.internals.StreamThread:225)
[2021-06-16 05:24:23,859] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-1] Processed 0 total records, ran 0 punctuators, and committed 0 total tasks since the last update (org.apache.kafka.streams.processor.internals.StreamThread:732)
[2021-06-16 05:24:23,863] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-4] Processed 0 total records, ran 0 punctuators, and committed 0 total tasks since the last update (org.apache.kafka.streams.processor.internals.StreamThread:732)
[2021-06-16 05:24:23,864] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-2] Processed 0 total records, ran 0 punctuators, and committed 0 total tasks since the last update (org.apache.kafka.streams.processor.internals.StreamThread:732)
[2021-06-16 05:24:23,864] INFO stream-thread [_confluent-ksql-default_transient_7238284644999143506_1623821063655-e15ceb24-3774-426c-b6c2-d3f5e3047aa5-StreamThread-3] Processed 0 total records, ran 0 punctuators, and committed 0 total tasks since the last update (org.apache.kafka.streams.processor.internals.StreamThread:732)
[2021-06-16 05:24:49,461] INFO Retrying request. Retry no: 1 Cause: 'org.apache.kafka.common.errors.TimeoutException: Call(callName=findCoordinator, deadlineMs=1623821089460, tries=596, nextAllowedTryMs=1623821089561) timed out at 1623821089461 after 596 attempt(s)' (io.confluent.ksql.util.ExecutorUtil:95)
@mjsax Can you help?
Can you share the CREATE STREAM command and the query you issue? Also, the command to change auto.offset.set (it must use quotes to be correct…)
The log you show are only INFO logs…
This topic was automatically closed after 30 days. New replies are no longer allowed.