Create a stream with no columns just to count events

[ortizfabio]

Hi,
I needed to create a stream for the sole purpose of create events per hour. I am trying to avoid the AVRO deserialization of the message for performance reasons. It looks like KSQLDB enforces a column. I tried:

CREATE STREAM stream_test WITH ( KAFKA_TOPIC = 'test_topic', VALUE_FORMAT = 'NONE');
CREATE STREAM stream_test (ROWTIME TIMESTAMP) WITH ( KAFKA_TOPIC = 'test_topic', VALUE_FORMAT = 'NONE');
Finally this is the only one that worked:
CREATE STREAM my_stream (eventKey VARCHAR KEY) WITH ( KAFKA_TOPIC = 'my_topic', VALUE_FORMAT = 'NONE');

However the topic does not have keys.
Is there any syntax for this case?

[mjsax]

It looks like KSQLDB enforces a column.

Yes it does – Like any other SQL-based system…

I guess the best thing would be to use BYTES types (not VARCHAR) (as introduced in 0.21 via KLIP-52) as it would not do any deserialization.

There is also no need to define a key column, but you can just declare a single value column with any arbitrary name using KAFKA format:

CREATE STREAM my_stream (someRandomName BYTES)
  WITH (KAFKA_TOPIC = ‘my_topic’, VALUE_FORMAT = ‘KAFKA’);

[ortizfabio]

That command returned error:

Value format does not support schema.

format: KAFKA
schema: Persistence{columns=[`ANYVALUE` BYTES], features=[]}
reason: The 'KAFKA' format does not support type 'BYTES'

So I tried this one and it worked:
CREATE STREAM my_stream (ANYVALUE BYTES) WITH (KAFKA_TOPIC='my_topic', KEY_FORMAT='KAFKA', VALUE_FORMAT='JSON');

Thanks

[mjsax]

@ortizfabio

The ‘KAFKA’ format does not support type ‘BYTES’

Interesting. Not sure why that is. Let me follow up internally on it.

[jzaralim]

That is a bug: KAFKA format does not support BYTES · Issue #8693 · confluentinc/ksql · GitHub

You can still do the same thing with STRING though:

CREATE STREAM my_stream (someRandomName STRING)
  WITH (KAFKA_TOPIC = ‘my_topic’, VALUE_FORMAT = ‘KAFKA’);

[ortizfabio]

Ok, thanks for confirming.