Tombstone messages not propagated

Stream Processing ksqlDB
1

Hello,
I got a problem with something that looks like very basic functionality of KSQL.
I’m creating a table as select, from the stream:

CREATE OR REPLACE TABLE t1
WITH (KAFKA_TOPIC='pipeline1', PARTITIONS=12, REPLICAS=3, TIMESTAMP='timestamp') AS SELECT 
id1, 
latest_by_offset(v1),
latest_by_offset(v2)
FROM s1
GROUP by id1
EMIT CHANGES;

In stream s1, I’ve got 3 records for key (id) 8 with offset 0, 1, 2
0 is creation, 1 is update, 2 is tombstone record with value null
However I still see that record in KSQL table (it wasn’t removed). I don’t see tombstone record in topic ‘pipeline1’. How to enable propagation of tombstone record to downstream tables?
PS. I’m using ksqldb Version 0.20.0

2

Because your input s1 is a STREAM there is no tombstone semantics… The aggregation will treat the <key,null> record as invalid and drop it (an STREAM-aggregation cannot semantically “forward/process” a delete because the input record is no delete to begin with).

If you want to materialize a TABLE, you should instead create two TABLES:

CREATE TABLE myTable <schema> WITH (...);
CREATE TALBE materializedTable AS SELECT * FROM myTable;

This way, tombstone semantics are preserved.

Btw: we are currently working on a new feature (KLIP-49) that allows you to do the same in a single statement:

CREATE SOURCE TABLE materializedTable <schema> WITH (...);