I’m trying to configure and see how compact topics work, but all my attempts have been unsuccessful.
1st I did with data that came from Kafka-connect, then I rekey one of the new topics (so I can have a key), but I never got only the last state of the same key!
Today I’ve been playing with an example that I found on the internet.
I’m going to share one of the examples that I found to see if someone can help me find what is wrong with the configuration.
I’ve been testing Kafka within docker containers, using the latest version on each image (images from confluent).
In here I guess that I have all the required configurations (cleanup.policy, delete.retention, min.cleanable.dirty.ratio) to make compact topics work
kafka-topics --create --zookeeper zookeeper:2181 --topic latest-product-price --replication-factor 1 --partitions 1 --config "cleanup.policy=compact" --config "delete.retention.ms=100" --config "segment.ms=100" --config "min.cleanable.dirty.ratio=0.01"
Then I’ve started my producer with
kafka-console-producer --broker-list localhost:9092 --topic latest-product-price --property parse.key=true --property key.separator=:
I’ve inserted this in the producer
>1:Tiago >2:Silvia >3:Ruizinho >3:Rui >1:Tiago >2:Silvia >3:Ruizinho >3:Rui
Then I started my consumer with
kafka-console-consumer --bootstrap-server localhost:9092 --topic latest-product-price --property print.key=true --property key.separator=: --from-beginning
and I got the this
1:Tiago 2:Silvia 3:Ruizinho 3:Rui (...)
For key 3 I should have Rui instead of Ruizinho.
What am I doing wrong here?
Thank you in advance!
Meanwhile, I’ve been testing it and I notice that now the consumer shows (using the same command that I’ve showed before)
3:Rui 1:um 2:dois 3:tres
Is this an issue only with the representation of the data?
I’ve tested it with KsqlDB, but I also got duplicates for the lat value that I’ve updated.
Why is it showing the values Rui and tres for the key 3?.