I had a question about having a cleanup policy of compact AND delete.
I also asked in the slack channel (see link) but no clear result was found.
Here is my example topic configuration
kafka-topics.sh --describe --bootstrap-server $KAFKA_STAGING --topic my-topic Topic:my-topic PartitionCount:10 ReplicationFactor:2 Configs:min.insync.replicas=2,cleanup.policy=compact,delete,retention.ms=2592000000,message.format.version=2.6-IV0,max.message.bytes=6291456,unclean.leader.election.enable=true,segment.ms=1200000
With this configuration, I had messages which were several months old in my topic.
This leads me to believe that topics that are “compacted and deleted” only delete segments, which are not compacted and not written to at all (so more like a whole topic cleanup instead of individual messages).
Does someone with more experience have the use cases in mind, which lead to the implementation of the current compact and delete behavior? Because I somehow fail to understand good examples when I should use compaction and deletion with the current behavior.