Altering the cleanup.policy on Kafka topics in Production

Ops
1

If I were update the clean up policy on a number of topics in a production Kafka instance, would it be possible without restarting the brokers? Would I need to rebalance the cluster afterwords?

Thanks,

2

hi @jcrichfield

welcome :slight_smile:

you could change per topic settings without restarting the cluster.

example from my local docker test env:

default settings

docker exec -it kafka-1 kafka-topics --describe --topic "mytest" --bootstrap-server kafka-1:19092
Topic: mytest	TopicId: cUITkrqwQx29sRbcuPCI8Q	PartitionCount: 1	ReplicationFactor: 1	Configs: message.timestamp.type=LogAppendTime
	Topic: mytest	Partition: 0	Leader: 1	Replicas: 1	Isr: 1

change settings

docker exec -it kafka-1 kafka-configs --alter --bootstrap-server kafka-1:19092 --entity-type topics --entity-name mytest --add-config cleanup.policy=compact

new settings

docker exec -it kafka-1 kafka-topics --describe --topic mytest --bootstrap-server kafka-1:19092
Topic: mytest	TopicId: cUITkrqwQx29sRbcuPCI8Q	PartitionCount: 1	ReplicationFactor: 1 Configs: cleanup.policy=compact,message.timestamp.type=LogAppendTime
	Topic: mytest	Partition: 0	Leader: 1	Replicas: 1	Isr: 1

hth,
michael

1 Like
3

Thanks for the reply. Can the --add-config modify a pre-existing setting as well?

4

mhh afaik the best way is to
delete and readd the config you’d like to change

best,
michael