We are running a dev environment on Windows with Kafka Broker and our own Kafka Producer and Consumer implementation using Java.
After running the broker for a day or so, the broker just goes down. Due to logs folder getting filled up. We manage by cleaning up the tmp\kafka-logs folder and then restarting Kafka Broker.
Obviously, this is not an option in production. Please suggest how we could fix this. We are running Kafka on windows.
I googled this and some solutions suggested were using Kafka as a docker container or running it on Linux. We could also have an automatic batch file to periodically restart Kafka/clean up the logs. But that is not a good idea at all in production.
Please suggest if there is any config change or any other solution to solve this.