Is it okay to have the producer running indefinitely once a connection is established?
There are couple of reasons for asking the question?
- What is the implication on the underlying resource/s (leader replica) when there is always an active open connection at all times?
- Idempotency only lasts for the lifetime of a producer instance. So restarting a new instance takes away a key feature/guarantee of the Kafka system
- What are the suggested best practices when it comes to maintaining the producer connection?
- For an application (like click streams that is constantly producing message), is there ever a need to close the producer connection?
- If for (4) there is no need to close the producer, can this be made generic to ALL applications and not close the producer connection at all?