# Handling of kafka stream if kafka broker goes down **URL:** \ **Category:** ksqlDB\ **Created:** [11 November 2021 15:06 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350 "2021-11-11T15:06:47Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![lucky2045](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/lucky2045/32/1309_2.png) [@lucky2045](https://forum.confluent.io/u/lucky2045)\ **Post date:** [11 November 2021 15:06 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/1 "2021-11-11T15:06:47Z")
Hi, I am facing one issue if any of my kafka broker from kafka cluster is getting down, KSQL stream is not able to push data to the topic. How can we use ksql stream if any broker is not available. ---
**Author:** ![mjsax](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/mjsax/32/3113_2.png) [@mjsax](https://forum.confluent.io/u/mjsax)\ **Post date:** [13 November 2021 03:24 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/2 "2021-11-13T03:24:16Z")
Seems there is a configuration problem. If you configure your brokers and topics correctly, a single broker being offline should not prevent ksqlDB to make progress. What is the replication factor of your topics? If the replication factor is only 1, it would explain the issue and you should increase it to three, to allow a different broker to take over if one broker goes offline. ---
**Author:** ![lucky2045](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/lucky2045/32/1309_2.png) [@lucky2045](https://forum.confluent.io/u/lucky2045)\ **Post date:** [15 November 2021 05:42 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/3 "2021-11-15T05:42:54Z")
I have a replication factor of 3 and have 10 partitions on each topic. _Command -_ ./bin/kafka-topics.sh --describe --topic \*\*\*\*\*\*\* --bootstrap-server _ **:9092,** _:9092,\*\*\*\*\*\*\*:9092 _Output -_ Topic: \*\*\*\*\*\*\*\*\* PartitionCount: 10 ReplicationFactor: 3 Configs: compression.type=lz4,min.insync.replicas=1,cleanup.policy=delete,segment.bytes=1073741824,retention.ms=900000,unclean.leader.election.enable=true ---
**Author:** ![lucky2045](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/lucky2045/32/1309_2.png) [@lucky2045](https://forum.confluent.io/u/lucky2045)\ **Post date:** [15 November 2021 05:46 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/4 "2021-11-15T05:46:35Z")
Also, I have another issue, how can I handle NullPointerException exception. Error Details : { “type”: 1, “deserializationError”: null, “recordProcessingError”: { “errorMessage”: “Error computing expression M\_BEFORE-\>IS\_AVAILABLE\_JCB for column M\_BEFORE\_IS\_AVAILABLE\_JCB with index 55”, “record”: null, “cause”: [ “java.lang.NullPointerException” ] }, “productionError”: null, “serializationError”: null, “kafkaStreamsThreadError”: null } In insert state, before value will be null, so while creating a stream it’s giving NullPointerException. How can i handle it. ---
**Author:** ![mjsax](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/mjsax/32/3113_2.png) [@mjsax](https://forum.confluent.io/u/mjsax)\ **Post date:** [15 November 2021 16:16 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/5 "2021-11-15T16:16:33Z")
Can you inspect if the leader for the topic partition changes broker side? Also, does kslqDB refresh its metadata about partition leaders (you would need to inspect the logs). > Error computing expression M\_BEFORE-\>IS\_AVAILABLE\_JCB for column M\_BEFORE\_IS\_AVAILABLE\_JCB with index 55 It seems `M_BEFORE` is `null`? It’s a known issue. You would need use `CASE` to first check for `NULL` and return `NULL` for this case, and evaluate `M_BEFORE->IS_AVAILABLE_JCB` only if not `NULL`. ---
**Author:** ![lucky2045](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/lucky2045/32/1309_2.png) [@lucky2045](https://forum.confluent.io/u/lucky2045)\ **Post date:** [18 November 2021 19:43 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/6 "2021-11-18T19:43:10Z")
Hi @mjsax I am getting the below logs from KSQL logs if one of my Kafka broker goes down for the cluster. I have a 3 node cluster setup. { “type”: 4, “deserializationError”: null, “recordProcessingError”: null, “productionError”: null, “serializationError”: null, “kafkaStreamsThreadError”: { “errorMessage”: “Unhandled exception caught in streams thread”, “threadName”: “_confluent-ksql-\*\*\*\*\*\*\*\*\*\*\*\*query\_CSAS_**\_247-0c0fec9f-aa6b-4de7-83cf-c8f45dcaf795-StreamThread-1", “cause”: [ "Error encountered sending record to topic _confluent-ksql-\*\*\*\*\*\*\*\*\*\*\*\*\*\*query\_CSAS_\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\_247-Join-left-repartition for task 0\_6 due to:\norg.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for _confluent-ksql-\*\*\*\*\*\*\*\*\*\*query\_CSAS_**\_247-Join-left-repartition-4:120001 ms has passed since batch creation\nThe broker is either slow or in bad state (like not having enough replicas) in responding the request, or the connection to broker was interrupted sending the request or receiving the response. \nConsider overwriting `max.block.ms` and /or `delivery.timeout.ms` to a larger value to wait longer for such scenarios and avoid timeout errors\nException handler choose to FAIL the processing, no more records would be sent.”, “Expiring 1 record(s) for _confluent-ksql-\*\*\*\*\*\*\*\*\*\*query\_CSAS_\*\*\*\*\*\*\*\*\*\*\*\*\_247-Join-left-repartition-4:120001 ms has passed since batch creation” ] } } WARN [Consumer clientId=_confluent-ksql-\*\*\*\*\*\*\*\*\*\*\*query\_CSAS_\*\*\*\*\*\*\*\*\*\*\*\*\*\*\_247-0c0fec9f-aa6b-4de7-83cf-c8f45dcaf795-StreamThread-3-consumer, groupId=_confluent-ksql-\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*query\_CSAS_\*\*\*\*\*\*\*\*\*_\_247] Connection to node 1 (/..._:9092) could not be established. Broker may not be available. (org.apache.kafka.clients.NetworkClient:796) If you need any more details please let me know. ---
**Author:** ![lucky2045](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/lucky2045/32/1309_2.png) [@lucky2045](https://forum.confluent.io/u/lucky2045)\ **Post date:** [18 November 2021 19:44 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/7 "2021-11-18T19:44:04Z")
Thanks @mjsax CASE solved my nullPointerException. ---
**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/c49438c90c9df282e9996fdf6971be890c71b65a.svg) [@system](https://forum.confluent.io/u/system)\ **Post date:** [11 December 2021 15:06 UTC](https://forum.confluent.io/t/handling-of-kafka-stream-if-kafka-broker-goes-down/3350/8 "2021-12-11T15:06:50Z")
This topic was automatically closed after 30 days. New replies are no longer allowed.