# Spring cloud stream kafka is very slow

**URL:** <https://forum.confluent.io/t/spring-cloud-stream-kafka-is-very-slow/9678>\
**Category:** Kafka Streams\
**Created:** [27 December 2023 03:13 UTC](https://forum.confluent.io/t/spring-cloud-stream-kafka-is-very-slow/9678 "2023-12-27T03:13:05Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![himanshu1063](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/himanshu1063/32/3462_2.png) [@himanshu1063](https://forum.confluent.io/u/himanshu1063)\
**Post date:** [27 December 2023 03:13 UTC](https://forum.confluent.io/t/spring-cloud-stream-kafka-is-very-slow/9678/1 "2023-12-27T03:13:05Z")

</div>

I am using spring cloud stream with Kafka and Avro.

I want to messages to be consumed very fast but unfortunately it is not meeting my desired expectation.  
I have over 100 millions messages being consumed. I have 20 pods. It takes around 6 hours for complete consumption which is too much. I am just consuming, merging with some other object and sending it to a new topic. I have increased instances of my consumer but still not able to achieve fast processing.

I don’t care if messages are lost or they are consumed twice.

I am adding the configurations I have added.

Please suggest how can I make it faster ? Any tips on spawning more threads or using some Kafka property ?

PS : I have 5 different listeners listening from different topics

```auto
spring:
  cloud:
    config:
    function:
      definition: RiskProcessor1;RiskProcessor2;RiskProcessor3;RiskProcessor4;RiskProcessor5
    stream:
      bindings:
        RiskProcessor1-in-0:
          destination: ******
        RiskProcessor1-out-0:
          destination: *****
        RiskProcessor2-in-0:
          destination: *****
        RiskProcessor2-out-0:
          destination: *****
        RiskProcessor3-in-0:
          destination: *****
        RiskProcessor3-out-0:
          destination: *****
        RiskProcessor4-in-0:
          destination: *****
        RiskProcessor4-out-0:
          destination: *****
        RiskProcessor5-in-0:
          destination: *****
        RiskProcessor5-out-0:
          destination: *****
      kafka:
        streams:
          binder:
            brokers: kaas-int.nam.nsroot.net:9093
            functions:
              RiskProcessor1:
                applicationId: RiskProcessor1_development
              RiskProcessor2:
                applicationId: RiskProcessor2_development
              RiskProcessor3:
                applicationId: RiskProcessor3_development
              RiskProcessor4:
                applicationId: RiskProcessor4_development
              RiskProcessor5:
                applicationId: RiskProcessor5_development
            configuration:
              commit.interval.ms: 1000
              security.protocol: SSL
              default:
                deserialization:
                  exception:
                    handler: org.apache.kafka.streams.errors.LogAndContinueExceptionHandler
              schema:
                registry:
                  url: ***** :9081
          default:
            consumer:
              keySerde: ****
              valueSerde: ****
            producer:
              keySerde: ****
              valueSerde: ****

```

---

<div class="post-metadata">

**Author:** ![himanshu1063](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/himanshu1063/32/3462_2.png) [@himanshu1063](https://forum.confluent.io/u/himanshu1063)\
**Post date:** [27 December 2023 15:45 UTC](https://forum.confluent.io/t/spring-cloud-stream-kafka-is-very-slow/9678/2 "2023-12-27T15:45:12Z")

</div>

If anyone if wondering. I have 20 partitions of topic and have 20 pods where each pods will have it’s consumer

---

<div class="post-metadata">

**Author:** ![mjsax](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/mjsax/32/3113_2.png) [@mjsax](https://forum.confluent.io/u/mjsax)\
**Post date:** [3 January 2024 22:01 UTC](https://forum.confluent.io/t/spring-cloud-stream-kafka-is-very-slow/9678/3 "2024-01-03T22:01:31Z")

</div>

> [@himanshu1063](#):
>
> ```auto
> commit.interval.ms: 1000
> 
> ```

Increasing commit interval could help (committing is an expensive blocking operation). Why do you reduce from 30sec (default) to 1sec?

---

<div class="post-metadata">

**Author:** ![himanshu1063](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/himanshu1063/32/3462_2.png) [@himanshu1063](https://forum.confluent.io/u/himanshu1063)\
**Post date:** [3 January 2024 22:29 UTC](https://forum.confluent.io/t/spring-cloud-stream-kafka-is-very-slow/9678/4 "2024-01-03T22:29:07Z")

</div>

Thanks for your response sir.  
No reason. It was like that when I took over project. I am still learning kafka and it’s working.  
I increased it to 30000ms but no difference

Is there any other configuration which I can add here I might not be aware of which can increase the speed ?

[Please note that when I tested the time intervals 30000, I have 20 partitions and 2 consumer pods]

---

<div class="post-metadata">

**Author:** ![mjsax](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/mjsax/32/3113_2.png) [@mjsax](https://forum.confluent.io/u/mjsax)\
**Post date:** [4 January 2024 09:12 UTC](https://forum.confluent.io/t/spring-cloud-stream-kafka-is-very-slow/9678/5 "2024-01-04T09:12:31Z")

</div>

The first question is: where is the bottleneck? CPU client side (-\> maybe add more pods)? Network (maybe change consumer “fetch” configs)?