When increasing task size of application takes one hour to connect

Kafka Streams
1

Hello ,
brief on the issue im having is on second deployment of my application after all internal topics have been created and when having more than one task of said application in my case i have 3 tasks. the application takes between 30 minutes to one hour to start consuming events again. I am connecting with IAM, my topics have 20 partitions and below i will link my configuration if someone has any idea would be greatly appreciated

Properties createStreamConfiguration(){
    final String bootstrapServers = Config.getKafkaClusterEndpoint();
    final Properties streamsConfiguration = new Properties();
    streamsConfiguration.put(StreamsConfig.APPLICATION_ID_CONFIG,Config.getKafkaApplicationIdConfig());
    streamsConfiguration.put(StreamsConfig.CLIENT_ID_CONFIG, Config.getKafkaClientIdConfig());
    streamsConfiguration.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
    streamsConfiguration.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
    streamsConfiguration.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, EventSerdes.serdes().getClass());
    

    if (Config.getIsKafkaIamConnection()){
        streamsConfiguration.put(CommonClientConfigs.SECURITY_PROTOCOL_CONFIG,"SASL_SSL");
        streamsConfiguration.put(SaslConfigs.SASL_MECHANISM, "AWS_MSK_IAM");
        streamsConfiguration.put(SaslConfigs.SASL_JAAS_CONFIG, "software.amazon.msk.auth.iam.IAMLoginModule required;");
        streamsConfiguration.put("sasl.client.callback.handler.class", software.amazon.msk.auth.iam.IAMClientCallbackHandler.class.getName());
    }
    
    streamsConfiguration.put(StreamsConfig.COMMIT_INTERVAL_MS_CONFIG, 100);

    return streamsConfiguration;
}
2

Hi @Javitan0309 ,

From what you’re describing, it sounds like the application is restoring state. Can you share any logs? Do you have stateful operations in the topology?