# Rebalancing tasks when new kafka connect is started **URL:** \ **Category:** Kafka Connect\ **Created:** [15 February 2021 13:24 UTC](https://forum.confluent.io/t/rebalancing-tasks-when-new-kafka-connect-is-started/876 "2021-02-15T13:24:46Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![Sirius\_Dour](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/sirius_dour/32/454_2.png) [@Sirius\_Dour](https://forum.confluent.io/u/Sirius_Dour)\ **Post date:** [15 February 2021 13:24 UTC](https://forum.confluent.io/t/rebalancing-tasks-when-new-kafka-connect-is-started/876/1 "2021-02-15T13:24:47Z")
Hi Could you help me to understand how rebalancing is working? I have next scenario: 1. kafka-connect-1 is started 2. add connector with 4 tasks via POST /connectors 3. checking status via GET /connectors/myConnector/status i can see my connector and 4 tasks for it 4. i am starting kafka-connect-2 5. checking status again and can see that all tasks still on kafka-connect-1 6. now if i do DELETE /connectors/myConnector/ then POST /connectors and check status again then i can see that now 4 tasks are spread between 2 connects My questions are - is there any way to trigger rebalance automatically after new kafka-connect joining cluster? - is this del+add connector is correct way to trigger rebalance manually? p.s. - kafka-connect-1 and kafka-connect-2 in the same cluster - [restapi](https://docs.confluent.io/platform/current/connect/references/restapi.html#) ---
**Author:** ![gianlucanatali](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/gianlucanatali/32/302_2.png) [@gianlucanatali](https://forum.confluent.io/u/gianlucanatali)\ **Post date:** [15 February 2021 14:36 UTC](https://forum.confluent.io/t/rebalancing-tasks-when-new-kafka-connect-is-started/876/2 "2021-02-15T14:36:23Z")
Hi @Sirius_Dour , you can find some details in this blog post: [Incremental Cooperative Rebalancing in Apache Kafka](https://www.confluent.io/blog/incremental-cooperative-rebalancing-in-kafka/) Can you try adding the configuration property `connect.protocol=eager` and see if that makes it work as you expect? Keep us posted! ---
**Author:** ![Sirius\_Dour](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/sirius_dour/32/454_2.png) [@Sirius\_Dour](https://forum.confluent.io/u/Sirius_Dour)\ **Post date:** [16 February 2021 07:34 UTC](https://forum.confluent.io/t/rebalancing-tasks-when-new-kafka-connect-is-started/876/3 "2021-02-16T07:34:39Z")
Hi @gianlucanatali Looks like it is working as i wanted: i added `connect.protocol=eager` in my .properties file for kafka-connect-1 and kafka-connect-2 i started kafka-connect-1 (cube) and added connector with 4 tasks on it then i just started kafka-connect-2 (smart) and checked status ![image](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/505d701eaf2645c84bcde66cad3b8c3e4d491536.png) i can see that all tasks are rebalanced between these 2 nodes Thank you! ---
**Author:** ![Sirius\_Dour](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/sirius_dour/32/454_2.png) [@Sirius\_Dour](https://forum.confluent.io/u/Sirius_Dour)\ **Post date:** [16 February 2021 08:39 UTC](https://forum.confluent.io/t/rebalancing-tasks-when-new-kafka-connect-is-started/876/4 "2021-02-16T08:39:49Z")
I have read [Incremental Cooperative Rebalancing in Apache Kafka](https://www.confluent.io/blog/incremental-cooperative-rebalancing-in-kafka/) And i have additional questions: so my kafka-connect-1 and kafka-connect-2 are just “workers”. So it mean that case that i described is exact representation of **1. A new worker joins (Figure 1).** and it says that during 2-d rebalance some tasks should be moved to this new worker (kafka-connect-2/smart in my case) and other tasks should not be interrupted. But it isn’t happen: without `connect.protocol=eager` property all tasks remains assigned to 1-st worker (cube). ![image](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/767962cad586ef9a0a209efdf6d253008808b240.png) (on the left status from worker-1 on the right status from worker-2) And this property forces my worker to use _Eager Rebalancing_. Am i correct? Why _Incremental Cooperative Rebalancing_ from this Figure 1 is not working? Will this rebalance trigger tasks revocations and interruptions as result? I tried to find more details about this property but found only this [https://docs.confluent.io/platform/current/installation/configuration/connect/index.html#connect.protocol](https://docs.confluent.io/platform/current/installation/configuration/connect/index.html#connect.protocol) ---
**Author:** ![Sirius\_Dour](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/sirius_dour/32/454_2.png) [@Sirius\_Dour](https://forum.confluent.io/u/Sirius_Dour)\ **Post date:** [16 February 2021 12:20 UTC](https://forum.confluent.io/t/rebalancing-tasks-when-new-kafka-connect-is-started/876/5 "2021-02-16T12:20:44Z")
Maybe my test wasn’t very accurate: i realized that there was 3 connectors but i checked status only for one of them. Now i repeated this test with only 1 connector onboard and without this property here is the result ![image](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/fc254a502a8b44ff73b6a96a65528908e4732ce9.png) look like 2 tasks moved to new worker recheck with all 3 connectors (without property) also shows correct results if check status of each connector: after worker-2 joined i got next picture: - worker-1: 4x connector-1 + 2x connector-3 = 6 tasks in total - worker-2: 4x connector-2 + 1x connector-3 = 5 tasks in total it looks balanced (based on amount of on each worker) recheck with all 3 connectors (with property) shows these results: - worker-1: 2x connector-1 + 2x connector-2 + 1x connector-3 = 5 tasks in total - worker-2: 2x connector-1 + 2x connector-2 + 2x connector-3 = 6 tasks in total this is also balanced but in more strict way ---
**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/c49438c90c9df282e9996fdf6971be890c71b65a.svg) [@system](https://forum.confluent.io/u/system)\ **Post date:** [23 February 2021 12:21 UTC](https://forum.confluent.io/t/rebalancing-tasks-when-new-kafka-connect-is-started/876/6 "2021-02-23T12:21:40Z")
This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.