I have two applications one write message and second consumer consum messages/
when consume message I send it to function that may take long time.
it say that the consumer wait untill get response from function.so I want to know if the way of partions correct
I config my application each server listen to one topic partition.
so if I have 1000 messages and two consumers and two partitions each consumer listen to 500 messages.if the first consumer wait long time’ the messages of this partions will not read untill the response arrive.
I want only one consumer consume message
so I want to know maybe I need another design
Your question is a little difficult to understand so I’m not certain what design advice to give you. I will recommend the Confluent Developer Apache Kafka 101 course which provides basic guidance for topics, partitions, consumers, producers, and other topics. You might find that this free online course could help you understand Kafka basics and provide an answer to your question.
Write 1000 messages to topic A
each partition hold 500 messages.
I created two consumers consumerA listen to Partition_1 and consumerB listen to Partition_2
the code look like:
…
var result = consumerA.consume(parameters…)
httpClient request ->send request to calc whatever // may take 5 minutes until response
my question is:
now the application wait 5 minutes untill the response arrived and not consume messages from partition_1
Are you asking why your client waits for the http request to complete before processing the next message from the Kafka Consumer? If so, it’s because of synchronous vs asynchronous design of your code and nothing specific to the Kafka consumer. If this is not what you mean, please elaborate on the question.
Hi,
I want to understand if the implement of partitions is right for me or need a different implementation.
.Each message should be read only onc*,
.I do not want to increase the number of requests to the server at a time(http request) - Therefore asynchronous requests are not good for them *.
If I choose to implement partitions and I am assigned let’s say 5 consumers then I have 5 parallel requests to the server.
If one consumer delay all messages in his partition are delayed and no one else will read them.
And if I work without partitions then the corresponding consumers will continue to read the other messages while the specific consumer is in delay.
But then, if I work without partitions I have a problem. Consumers read all the messages and I want each message to be read only once.