we are using confluent kafka platform and java to stream binary messages from IOT devices to the kafka.
The IOT devices encrypts the part of a message using an aes key. The java app needs to do custom security handshake and get the key as a message from the kafka topic and do decryption, if the security handshake exchange fails ,then IOT will never place the data until the handshake is successful.
iot (AMQP queue starts handshake process when client connects or amqp broker reconnects.) —> gateway (dumb one K8S java app write to kafka topic) —> decoder( K8S kstream app decodes as per avro schema,decrypted sub message) —> decryption(? where to fit in /handshake message can come at anytime in this topic/ combine logic in decoder ) ----> sub msg decoder(parse business events) —> downstream app
my initial design is to use Request async to do in a separate flow and then re flow the unencrypted message to sub msg decoder. how can better be implemented in Kafka