Use MQTT Proxy in Confluent Cloud

https://docs.confluent.io/platform/current/kafka-mqtt/intro.html#
Is this available in Confluent Cloud and how do I connect to it?
I only see it here in on-prem.
I want to go from client directly to Confluent Cloud.

hey @donnie

did you try the solution offered in

best,
michael

Thank you @mmuehlbeyer . Glad to know it also should work in cloud version.
I’m still waiting to find out how to pass these settings from the browser using mqtt.js and/or this Vue.js wrapper.

The docs on mqtt.js do not define how to make these settings in the Client constructor.
It doesn’t work with just username and password but it isn’t clear how to set the authenticationMethod and authenticationData or whichever you need to use.

On the Confluent Cloud MQTT Proxy side, will the MQTT over Websockets still work? (assuming I can get the right auth settings added to client)

This is what I’m trying. I tried using mqtt or wss but I don’t know why it is failing other than I’m not explicitly including SASL_SSL and PLAIN.
I’m am new to Confluent, so please be patient with me. :slight_smile:

hey @donnie

could you provide some details how you try to run your mqtt proxy?
I guess it’s running locally and tries to connect to confluent cloud right?
could you share the confs?

best,
michael

@mmuehlbeyer the purpose of this post was to ask if the MQTT Proxy work in cloud version of confluent by sending message to the cluster. I do not know how to turn that on in the cloud version. If the cluster doesn’t work with it, then it is dead in the water. I don’t want to stand up another server. I can just add a broker in the mix if I have to do that.

How does the Confluent Cloud front-end receive messages in the browser? what client does it use in the browser and how does it talk to the cluster?

This implies to me that the mqtt client sends directly to the cluster’s bootstrap.servers, which I thought handled the MQTT proxy built in. I could be wrong. That is what I was trying to find out in this thread. Clearly I misunderstood that you have to stand up your own proxy.

hi donnie,

not fully sure if I understand your plans correctly :wink:

if you’d like to connect to Confluent Cloud to an MQTT proxy you could use MQTT source connector.

though if my understanding is right you would like to send data directly to the cluster from an app correct?

best,
michael

Correct, I wanted to connect an IoT device and the browser to Confluent using an MQTT client. Without having to add an extra broker in the mix. I’m looking into Waterstream but I also wanted to learn how to use the MQTT Proxy option and go direct to Confluent Cloud.
I tried MQTT source connector with HiveMQ but I couldn’t get it to connect. They force you to use their connector, which is not a fully-managed Confluent connector that works in cloud version. But, again, a separate broker shouldn’t be required, from my perspective.
I appreciate your help

This could be documented better. But MQTT proxy requires explicit configuration of each of the Kafka clients used.

So for example, this is a working MQTT proxy configuration for Confluent Cloud (obviously, update bootstrap server and creds accordingly.

bootstrap.servers=<BOOTSTRAP_SERVER>
ssl.endpoint.identification.algorithm=https
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<API_KEY>" password="<API_SECRET>";

producer.bootstrap.servers=<BOOTSTRAP_SERVER>
producer.ssl.endpoint.identification.algorithm=https
producer.security.protocol=SASL_SSL
producer.sasl.mechanism=PLAIN
producer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<API_KEY>" password="<API_SECRET>";

consumer.bootstrap.servers=<BOOTSTRAP_SERVER>
consumer.ssl.endpoint.identification.algorithm=https
consumer.security.protocol=SASL_SSL
consumer.sasl.mechanism=PLAIN
consumer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<API_KEY>" password="<API_SECRET>";

confluent.topic.bootstrap.servers=<BOOTSTRAP_SERVER>
confluent.topic.ssl.endpoint.identification.algorithm=https
confluent.topic.security.protocol=SASL_SSL
confluent.topic.sasl.mechanism=PLAIN
confluent.topic.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="<API_KEY>" password="<API_SECRET>";

I’m not 100% sure you need all four of them, but I do know that this works.

1 Like

Their Confluent connector is forced upon you, and it isn’t a fully managed Cloud version of Confluent connector.However, I don’t believe that a separate broker should be necessary. Your assistance is greatly appreciated