Control Center - Enabling HTTPS

Hi,

I am using confluent platform local Mac setup.

  1. I have followed GitHub - confluentinc/confluent-platform-security-tools: Security tools for the Confluent Platform.

In the above mentioned script, I just exported the necessary variables and triggered it, it has created key and trust stores

OR

  1. I followed this page Security Tutorial | Confluent Documentation

Same result as in the first step.

In both attempts, I have added the below properties to etc/control-center.properties

confluent.controlcenter.rest.listeners=https://localhost:9021
confluent.controlcenter.rest.ssl.keystore.location=/Users/gibby/Documents/data/certs
confluent.controlcenter.rest.ssl.keystore.password=abcd12
confluent.controlcenter.rest.ssl.key.password=abcd12
confluent.controlcenter.rest.ssl.truststore.location=/Users/gibby/Documents/data/certs
confluent.controlcenter.rest.ssl.truststore.password=abcd12
  1. Then I restarted the control center

confluent local services control-center stop confluent local services control-center start

OR

  1. I tried restarting the entire platform

confluent local services stop confluent local services start

Analysis:

I checked the logs, no error in control-center.stdout

I noticed that logs in that file doesn’t have the above mentioned properties under control-center-configs.

2022-06-09 19:43:41,007] INFO [main] ControlCenterConfig values: 
    auth.bearer.roles.claim = 
    bootstrap.servers = [localhost:9092]
    confluent.controlcenter.alert.cluster.down.autocreate = false
    confluent.controlcenter.alert.cluster.down.send.rate = 12
    confluent.controlcenter.alert.cluster.down.to.email = 
    confluent.controlcenter.alert.cluster.down.to.pagerduty.integrationkey = 
    confluent.controlcenter.alert.cluster.down.to.webhookurl.slack = 
    confluent.controlcenter.alert.max.trigger.events = 1000
    confluent.controlcenter.auth.bearer.issuer = Confluent
    confluent.controlcenter.auth.restricted.roles = []
    confluent.controlcenter.auth.session.expiration.ms = 0
    confluent.controlcenter.broker.config.edit.enable = true
    confluent.controlcenter.command.streams.start.timeout = 300000
    confluent.controlcenter.command.topic = _confluent-command
    confluent.controlcenter.command.topic.replication = 1
    confluent.controlcenter.command.topic.retention.ms = 259200000
    confluent.controlcenter.consumer.metadata.timeout.ms = 15000
    confluent.controlcenter.consumers.view.enable = true
    confluent.controlcenter.data.dir = /var/folders/25/3y752g4x77j_ps4gxz27yy8m0000gn/T/confluent.006186/control-center/data
    confluent.controlcenter.deprecated.views.enable = false
    confluent.controlcenter.disk.skew.warning.min.bytes = 1073741824
    confluent.controlcenter.hostedmonitoring.enable = false
    confluent.controlcenter.id = 1
    confluent.controlcenter.internal.streams.start.timeout = 21600000
    confluent.controlcenter.internal.topics.changelog.segment.bytes = 134217728
    confluent.controlcenter.internal.topics.partitions = 2
    confluent.controlcenter.internal.topics.replication = 1
    confluent.controlcenter.internal.topics.retention.bytes = -1
    confluent.controlcenter.internal.topics.retention.ms = 604800000
    confluent.controlcenter.ksql.enable = true
    confluent.controlcenter.license.manager = _confluent-controlcenter-license-manager-6-2-0
    confluent.controlcenter.license.manager.enable = true
    confluent.controlcenter.mail.bounce.address = 
    confluent.controlcenter.mail.enabled = false
    confluent.controlcenter.mail.from = c3@confluent.io
    confluent.controlcenter.mail.host.name = localhost
    confluent.controlcenter.mail.password = 
    confluent.controlcenter.mail.port = 587
    confluent.controlcenter.mail.ssl.checkserveridentity = false
    confluent.controlcenter.mail.starttls.required = false
    confluent.controlcenter.mail.username = 
    confluent.controlcenter.name = _confluent-controlcenter
    confluent.controlcenter.proactive.support.ui.cta.enable = true
    confluent.controlcenter.purge.stale.cluster.enable = false
    confluent.controlcenter.request.buffer.size.bytes = 10000
    confluent.controlcenter.rest.advertised.url = 
    confluent.controlcenter.rest.compression.enable = true
    confluent.controlcenter.rest.csrf.prevention.enable = false
    confluent.controlcenter.rest.csrf.prevention.token.endpoint = /csrf
    confluent.controlcenter.rest.csrf.prevention.token.expiration.minutes = 30
    confluent.controlcenter.rest.hsts.enable = true
    confluent.controlcenter.rest.port = 9021
    confluent.controlcenter.sbk.ui.enable = true
    confluent.controlcenter.schema.registry.enable = true
    confluent.controlcenter.schema.registry.url = [http://localhost:8081]
    confluent.controlcenter.service.healthcheck.interval.sec = 20
    confluent.controlcenter.streams.cache.max.bytes.buffering = 1073741824
    confluent.controlcenter.streams.consumer.session.timeout.ms = 60000
    confluent.controlcenter.streams.num.stream.threads = 12
    confluent.controlcenter.streams.producer.compression.type = lz4
    confluent.controlcenter.streams.producer.delivery.timeout.ms = 2147483647
    confluent.controlcenter.streams.producer.linger.ms = 500
    confluent.controlcenter.streams.producer.max.block.ms = 9223372036854775807
    confluent.controlcenter.streams.producer.retries = 2147483647
    confluent.controlcenter.streams.producer.retry.backoff.ms = 100
    confluent.controlcenter.streams.retries = 2147483647
    confluent.controlcenter.streams.upgrade.from = 2.3
    confluent.controlcenter.topic.inspection.enable = true
    confluent.controlcenter.trigger.active-controller-count.enable = false
    confluent.controlcenter.ui.autoupdate.enable = false
    confluent.controlcenter.ui.controller.chart.enable = false
    confluent.controlcenter.ui.data.expired.threshold = 120
    confluent.controlcenter.ui.replicator.monitoring.enable = true
    confluent.controlcenter.usage.data.collection.enable = true
    confluent.controlcenter.webhook.enabled = true
    confluent.license = 
    confluent.metadata.basic.auth.user.info = [hidden]
    confluent.metadata.bootstrap.server.urls = []
    confluent.metadata.cluster.registry.enable = false
    confluent.metadata.cluster.registry.merge.configuration.enable = true
    confluent.metrics.topic = _confluent-metrics
    confluent.metrics.topic.config.validate = false
    confluent.metrics.topic.max.message.bytes = 10485760
    confluent.metrics.topic.partitions = 12
    confluent.metrics.topic.replication = 1
    confluent.metrics.topic.retention.bytes = -1
    confluent.metrics.topic.retention.ms = 259200000
    confluent.metrics.topic.skip.backlog.minutes = 15
    confluent.monitoring.interceptor.topic = _confluent-monitoring
    confluent.monitoring.interceptor.topic.config.validate = false
    confluent.monitoring.interceptor.topic.partitions = 2
    confluent.monitoring.interceptor.topic.replication = 1
    confluent.monitoring.interceptor.topic.retention.bytes = -1
    confluent.monitoring.interceptor.topic.retention.ms = 259200000
    confluent.monitoring.interceptor.topic.skip.backlog.minutes = 15
    confluent.support.metrics.enable = true
    confluent.support.metrics.segment.id = MORqDG61F2eE5mfxAXVqpEblmFG18nbv
    public.key.path = 
    zookeeper.connect = localhost:2181
 (io.confluent.controlcenter.ControlCenterConfig)

But I see SSL related configs under AdminClientConfig but not control-centre-rest related.

Goal: I am just trying to get https for Control Center. What am I missing here?

Hi @gibby

welcome :slight_smile:

one question did you follow the following to install?

just to understand your starting point.
what does

confluent local services status

say?

best,
michael

Hi,

REST is down, could that be the reason?

gopir-mac-1:confluent-6.2.0 gopir$ confluent local services status

The local commands are intended for a single-node development environment only,

NOT for production usage. https://docs.confluent.io/current/cli/index.html

Using CONFLUENT_CURRENT: /var/folders/25/3y752g4x77j_ps4gxz27yy8m0000gn/T/confluent.006186

Connect is [UP]

Control Center is [UP]

Kafka is [UP]

Kafka REST is [DOWN]

ksqlDB Server is [UP]

Schema Registry is [UP]

ZooKeeper is [UP]

Thanks for your time,
Gopi.

I have downloaded confluent-platform 6.2.0 from archives

Started all the services, added mongo-sink connector, produced and consumed 10k+ events.

Thought of exploring SSL/TLS for C3, ended up with the above error. Default logging is enabled for all services, So I could see only INFO+ messages not DEBUG.

I did the below, REST is UP now.

confluent local services kafka-rest status

The local commands are intended for a single-node development environment only,

NOT for production usage. https://docs.confluent.io/current/cli/index.html

Using CONFLUENT_CURRENT: /var/folders/25/3y752g4x77j_ps4gxz27yy8m0000gn/T/confluent.006186

Kafka REST is [DOWN]

confluent local services kafka-rest start

The local commands are intended for a single-node development environment only,

NOT for production usage. https://docs.confluent.io/current/cli/index.html

thanks for providing the details

so just to be sure:
you would like to make the control center available via tls/https ?
but no interbroker tls encryption, right?

best,
michael

Yes, You are correct. Only HTTPS for C3, No inter broker encryption.

ok I see

you changed the config in /etc/kafka/ right?

I think you have to change the conf in
/var/folders/25/3y752g4x77j_ps4gxz27yy8m0000gn/T/confluent.006186/

there should be a subfolder called control-center and there should reside a field control-center.properties

best,
michael

Hi,

Yes, I changed in /etc/kafka.

But the below attempt works

  1. modify etc/control-center-dev.properties not control-center.properties
  2. When you restart the service, this property file is placed in /var/… folder

Thanks @mmuehlbeyer for your hint.

It shows that certificate is invalid, considers it as self-signed one. But it shows as below when I see it in the browser for certificate details

Self-signed root certificate
The certificate has not been verified by a third party

Kindly shed some light on this.

hi @gibby

I think the warning/error message is exptected as its a self signed cert.

are you able to start control center ui?

best,
michael

Hi @mmuehlbeyer

Yes, it works as expected. But there is that self-signed cert warning. Is it ok to go to production with this warning in the private network or should I go for other mechanisms like GSSAPI?
Kindly suggest.

Thanks.

Hi @gibby

I would recommend to sign the certificate by your internal CA.

best,
michael

Thanks @mmuehlbeyer

It helps.