How to setup ACL between Kafka and ZooKeeper

Ops
1

Need to setup acl b/w kafka and zookeeper please help with the steps

2

Hi @sagarbhat

welcome to the forum :slight_smile:
did you check the following docs and howtos:

best,
michael

1 Like
3

Acl is setup but how to test it please provide the steps for testing

[zk: localhost:2181(CONNECTED) 0] getAcl /kafka
'sasl,'kafka
: cdrwa
'world,'anyone
: r
[zk: localhost:2181(CONNECTED) 2] deleteall /kafka
Authentication is not valid : /kafka/config/brokers
[zk: localhost:2181(CONNECTED) 3] addauth digest kafka:******
[zk: localhost:2181(CONNECTED) 4] deleteall /kafka
Authentication is not valid : /kafka/config/brokers

once auth is set not able to delete kafka folder even after providing the auth.

4

could you please share what you would like to do?

and share the config as well?

Iā€™m not 100% sure what you would like to achieve

Best,
michael

5

Below configs are added

kafka jaas config 
Client {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

zookeeper jaas config
KafkaServer {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="super" password="123456" user_super="super";
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

QuorumServer {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

QuorumLearner {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

vi /mnt1/zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/mnt1/data/zookeeper
clientPort=2181
dataLogDir=/mnt1/zklog/datalog
server.0 = ******:2888:3888
maxClientCnxns=0
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
4lw.commands.whitelist=*
requireClientAuthScheme=sasl
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

vi /mnt1/kafka/config/server.properties

broker.id=1
host.name=*********
port=9092

message.max.bytes=10485760
num.partitions=30
auto.create.topics.enable=true

# Replication configurations
default.replication.factor=1
num.replica.fetchers=1
replica.fetch.max.bytes=10485760


log.dirs=/mnt1/data/kafka/kafka-logs-1
log.retention.hours=3
log.flush.interval.ms=10000
log.flush.interval.messages=20000
log.flush.scheduler.interval.ms=2000
log.cleanup.interval.mins=30

zookeeper.connect=**********:2181/kafka
zookeeper.connection.timeout.ms=1000000

# Socket server configuration
num.io.threads=8
num.network.threads=8
socket.request.max.bytes=104857600
socket.receive.buffer.bytes=1048576
socket.send.buffer.bytes=1048576
queued.max.requests=16
fetch.purgatory.purge.interval.requests=100
producer.purgatory.purge.interval.requests=100

delete.topic.enable=true
zookeeper.set.acl=true

vi /mnt1/zookeeper/conf/java.env
export JVMFLAGS=" -Xms7g -Xmx7g "
export ZOO_LOG4J_PROP="INFO, ROLLINGFILE"
export ZOO_LOG_DIR="/mnt1/logs/zookeeper"
export SERVER_JVMFLAGS="-Djava.security.auth.login.config=/mnt1/apache-zookeeper-3.5.9-bin/conf/jaas.conf"
export CLIENT_JVMFLAGS="-Djava.security.auth.login.config=/mnt1/apache-zookeeper-3.5.9-bin/conf/jaas.conf"

export KAFKA_OPTS="-Djava.security.auth.login.config=/opt/kafka/kafka/config/jaas.conf" added in startup script

what you would like to achieve?
how can i revert it back once acl is set on kafka is there a way to delete and test

[zk: localhost:2181(CONNECTED) 2] ls  /
[kafka, zookeeper]

[zk: localhost:2181(CONNECTED) 0] getAcl /kafka
'sasl,'kafka
: cdrwa
'world,'anyone
: r
6

you could use the kafka-acl tool

see
https://jaceklaskowski.gitbooks.io/apache-kafka/content/kafka-tools-kafka-acls.html

and

hth,
michael