How to setup ACL between Kafka and ZooKeeper

Need to setup acl b/w kafka and zookeeper please help with the steps

Hi @sagarbhat

welcome to the forum :slight_smile:
did you check the following docs and howtos:

best,
michael

1 Like

Acl is setup but how to test it please provide the steps for testing

[zk: localhost:2181(CONNECTED) 0] getAcl /kafka
'sasl,'kafka
: cdrwa
'world,'anyone
: r
[zk: localhost:2181(CONNECTED) 2] deleteall /kafka
Authentication is not valid : /kafka/config/brokers
[zk: localhost:2181(CONNECTED) 3] addauth digest kafka:******
[zk: localhost:2181(CONNECTED) 4] deleteall /kafka
Authentication is not valid : /kafka/config/brokers

once auth is set not able to delete kafka folder even after providing the auth.

could you please share what you would like to do?

and share the config as well?

Iā€™m not 100% sure what you would like to achieve

Best,
michael

Below configs are added

kafka jaas config 
Client {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

zookeeper jaas config
KafkaServer {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="super" password="123456" user_super="super";
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

QuorumServer {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

QuorumLearner {
    org.apache.zookeeper.server.auth.DigestLoginModule required username="kafka" password="123456" user_kafka="kafka";
};

vi /mnt1/zookeeper/conf/zoo.cfg
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/mnt1/data/zookeeper
clientPort=2181
dataLogDir=/mnt1/zklog/datalog
server.0 = ******:2888:3888
maxClientCnxns=0
autopurge.snapRetainCount=3
autopurge.purgeInterval=1
4lw.commands.whitelist=*
requireClientAuthScheme=sasl
authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider

vi /mnt1/kafka/config/server.properties

broker.id=1
host.name=*********
port=9092

message.max.bytes=10485760
num.partitions=30
auto.create.topics.enable=true

# Replication configurations
default.replication.factor=1
num.replica.fetchers=1
replica.fetch.max.bytes=10485760


log.dirs=/mnt1/data/kafka/kafka-logs-1
log.retention.hours=3
log.flush.interval.ms=10000
log.flush.interval.messages=20000
log.flush.scheduler.interval.ms=2000
log.cleanup.interval.mins=30

zookeeper.connect=**********:2181/kafka
zookeeper.connection.timeout.ms=1000000

# Socket server configuration
num.io.threads=8
num.network.threads=8
socket.request.max.bytes=104857600
socket.receive.buffer.bytes=1048576
socket.send.buffer.bytes=1048576
queued.max.requests=16
fetch.purgatory.purge.interval.requests=100
producer.purgatory.purge.interval.requests=100

delete.topic.enable=true
zookeeper.set.acl=true

vi /mnt1/zookeeper/conf/java.env
export JVMFLAGS=" -Xms7g -Xmx7g "
export ZOO_LOG4J_PROP="INFO, ROLLINGFILE"
export ZOO_LOG_DIR="/mnt1/logs/zookeeper"
export SERVER_JVMFLAGS="-Djava.security.auth.login.config=/mnt1/apache-zookeeper-3.5.9-bin/conf/jaas.conf"
export CLIENT_JVMFLAGS="-Djava.security.auth.login.config=/mnt1/apache-zookeeper-3.5.9-bin/conf/jaas.conf"

export KAFKA_OPTS="-Djava.security.auth.login.config=/opt/kafka/kafka/config/jaas.conf" added in startup script 

what you would like to achieve?
how can i revert it back once acl is set on kafka is there a way to delete and test

[zk: localhost:2181(CONNECTED) 2] ls  /
[kafka, zookeeper]

[zk: localhost:2181(CONNECTED) 0] getAcl /kafka
'sasl,'kafka
: cdrwa
'world,'anyone
: r

you could use the kafka-acl tool

see
https://jaceklaskowski.gitbooks.io/apache-kafka/content/kafka-tools-kafka-acls.html

and

hth,
michael