GroupAuthorizationException while all ACLs are done

Kafka Streams
1

Hello to everyone.
I have bare metal kafka cluster with sasl_plaintext authorization between clients and brokers. There are no autorization between kafka brokers and zookeeper. I’m facing with a problem of GroupAuthorizationException.

Kafka version: 2.4.1
Kafka commitId: c57222ae8cd7866b

The problem can be reproduced on my cluster. All next bash commands are being called from one of broker.

export USERNAME="test-app"
export PASSWORD="password"
export TOPIC="test-app-logs"
export CONSUMER_GROUP="test-app-consumer-group"

I created topic test-app-logs, created user test-app in jaas.conf, setup ACL with built-in bash scripts. Here are they called with --list:

$ bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper:2181 --list --topic $TOPIC
Current ACLs for resource `Topic:LITERAL:test-app-logs`: 
        User:test-app has Allow permission for operations: Read from hosts: *
        User:test-app has Allow permission for operations: Describe from hosts: *
        User:test-app has Allow permission for operations: All from hosts: *
        User:test-app has Allow permission for operations: Write from hosts: * 

$ bin/kafka-acls.sh --authorizer kafka.security.auth.SimpleAclAuthorizer --authorizer-properties zookeeper.connect=zookeeper:2181 --list --group $CONSUMER_GROUP
Current ACLs for resource `Group:LITERAL:test-app-consumer-group`: 
        User:test-app has Allow permission for operations: Read from hosts: *
        User:test-app has Allow permission for operations: Describe from hosts: *
        User:test-app has Allow permission for operations: All from hosts: *

Now I want to produce some message and then consume it.

This works as expected.

echo "hello" | /opt/kafka/bin/kafka-console-producer.sh \
--broker-list localhost:9092 \
--topic $TOPIC \
--producer-property security.protocol=sasl_plaintext \
--producer-property sasl.mechanism=PLAIN \
--producer-property sasl.jaas.config="org.apache.kafka.common.security.plain.PlainLoginModule required username='$USERNAME' password='$PASSWORD';"

This doesn’t work:

bin/kafka-console-consumer.sh --from-beginning \
--bootstrap-server localhost:9092 \
--group $CONSUMER_GROUP \
--topic $TOPIC \
--consumer-property security.protocol=sasl_plaintext \
--consumer-property sasl.mechanism=PLAIN \
--consumer-property sasl.jaas.config="org.apache.kafka.common.security.plain.PlainLoginModule required username='$USERNAME' password='$PASSWORD';"

The problem is that I can’t consume topic with group.id test-app-consumer-group, because of the error:

org.apache.kafka.common.errors.GroupAuthorizationException: Not authorized to access group: test-app-consumer-group

I still can use partition instead of group, this works fine:

bin/kafka-console-consumer.sh --from-beginning \
--bootstrap-server localhost:9092 \
--topic $TOPIC \
--consumer-property security.protocol=sasl_plaintext \
--consumer-property sasl.mechanism=PLAIN \
--consumer-property sasl.jaas.config="org.apache.kafka.common.security.plain.PlainLoginModule required username='$USERNAME' password='$PASSWORD';" \
--partition 0

What am I doing wrong? What are correct configuration of ACL should I have to be able to produce and consume message in new topic?

I also can show my conf files if needed.

2

The question is not related to Kafka Streams.

Maybe try to post in Ops - Confluent Community, Tools - Confluent Community, or Clients - Confluent Community instead.