Kafka connect s3 sink failing

achar007 · 23 July 2024 13:58

I’m trying to run a Kafka Connect S3 sink container using confluent s3 plugin

DockerFile


FROM bitnami/kafka:latest

# Create the plugins directory
RUN mkdir -p /opt/bitnami/kafka/plugins

# Set the command to start Kafka Connect with the provided configuration
CMD ["/opt/bitnami/kafka/bin/connect-standalone.sh", "/bitnami/kafka/config/connect-standalone.properties", "/bitnami/kafka/config/s3-sink.properties"]

build command

docker build -t kafka-connect-s3:10.5.13 .

downloaded all the plugins from confluent

list of plugins

ls -lrt /root/k-connect
total 45288
-rwxr-xr-x 1 root root   29904 Jul 23 07:48 accessors-smart-2.4.9.jar
-rwxr-xr-x 1 root root  122176 Jul 23 07:48 asm-9.3.jar
-rwxr-xr-x 1 root root   20891 Jul 23 07:48 audience-annotations-0.12.0.jar
-rwxr-xr-x 1 root root  644626 Jul 23 07:48 avro-1.11.3.jar
-rwxr-xr-x 1 root root 1035393 Jul 23 07:48 aws-java-sdk-core-1.12.650.jar
-rwxr-xr-x 1 root root  717066 Jul 23 07:48 aws-java-sdk-kms-1.12.650.jar
-rwxr-xr-x 1 root root 1271272 Jul 23 07:48 aws-java-sdk-s3-1.12.650.jar
-rwxr-xr-x 1 root root  154476 Jul 23 07:48 aws-java-sdk-sts-1.12.650.jar
-rwxr-xr-x 1 root root  246918 Jul 23 07:48 commons-beanutils-1.9.4.jar
-rwxr-xr-x 1 root root   53820 Jul 23 07:48 commons-cli-1.4.jar
-rwxr-xr-x 1 root root  353793 Jul 23 07:48 commons-codec-1.15.jar
-rwxr-xr-x 1 root root  588337 Jul 23 07:48 commons-collections-3.2.2.jar
-rwxr-xr-x 1 root root 1079377 Jul 23 07:48 commons-compress-1.26.1.jar
-rwxr-xr-x 1 root root  657516 Jul 23 07:48 commons-configuration2-2.10.1.jar
-rwxr-xr-x 1 root root  285424 Jul 23 07:48 commons-io-2.8.0.jar
-rwxr-xr-x 1 root root  284220 Jul 23 07:48 commons-lang-2.6.jar
-rwxr-xr-x 1 root root  587402 Jul 23 07:48 commons-lang3-3.12.0.jar
-rwxr-xr-x 1 root root   61829 Jul 23 07:48 commons-logging-1.2.jar
-rwxr-xr-x 1 root root 1599627 Jul 23 07:48 commons-math3-3.1.1.jar
-rwxr-xr-x 1 root root  111119 Jul 23 07:48 commons-pool-1.6.jar
-rwxr-xr-x 1 root root  238400 Jul 23 07:48 commons-text-1.10.0.jar
-rwxr-xr-x 1 root root   17385 Jul 23 07:48 common-utils-7.7.0-1115.jar
-rwxr-xr-x 1 root root 2983237 Jul 23 07:48 curator-client-5.2.0.jar
-rwxr-xr-x 1 root root  336384 Jul 23 07:48 curator-framework-5.2.0.jar
-rwxr-xr-x 1 root root  315569 Jul 23 07:48 curator-recipes-5.2.0.jar
-rwxr-xr-x 1 root root  307637 Jul 23 07:48 dnsjava-2.1.7.jar
-rwxr-xr-x 1 root root  249277 Jul 23 07:48 gson-2.9.0.jar
-rwxr-xr-x 1 root root   25106 Jul 23 07:48 hadoop-annotations-3.3.6.jar
-rwxr-xr-x 1 root root  106145 Jul 23 07:48 hadoop-auth-3.3.6.jar
-rwxr-xr-x 1 root root 4603101 Jul 23 07:48 hadoop-common-3.3.6.jar
-rwxr-xr-x 1 root root  780321 Jul 23 07:48 httpclient-4.5.13.jar
-rwxr-xr-x 1 root root  326724 Jul 23 07:48 httpcore-4.4.4.jar
-rwxr-xr-x 1 root root   78480 Jul 23 07:48 jackson-annotations-2.16.1.jar
-rwxr-xr-x 1 root root  578125 Jul 23 07:48 jackson-core-2.16.1.jar
-rwxr-xr-x 1 root root 1637611 Jul 23 07:48 jackson-databind-2.16.1.jar
-rwxr-xr-x 1 root root   69928 Jul 23 07:48 jackson-dataformat-cbor-2.16.1.jar
-rwxr-xr-x 1 root root  102033 Jul 23 07:48 jackson-dataformat-csv-2.16.1.jar
-rwxr-xr-x 1 root root   44399 Jul 23 07:48 jakarta.activation-api-1.2.1.jar
-rwxr-xr-x 1 root root   26586 Jul 23 07:48 javax.annotation-api-1.3.2.jar
-rwxr-xr-x 1 root root   95806 Jul 23 07:48 javax.servlet-api-3.1.0.jar
-rwxr-xr-x 1 root root  125632 Jul 23 07:48 jaxb-api-2.3.0.jar
-rwxr-xr-x 1 root root  890168 Jul 23 07:48 jaxb-impl-2.2.3-1.jar
-rwxr-xr-x 1 root root  436731 Jul 23 07:48 jersey-core-1.19.4.jar
-rwxr-xr-x 1 root root  158695 Jul 23 07:48 jersey-json-1.20.jar
-rwxr-xr-x 1 root root  705276 Jul 23 07:48 jersey-server-1.19.4.jar
-rwxr-xr-x 1 root root  128990 Jul 23 07:48 jersey-servlet-1.19.4.jar
-rwxr-xr-x 1 root root   90184 Jul 23 07:48 jettison-1.5.4.jar
-rwxr-xr-x 1 root root  213911 Jul 23 07:48 jline-2.12.1.jar
-rwxr-xr-x 1 root root   27588 Jul 23 07:48 jmespath-java-1.12.650.jar
-rwxr-xr-x 1 root root  640724 Jul 23 07:48 joda-time-2.10.1.jar
-rwxr-xr-x 1 root root  282591 Jul 23 07:48 jsch-0.1.55.jar
-rwxr-xr-x 1 root root  120206 Jul 23 07:48 json-smart-2.4.10.jar
-rwxr-xr-x 1 root root  100636 Jul 23 07:48 jsp-api-2.1.jar
-rwxr-xr-x 1 root root   19936 Jul 23 07:48 jsr305-3.0.2.jar
-rwxr-xr-x 1 root root   46367 Jul 23 07:48 jsr311-api-1.1.1.jar
-rwxr-xr-x 1 root root   37262 Jul 23 07:48 kafka-avro-serializer-7.7.0-827.jar
-rwxr-xr-x 1 root root   47038 Jul 23 07:48 kafka-connect-avro-data-7.7.0-827.jar
-rwxr-xr-x 1 root root  133706 Jul 23 07:48 kafka-connect-s3-10.5.13.jar
-rwxr-xr-x 1 root root    7497 Jul 23 07:48 kafka-connect-storage-common-11.2.15.jar
-rwxr-xr-x 1 root root 3783853 Jul 23 07:48 kafka-connect-storage-common-hadoop-shaded-guava-11.2.15.jar
-rwxr-xr-x 1 root root   31612 Jul 23 07:48 kafka-connect-storage-core-11.2.15.jar
-rwxr-xr-x 1 root root 2348886 Jul 23 07:48 kafka-connect-storage-format-11.2.15.jar
-rwxr-xr-x 1 root root   27627 Jul 23 07:48 kafka-connect-storage-partitioner-11.2.15.jar
-rwxr-xr-x 1 root root    7239 Jul 23 07:48 kafka-schema-converter-7.7.0-827.jar
-rwxr-xr-x 1 root root  294139 Jul 23 07:48 kafka-schema-registry-client-7.7.0-827.jar
-rwxr-xr-x 1 root root   61359 Jul 23 07:48 kafka-schema-serializer-7.7.0-827.jar
-rwxr-xr-x 1 root root   80980 Jul 23 07:48 kerb-admin-1.0.1.jar
-rwxr-xr-x 1 root root  113017 Jul 23 07:48 kerb-client-1.0.1.jar
-rwxr-xr-x 1 root root   65464 Jul 23 07:48 kerb-common-1.0.1.jar
-rwxr-xr-x 1 root root  226672 Jul 23 07:48 kerb-core-1.0.1.jar
-rwxr-xr-x 1 root root  116120 Jul 23 07:48 kerb-crypto-1.0.1.jar
-rwxr-xr-x 1 root root   20046 Jul 23 07:48 kerb-identity-1.0.1.jar
-rwxr-xr-x 1 root root   82756 Jul 23 07:48 kerb-server-1.0.1.jar
-rwxr-xr-x 1 root root   20409 Jul 23 07:48 kerb-simplekdc-1.0.1.jar
-rwxr-xr-x 1 root root   36708 Jul 23 07:48 kerb-util-1.0.1.jar
-rwxr-xr-x 1 root root  102174 Jul 23 07:48 kerby-asn1-1.0.1.jar
-rwxr-xr-x 1 root root   30674 Jul 23 07:48 kerby-config-1.0.1.jar
-rwxr-xr-x 1 root root  204650 Jul 23 07:48 kerby-pkix-1.0.1.jar
-rwxr-xr-x 1 root root   40554 Jul 23 07:48 kerby-util-1.0.1.jar
-rwxr-xr-x 1 root root   29134 Jul 23 07:48 kerby-xdr-1.0.1.jar
-rwxr-xr-x 1 root root   30604 Jul 23 07:48 logredactor-1.0.12.jar
-rwxr-xr-x 1 root root    3879 Jul 23 07:48 logredactor-metrics-1.0.12.jar
-rwxr-xr-x 1 root root  136314 Jul 23 07:48 metrics-core-3.2.4.jar
-rwxr-xr-x 1 root root   34221 Jul 23 07:48 minimal-json-0.9.5.jar
-rwxr-xr-x 1 root root  683101 Jul 23 07:48 parquet-avro-1.11.2.jar
-rwxr-xr-x 1 root root 1619624 Jul 23 07:48 parquet-column-1.11.2.jar
-rwxr-xr-x 1 root root   96209 Jul 23 07:48 parquet-common-1.11.2.jar
-rwxr-xr-x 1 root root  848727 Jul 23 07:48 parquet-encoding-1.11.2.jar
-rwxr-xr-x 1 root root  702080 Jul 23 07:48 parquet-format-structures-1.11.2.jar
-rwxr-xr-x 1 root root  437558 Jul 23 07:48 parquet-hadoop-1.11.2.jar
-rwxr-xr-x 1 root root 1838876 Jul 23 07:48 protobuf-java-3.24.4.jar
-rwxr-xr-x 1 root root  128414 Jul 23 07:48 re2j-1.1.jar
-rwxr-xr-x 1 root root   41125 Jul 23 07:48 slf4j-api-1.7.36.jar
-rwxr-xr-x 1 root root    9824 Jul 23 07:48 slf4j-reload4j-1.7.36.jar
-rwxr-xr-x 1 root root  334803 Jul 23 07:48 snakeyaml-2.0.jar
-rwxr-xr-x 1 root root 2112099 Jul 23 07:48 snappy-java-1.1.10.4.jar
-rwxr-xr-x 1 root root  195842 Jul 23 07:48 stax2-api-4.2.jar
-rwxr-xr-x 1 root root   34318 Jul 23 07:48 swagger-annotations-2.1.10.jar
-rwxr-xr-x 1 root root   18763 Jul 23 07:48 token-provider-1.0.1.jar
-rwxr-xr-x 1 root root  522679 Jul 23 07:48 woodstox-core-5.4.0.jar
-rwxr-xr-x 1 root root 1323991 Jul 23 07:48 zookeeper-3.8.4.jar
-rwxr-xr-x 1 root root  254932 Jul 23 07:48 zookeeper-jute-3.8.4.jar

connect-standalone.properties

bootstrap.servers=<brokerip>:9094
plugin.path=/opt/bitnami/kafka/plugins
key.converter=org.apache.kafka.connect.storage.StringConverter
value.converter=org.apache.kafka.connect.storage.StringConverter
internal.key.converter=org.apache.kafka.connect.json.JsonConverter
internal.value.converter=org.apache.kafka.connect.json.JsonConverter
internal.key.converter.schemas.enable=true
internal.value.converter.schemas.enable=true
offset.storage.file.filename=/tmp/connect.offsets
rest.port=8083
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required \
    username="kafka01" \
    password="mypassword";

s3-sink.properties

name=s3-sink-connector
connector.class=io.confluent.connect.s3.S3SinkConnector
tasks.max=1
topics=mytopic2
s3.bucket.name=kafka-backup
s3.part.size=5242880
flush.size=3
storage.class=io.confluent.connect.s3.storage.S3Storage
format.class=io.confluent.connect.s3.format.json.JsonFormat
schema.compatibility=NONE
partitioner.class=io.confluent.connect.storage.partitioner.DefaultPartitioner
aws.access.key.id="admin"
aws.secret.access.key="mypassword"
s3.endpoint="http://<minio-ip>:9001"

run container

podman run -d --name my-kafka-connect-s3 --restart=unless-stopped -v /root/kafka-connect:/bitnami/kafka/config -v /root/k-connect:/opt/bitnami/kafka/plugins -e CLASSPATH=/opt/bitnami/kafka/plugins/* kafka-connect-s3:10.5.13

error from container log

podman logs kafka-connect-s3:10.5.13

[2024-07-23 08:09:57,660] INFO Kafka Connect worker initialization took 6771ms (org.apache.kafka.connect.cli.AbstractConnectCli:141)
[2024-07-23 08:09:57,660] INFO Kafka Connect starting (org.apache.kafka.connect.runtime.Connect:51)
[2024-07-23 08:09:57,662] INFO Herder starting (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:107)
[2024-07-23 08:09:57,662] INFO Worker starting (org.apache.kafka.connect.runtime.Worker:231)
[2024-07-23 08:09:57,662] INFO Starting FileOffsetBackingStore with file /tmp/connect.offsets (org.apache.kafka.connect.storage.FileOffsetBackingStore:63)
[2024-07-23 08:09:57,664] INFO Worker started (org.apache.kafka.connect.runtime.Worker:241)
[2024-07-23 08:09:57,664] INFO Herder started (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:110)
[2024-07-23 08:09:57,664] INFO Initializing REST resources (org.apache.kafka.connect.runtime.rest.RestServer:219)
[2024-07-23 08:09:57,686] INFO Adding admin resources to main listener (org.apache.kafka.connect.runtime.rest.RestServer:234)
[2024-07-23 08:09:57,728] INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session:334)
[2024-07-23 08:09:57,728] INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session:339)
[2024-07-23 08:09:57,729] INFO node0 Scavenging every 660000ms (org.eclipse.jetty.server.session:132)
[2024-07-23 08:09:57,879] WARN unavailable (org.eclipse.jetty.server.handler.ContextHandler.ROOT:2374)
java.lang.IllegalArgumentException
        at jersey.repackaged.org.objectweb.asm.ClassVisitor.<init>(ClassVisitor.java:79)
        at jersey.repackaged.org.objectweb.asm.ClassVisitor.<init>(ClassVisitor.java:64)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener$AnnotatedClassVisitor.<init>(AnnotationAcceptingListener.java:172)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener$AnnotatedClassVisitor.<init>(AnnotationAcceptingListener.java:156)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener.<init>(AnnotationAcceptingListener.java:124)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener.newJaxrsResourceAndProviderListener(AnnotationAcceptingListener.java:96)
        at org.glassfish.jersey.server.ResourceConfig.scanClasses(ResourceConfig.java:903)
        at org.glassfish.jersey.server.ResourceConfig._getClasses(ResourceConfig.java:864)
        at org.glassfish.jersey.server.ResourceConfig.getClasses(ResourceConfig.java:773)
        at org.glassfish.jersey.server.ResourceConfig$RuntimeConfig.<init>(ResourceConfig.java:1216)
        at org.glassfish.jersey.server.ResourceConfig$RuntimeConfig.<init>(ResourceConfig.java:1187)
        at org.glassfish.jersey.server.ResourceConfig.createRuntimeConfig(ResourceConfig.java:1183)
        at org.glassfish.jersey.server.ApplicationHandler$RuntimeConfigConfigurator.init(ApplicationHandler.java:184)
        at org.glassfish.jersey.server.ApplicationHandler.lambda$initialize$0(ApplicationHandler.java:294)
        at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4204)
        at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:294)
        at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:261)
        at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:311)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:154)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:339)
        at javax.servlet.GenericServlet.init(GenericServlet.java:244)
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:632)
        at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:415)
        at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
        at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)
        at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:774)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.apache.kafka.connect.runtime.rest.RestServer.initializeResources(RestServer.java:285)
        at org.apache.kafka.connect.runtime.rest.ConnectRestServer.initializeResources(ConnectRestServer.java:44)
        at org.apache.kafka.connect.runtime.Connect.start(Connect.java:55)
        at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:143)
        at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:94)
        at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:182)
[2024-07-23 08:09:57,884] ERROR Failed to start Connect (org.apache.kafka.connect.cli.AbstractConnectCli:145)
org.apache.kafka.connect.errors.ConnectException: Unable to initialize REST resources
        at org.apache.kafka.connect.runtime.rest.RestServer.initializeResources(RestServer.java:287)
        at org.apache.kafka.connect.runtime.rest.ConnectRestServer.initializeResources(ConnectRestServer.java:44)
        at org.apache.kafka.connect.runtime.Connect.start(Connect.java:55)
        at org.apache.kafka.connect.cli.AbstractConnectCli.startConnect(AbstractConnectCli.java:143)
        at org.apache.kafka.connect.cli.AbstractConnectCli.run(AbstractConnectCli.java:94)
        at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:182)
Caused by: javax.servlet.ServletException: org.glassfish.jersey.servlet.ServletContainer-112ca081==org.glassfish.jersey.servlet.ServletContainer@3617b3c7{jsp=null,order=-1,inst=true,async=true,src=EMBEDDED:null,STARTED}
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:650)
        at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:415)
        at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
        at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
        at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510)
        at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
        at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)
        at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
        at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
        at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:774)
        at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
        at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
        at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
        at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.apache.kafka.connect.runtime.rest.RestServer.initializeResources(RestServer.java:285)
        ... 5 more
Caused by: java.lang.IllegalArgumentException
        at jersey.repackaged.org.objectweb.asm.ClassVisitor.<init>(ClassVisitor.java:79)
        at jersey.repackaged.org.objectweb.asm.ClassVisitor.<init>(ClassVisitor.java:64)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener$AnnotatedClassVisitor.<init>(AnnotationAcceptingListener.java:172)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener$AnnotatedClassVisitor.<init>(AnnotationAcceptingListener.java:156)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener.<init>(AnnotationAcceptingListener.java:124)
        at org.glassfish.jersey.server.internal.scanning.AnnotationAcceptingListener.newJaxrsResourceAndProviderListener(AnnotationAcceptingListener.java:96)
        at org.glassfish.jersey.server.ResourceConfig.scanClasses(ResourceConfig.java:903)
        at org.glassfish.jersey.server.ResourceConfig._getClasses(ResourceConfig.java:864)
        at org.glassfish.jersey.server.ResourceConfig.getClasses(ResourceConfig.java:773)
        at org.glassfish.jersey.server.ResourceConfig$RuntimeConfig.<init>(ResourceConfig.java:1216)
        at org.glassfish.jersey.server.ResourceConfig$RuntimeConfig.<init>(ResourceConfig.java:1187)
        at org.glassfish.jersey.server.ResourceConfig.createRuntimeConfig(ResourceConfig.java:1183)
        at org.glassfish.jersey.server.ApplicationHandler$RuntimeConfigConfigurator.init(ApplicationHandler.java:184)
        at org.glassfish.jersey.server.ApplicationHandler.lambda$initialize$0(ApplicationHandler.java:294)
        at java.base/java.util.Arrays$ArrayList.forEach(Arrays.java:4204)
        at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:294)
        at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:261)
        at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:311)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:154)
        at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:339)
        at javax.servlet.GenericServlet.init(GenericServlet.java:244)
        at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:632)
        ... 19 more
[2024-07-23 08:09:57,884] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:67)
[2024-07-23 08:09:57,885] INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer:354)
[2024-07-23 08:09:57,887] INFO Stopped o.e.j.s.ServletContextHandler@6a9164d1{/,null,STOPPED} (org.eclipse.jetty.server.handler.ContextHandler:1159)
[2024-07-23 08:09:57,894] INFO Stopped http_8083@1f50f75{HTTP/1.1, (http/1.1)}{0.0.0.0:8083} (org.eclipse.jetty.server.AbstractConnector:383)
[2024-07-23 08:09:57,894] INFO node0 Stopped scavenging (org.eclipse.jetty.server.session:149)
[2024-07-23 08:09:57,898] INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer:383)
[2024-07-23 08:09:57,898] INFO Herder stopping (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:115)
[2024-07-23 08:09:57,899] INFO Worker stopping (org.apache.kafka.connect.runtime.Worker:248)
[2024-07-23 08:09:57,899] INFO Stopped FileOffsetBackingStore (org.apache.kafka.connect.storage.FileOffsetBackingStore:71)
[2024-07-23 08:09:57,900] INFO Metrics scheduler closed (org.apache.kafka.common.metrics.Metrics:684)
[2024-07-23 08:09:57,900] INFO Closing reporter org.apache.kafka.common.metrics.JmxReporter (org.apache.kafka.common.metrics.Metrics:688)
[2024-07-23 08:09:57,900] INFO Metrics reporters closed (org.apache.kafka.common.metrics.Metrics:694)
[2024-07-23 08:09:57,900] INFO App info kafka.connect for 100.64.0.23:8083 unregistered (org.apache.kafka.common.utils.AppInfoParser:88)
[2024-07-23 08:09:57,900] INFO Worker stopped (org.apache.kafka.connect.runtime.Worker:269)
[2024-07-23 08:09:57,901] INFO Herder stopped (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:126)
[2024-07-23 08:09:57,901] INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect:72)

Need your help on fixing this.

dtroiano · 23 July 2024 15:49

Should the worker and connector properties in your Dockerfile be /bitnami/kafka/config/... or /opt/bitnami/kafka/config/..?

If that isn’t the issue, I’d recommend changing the CMD in your Dockerfile so that you can exec into the container to test the connect-standalone.sh command directly. Since ConnectStandalone.main is in the stack trace I am hoping that it’ll be something obvious like a path or properties file availability issue if you can get into the container to test.

Topic		Replies	Views
Kafka connect s3 sink not able to create connectors Kafka Connect	3	3379	27 January 2022
KafkaConnect with Amazon Sink S3 Sink Connect is not working Self-Managed Connectors	12	10649	2 July 2022
Self-Managed S3 Sink Connector by docker doesn't work properly Confluent Cloud	4	3294	2 February 2023
Confluent Kafka Self managed Lambda sink connector configuration issues Kafka Connect	3	5006	10 May 2022
Kafkaconnect not seeing S3Sinkconnector Kafka Connect	7	378	23 June 2024