# Kafka connect to s3 running but i see no data

**URL:** <https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172>\
**Category:** Kafka Connect\
**Created:** [30 May 2022 21:22 UTC](https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172 "2022-05-30T21:22:04Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ighi](https://avatars.discourse-cdn.com/v4/letter/i/ba9def/32.png) [@ighi](https://forum.confluent.io/u/ighi)\
**Post date:** [30 May 2022 21:22 UTC](https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172/1 "2022-05-30T21:22:04Z")

</div>

Hi  
I am hoping to get some help. I am quite new to kafka. So far i have installed confluent cp-all on docker on an ec2 instance. The schema registry is working. My kafka server is on aws msk. I have setup kafka connect with the aws s3 sink. it appears to be running and there are no errors in the logs but i cannot see any data being written to my bucket. Does anyone have any tips or pointers

Thanks  
Ighi

---

<div class="post-metadata">

**Author:** ![kuro](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/kuro/32/1809_2.png) [@kuro](https://forum.confluent.io/u/kuro)\
**Post date:** [31 May 2022 14:04 UTC](https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172/2 "2022-05-31T14:04:27Z")

</div>

Hello @ighi

Is your MSK using autentication? Can you share the connect worker and S3 connector configurations?

Cheers,  
Brais

---

<div class="post-metadata">

**Author:** ![ighi](https://avatars.discourse-cdn.com/v4/letter/i/ba9def/32.png) [@ighi](https://forum.confluent.io/u/ighi)\
**Post date:** [31 May 2022 14:21 UTC](https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172/3 "2022-05-31T14:21:38Z")

</div>

```auto
connector.class = io.confluent.connect.s3.S3SinkConnector
s3.region = <s3region>
format.class = io.confluent.connect.s3.format.bytearray.ByteArrayFormat
task.class = io.confluent.connect.s3.S3SinkTask
flush.size = 2
topics = plainone
name = plainone
aws.access.key.id = **************************
storage.class = io.confluent.connect.s3.storage.S3Storage
aws.secret.access.key = **************************
s3.bucket.name = mybucket

```

---

<div class="post-metadata">

**Author:** ![ighi](https://avatars.discourse-cdn.com/v4/letter/i/ba9def/32.png) [@ighi](https://forum.confluent.io/u/ighi)\
**Post date:** [31 May 2022 14:24 UTC](https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172/4 "2022-05-31T14:24:46Z")

</div>

Thanks @kuro  
my msk is using authentication  
the s3 config is

```auto
connector.class = io.confluent.connect.s3.S3SinkConnector
s3.region = <s3region>
format.class = io.confluent.connect.s3.format.bytearray.ByteArrayFormat
task.class = io.confluent.connect.s3.S3SinkTask
flush.size = 2
topics = plainone
name = plainone
aws.access.key.id = **************************
storage.class = io.confluent.connect.s3.storage.S3Storage
aws.secret.access.key = **************************
s3.bucket.name = mybucket

```

I am using docker-compose on an EC2 instance - i ma not sure how to change the defaults to point to my cluster

---

<div class="post-metadata">

**Author:** ![kuro](https://sea1.discourse-cdn.com/flex019/user_avatar/forum.confluent.io/kuro/32/1809_2.png) [@kuro](https://forum.confluent.io/u/kuro)\
**Post date:** [31 May 2022 17:27 UTC](https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172/5 "2022-05-31T17:27:57Z")

</div>

@ighi , your configuration seems to be fine.

You are using personal IAM credentials autentication, does your IAM account have a policy with the permissions that allows you to use the bucket?

You can find an example of a policy for the S3 sink connector here: [https://docs.confluent.io/kafka-connect-s3-sink/current/overview.html#iam-policy-for-s3](https://docs.confluent.io/kafka-connect-s3-sink/current/overview.html#iam-policy-for-s3)

Regards,  
Brais

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex019/uploads/confluentcommunity/original/1X/c49438c90c9df282e9996fdf6971be890c71b65a.svg) [@system](https://forum.confluent.io/u/system)\
**Post date:** [7 June 2022 17:28 UTC](https://forum.confluent.io/t/kafka-connect-to-s3-running-but-i-see-no-data/5172/6 "2022-06-07T17:28:27Z")

</div>

Closed on 7 June 2022 17:28 UTC

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.