Golang app not compiling in docker

Clients Non-Java Clients
1

Hi all

Please see attached. if I compile my code directly on system, it works, when I compile/build into a docker images I get errors as per the attached.

Anyone seen this, I’m also attaching my docker file.

G

Dockerfile

FROM golang:alpine as build-env

RUN apk update && apk add bash ca-certificates git gcc g++ libc-dev

WORKDIR /scrubber

COPY . .

RUN go build -o scrubber .

CMD ./scrubber

Err.log → Output from above build.

Sending build context to Docker daemon 41.54MB

Step 1/6 : FROM golang:alpine as build-env

---> 14ee78639386

Step 2/6 : RUN apk update && apk add bash ca-certificates git gcc g++ libc-dev

---> Using cache

---> 159487736117

Step 3/6 : WORKDIR /scrubber

---> Using cache

---> 04f9d5590771

Step 4/6 : COPY . .

---> Using cache

---> 313c9ab381f8

Step 5/6 : RUN go build -o scrubber .

---> Running in 19c14cb0cbd2

e[91m# github.com/confluentinc/confluent-kafka-go/kafka

/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: vendor/github.com/confluentinc/confluent-kafka-go/kafka/librdkafka_vendor/librdkafka_glibc_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_set_fatal_error':

(.text+0x149): undefined reference to `__strdup'

/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: vendor/github.com/confluentinc/confluent-kafka-go/kafka/librdkafka_vendor/librdkafka_glibc_linux.a(rdkafka_txnmgr.o): in function `rd_kafka_txn_set_abortable_error':

(.text+0x683): undefined reference to `__strdup'

/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: vendor/github.com/confluentinc/confluent-kafka-go/kafka/librdkafka_vendor/librdkafka_glibc_linux.a(rdkafka_admin.o): in function `rd_kafka_NewTopic_new':

and this above message is repeated probably 50+ times

2

update,

I’ve added “librdkafka-dev pkgconf” to the docker file as modules to pull in, still getting same errors.

G

3

@georgelza maybe this GH issue has some clues to your compilation issue?

4

so being the noob at this. where do I add the “-tags dynamic” tags.

looking at some dogs it implies with the go build command, but with us doing a docker build, using a docker file, i never issue a go build command.

G

5

I think next I might try the golang docs, in particular the Docker information. Maybe start here?

https://hub.docker.com/_/golang

6

don’t think this is golang related, this is more docker and have feeling the combination of docker/golang and kafka.

surely someone else have deployed a go app accessing CP Kafka in K8s. be keen to see their docker file.

G

7

… let me add, my other client → receiving app is also golang and inside docker.
App 1, read text file, do some logic, drop onto Kafka topic.
App 2, consume topic (This app that I have problem with) more logic, and then send via gRPC to App 3, some more logic and then into a unstructured DB ( This app in docker works)
App 2, if just direct compiled/executed, it works, when attempting to build into docker image then have these problems.

G

8

checking, but looks like i got it build.

changed
RUN apk update && apk add bash ca-certificates git gcc g++ libc-dev
to
RUN apk update && apk add bash ca-certificates git gcc g++ libc-dev librdkafka-dev pkgconf

change
RUN go build -o scrubber .
to
RUN go build -tags musl -o scrubber .

G

2 Likes
9

God you saved my life georgelza. Thank you so much!

1 Like
10

I took this a bit further, since Go requires disk space, and created multistage.Dockerfile

# syntax=docker/dockerfile:1

## Build
FROM golang:1.19.3-alpine AS build

RUN apk update && apk add gcc g++ libc-dev librdkafka-dev pkgconf

WORKDIR /app

COPY go.mod go.sum ./

RUN go mod download

COPY *.go ./

RUN go build -tags musl -o myapp .

## Deploy
FROM alpine:latest

RUN apk update && apk add gcc g++ libc-dev librdkafka-dev pkgconf

COPY --from=build /app/myapp /

ENTRYPOINT ["/myapp"]

Image size ~250Mb + “myapp” size.

Open question: is it possible to avoid loading gcc, g++, etc in Deploy section of multistage Dockerfile?

1 Like
11

Thank you all, you saved me a lot of time)

1 Like
12

Also note there are other go native libraries like franz and (now deprecated) sarama.