Docker build fails due to kafka undefined error

user_16761772 0
Viewed 623

When I am trying to build our go application we are getting the below error.

=> ERROR [builder 7/7] RUN CGO_ENABLED=0 GOOS=linux go build -o myapp
 > [builder 7/7] RUN CGO_ENABLED=0 GOOS=linux go build -o myapp:
#14 6.962 # main
#14 6.962 ./kafkaproducer.go:12:12: undefined: kafka.NewProducer
#14 6.962 ./kafkaproducer.go:12:31: undefined: kafka.ConfigMap
#14 6.962 ./kafkaproducer.go:23:10: undefined: kafka.Message
#14 6.962 ./kafkaproducer.go:39:13: undefined: kafka.Message

My Docker file is

FROM golang:1.16-alpine AS builder
RUN mkdir /app
ADD . /app
WORKDIR /app
RUN go mod tidy
RUN CGO_ENABLED=0 GOOS=linux go build -o myapp
FROM busybox AS prod
COPY --from=builder /app .
CMD ["./myapp"]

In my kafkaProducer go file I am importing this

import (
    "fmt"
    "log"

    "github.com/confluentinc/confluent-kafka-go/kafka"
)

My application is built successfully when I build it locally but through docker build is getting failed as it is unable to download kafka dependency and undefined error is thrown. Please help me fix this issue

dockerfile confluent-platform apache-kafka docker go
1 Answers

So somehow this seems to depend on cgo when building. I managed to work around this by compiling with cgo, but linking the dependencies statically.

FROM golang AS builder

WORKDIR /app
COPY . .
RUN go mod download && \
    go build -o myapp -ldflags '-linkmode external -w -extldflags "-static"'


# works also with alpine
FROM busybox 

COPY --from=builder /app/myapp myapp
CMD ["./myapp"]

I have used the code below, from their repo, to test. Which gave me initially the kind of error you have shown.

package main

import (
    "fmt"

    "github.com/confluentinc/confluent-kafka-go/kafka"
)

func main() {

    p, err := kafka.NewProducer(&kafka.ConfigMap{"bootstrap.servers": "localhost"})
    if err != nil {
        panic(err)
    }

    defer p.Close()

    // Delivery report handler for produced messages
    go func() {
        for e := range p.Events() {
            switch ev := e.(type) {
            case *kafka.Message:
                if ev.TopicPartition.Error != nil {
                    fmt.Printf("Delivery failed: %v\n", ev.TopicPartition)
                } else {
                    fmt.Printf("Delivered message to %v\n", ev.TopicPartition)
                }
            }
        }
    }()

    // Produce messages to topic (asynchronously)
    topic := "myTopic"
    for _, word := range []string{"Welcome", "to", "the", "Confluent", "Kafka", "Golang", "client"} {
        p.Produce(&kafka.Message{
            TopicPartition: kafka.TopicPartition{Topic: &topic, Partition: kafka.PartitionAny},
            Value:          []byte(word),
        }, nil)
    }

    // Wait for message deliveries before shutting down
    p.Flush(15 * 1000)
}
user_92088870
Related