Working example of schema registry and Golang?

Clients Non-Java Clients
21

Correct. In my code, I created the Kafka records using the structs and functions generated by the Protobuf compiler for Go. In this case, there is no need actually to check if the bytes created are, in fact, coming from the local schema — since they have been created based on statically compiled files. This is slightly different from strategies from Avro, where you can generate the bytes dynamically (i.e., in runtime) from the local schema, where it does make sense to check for this since you are playing with fire. So I skipped this part altogether.

There are some strategies in Protobuf to create messages in runtime dynamically, but I have decided not to use them and prioritize generation statically. IMO it’s safer and produces a more compatible set of messages. The only responsibility left to the producer is to tell SR which schema was used to create the records :+1:

1 Like
22

Ye, and guess part of the protobuf / gRPC standard is to share the .proto and compiled/generated .pb.go file, which then ensures everyone that uses it is using the same struct…

G

1 Like
23

PS: love the little create topic function, hope you not mind, I’m “plagiarising it into my code”.

like the simplicity of your config load routine (had me wondering/appreciating the simplicity), but decided sticking to a export = for a shell script (or user param values specified in a container yaml file) with a os.GetEnv(“key”) as it works directly on the OS and also exactly the same in a container architecture where I can define the params into my yaml file and use the same os.GetEnv(… method.

G

24

bit off topic… I have the major part of the configmap thats used to create the kafka session.

	cm := kafka.ConfigMap{
		"bootstrap.servers":       fmt.Sprintf("%s:%s", props.bootstrapservers, props.bootstrapport),
		"broker.version.fallback": "0.10.0.0",
		"api.version.fallback.ms": 0,
	}

I now want to add the below values to the above config map based on a if condition, how would I add it though ?

			"sasl.mechanisms":   props.sasl_mechanisms,
			"security.protocol": props.security_protocol,
			"sasl.username":     props.sasl_username,
			"sasl.password":     props.sasl_password,
	*/

	adminClient, err := kafka.NewAdminClient(&cm)
25

The type ConfigMap is a pointer to a map[string][interface{}] so pretty much what you should do is creating it before using further down your code and depending on the condition — add more values to it:

if <CONDITION> {
   kafkaConfig["sasl.mechanisms"] = "VALUE"
}

What you can’t do, at least no so easily, is applying the condition during the map creation dynamically. You could explore approaches like returning a pointer to a function that could receive a func() that would apply the operation conditionally or perhaps creating a func() altogether that would perform the whole map creation (including the condition), and the receiver would use it. But I feel that just creating the map first, conditionally updating it, and later using it is simpler.

1 Like
26

agree => But I feel that just creating the map first, conditionally updating it, and later using it is simpler

Based on what variables are populated I want to build the map, if security values are specified then it implies it should be used/included, if not, then don’t include it in the map.

G