Registering a schema via C#

Schema Registry
1

I’m creating some simple console apps to learn Kafka and running into issues when trying to register a schema.

I set up my environment in docking using this:

version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    ports:
      - 22181:2181
  
  kafka:
    image: confluentinc/cp-kafka:latest
    depends_on:
      - zookeeper
    ports:
      - 29092:29092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1

  schema-registry:
    image: confluentinc/cp-schema-registry:latest
    depends_on: 
      - kafka
    ports:
      - 28081:28081
    environment:
      SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: 'zookeeper:2181'
      SCHEMA_REGISTRY_HOST_NAME: schema-registry
      SCHEMA_REGISTRY_KAFKASTORE_BOOTSTRAP_SERVERS: 'PLAINTEXT://kafka:9092'
      # SCHEMA_REGISTRY_LISTENERS: http://0.0.0.0:28081

Then try and add a schema in a console app:

        static async Task Main(string[] args)
        {
            var schemaRegistryConfig = new SchemaRegistryConfig
            {
                Url = "http://localhost:29092"
            };

            var schemaRegistry = new CachedSchemaRegistryClient(schemaRegistryConfig);

            //var schema = @"{
            //    ""type"": ""record"",
            //    ""name"": ""TotallyCoolCustomClass"",
            //    ""properties"": {
            //        ""FavouriteQuote"": {""type"": ""string""},
            //        ""FavouriteNumber"": {""type"": ""integer""}
            //    }
            //}";

            var schema = @"{
                ""type"": ""record"",
                ""name"": ""TotallyCoolCustomClass"",
                ""fields"": [
                    {""name"": ""FavouriteQuote"", ""type"": ""string""},
                    {""name"": ""FavouriteNumber"", ""type"": ""int""}
                ]
            }";

            var subject = "ComplexTest"; // Replace with your topic name and value type
            var schemaId = await schemaRegistry.RegisterSchemaAsync(subject, schema);
        }

This throws an error on the register schema line saying ‘[http://localhost:29092/] HttpRequestException: An error occurred while sending the request.’

And my docker kafka log shows:

Unexpected error from /172.23.0.1 (channelId=172.23.0.3:29092-172.23.0.1:41768-1); closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 1347375956 larger than 100001200)

Obviously my message isn’t bigger than the 100mb and certianly not 13gb!

Any pointers on where I’m going wrong?

Thanks

2

Your schema registry isn’t running on port 29092, so you’re sending an HTTP request to the broker and it’s denying the large TCP payload of that request