Schema Registry - Bulk Schema Creation

Schema Registry
1

Hi everyone,

I searched the forum but didn’t find what I’m looking for.

I am working on an application that will need to create thounsands of subjects/schemas at once.
I plan to submit smaller chuncks but still want to bulk/batch the creation.

I’m trying to figure out a performing way, using REST API to do that.

Is there a way to provide in a POST a body containing not only one but rather a list of schema to be processed ?

Thanks for any help !

2

Hi @Rudriko

welcome to the forum.

Not sure whether it works with the REST API.

Nevertheless I’d like to share an idea which came to my mind:

use the confluent cli, e.g.
confluent schema-registry schema create --subject payments --schema payments.avro --type AVRO

and use a bash loop to create your 1000 of schemas/subjects.

for full reference see
https://docs.confluent.io/confluent-cli/current/command-reference/schema-registry/schema/confluent_schema-registry_schema_create.html#confluent-schema-registry-schema-create

best,
michael

3

Hi,

Thanks Michael, indeed that was my initial lead but we are implementing SaaS and do not want any process on server side this is why REST API is investigated.

It looks like I would need to allow REST Proxy server access to do so but customer usually don’t like the idea ^^

JL

4

Hi,

ok I see.
Something I stumbled over some time ago but never used by myself.
It may be worth to have look at

best,
michael

5

Hi,

Thanks again Michael,
I am exploring the HttpsUrlConnection class that seems to provide support for security features.

JL

1 Like