Hi,
I have deployed confluent schema registry version 7.6.0 on my on-premise Kubernetes cluster,and it is pointing to _schemas topic in Kafka cluster hosted to Confluent cloud on GCP.
The_schemas topic is having around 21000+ schema entries.
When the schema registry instance is started for the first,it is taking almost 6-7 mins to start.From the logs I can see that this time is taken to catch up with those 21000+ offsets.Below is the logs snippet:
=====================================
[2024-06-21 18:00:22,567] INFO Kafka store reader thread starting consumer (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread:135)
[2024-06-21 18:00:23,127] INFO Seeking to beginning for all partitions (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread:181)
[2024-06-21 18:00:23,129] INFO Initialized last consumed offset to -1 (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread:185)
[2024-06-21 18:00:23,132] INFO [kafka-store-reader-thread-_schemas]: Starting (io.confluent.kafka.schemaregistry.storage.KafkaStoreReaderThread:127)
[2024-06-21 18:00:24,012] INFO Wait to catch up until the offset at 21291 (io.confluent.kafka.schemaregistry.storage.KafkaStore:313)
After the last line in above logs,its stuck for about 6-7 mins,after which the schema registry finally starts listening to requests.
Could you please let us know what could be the reason for such a delay,and how to fix it?
FYI,we have dedicated connection between our on-prem environment and the confluent cloud hosted on GCP.