Hi,
I wonder if there is any easy tooling that can be used to export topics (Avro + SR) from a Kafka cluster and than to re-import it in another Kafka cluster (Avro+SR) in “Air Gap” Scenarios where alle the Schemas (Multi-Schema Topic) are carried over and get looked up or registered on the target cluster’s SR.
Use Case: There is a Kafka Cluster containing that topics which is not accessible directly as it is a “high security” setup (critical infrastructure). The only way to get data in or out is via File Transfers via a bastion host that do not allow any kind of port forwarding.
To reproduce an issue or to do some tests with real data you want to copy parts of that topic (all records between two points in time) 1:1 to your local Kafka cluster.
For Topics that don’t use SR or where the Schema is the same for all records this can be done with the Kafka CLI tools (ConsoleConsumer → File → ConsoleProducer) but for topics that use SR and in the worst case contain different schemas or schema versions I’m not aware about any easy to use tooling. (One option might be to also copy over the schemas topic 1:1 but this isn’t an option in this case)
Anyone out there already had this issue and found a easy-to-use solution for it?