Hey guys!
I created a script that allows us to use the ksq-migrations tool to initialize the metadata to create the KSQL Streams and the KSQL Table. Upon creating this, I am now able to apply the migrations. This allows me to create the KSQL Queries, drop the query etc…
While I am able to get the script to work by ignoring the errors like
Failed to create stream <MIGRATION_STREAM_NAME>: io.confluent.ksql.api.client.exception.KsqlClientException: Received 400 response from server: Cannot add stream ‘<MIGRATION_STREAM_NAME’: A stream with the same name already exists. Error code: 40001
or
Validating current migration state before applying new migrations
Loading migration files
No eligible migrations found.
I am able to continue the script thanks to the “if !..” (the full script is attached in the chat). While I am able to continue the script, having the “Failed to create stream…” may look unprofessional if we are planning to place this in Source Control. Is there a method in which I am able to use bash in order to hide the word “Failed” if we get the error "
Failed to create stream <Stream_Name>: io.confluent.ksql.api.client.exception.KsqlClientException: Received 400 response from server: Cannot add stream ‘<Stream_Name>S’: A stream with the same name already exists. Error code: 40001"
Thanks! Feel free to use the script if you guys need as well. This assumes that I have the confluent binaries installed as well.
sudo chmod+x ksql-migrations.sh
ksql-migrations.sh
CONFLUENT_HOME=“/Source/confluent-6.2.0/bin”
for k in .///ksql-migrations.properties; do
if ! $CONFLUENT_HOME/ksql-migrations -c $k initialize-metadata ; then
echo “We have this stream already!”
fi
done
for j in .///ksql-migrations.properties; do
if ! $CONFLUENT_HOME/ksql-migrations apply -a -c $j ; then
echo “KSQL is amazing!”
fi
done