How to allow in-place upgrade of persistent query with variable underlying data

ksqlDB
1

Is there a solution or workaround to evolving a ksql stream in-place (CREATE OR REPLACE) that has a json object with variable fields?
eg: if the incoming topic has a field like:

{
  "metadata": {
    "fieldA": "string val",
    "fieldB": 123
  }
}

Later, we get decide that metadata can have a new field fieldC that is a string value.
From what I can tell STRUCT is not a good fit here because it doesn’t allow in-place update where you add a new field to the struct :disappointed::melting_face:

And MAP is not quite the best either because it requires a consistent value type (eg: fieldB`` would end up getting emitted by our stream as a VARCHARrather than anINT`) :disappointed:

I’ve been playing around with the various JSON scalar functions but nothing seems to allow me to emit the values in their original types, while allowing future in-place replacements

2

This topic was automatically closed after 30 days. New replies are no longer allowed.