How to remove deduplication upon struct in ksqldb

Hi ,
Step 1 i am creating stream with create Type with struct format as json

CREATE TYPE contact AS STRUCT<"otherPhone" VARCHAR, "workPhone" VARCHAR, "address1" VARCHAR, "city" VARCHAR, "country" VARCHAR, "zipCode" VARCHAR>;

CREATE STREAM event_load(KEY VARCHAR KEY,"eventType" VARCHAR, "platform" VARCHAR, "billingAccount" BILLINGACCOUNT, "soldToContact" CONTACT, "billToContact" CONTACT, "subscription" "subscription") WITH (KAFKA_TOPIC = '', VALUE_FORMAT = 'JSON',PARTITIONS=1);
upon this i am applying  select * 

CREATE STREAM dsc_events_supply WITH (KAFKA_TOPIC = ', VALUE_FORMAT = 'JSON',PARTITIONS=1) AS select * from event_load where "eventType" in ('SubscriptionCreated') ;

so here is the problem: when I alter the stream event_load with add column and dropping stream dsc_events_supply and recreating it for addition of new column from event_load in my topic data is loaded twice: 1st set of data which was already existing and second set of data with new added column, so duplication is happening. I am not able use this How to find distinct values in a stream of events using ksqlDB because my data is struct making group by columns into primary key. So how to fix this issue

This topic was automatically closed after 30 days. New replies are no longer allowed.