InsertField using part of another existing field

Lets say I use the SpoolDir Source connector for disgesting a CSV files. This SCV file has a field named EventTime. This field is formatted as yyyy-MM-dd HH:mm:ss.
Is it possible to use either a SMT or something else to break this field up in 2 (e.g. Date & Time) and populate these fields using Date part and Time part of the EventTime field?

Raoul

Can you give some context as to why youโ€™d want to do this?
If you have a perfectly good timestamp it doesnโ€™t strike me as the best idea to break it apart at ingest; instead use a function when processing to extract the date / time as required.

Robin,

We are planning to use Kafka in combination with Spark and there was a request to have the date time split. I thought it might be easiest to do this as close to the source as possible. But instead I believe we will do this on the spark side when we consume the data from Kafka.
Was just pushing out a random thought.

Thank you.
Raoul.

1 Like

This topic was automatically closed 7 days after the last reply. New replies are no longer allowed.