How do I package a custom transform?
I’m trying to write a custom transform SMT. I’ve coded the SMT based upon the examples already provided in GIT. The source code all makes sense. I’m building / packaging this using maven and the kafka-connect-maven plugin. The plugin seems to work and generates the output to the Target folder. Our Kafka-Connect cluster (K8s) fails to load my transformation.
There is no error information shown to indicate the problem. I’ve extracted the plugin to the correct Path. I downloaded a random ZIP from Confluent Hub to check that I had the right path and process. For some reason Kafka-Connect fails to load “My” component.
My component is composed of :
root folder
/lib
manifest.json
Inside the /lib folder are the dependent JAR files along with my components jar file.
How does Kafka-Connect know to ingest my custom smt? Is there something else I need to set so it can see my component?
Thanks in advance!!!
Adding a little more info…
An error when trying to use my SMT is all I can find. I removed our domain manually here.
Invalid value com.[domainremoved].kafka.connect.date2timestamp for configuration transforms.AccountEffective.type: Class com.[domainremoved].kafka.connect.date2timestamp could not be found.
POM Settings I am using to build.
io.confluent
kafka-connect-maven-plugin
0.12.0
<configuration>
<name>date2timestamp</name>
<version>0.1</version>
<ownerUsername>ctheis</ownerUsername>
<description>Converts Date epoch to a Timestamp Epoch</description>
<componentTypes>
<componentType>transform</componentType>
</componentTypes>
</configuration>
<executions>
<execution>
<goals>
<goal>kafka-connect</goal>
</goals>
<configuration>
<outputDirectory>${project.build.directory}</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
And just like that I figured it out. It was all correct except I needed an additional dependency included. There was nothing to help me find that. I just managed through trial and error. I copied the entire code for one of your transforms on git hub and just replaced the class name with mine. Then it worked. I then worked backward until I could get mine to work.