Metadata in Clover ETL

CloverDX Platform
1

Hi there!
I am pretty new to CloverETL. So if this question sounds silly, please forgive me.
Should a transformer component use the same metadata for both input and output ports? If so why?
For example, I have a reader component that reads the employee id and gives the records to another DBJoin component. This DBJoin component retrieves the employee’s dependent as declared by him. This DBJoin component will add the dependent’s details to the empoyee id and pass it on to the next component.
In my case, I added a metadata that describes the employee id (which is a string) to the edge that connects the DBInputTable and the DBJoin components. For the edge that connected the DBJoin and it’s successor I used another metadata component that had the dependent details fields along with the employee id field. I’m getting an error saying the two meta data (inbound and outbound of DBJoin) are not the same.

What should I do now? Also I’m pretty much confused of the concepts used in the transformations and lost in nowhere. So If you can point me to some documentation that clearly explains the concepts, it will be great. Of course I read the wikis. But that didn’t help me to understand the basic principles.

Thanks in advance,
- mseethar

2

Hello, DBJoin requires the same metadata on the input and on the 1st output port (if connectected); on the 0th port the metadata can be different. Can you show your graph? Problem is probably somewhere else.

3

In CloverETL, any two components have to be connected with the same metadata - actually, the edge which connects them has only one metadata set.

If you need to add or drop field in your flow of data, you have to use component specially designed to do that - one of Reformat, Joiner (any) and some others.

To your questions - transformer component is usually used to change the structure of data - therefore it has different metadata on its input and output.

In your situation, the DBJoin will read some data from its only input port and if able to find corresponding record in DB, then it will call the “transform” function where you can define how your input port data and your data from DB should map on output metadata - the structure there can be totally different from input.