Adding Prefix to Metadata Created from Excel File

CloverDX Platform
1

I create metadata by reading the headline from an excel file, however, I want to add prefix to the headline. Is there a way to accomplish this? Right now, I will add prefix in the excel file then create metadata from it.

Thanks,
Perri

2

Hi Perri,

You can of course edit the result metadata as the last step of the metadata extraction process. In case this is not the right solution for you, can you please give an example of the desired input and output?

Thanks.

3

I attached a graph where I have this need. Take a look at the metadata: Input_LegalName and Input_PrimaryName. These two meta data are derived from the same excel file. However, I need to separate LegalName and PrimaryName then merge them back to do further transformation. Adding a prefix to one of the metadata can be tedious if were to do it manually. Is this something can be achieved within Clover?

4

I am affraid there is no direct way of achieving this in CloverETL Designer. But there is one indirect way.

You can externalize the metadata. The result is an XML file (*.fmt). This file can be read by UniversalDataReader as one string value and used as an argument of replace function in CTL.

$out.0.field1=replace($in.0.field1,"app","LEG");

The result of this function can be saved into another fmt file and used by other graphs.

By the way, looking into your graph, I recommend replacing ExtHashJoin components with ExtMergeJoins which are more suitable for joining two inputs of similar size.

I hope this helps

5

Thanks! What is the diff between merge join and ExtHashJoin? Merge join requires same data structure?

6

ExtHashJoin

Number of records on a slave port should be small enough because all slave records are stored in memory during the whole joining process. This speeds up the process significantly but it also has increased memory requirements. A common example is let’s say 10 million records on the master port and 10 thousand records on the slave port. 10 thousand records can be stored in memory which causes the 10 million master records to be processed faster than in case of ExtMergeJoin. An advantage is also the fact that the input records do not have to be sorted first. You can find more information on our ExtHashJoin documentation page.

ExtMergeJoin

Both input sets can be larger than your memory because none of them is held in memory as a whole. This fact, on the other hand, enforces that both inputs have to be sorted first. But it has an advantage as well because sorted inputs can be joined much faster than unsorted inputs. More information is again available on ExtMergeJoin documentation page.