CloverETL dynamic output file

CloverDX Platform
1

Hi,

CloverETL helped me lot in bulk data handling. But still I can’t find a solution for this issue. In one of my graph the output file name will be generated by embedded java and the output file has to be in that name.

Can anyone tell me whether this can be done?

Regards,
Mari

2

Hello,
most of the flat file writers support partitioning (see partitionKey, partition, partitionFileTag and partitionOutFields attributes of eg. Data Writer component). You can split records to different files depending on the value of given field. The name of output file can be stored directly in data record or in look up table.
Following example could help you:

<?xml version="1.0" encoding="UTF-8"?>
<Graph id="1238657413863" name="partionOutput" >
<Global>
<Metadata id="Metadata0" >
<Record fieldDelimiter=";" name="input_record" recordDelimiter="\r\n" type="delimited">
<Field name="email" type="string"/>
<Field auto_filling="source_name" name="file" type="string"/>
</Record>
</Metadata>
<Property fileURL="workspace.prm" id="GraphParameter0"/>
</Global>
<Phase number="0">
<Node fileURL="${DATAIN_DIR}/email/*.txt" id="DATA_READER0" type="DATA_READER"/>
<Node fileURL="${DATAOUT_DIR}/"  id="DATA_WRITER0" partitionFileTag="keyNameFileTag" partitionKey="file" type="DATA_WRITER"/>
<Node  id="REFORMAT0" type="REFORMAT">
<attr name="transform"><![CDATA[import java.io.File;

import org.jetel.component.DataRecordTransform;
import org.jetel.data.DataRecord;
import org.jetel.exception.TransformException;


public class getFileName extends DataRecordTransform {

	public int transform(DataRecord[] arg0, DataRecord[] arg1)
			throws TransformException {
		arg1[0].getField(0).setValue(arg0[0].getField(0));
		File output = new File(arg0[0].getField(1).toString());
		arg1[0].getField(1).setValue(output.getName());
		return 0;
	}

}
]]></attr>
</Node>
<Edge fromNode="DATA_READER0:0" id="Edge0" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (output)" toNode="REFORMAT0:0"/>
<Edge fromNode="REFORMAT0:0" id="Edge1" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (out)" toNode="DATA_WRITER0:0"/>
</Phase>
</Graph>