Duplicate records

CloverDX Platform
1

Hi,

Currently I’d like to duplicate records without using copy.
In my example, I have 13 records that I’d like to multiply by 10’000. In order to get 130’000 records.
I cannot do it in the input file as it’s an Excel file and doesn’t support more than 65K lines.

What do you suggest?
Thanks

2

Hello,
I see two simple solutions:

  • copy these records many times to the flat file:
    [list:1pwkoc8m][*:1pwkoc8m]XLSReader → DataWriter with append=“true”

  • DataGenerator → RunGraph
    [/*:m:1pwkoc8m]

  • Use normalizer, that just duplicates the input records:

//#CTL2  
function integer count() {  
	return ${rec_no};  
}  
function integer transform(integer idx) {  
	$0.* = $0.*;  
    return OK;  
}

[/list:o:1pwkoc8m]

3

Thanks I used the second solution and it works fine.

FYI I created around 1’000’000 records from 13. with the following functions:

function int count() {
	return 75000;
}

and

function int transform(int idx) {
	$0.* = $0.*;
	if(idx != 0){
		$0.positionType = $0.positionType + '-gen-' + num2str(idx);
	}
    return 0;
}

My process is generating CSV files and run in 150 seconds on a powerful machine.