Some questions about posibilities in using CloverETL Desig

CloverDX Platform
1

Hello,

I have some questions about programming in CloverETL Designer:

  1. How to set-up the graph? I want my graph to calculate according user’s settings. The user should the graph set up in some dialog window (choose, what he wants to be calculated from some options) . Is it possible?
    I read manual and found some possiblities (parameters, dictionary). Is some of them useful, or is there another (better) way, how to do it?

  2. Is some component (or is it possible to do it), which reads some records from one input port, make some operations above it and write another records into the output ports, but with different length (for example it reads records about employees and write on the port 1 employees names and on the port 0 count of records, which is only one number actually)?

Thanks for answers.
Jana

2

Hello Jana,
user can edit component’s attributes and dictionary values directly in CloverETL Designer. But the better way, is to use parameters as they can be also passed, when executing the graph from command line. Theoretically you can write an eclipse plug-in with the wizard, that creates a graph with user provided options, but it is undoubtedly not easy.
The answer to your 2nd question is Reformat - it receives one record and can create from it one record, that will be sent to specified port or can create array of different records, that will be sent to output ports, or can sent nothing to any output ports (see Return Values of Transformations).
In your example, that transformation could look as follows:

//#CTL2
integer counter = 0;
// Transforms input record into output record.
function integer transform() {
	$0.Employee = $0.Employee;
	$1.counter = counter++;

	return ALL;
}
3

Hi,

1)I tried it with the Reformat component, but it doesn’t work properly.
Actually, if I copied the code, the errors occured (The variable ALL is not declared.) I don’t understand, why it does not work, in the link
http://www.cloveretl.com/documentation/ … alues.html
is written, that ALL do not need do be declared.

I noticed one difference, when I open the source, there is impicitly written, not //#CTL2, but //#TL. Could it be the the cause, why it does not work? I use the Designer Version: 2.9.7.

I tried do it in another way, but the result was, that on one port were employees and on another count, but not as one number, but in each pass there was sent the count of executed records, but it was not, what I wanted.

2)And I have one another problem. I need some good documentation for CloverETL. I found only JavaDoc and user guide. Anything else exists to get information, how it work all together? Some manual for developers?

3)And one another question. I don’t understand the overview. Could be the transformation component written in Java and have many functions, not only the transform(), finish(), initiate()? I don’t see, how it works…

I need to do one component, which calculates statictics from one file (some should be implicit(number of records, average for numeric column…) and user can choose some another–f.e. quantile, diversion…). the component sholud give the result in xsd (or dtd). It should be easily extensible.
The result should be read by another java utility (so I have chosen xsd for the result), which displayes everything in user-friendly way – some histograms, graphs…

  1. Could be the result only one string (one record, one field)? I tried it, but only 1MB passed. I have changed defaultProperties. There was the information, that the string could be 2^31.
4

Hello Jana,

  • The transformation is written in CTL2, that is available since CloverETL 3.0. Corresponding transformation in CTL1 (TL) looks as follows:
//#TL  
int counter = 0;  
// Transforms input record into output record.  
function transform() {  
   $0.Employee := $0.Employee;  
   $1.counter := counter++;  
   return ALL  
}

The above transformation copies the field Employee from input port 0 to output port 0 and sends to output port 1 number of currently processed records.

  • All available resources and documentation can be reached from Resources page

  • Examples of graph for statistic calculations can be found among our examples (see DataProfiling project). If such graph is not enough for you, you can create your own component.

  • I’m sorry, but I don’t understand the question. What does mean “only 1MB passed”? Have you got an error?