How does clover ETL Write lines during DataTransform

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

naveenvijay
Posts: 12
Joined: Thu Oct 13, 2011 6:49 pm

How does clover ETL Write lines during DataTransform

Postby naveenvijay » Tue Oct 18, 2011 2:03 am

I have created a graph file which reads data from Database and I am sending it to a reformat component and I am writing it to a file from ReFormat Component. I am little confused the way Clover ETL Is reading and writing the records. When It writes the records is it processing one line at a time.

Where can I get more information about public int transform (DataRecord[] source, DataRecord[] target)

In Reformat Component will I be able to write a custom call to Database to insert any records..

Source Database -> Reformat -> If Value is "123" Send it to file -> File Output
-> If Value is "234" Write to Database -> File Output


Basically I want to understand what is the right way to do Database Call.

Thanks,
Naveen

avackova
Posts: 841
Joined: Fri Jul 20, 2007 9:28 am

Re: How does clover ETL Write lines during DataTransform

Postby avackova » Tue Oct 18, 2011 9:54 am

Hello Naveen,
  • Why do you think that CloverETL "is processing one line at a time"? DataParser (used by UniversalDataWriter) formates one record to byte buffer and after it is full it flushes it to a file.
  • The best information about the interfaces and classes that can be used by users can be found in cloverETL.javadoc.rel-3-1-2.zip
  • The right way to database call is to use DBOutputTable or DBExecute:
    graphDBJoin.png
    graphDBJoin.png (48.16 KiB) Viewed 2643 times

    You can of course insert records directly from Reformat component also (see Problem with $ character for inspiration).
Agata Vackova
Javlin a.s.
agata.vackova@javlin.eu