Filtering Trailer Record

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

skaszyk
Posts: 12
Joined: Mon Apr 14, 2014 9:52 pm

Filtering Trailer Record

Postby skaszyk » Wed Jun 18, 2014 4:13 pm

I'm using a UniversalDataReader to read a fixed length file that contains both a header and trailer record. I can skip the header record by setting 'Skip first line' to true (I see this is deprecated but can't find a better solution).

However, I'm unable to skip the trailer record. As a result I get an error stating 'Unexpected record delimiter, probably record is too short."

How can I skip trailer records? The files have a variable number of records so I can't control it by the # of records in the file.

slechtaj
Posts: 192
Joined: Wed Aug 15, 2012 8:18 am

Re: Filtering Trailer Record

Postby slechtaj » Thu Jun 19, 2014 2:55 pm

Hi skaszyk,

Deprecated attribute "Skip first line" can be replaced with "Number of skipped records" (or "Number of skipped records per source" if you read multiple files). Regarding trailing records, you may simply use "Max error count" attribute and set Data policy attribute to Controlled. If you set Max error count to accept just one record that does not match the metadata, it will work the way you want to.

For more information about aforementioned attributes, please refer to our documentation:
Data Policy - http://doc.cloveretl.com/documentation/ ... olicy.html
UniversalDataReader - http://doc.cloveretl.com/documentation/ ... eader.html

Hope it helps.
Jan Slechta
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com

How to speed up communication with CloverCARE support

skaszyk
Posts: 12
Joined: Mon Apr 14, 2014 9:52 pm

Re: Filtering Trailer Record

Postby skaszyk » Fri Jun 20, 2014 4:21 pm

Worked perfectly! Thanks for the reply and the detailed links, very helpful.