Filtering Trailer Record

[skaszyk]

I'm using a UniversalDataReader to read a fixed length file that contains both a header and trailer record. I can skip the header record by setting 'Skip first line' to true (I see this is deprecated but can't find a better solution).

However, I'm unable to skip the trailer record. As a result I get an error stating 'Unexpected record delimiter, probably record is too short."

How can I skip trailer records? The files have a variable number of records so I can't control it by the # of records in the file.

[slechtaj]

Hi skaszyk,

Deprecated attribute "Skip first line" can be replaced with "Number of skipped records" (or "Number of skipped records per source" if you read multiple files). Regarding trailing records, you may simply use "Max error count" attribute and set Data policy attribute to Controlled. If you set Max error count to accept just one record that does not match the metadata, it will work the way you want to.

For more information about aforementioned attributes, please refer to our documentation:
Data Policy - http://doc.cloveretl.com/documentation/ ... olicy.html
UniversalDataReader - http://doc.cloveretl.com/documentation/ ... eader.html

Hope it helps.

[skaszyk]

Worked perfectly! Thanks for the reply and the detailed links, very helpful.