Reading Column Headers from an Incoming CSV file

CloverDX Platform
1

Hello,

First time post but glad I found the site!

I am having trouble with reading column headers from an incoming CSV file and grabbing the data in the rows below.

My code correctly picks up the column headers and identifies the data below for the specific columns into my metadata. However, it always leaves off the last column and it’s data. I’ve used print log to verify the number of columns that my code is picking up and it’s picking it up correctly. However, it is adding 2 more spaces to the data and not moving to the out port for the metadata.

Here’s a snapshot of the printLog:

12:07:03,858 INFO [MAP_COLUMNS_7] Fields=64
12:07:03,858 INFO [MAP_COLUMNS_7] Last Field:Excluded EE NonMatch

It correctly identifies there are 64 fields in the file and that the last field is Excluded EE NonMatch

12:07:03,889 INFO [MAP_COLUMNS_7] Fields=64 Field: Excluded EE Deferral Value: Length: 0
12:07:03,889 INFO [MAP_COLUMNS_7] Fields=64 Field: Excluded EE Match Value: Y Length: 1
12:07:03,889 INFO [MAP_COLUMNS_7] Fields=64 Field: Excluded EE NonMatch
Value: Y
Length: 3

It incorrectly shows that the length of the last field is 3 when it really is 1. Could it be something to do with the end of line delimiter? The datafile itself only has Y in it so I know the length of the field is not 3.

Any help is much appreciated!

Thanks!

2

Imohtep86,

You mentioned that you are reading the Column headers and extracting data from a CSV file:

  • Are you using the the FlatFileReader component to read this File?

  • Are you creating the metadata manually or are you using the Extract metadata?

  • If possible please send me your graph, with the basic structure of the CSV file (it can be dummy data)

I would like to take a closer look on how the CSV file structure looks and how you configured the metadata.