Data Reader with Comma at EOL

CloverDX Platform
1

I’m trying to read a .csv with the Universal Data Reader that has a comma a /r and /n for each end of line(EOL). I’m not able to get the reader to go past the first record, it always fails on the last field. Here is the error message.

Parse error: The size of data buffer for data record is only 12288. Set appropriate parameter in defautProperties file. when parsing record #1 field blank

I’ve tried setting the last field to ,\r\n for the delimiter and tried creating a bogus blank field with just \r\n. Both return the same error which seems kind of strange to me.

Here is a sample of the data.

0,13.65,13.65,“”,“”,“”,“”,“”,“Nordrhein-Westfalen”,

Any help would be great, thank you!

Glenn

2

You need to add a 10 field to your metadata, type integer, delimiter \r\n

Name Type Delimiter
Field1 Integer ,
…
Field8 String ,
Field9 String ,
Field10 integer \r\n

3

Thanks for the idea, that worked great. Of course I had two issues which was compounding the issue.

The other issue is I’m having trouble reading the file directly from the customer. It is coming from a Linux box, probably output from some sort of scripting language. The format is UTF-8 and although it reads fine in the preview the Java engine has issues with it, ex. you can’t see the field preview when editing the metadata. If I open on my windows computer in VIM and copy and paste it to a new file it works great. That workaround is OK for now, but this is a daily process that needs to be automated.

Once again, any ideas appreciated!

Glenn

4

Hi Glenn,

could you please describe your problem in more details please? Do I understand it correctly that on the same computer the preview works OK (when creating metadata) but it doesn’t work when editing metadata? Do you see some error message?

On first thought I would expect some problems with line delimiters - can you try just \n or just \n?

Best regards,
Jaro

5

Hello Glenn,
the problem is probably with line breaks. To make your graph working with both, linux and windows ends of line, you can use alternative delimiter \n\\|\r\n (for full information see Changing and Defining Delimiters)