MySQLBulkWriter empty rows

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

Nismon8er
Posts: 10
Joined: Thu Apr 01, 2021 7:51 pm

MySQLBulkWriter empty rows

Postby Nismon8er » Thu Apr 01, 2021 8:07 pm

Hello,
I have implemented the MySQLBulkWriter and it is working like a charm. My issue is that there are 713 records that are being read via DatabaseReader and when the MySQLBulkWriter writes to my other table it ends up adding 45 additional blank rows. When I use DatabaseWriter it works fine and only adds the 713. Also I know I shouldn't be using MySQLBulkWriter for small datasets but this is just a trial run to make sure I am configuring it properly. Any ideas what the issue could be?

jandikovae
Posts: 64
Joined: Fri Nov 04, 2016 8:51 am

Re: MySQLBulkWriter empty rows

Postby jandikovae » Thu Apr 22, 2021 2:22 pm

Hi,

From my perspective, I can't see any reason for the behavior you have described. In my testing graph (attached), I am generating 713 records, loading them to my local database using MySQLBulkWriter and I can see in the database that there is the same number of records then. Therefore I believe that it might be related to some data issue.

To be able to see the script and data being processed by the component, I would suggest you enter "Path to control script" and "Loader input file" attributes in the MySQLBulkWriter component. As long as you enter a filename that does not exist yet at the selected location, it will create the file and leave it at the location after the graph run (both attributes work the same). It means that you can run the graph and then review the data that has been used by the component. (Also, don't forget to delete the files or change the attributes before you run the graph again, otherwise, these files are taken instead of creating and using the new ones).

Also, the MySQLBulkWriter is using LOAD DATA Statement. You can review all the default values and also in the "Path to control script" file you can see what statement parameters are sent to the database. The script can be adjusted using the "Control script parameters" attribute to make sure the mysql utility is getting the correct setups and interpreting your data correctly.

In case you are still getting the extra rows and none of the above helped to resolve the situation, please, provide us with a sample of your input data and the script that the component is using (files from "Path to control script" and "Loader input file" attributes) as we can take a further look.

Thank you. Best Regards,
Eva
Attachments
BulkWriter.grf
(2.47 KiB) Downloaded 26 times
---
Eva Jandikova
CloverCARE Support
CloverDX

Visit us online at http://www.cloverdx.com

Nismon8er
Posts: 10
Joined: Thu Apr 01, 2021 7:51 pm

Re: MySQLBulkWriter empty rows

Postby Nismon8er » Mon Apr 26, 2021 2:58 pm

So I figured it out..

There were some line breaks "\n" in one of my fields and for some reason that threw EVERYTHING off when the bulkwriter started working. I removed any "\n" using a Reformat and that fixed my issue.

Thank you!