Page 1 of 1

MySQLBulkWriter empty rows

Posted: Thu Apr 01, 2021 8:07 pm
by Nismon8er
Hello,
I have implemented the MySQLBulkWriter and it is working like a charm. My issue is that there are 713 records that are being read via DatabaseReader and when the MySQLBulkWriter writes to my other table it ends up adding 45 additional blank rows. When I use DatabaseWriter it works fine and only adds the 713. Also I know I shouldn't be using MySQLBulkWriter for small datasets but this is just a trial run to make sure I am configuring it properly. Any ideas what the issue could be?

Re: MySQLBulkWriter empty rows

Posted: Thu Apr 22, 2021 2:22 pm
by jandikovae
Hi,

From my perspective, I can't see any reason for the behavior you have described. In my testing graph (attached), I am generating 713 records, loading them to my local database using MySQLBulkWriter and I can see in the database that there is the same number of records then. Therefore I believe that it might be related to some data issue.

To be able to see the script and data being processed by the component, I would suggest you enter "Path to control script" and "Loader input file" attributes in the MySQLBulkWriter component. As long as you enter a filename that does not exist yet at the selected location, it will create the file and leave it at the location after the graph run (both attributes work the same). It means that you can run the graph and then review the data that has been used by the component. (Also, don't forget to delete the files or change the attributes before you run the graph again, otherwise, these files are taken instead of creating and using the new ones).

Also, the MySQLBulkWriter is using LOAD DATA Statement. You can review all the default values and also in the "Path to control script" file you can see what statement parameters are sent to the database. The script can be adjusted using the "Control script parameters" attribute to make sure the mysql utility is getting the correct setups and interpreting your data correctly.

In case you are still getting the extra rows and none of the above helped to resolve the situation, please, provide us with a sample of your input data and the script that the component is using (files from "Path to control script" and "Loader input file" attributes) as we can take a further look.

Thank you. Best Regards,
Eva

Re: MySQLBulkWriter empty rows

Posted: Mon Apr 26, 2021 2:58 pm
by Nismon8er
So I figured it out..

There were some line breaks "\n" in one of my fields and for some reason that threw EVERYTHING off when the bulkwriter started working. I removed any "\n" using a Reformat and that fixed my issue.

Thank you!