Hello,
I have several large Clover jobs which include multiple queries that return millions of rows. When running them within the DBInputTable components, I’ve noticed the the CPU load (indicated in the tracking tab in the clover portal) seems to vary quite a bit form day to day.
Can anyone give me a good idea on what I can do to maximize the CPU utilization on the specific threads that are running these components?
Thank you,
Hi,
Here are some options how to possibly increase the CPU load:
1. Increase the value of the Fetch size property of the DBInputTable component.
2. Use more DBInputTable components (in single graph) and load only some part of the table in each of them (parallel loading). When the loading is done, you can merge the results by for example a SimpleGather component.
However, I would recommend to focus more on the speed of the loading instead of the CPU load as a higher CPU load isn’t always the same as faster input from a database. You can try setting different values to the Fetch size property and play with the number of DBInputTables in order to achieve the best possible speed. Also, if you have a Cluster license and at least two cluster nodes, you can use the Cluster components to distribute the reading process to these nodes (in order to increase the speed). You can find more information about clustering here and here.
Hope this helps.