Execute a block of components in a loop

Hi,
I am working on generating a report where in:

1. Retrieve a set of data from database A.
2. Retrieve a list of IDs from another database B.
3. For each item in the list, retrieve the details from database B.
4. Join the details from database B with the data from database A.
5. Write the report to a spreadsheet.

# 1 & 2 need to be performed only once.
# 3, 4 & 5 need to be performed in a loop.

I tried to push the data one by one with a delay from ‘Data Generator’ to the ‘DB Reader’; but DB Reader just takes the data pushed last & works on it.

I am able to achieve this for a single ID from database B; but unable to perform the series actions for the data from database B. Can someone please help how to achieve this ?

Any help on this is highly appreciated.

Thanks !

Hi Balaji,

We might need a bit more information here: could you please explain in more detail what you are trying to achieve? If you could explain the desired process flow on examples and also explain why steps 3-5 need to run in a loop and why you are using the DataGenerator component in your project, we can then assist you further.

Sure. Let me try to explain further:

1. Retrieve a set of data from database A.
2. Retrieve a list of IDs from another database B.
3. For each item in the list, retrieve the details from database B.
4. Join the details from database B with the data from database A.
5. Write the report to a spreadsheet.

# 1 is loading static data which needs to be loaded from database A once during the report generation.

# 2 is getting the list of IDs from database B. Let say, this is like getting the list of Department IDs.

# 3, 4 & 5 are consolidating/ joining the data from database A & B and generating the report. In analogy to the above: For each department, we need to generate the report with information from 2 databases.

Please let me know if you need more information.

Can someone please check my query & respond ?

Hi Balaji,

You should be able to achieve what you need by using the following components:

1. Steps 1 and 2 from the list you provided in your previous post can be done by using two DatabaseReader components (i.e. to get data from database A and a list of IDs from database B).
2. Step 3 can be achieved by using the DBJoin component. This way you can retrieve details from database B based on the list of IDs you retrieved from the DatabaseReader.
3. Step 4 (join data from databases A and B) can be performed by using either the ExtMergeJoin or ExtHashJoin components.

If you use this approach, all data will be processed and you should not be hitting the problem where only the last record is processed.

I am attaching a sample graph to demonstrate how the components can be used.

If this solution does not meet your requirements and you need prompt assistance with making your graph work, please contact the CloverDX support team (you can use the link at the very bottom of this page). We do our best to respond to forum posts as soon as we can, however, we cannot guarantee timely responses here on the forum.

Thank you