Dynamic Metadata

CloverDX Platform
1

Hi,

I want to know how to create dynamic metadata for an edge in a loop.

I want the sql query on the metadata to refer to certain outputs of the query.

I know that we can use change the metadata with an sql query but I want the table which we select from (in blue) to change for each loop iteration depending on the INPUT of the query children component. This is because the query preparation will change the output of the query each loop iteration and the metadata will be different.


<Metadata connection="JDBC0" id="DynamicMetadata0" name="users"
          sqlQuery="SELECT * FROM [color=#804080]users[/color] LIMIT 1"/>

Above code is from the documentation page

http://doc.cloveretl.com/documentation/ … adata.html

Thank you very much.

2

Hello Fyrezx,
in theory, you can apply dynamic metadata to an edge inside of a loop in the same way as you normally would in the rest of the graph. To do that, you would have to switch to the ‘Source’ tab in the CloverETL Designer view and add a code that resembles what you mentioned in your post. Then you can simply drag and drop the metadata from the Outline pane into your graph (after switching back to the ‘Graph’ view).
However, the practical usage of this feature inside of a loop is significantly less applicable. If my understanding is correct, what you are trying to achieve here is that the metadata will change during every iteration of the loop. However, the dynamic metadata is resolved at the beginning of the graph run and cannot be changed during the run. This is how CloverETL is currently designed to work.
Moreover, although I don’t see the details of you graphs besides the provided screenshot, I dare to presume that you might also face issues with your token flow due to the DBInputTable component. Each token sent to the loop body must be navigated back to the Loop component. The token cannot be duplicated or cannot disappear. If these rules are not strictly followed, your jobflow could get stuck in a deadlock situation. Also, there has to be the ‘iterationNumber’ field for the purpose of the proper loop iterating. One of these problems could be indicated by clicking on the red X symbol on the Loop component.
Overall, I would like to understand your use case in more detail in order to be able to suggest an optimal approach in Clover. Please provide us with additional items of information such as:

  • what your overall goal is

  • what sort of input data you need to process

  • your version of CloverETL Designer

  • your version of CloverETL Server, if you have one

  • your jobflow/graph (if it contains sensitive data, please remove or mask it)

Kind regards,