As graphs run, each component posts input/output counts. These can be seen in the designer’s graph tracking pane and also in its console pane. These counts are also available (at least for viewing) from ETL Server’s graph execution history.
Is there a way to access these counts from within a graph? Like, I want to be able to access input and output counts for a normalizer component that is run during phase 0, and then log those counts to some external store during phase 1 (or later).
Possible?
Hi sances,
This information is not available during graph run. You can workaround this by duplicating records from input/output edge by SimpleCopy and count them by Aggregate component. Then Aggregate result can be used in later phases of graph.
Other possibility (within single component) is to create global variable (defined above first CTL2 function).
I don’t know problem you are solving, but if it is transformation M:N then maybe http://doc.cloveretl.com/documentation/ … ollup.html may be helpful.
Thanks.
So, is it possible to access these results in a subsequent graph? How does the server retain them? Can I not obtain these results via web service?
Answer highly depends on version of Server you use.
For 3.3+: In case of JobFlow containing 2 subsequent ExecuteGraph (http://doc.cloveretl.com/documentation/ … graph.html) components passing records count from first to second; information about passed records can be obtained in output mapping of first ExecuteGraph and passed by input mapping of second. In case graph is executing subsequent graph from within itself; you can obtain passed records counts by methods mentioned in last mail and pass them via input mapping.
For older: you can call “graph_status” or “executions_history” by http://doc.cloveretl.com/documentation/ … p-api.html from subsequent graph, but you need to know runID. This is stored in parameter “RUN_ID” of parent graph and need to be passed to child/subsequent graph which uses HttpConnector for getting data.
I hope this helps.