Hello,
I’m curious to know if there exists a way in which a user could export a graph’s source data out of Clover.
The use-case I’m trying to accommodate is so I can parse ALL transformations that occur to a specific data field throughout a graph, which often have upwards of 10 stages (or individual graphs within a master graph).
I’ve currently been working around this, while obtaining the same results, by simply copying and pasting the source data from each graph into a master text document. For graphs with sub-graphs, I have to open each one individually. This process is getting increasingly complex as the complexity of our graphing increases.
Any and all help/suggestions/comments would be appreciated.
Regards,
- Tyler
Hi Tyler,
I am afraid that I am not familiar with the business background of your question, therefore I’ll try to answer as general as possible.
The whole source data that a graph uses (graph itself, data-in directory, metadata, parameters) can be viewed in the Navigator pane in the Designer and out of CloverETL they are located in a workspace directory that a user specifies when a Designer is started for the first time (by default at C:\Users\Eva\workspace on a Windows machine). If the project is a Server project, the data is synchronized to a Sandbox directory (you can control the location of Sandboxes in the Server console “Sandboxes” section.
As you mentioned also the fact that you are interested in data within the graph run (“ALL transformations that occur to a specific data field throughout a graph”) I would like to bring to your attention a debug data that are usually stored in temporary directories.
Local projects store their debug data on the following location by default: C:\Users\Eva\workspace\.metadata\.plugins\com.cloveretl.gui.runtime\runtime-work\temp\clover_temp_node01\<graph_run>\clover-debug
On the server (for server projects), these data are stored in C:\ApacheTomcat8\clover_temp_node01\<graph_run>\clover-debug (if you use Tomcat as an application server). On the location mentioned above, you can find a file for each edge.
There is a file for each edge in the graph that has a debug turned on (on newer versions of CloverETL the debug mode is by default turned on for all edges). The files have extension .dbg and can be processed by CloverETL graphs.
Also, if checking/storing data being processed in a graph run is a part of your process flow, you can also add FlatFileWriters in your graph design and create those files automatically (e.g. add a SimpleCopy component when one output port goes to the next stage and the other output port writes the current data to a file). The complexity of your graphs might increase, but this way CloverETL can do automatically what you currently do manually.
Please let me know if this answers your question or if I have misunderstood the situation.
Thank you and have a nice day, Eva