Progress Indicator on Transform

CloverDX Platform
1

Hi,

I have written a transform that sucks in some Excel data and then transforms it to a database. Question I have is:

Is it possible to somehow provide feedback that can be used to indicate the progress of a particular phase (I have 2)?

My intention is to provide the percentage completion of a phase as a visual progress bar. Two things I would need is the total records being transformed in the phase and then more importantly, the completed records so far. The Console output seems to provide information, but I’m not sure how I can tap into that programatically?

I use the following code to start my ETL jobs:

// Initiate the graph
EngineInitializer.initGraph(graph, runtimeContext);

Future result;
try{
result = runGraph.executeGraph(graph, runtimeContext);
while (result.isDone()) {;}
if (!result.get().equals(Result.FINISHED_OK)){
System.out.println(“Failed graph execution!\n”);
return false;
}
}catch (Exception e) {
System.out.println(“Failed graph execution!\n” + e.getMessage());
return false;
}

The output is similar to the Console below and I was hoping I could tap into this info in some way:

2009-07-20 19:52:19,150 INFO WatchDog org.jetel.graph.runtime.WatchDog - Sucessfully started all nodes in phase!
2009-07-20 19:52:24,152 INFO WatchDog Tracking - ---------------------** Start of tracking Log for phase [1] **-------------------
2009-07-20 19:52:24,152 INFO WatchDog Tracking - Time: 20/07/09 19:52:24
2009-07-20 19:52:24,152 INFO WatchDog Tracking - Node Status Port #Records #KB Rec/s KB/s
2009-07-20 19:52:24,153 INFO WatchDog Tracking - ---------------------------------------------------------------------------------
2009-07-20 19:52:24,155 INFO WatchDog Tracking - DB_OUTPUT_TABLE1 RUNNING
2009-07-20 19:52:24,155 INFO WatchDog Tracking - %cpu:0.03 In:0 736 56 0 0
2009-07-20 19:52:24,155 INFO WatchDog Tracking - ---------------------------------** End of Log **--------------------------------
2009-07-20 19:52:29,157 INFO WatchDog Tracking - ---------------------** Start of tracking Log for phase [1] **-------------------
2009-07-20 19:52:29,157 INFO WatchDog Tracking - Time: 20/07/09 19:52:29
2009-07-20 19:52:29,157 INFO WatchDog Tracking - Node Status Port #Records #KB Rec/s KB/s
2009-07-20 19:52:29,157 INFO WatchDog Tracking - ---------------------------------------------------------------------------------
2009-07-20 19:52:29,157 INFO WatchDog Tracking - DB_OUTPUT_TABLE1 RUNNING
2009-07-20 19:52:29,157 INFO WatchDog Tracking - %cpu:0.04 In:0 2068 158 266 20
2009-07-20 19:52:29,157 INFO WatchDog Tracking - ---------------------------------** End of Log **--------------------------------
2009-07-20 19:52:29,296 INFO WatchDog Tracking - ----------------------** Final tracking Log for phase [1] **---------------------
2009-07-20 19:52:29,296 INFO WatchDog Tracking - Time: 20/07/09 19:52:29
2009-07-20 19:52:29,296 INFO WatchDog Tracking - Node Status Port #Records #KB Rec/s KB/s
2009-07-20 19:52:29,296 INFO WatchDog Tracking - ---------------------------------------------------------------------------------
2009-07-20 19:52:29,296 INFO WatchDog Tracking - DB_OUTPUT_TABLE1 FINISHED_OK
2009-07-20 19:52:29,296 INFO WatchDog Tracking - %cpu:0.04 In:0 2104 160 0 0
2009-07-20 19:52:29,296 INFO WatchDog Tracking - ---------------------------------** End of Log **--------------------------------

Thanks
Des

2

Hi,

first of all you shouldn’t use this type of statement

while (result.isDone()) {;}

This is active waiting and really undesirable. Probably it takes most of your processing time. For this purpose, please use

result.get()

This method returns immediately the result of graph is available.

Regarding your question, probably the best starting point for your studying is our class TrackingLogger, which is simple NotificationListener. You can register your own listener and via this object you will be informed about everything what happening in the running graph.

Martin