RunGraph component problem

evergreen · December 2, 2009, 12:00am

Hi,

I am using the Rungraph component to run a two other graphs.The graph runs successfully yet I am unable to achieve the result which the two graphs have to be performed.I have also pasted the Log file details after the Node code
Following is the node code.Please suggest

NODE CODE


<?xml version="1.0" encoding="UTF-8"?>
<Graph author="avackova" created="Thu May 07 09:41:13 CEST 2009" guiVersion="2.8.1" id="1241683270406" licenseCode="evaluation" licenseType="Evaluation" modified="Wed Dec 02 10:47:37 IST 2009" modifiedBy="vinutha.harishankar" name="runAllGraphs" revision="1.56">
<Global>
<Metadata id="Metadata0" previewAttachmentCharset="ISO-8859-1">
<Record fieldDelimiter="|" name="graph" previewAttachmentCharset="ISO-8859-1" recordDelimiter="\n" type="delimited">
<Field delimiter="\n" name="graphName" type="string"/>
</Record>
</Metadata>
<Metadata id="Metadata1" previewAttachmentCharset="ISO-8859-1">
<Record name="outdata" previewAttachmentCharset="ISO-8859-1" recordSize="-1" type="delimited">
<Field delimiter=";" name="graph" nullable="true" type="string"/>
<Field delimiter=";" name="result" nullable="true" type="string"/>
<Field delimiter=";" name="description" nullable="true" type="string"/>
<Field delimiter=";" name="message" nullable="true" type="string"/>
<Field delimiter="\n" name="duration" nullable="true" type="decimal"/>
</Record>
</Metadata>
<Property fileURL="workspace.prm" id="GraphParameter0"/>
<Note alignment="1" backgroundColorB="220" backgroundColorG="255" backgroundColorR="255" folded="false" height="86" id="Note0" textColorB="0" textColorG="0" textColorR="0" textFontSize="8" title="Running all requested graphs at once" titleColorB="0" titleColorG="0" titleColorR="0" titleFontSize="10" width="541" x="58" y="122">
<attr name="text"><![CDATA[This graph runs all the graphs that are listed in ${DATAIN_DIR}/example_etl_process.txt. 
Data Reader reads names of graphs to execute, Reformat adds path to graph names and sends them to RunGraph component. Then each graph is executed and the result is sent to Trash. Graphs execution log and results are saved in ${DATAOUT_DIR} directory (runAllGraps.log and allGraphs.txt respectively). ]]></attr>
</Note>
</Global>
<Phase number="0">
<Node enabled="enabled" fileURL="${DATAIN_DIR}/example_etl_process.txt" guiHeight="0" guiName="Read graphs" guiWidth="0" guiX="59" guiY="35" id="DATA_READER0" type="DATA_READER"/>
<Node enabled="enabled" guiHeight="0" guiName="Reformat" guiWidth="0" guiX="218" guiY="35" id="REFORMAT0" type="REFORMAT">
<attr name="transform"><![CDATA[//#TL

// Transforms input record into output record.
function transform() {
	$0.graphName := "${GRAPH_DIR}/"+$0.graphName;
}

// Called to return a user-defined error message when an error occurs.
// function getMessage() {}

// Called during component initialization.
// function init() {}

// Called after the component finishes.
// function finished() {}
]]></attr>
</Node>
<Node alternativeJavaCmdLine="java -Xmx1g -cp" enabled="enabled" guiHeight="0" guiName="RunGraph" guiWidth="0" guiX="370" guiY="35" id="RUN_GRAPH0" ignoreGraphFail="true" logAppend="false" logFile="${DATAOUT_DIR}/runAllGraps.log" sameInstance="false" type="RUN_GRAPH"/>
<Node debugFilename="${DATAOUT_DIR}/allGraphs.txt" debugPrint="true" enabled="enabled" guiHeight="0" guiName="Trash" guiWidth="0" guiX="546" guiY="35" id="TRASH0" type="TRASH"/>
<Edge debugMode="false" fromNode="DATA_READER0:0" guiBendpoints="" id="Edge0" inPort="Port 0 (in)" metadata="Metadata0" outPort="Port 0 (output)" toNode="REFORMAT0:0"/>
<Edge fromNode="REFORMAT0:0" guiBendpoints="" id="Edge2" inPort="Port 0 (in regular mode: names of the graphs to be executed, otherwise nothing)" metadata="Metadata0" outPort="Port 0 (out)" toNode="RUN_GRAPH0:0"/>
<Edge debugMode="false" fromNode="RUN_GRAPH0:0" guiBendpoints="" id="Edge1" inPort="Port 0 (in)" metadata="Metadata1" outPort="Port 0 (regular mode: status messages, pipeline mode: an information record in case of successful termination, otherwise nothing)" toNode="TRASH0:0"/>
</Phase>
</Graph>

THE LOG STATUS

INFO [WatchDog] - [Clover] phase: 0 initialized successfully.
INFO [WatchDog] - Starting up all nodes in phase [0]
DEBUG [WatchDog] - DATA_READER0 … started
DEBUG [WatchDog] - REFORMAT0 … started
DEBUG [WatchDog] - RUN_GRAPH0 … started
DEBUG [WatchDog] - TRASH0 … started
INFO [WatchDog] - Successfully started all nodes in phase!
INFO [RUN_GRAPH0] - Running graph ./graph/A01_Uportal_Transfer.grf
A02_Uportal_Transfer.grf in separate instance of clover.
INFO [RUN_GRAPH0] - Executing command: “[java, -Xmx1g, -cp, /D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/cloveretl.engine.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/annotations.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/cloveretl.engine.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/commons-cli-1.1.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/commons-logging.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/commons-net-1.4.1.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/dom4j-1.6.1.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/icu4j-normalizer.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/jakarta-oro-2.0.8.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/janino.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/javolution.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/jms.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/joda-time-1.6.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/jsch-0.1.39.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/jsr173_1.0_api.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/jxl.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/log4j-1.2.12.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/ooxml-schemas-1.0.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/openxml4j-1.0-beta.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/poi-3.5-beta4-20081128.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/poi-ooxml-3.5-beta4-20081128.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/tar.jar;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/lib/xmlbeans-2.3.0.jar;C:\clover_update1\uportal\trans, org.jetel.main.runGraph, -plugins, /D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.bulkloader/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.compiler.commercial/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.component/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.component.commercial/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.connection/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.ctlfunction/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.engine/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.jdbc/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.lookup/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.lookup.commercial/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.sequence/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.thirdparty/;/D:/softwares/Clover_eval_update/eclipse/plugins/com.cloveretl.gui_2.8.1/lib/plugins/org.jetel.tlfunction/;/D:/softwares/Clover_eval_update/eclipse/configuration/org.eclipse.osgi/bundles/277/1/.cp/lib/plugins/org.jetel.quickbase.commercial/, , -nodebug, ./graph/A01_Uportal_Transfer.grf
A02_Uportal_Transfer.grf]”
INFO [RUN_GRAPH0] - ./graph/A01_Uportal_Transfer.grf
A02_Uportal_Transfer.grf: Processing with an ERROR:
: Process exit value = -1
WARN [RUN_GRAPH0] - Some graph(s) finished with error.
INFO [WatchDog] - ----------------------** Final tracking Log for phase [0] **---------------------
INFO [WatchDog] - Time: 01/12/09 19:46:24
INFO [WatchDog] - Node Status Port #Records #KB Rec/s KB/s
INFO [WatchDog] - ---------------------------------------------------------------------------------
INFO [WatchDog] - DATA_READER0 FINISHED_OK
INFO [WatchDog] - %cpu:… Out:0 1 0 0 0
INFO [WatchDog] - REFORMAT0 FINISHED_OK
INFO [WatchDog] - %cpu:… In:0 1 0 0 0
INFO [WatchDog] - Out:0 1 0 0 0
INFO [WatchDog] - RUN_GRAPH0 FINISHED_OK
INFO [WatchDog] - %cpu:… In:0 1 0 0 0
INFO [WatchDog] - Out:0 1 0 0 0
INFO [WatchDog] - TRASH0 FINISHED_OK
INFO [WatchDog] - %cpu:… In:0 1 0 0 0
INFO [WatchDog] - ---------------------------------** End of Log **--------------------------------
INFO [WatchDog] - Execution of phase [0] successfully finished - elapsed time(sec): 1
INFO [WatchDog] - -----------------------** Summary of Phases execution **---------------------
INFO [WatchDog] - Phase# Finished Status RunTime(sec) MemoryAllocation(KB)
INFO [WatchDog] - 0 FINISHED_OK 1 3604
INFO [WatchDog] - ------------------------------** End of Summary **---------------------------
INFO [WatchDog] - WatchDog thread finished - total execution time: 1 (sec)
INFO [main] - Freeing graph resources.
INFO [main] - Execution of graph successful !

avackova · December 2, 2009, 8:36am

Hello,
the reason of the failure must be in the log file (runAllGraps.log). Do the graphs execute successfully if you run them alone?

evergreen · December 2, 2009, 10:29am

Yeah avackova the graphs run successfully if I run them individually.
Can you suggest me

avackova · December 2, 2009, 11:45am

What is in runAllGraps.log and allGraphs.txt? One of thees files has to contain the error.

evergreen · December 3, 2009, 6:52am

Hi avackova

Your suggestions were helpful to fix my problem.Thanks very much for the help.
The file from where I was reading the multiple graphs need to be formatted.

There was no delimiter between the graphNames.
I included a semicolon as the delimiter.The graph ran successfully.

mkansal · August 8, 2011, 7:38pm

Hello there,

Could you please let me know how to format the example_etl_process.txt file? I tried to add the file names as ${GRAPH_DIR}/test2/eig_import_match_candidate_parameter_test.grf;

However, it throws the following exception. Also, if I need to pass parameters to the listed graphs, can I use the same text file to pass those? If so, what should be the format of the text file?

2011-08-08 14:33:45,491 ERROR 21562 [WatchDog] Node DATA_READER0 finished with status: ERROR caused by: Parsing error: Unexpected end of file in field # 1 of record # 1, value: ‘SimpleDataParser does not provide raw record.’
2011-08-08 14:33:45,491 ERROR 21562 [WatchDog] Node DATA_READER0 error details:
org.jetel.exception.BadDataFormatException: Parsing error: Unexpected end of file in field # 1 of record # 1, value: ‘SimpleDataParser does not provide raw record.’
at org.jetel.data.parser.SimpleDataParser.parsingErrorFound(SimpleDataParser.java:407)

Thanks!

avackova · August 9, 2011, 7:23am

Hello,
if you have only graph names in the input file, the metadata should not have defaultFieldDelimiter and EOL as record delimiter:OnlyGraph.png If you want to pass the parameters also, you need to add one more field:GraphAndParameters.png In such case, your input file should look as follows:

test1.grf|-P:my_param=my_value -P:another_param=value1
test2.grf|-P:my_param=my_value2 -P:another_param=value2

Topic		Views
Query on RunGraph component CloverDX Platform	2	July 12, 2021
How to detect a failed graph inside a RUN_GRAPH component CloverDX Platform	1	April 9, 2008
ENGINE: Error during graph initialization(null) CloverDX Platform	4	September 30, 2008
GraphExecutor class? CloverDX Platform	2	September 27, 2008
Graph run fails CloverDX Platform	1	June 23, 2014

RunGraph component problem

Related topics