JobFlow: Issues with propagation of dictionary values from

CloverDX Platform
1

Hi,

I’m working with the Clover demo server’ sandbox playground2. I’ve created the jira.jbf and the jira_retrieve_page.grf graphs. The first one invokes the other one. I can’t pass the dictionary values jql_query, temp_max, and pager_start from the JobFlow to the graph. Whenever I execute the JobFlow and view the data on the first edge that goes from the inner graph’s GetJobInput component, I see, that ‘null’ are passed instead the dictionary values that I’m setting in the outer graph.

Can you please help me to debug the graphs?

Thanks

ZD

2

Ok, just figured this out. The problem is that the “View Data” doesn’t work in the inner graph (this is quite annoying). Also the debug log that is shown in the Designer doesn’t contain the inner graph log events. I have to go to the server and take a look at the log there.

Now I would like to execute the inner graph in loop until the values returned from the inner graph last_entry <= total. How can I setup such while iteration in the JobFlow?

Thanks

ZD

3

Hi ZD,

Unfortunately currently is not possible to see data on edges of graph executed by JobFlow. This will be addressed in 3.5 release which is currently in development. Until this is done, simple workaround must be used:

* put SimpleCopy component on edge which should be debugged
* create Thrash component and connect it to SimpleCopy
* so records will be copied also to Thrash
* on Thrash component enable “Debug print”

Alternatively replace Thrash by writer (eg. UniversalDataWriter).

By inspecting you values I see that meaningful values was passed:


2013-05-30 08:17:40,263 INFO  202 [TRASH_202] +-------+------------+-----------+--------------+
2013-05-30 08:17:40,263 INFO  202 [TRASH_202] |Record |jql_query   |temp_max   |pager_start   |
2013-05-30 08:17:40,263 INFO  202 [TRASH_202] +-------+------------+-----------+--------------+
2013-05-30 08:17:40,263 INFO  202 [TRASH_202] |# 1    |            |10         |0             |
2013-05-30 08:17:40,264 INFO  202 [TRASH_202] +-------+------------+-----------+--------------+

But then graph fails on:


2013-05-30 08:17:42,094 ERROR 202 [WatchDog_202] Component [XMLReader:XMLREADER] finished with status ERROR.
 Unexpected null value.
2013-05-30 08:17:42,095 ERROR 202 [WatchDog_202] Error details:
org.jetel.exception.JetelRuntimeException: Component [XMLReader:XMLREADER] finished with status ERROR.
        at org.jetel.graph.Node.createNodeException(Node.java:535)
        at org.jetel.graph.Node.run(Node.java:514)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
        at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NullPointerException
        at org.jetel.component.tree.reader.XPathPushParser.handleFieldMapping(XPathPushParser.java:179)
        at org.jetel.component.tree.reader.XPathPushParser.applyContextMappings(XPathPushParser.java:167)
        at org.jetel.component.tree.reader.XPathPushParser.handleContext(XPathPushParser.java:94)
        at org.jetel.component.tree.reader.XPathPushParser.parse(XPathPushParser.java:83)
        at org.jetel.component.TreeReader$XPathProcessor.processInput(TreeReader.java:665)
        at org.jetel.component.TreeReader.execute(TreeReader.java:411)
        at org.jetel.graph.Node.run(Node.java:485)
        ... 3 more

Which is caused by invalid XmlReader configuration. You missed output port setting:


<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Context xpath="/rss" outPort="0">
  			<Mapping cloverField="last_entry" xpath="./channel/issue/@end"/>
  			<Mapping cloverField="total" xpath="./channel/issue/@total"/>
</Context>

I admit that error message should be improved here - so I created https://bug.javlin.eu/browse/CLD-4721 .

I hope this helps.

4

Thanks, I’ve figured it out and fixed the XmlReader. Can you please also hint how I can execute the inner graph in a while loop? I need to keep calling the inner graph with pager_start = last_entry{from previous run} until the last_entry < total .

How can I loop in JobFlow?

ZD

5

Hi ZD,

there is new component in development for that - https://bug.javlin.eu/browse/CLO-872 It should be part of 3.5 release.

For now, you can use ExecuteGraph and its property “Execute graph as daemon” for recursive calling of itself. Then when job is done just do not send token/record to ExecuteGraph and while loop ends.