I have a subgraph that, when I run it from within a graph take about 10 seconds to run. When I run the same subgraph from within a jobflow it takes about 7 minutes to run. Is there a setting somewhere that causes subgraphs to run differently when executed from within a jobflow?
This jobflow and subgraph are in a Clover server project.
Hi Bob,
Subgraphs can behave differently depending on whether they are run inside a graph or a jobflow due to the distinct nature of these two job types.
Graphs are designed for high-throughput data processing. They stream records efficiently between components within a single execution context, making them well-suited for operations involving large volumes of data.
Jobflows are designed for orchestration and use tokens as control units, each representing a single record or step in the execution flow. These tokens carry execution context and trigger downstream jobflow nodes or graphs. Tokens are intended to manage execution flow and coordinate steps in a process rather than to handle large-scale data processing like data records in graphs.
When a jobflow executes a subgraph, it treats each record as an individual token. Because jobflows handle each token separately, this leads to additional overhead related to token tracking, job control, and resource management, which can significantly slow down execution.
In addition to slower execution, using a jobflow for data processing can lead to unexpected results.
That said, Jobflows are not intended to perform data transformations directly. Incorporating subgraphs that include transformation logic within a jobflow may lead to unexpected behavior or execution patterns.
You can find more details regarding job types in our documentation here: Job types | CloverDX 7.0.1 Documentation
Kind regards,
Ladislav