Jobflow failures after upgrading to 4.0.1.20 from 3.4.2.11

CloverDX Platform
1

Subsequent to our upgrade from 3.4.2.11 to 4.0.1.20 version of the CloverETL server, an execute graph component on error is causing the entire job flow to fail. It used to only report a failure at the graph level but not cause the entire job flow to fail. I couldn’t find any settings that would help restore the previous behavior. Let me know if you need any additional details.

Thanks,
- Rama

2

Hello Rama,

ExecuteGraph has a property called Stop processing on fail which can ignore errors in the graph when set to False.

Hope this helps.

3

Hi,
We tested the “Stop processing on fail” flag even prior to starting this post. Unfortunately it doesn’t seem to impact the behavior, the entire job flow fails on the graph that is used in the execute graph component.

Thanks,
- Rama

4

This seems odd, it should not behave this way. Could you please send me the jobflow, the graph and the error log? I will look into them and hopefully find the issue.

Thanks.

5

Hi Lubos,
Here are the files that you requested the jobflow, graph and the corresponding error logs for them. Couldn’t add one of the log files because of the 3 file max limitation. I will add that in the next response.

Thanks,
- Rama

6

I could not upload the error log for the jobflow because of the size limitation. Let me know if you need the tail of that file with the failure details.

Thanks,
- Rama

7

Hi Rama,

It seems that your graph fails during initialization phase - so it is not runnable. That is not covered by “Stop processing on fail”.

The problem is visible in log:


java.sql.SQLException: Invalid column name 'visible_ind'.

There are dynamic metadata DynamicMetadata2 which does sql query “select ${FIELD_LIST} from ${TARGET_PROCESSED_TABLE_WITH_SCHEMA}” during initialization phase. Parameters for it are passed from JobFlow. So please inspect what parameters values are passed. Obviously it is “visible_ind” which is not available in DB.

I hope that helps.

8

Hi Lubos,
Thanks for your assistance thus far with this issue.

This is the same code that was used in 3.4.2.11 and even though the graph would fail in the initialization phase the job flow wasn’t impacted. We haven’t changed anything in this code since the upgrade roughly two weeks ago. We compare the source and target tables scoped for replication during this phase. This ETL process was developed by Clover’s consulting team and we haven’t modified anything in this graph ever since.

Thanks,
- Rama

9

Hi,

The change does not have to be in the graph, it can be in the DB as well. Did the database table change lately?

We did multiple tests here and the result was always as expected - if the graph fails during the initialization, the jobflow ends with an error no matter the version of CloverETL. Therefore, I would like to ask you to send tail of the jobflow run log.

Thanks.