We are using a dictionary entry of type “map”, to pass data from one element in one phase to another element in a subsequent phase. This data is used iteratively in the 2nd element, so we can’t use a join-type connection between them.
We noticed that this approach works, but only for a limited number of key-value pairs in the dictionary. Once we get to about 5,000 or so pairs, we get the following error:
14:47:19,262 INFO [JobFinalizer_1] Dictionary output: { *testMap={KEYFORID2=VALUEFORID2, ....
14:47:19,267 WARN [JobFinalizer_1] Updating modified RunRecord ...
14:47:19,315 WARN [JobFinalizer_1] SQL Error: 20000, SQLState: 22001
14:47:19,316 ERROR [JobFinalizer_1] A truncation error was encountered trying to shrink LONG VARCHAR 'UEsDBBQACAgIAOl1kUcAAAAAAAAAAAAAAAAAAAAAZN2/7i5qfpb3hREYYych&' to length 32700.
14:47:19,316 WARN [JobFinalizer_1] org.hibernate.exception.DataException: could not execute statement
javax.persistence.PersistenceException: org.hibernate.exception.DataException: could not execute statement
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1387)
at org.hibernate.ejb.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1310)
at org.hibernate.ejb.AbstractEntityManagerImpl.throwPersistenceException(AbstractEntityManagerImpl.java:1397)
...
Caused by: org.hibernate.exception.DataException: could not execute statement
at org.hibernate.exception.internal.SQLExceptionTypeDelegate.convert(SQLExceptionTypeDelegate.java:71)
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:49)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:125)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:110)
...
Caused by: java.sql.SQLDataException: A truncation error was encountered trying to shrink LONG VARCHAR 'UEsDBBQACAgIAOl1kUcAAAAAAAAAAAAAAAAAAAAAZN2/7i5qfpb3hREYYych&' to length 32700.
at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
...
14:47:19,318 WARN [JobFinalizer_1] Couldn't update RunRecord; org.hibernate.exception.DataException: could not execute statement
javax.persistence.PersistenceException: org.hibernate.exception.DataException: could not execute statement
...
at org.hibernate.ejb.AbstractQueryImpl.executeUpdate(AbstractQueryImpl.java:111)
at com.cloveretl.server.dao.RunRecordDao.update(Unknown Source)
at com.cloveretl.server.dao.PersistenceServiceImpl.updateRunRecord(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
...
I can provide more details and a sample graph to reproduce this error if needed.
Interestingly, running this same graph through the command-line engine works without any problems, it’s only in the Designer (4.1.1) that we see this failure.