I have been developing graphs with Java and tried to update from engine 2.0.4 to 2.1. I setup a simple graph that reads from a CloverData file and outputs a delimited version. (Node CloverDataReader → Node DelimitedDataWriter). In 2.0.4 this works but when I tried it with 2.1 I get a ‘java.nio.BufferUnderflowException’
Here is the Java that I use in 2.1:
(I have another class create the SimpleCopy object and call the run())
package edu.taylor.dw;
import java.io.FileInputStream;
import java.io.IOException;
import org.jetel.component.CloverDataReader;
import org.jetel.component.DelimitedDataWriter;
import org.jetel.exception.GraphConfigurationException;
import org.jetel.graph.Edge;
import org.jetel.graph.Node;
import org.jetel.graph.Phase;
import org.jetel.graph.Result;
import org.jetel.graph.TransformationGraph;
import org.jetel.main.runGraph;
import org.jetel.metadata.DataRecordMetadata;
import org.jetel.metadata.DataRecordMetadataXMLReaderWriter;
public class SimpleCopy {
private TransformationGraph copyGraph;
public SimpleCopy() {
copyGraph = new TransformationGraph();
}
public void run(){
runGraph.initEngine(“Z:\\workspace\\new_workspace\\data-warehouse-etl-2-1\\clover\\cloverETL\\plugins”,null);
Phase phaseOne = new Phase(0);
try {
copyGraph.addPhase(phaseOne);
} catch (GraphConfigurationException e1) {
e1.printStackTrace();
}
// METADATA ________________________________________________
DataRecordMetadata inputMeta;
DataRecordMetadataXMLReaderWriter reader = new DataRecordMetadataXMLReaderWriter();
try{
inputMeta = reader.read(new FileInputStream(“Z:\\workspace\\DataWarehouse\\stage\\spriden\\current.fmt”));
} catch(IOException ex){
System.err.println(“Error when reading metadata!!”);
throw new RuntimeException(ex);
}
if (inputMeta == null){
throw new RuntimeException(“No INPUT metadata”);
}
//_________________________METADATA
Node inputData = new CloverDataReader(“Input”,“Z:\\workspace\\DataWarehouse\\stage\\spriden\\current”,
“Z:\\workspace\\DataWarehouse\\stage\\spriden\\current.idx”);
Node outputData = new DelimitedDataWriter(“Output”,“Z:\\workspace\\DataWarehouse\\currentDel.txt”,false);
Edge inToOut = new Edge(“InToOut”,inputMeta);
inputData.addOutputPort(0,inToOut);
outputData.addInputPort(0,inToOut);
try {
phaseOne.addNode(inputData);
phaseOne.addNode(outputData);
phaseOne.addEdge(inToOut);
} catch (GraphConfigurationException e) {
e.printStackTrace();
}
copyGraph.checkConfig(null);
copyGraph.setTrackingInterval(1000 * 10);// in milliseconds
if (copyGraph.init()) {
System.out.println(“Init error”);
}
if (copyGraph.run() != Result.FINISHED_OK) {
System.out.println(“Run error”);
}
}
}
And here is the console output:
DEBUG [main] - Plugin cz.opensys.oracle loaded.
id - cz.opensys.oracle
version - 2.1.0
provider-name -
point-id - component - {className=cz.opensys.oracle.component.OracleDataReader, type=ORACLE_DATA_READER}
point-id - component - {className=cz.opensys.oracle.component.OracleDataWriter, type=ORACLE_DATA_WRITER}
DEBUG [main] - Plugin org.jetel.component loaded.
id - org.jetel.component
version - 2.1.0
provider-name - null
point-id - component - {type=DATA_READER, className=org.jetel.component.DataReader}
point-id - component - {type=DATA_WRITER, className=org.jetel.component.DataWriter}
point-id - component - {type=DELIMITED_DATA_READER, className=org.jetel.component.DelimitedDataReader}
point-id - component - {type=DELIMITED_DATA_WRITER, className=org.jetel.component.DelimitedDataWriter}
point-id - component - {type=SIMPLE_COPY, className=org.jetel.component.SimpleCopy}
point-id - component - {type=CONCATENATE, className=org.jetel.component.Concatenate}
point-id - component - {type=SIMPLE_GATHER, className=org.jetel.component.SimpleGather}
point-id - component - {type=REFORMAT, className=org.jetel.component.Reformat}
point-id - component - {type=DB_INPUT_TABLE, className=org.jetel.component.DBInputTable}
point-id - component - {type=SORT, className=org.jetel.component.Sort}
point-id - component - {type=DB_OUTPUT_TABLE, className=org.jetel.component.DBOutputTable}
point-id - component - {type=FIXLEN_DATA_WRITER, className=org.jetel.component.FixLenDataWriter}
point-id - component - {type=DEDUP, className=org.jetel.component.Dedup}
point-id - component - {type=FIXLEN_DATA_READER, className=org.jetel.component.FixLenDataReader}
point-id - component - {type=MERGE, className=org.jetel.component.Merge}
point-id - component - {type=MERGE_JOIN, className=org.jetel.component.MergeJoin}
point-id - component - {type=EXT_MERGE_JOIN, className=org.jetel.component.MergeJoin}
point-id - component - {type=SORTED_JOIN, className=org.jetel.component.MergeJoin}
point-id - component - {type=TRASH, className=org.jetel.component.Trash}
point-id - component - {type=DB_EXECUTE, className=org.jetel.component.DBExecute}
point-id - component - {type=HASH_JOIN, className=org.jetel.component.HashJoin}
point-id - component - {type=EXT_HASH_JOIN, className=org.jetel.component.HashJoin}
point-id - component - {type=CHECK_FOREIGN_KEY, className=org.jetel.component.CheckForeignKey}
point-id - component - {type=DBF_DATA_READER, className=org.jetel.component.DBFDataReader}
point-id - component - {type=EXT_FILTER, className=org.jetel.component.ExtFilter}
point-id - component - {type=EXT_SORT, className=org.jetel.component.ExtSort}
point-id - component - {type=PARTITION, className=org.jetel.component.Partition}
point-id - component - {type=DATA_INTERSECTION, className=org.jetel.component.DataIntersection}
point-id - component - {type=AGGREGATE, className=org.jetel.component.Aggregate}
point-id - component - {type=SYS_EXECUTE, className=org.jetel.component.SystemExecute}
point-id - component - {type=KEY_GEN, className=org.jetel.component.KeyGenerator}
point-id - component - {type=APROX_MERGE_JOIN, className=org.jetel.component.AproxMergeJoin}
point-id - component - {type=DBJOIN, className=org.jetel.component.DBJoin}
point-id - component - {type=XLS_READER, className=org.jetel.component.XLSReader}
point-id - component - {type=XLS_WRITER, className=org.jetel.component.XLSWriter}
point-id - component - {type=CLOVER_WRITER, className=org.jetel.component.CloverDataWriter}
point-id - component - {type=CLOVER_READER, className=org.jetel.component.CloverDataReader}
point-id - component - {className=org.jetel.component.MysqlDataReader, type=MYSQL_DATA_READER}
point-id - component - {className=org.jetel.component.MysqlDataWriter, type=MYSQL_DATA_WRITER}
point-id - component - {className=org.jetel.component.StructureWriter, type=STRUCTURE_WRITER}
point-id - component - {className=org.jetel.component.Normalizer, type=NORMALIZER}
point-id - component - {className=org.jetel.component.Denormalizer, type=DENORMALIZER}
point-id - component - {className=org.jetel.component.JmsReader, type=JMS_READER}
point-id - component - {className=org.jetel.component.JmsWriter, type=JMS_WRITER}
point-id - component - {className=org.jetel.component.LookupJoin, type=LOOKUP_JOIN}
point-id - component - {className=org.jetel.component.LookupTableReaderWriter, type=LOOKUP_TABLE_READER_WRITER}
point-id - component - {className=org.jetel.component.DataGenerator, type=DATA_GENERATOR}
point-id - component - {className=org.jetel.component.MultiLevelReader, type=MULTI_LEVEL_READER}
point-id - component - {className=org.jetel.component.SequenceChecker, type=SEQUENCE_CHECKER}
DEBUG [main] - Plugin org.jetel.connection loaded.
id - org.jetel.connection
version - 2.1.0
provider-name - null
point-id - connection - {className=org.jetel.connection.DBConnection, type=JDBC}
point-id - connection - {className=org.jetel.connection.JmsConnection, type=JMS}
DEBUG [main] - Plugin org.jetel.lookup loaded.
id - org.jetel.lookup
version - 2.1.0
provider-name - null
point-id - lookup - {className=org.jetel.lookup.SimpleLookupTable, type=simpleLookup}
point-id - lookup - {className=org.jetel.lookup.DBLookupTable, type=dbLookup}
DEBUG [main] - Plugin org.jetel.sequence loaded.
id - org.jetel.sequence
version - 2.1.0
provider-name - null
point-id - sequence - {className=org.jetel.sequence.SimpleSequence, type=SIMPLE_SEQUENCE}
point-id - sequence - {className=org.jetel.sequence.PrimitiveSequence, type=PRIMITIVE_SEQUENCE}
DEBUG [main] - Plugin org.jetel.thirdparty loaded.
id - org.jetel.thirdparty
version - 2.1.0
provider-name - null
point-id - component - {type=XML_EXTRACT, className=org.jetel.component.XMLExtract}
point-id - component - {type=FILTER, className=org.jetel.component.Filter}
point-id - component - {type=LDAP_READER, className=com.linagora.component.LdapReader}
point-id - component - {type=LDAP_WRITER, className=com.linagora.component.LdapWriter}
Init error
INFO [main] - Starting WatchDog thread …
INFO [WatchDog] - Thread started.
INFO [WatchDog] - Running on 2 CPU(s) max available memory for JVM 504896 KB
INFO [WatchDog] - [Clover] Initializing phase: 0
DEBUG [WatchDog] - initializing edges:
DEBUG [WatchDog] - all edges initialized successfully…
DEBUG [WatchDog] - initializing nodes:
DEBUG [WatchDog] - Input …OK
DEBUG [WatchDog] - Output …OK
INFO [WatchDog] - [Clover] phase: 0 initialized successfully.
INFO [WatchDog] - Starting up all nodes in phase [0]
DEBUG [WatchDog] - Input … started
DEBUG [WatchDog] - Output … started
INFO [WatchDog] - Sucessfully started all nodes in phase!
FATAL [WatchDog] - !!! Fatal Error !!! - graph execution is aborting
ERROR [WatchDog] - Node Input finished with status: ERROR caused by: null
DEBUG [WatchDog] - Node Input error details:
java.nio.BufferUnderflowException
at java.nio.Buffer.nextGetIndex(Buffer.java:404)
at java.nio.DirectByteBuffer.getInt(DirectByteBuffer.java:620)
at org.jetel.data.IntegerDataField.deserialize(IntegerDataField.java:431)
at org.jetel.data.DataRecord.deserialize(DataRecord.java:280)
at org.jetel.data.parser.CloverDataParser.getNext(CloverDataParser.java:229)
at org.jetel.component.CloverDataReader.execute(CloverDataReader.java:136)
at org.jetel.graph.Node.run(Node.java:364)
at java.lang.Thread.run(Thread.java:595)
WARN [WatchDog] - Interrupted node: Input
WARN [WatchDog] - Interrupted node: Output
INFO [WatchDog] - ---------------------** Start of tracking Log for phase [0] **-------------------
INFO [WatchDog] - Time: 01/03/07 10:53:59
INFO [WatchDog] - Node Status Port #Records #KB Rec/s KB/s
INFO [WatchDog] - ----------------------------------------------------------------------------------
INFO [WatchDog] - ---------------------------------** End of Log **--------------------------------
ERROR [WatchDog] - !!! Phase finished with error - stopping graph run !!!
INFO [main] - WatchDog thread finished - total execution time: 0 (sec)
ERROR [main] - !!! Graph execution finished with errors !!!
Run error