EBCDIC file format

Hi.
Does Clover suports EBCDIC file format?
I can’t mach it to any default charasets in Universal Data Reader.

Best Regards

Hi,
you can try the IBM 1047 encoding. Maybe it is what you are searching for.
Best regards

The IBM 1047 codepage is (according to iana.org) - EBCDIC Latin 1/Open Systems

Thanks for an answer.
I’m afraid that IBM 1047 is not supported by Clover.
It’s not on the Reader task Charset list, and gives error when trying to set it manualy.

INFO  [main] - Graph definition file: graph/Serat.grf
Exception in thread "main" ERROR [main] - Can't create object of type DATA_READER with reason: DATA_READER:DATA_READER0:IBM1047
java.lang.RuntimeException: Can't create object of type DATA_READER with reason: DATA_READER:DATA_READER0:IBM1047
	at org.jetel.component.ComponentFactory.createComponent(ComponentFactory.java:132)
	at org.jetel.graph.TransformationGraphXMLReaderWriter.instantiateNodes(TransformationGraphXMLReaderWriter.java:465)
	at org.jetel.graph.TransformationGraphXMLReaderWriter.instantiatePhases(TransformationGraphXMLReaderWriter.java:428)
	at org.jetel.graph.TransformationGraphXMLReaderWriter.read(TransformationGraphXMLReaderWriter.java:340)
	at org.jetel.graph.TransformationGraphXMLReaderWriter.read(TransformationGraphXMLReaderWriter.java:270)
	at org.jetel.graph.TransformationGraphXMLReaderWriter.loadGraph(TransformationGraphXMLReaderWriter.java:203)
	at org.jetel.main.runGraph.main(runGraph.java:308)

Well, it depends on platform, not Clover. Check your Java installation. On Mac (OSX) Clover offers more than 120 encodings, IBM1047 amongst them.

On Windows Jre 1.5 does not have it, Jdk 1.5 does.You may try just to borrow charset.jar from Jdk 1.5

Ok. J have changed JRE from jre1.6 to jdk1.5. Now, i’m getting this error:


ERROR [WatchDog] - Node DATA_READER0 finished with status: ERROR caused by: Parse error: The size of data buffer for data record is only 49152. Set appropriate parameter in defautProperties file. when parsing record #1 field zm19
DEBUG [WatchDog] - Node DATA_READER0 error details:
java.lang.RuntimeException: Parse error: The size of data buffer for data record is only 49152. Set appropriate parameter in defautProperties file. when parsing record #1 field zm19
	at org.jetel.data.parser.DataParser.parseNext(DataParser.java:454)
	at org.jetel.data.parser.DataParser.getNext(DataParser.java:147)
	at org.jetel.util.MultiFileReader.getNext(MultiFileReader.java:354)
	at org.jetel.component.DataReader.execute(DataReader.java:191)
	at org.jetel.graph.Node.run(Node.java:379)
	at java.lang.Thread.run(Thread.java:619)
ERROR [WatchDog] - !!! Phase finished with error - stopping graph run !!!

The metadata is fixed length, and summary it contains no more than 1274 characters per line.

For some reason, the parser can’t find record delimiter (probably). This message means that Clover tries to create a data record which is too large - usually means, that record/field boundaries are not found or properly defined.

Still nothing…

DBA said, that this is a binary fie, generated from DB2 database, with codepage 870, and 0D0A(hex) as EOL character.

Is there another way to read this file from Clover?

Is the file stored in IXF format? If you mean this as binary DB2 file, I’m afraid, Clover is not able to parse it. On DB2 documentation I’ve found that “IXF (Integration Exchange Format, PC version) is a binary format that is used exclusively by DB2.” (http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.cmd.doc/doc/r0008304.html). So I think that only way to get data from it, is to import them to DB2 database.

There is existing mechanism writen in SAS script language that reads this file to SAS table, so I belive it is readable outside DB2.
My job is to migrate from SAS to Java…

So maybe try to read the data directly from SAS by DBInputTable (http://support.sas.com/documentation/cdl/en/jdbcref/59666/HTML/default/n12a0cg9du45g2n1ib275rxpcusz.htm)

I can’t. The SAS licence is expiring, that is why I need to migrate reader for this binary file.

Only way is to write your own Reader. Clover parsers can be helpful then. Information about ixf format you can find in IBM pages ( http://publib.boulder.ibm.com/infocenter/db2luw/v9r5/index.jsp?topic=/com.ibm.db2.luw.admin.dm.doc/doc/r0004667.html)

One way to do that:


import java.io.*;

public class readEbcdic {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		// TODO Auto-generated method stub
		try {
			readFile("C:/input.txt");
		} catch (IOException e) {
			System.out.println(e.getMessage());
		}
	}

	static void readFile(String fileName) throws IOException
	{
		BufferedInputStream bis = null;

		try	{
			bis = new BufferedInputStream(new FileInputStream(fileName));
			
			FileWriter fstream = new FileWriter("C:/output.txt");
			BufferedWriter out = new BufferedWriter(fstream);
			
			byte[] recText;
			ByteArrayOutputStream baos = new ByteArrayOutputStream();
			
			while(bis.read() > -1) {
				
				recText = new byte[1273];	// 1273 is a fixed line length
				bis.read(recText);
				
				baos.write(recText);
				out.write(baos.toString("Cp037") + "\n");
				baos.reset();
			}
			out.close();
		}
		finally	{
			try {
				bis.close();
			}
			catch (Exception e) {}
		}
	}
}