What is the rationale for the enableIncomplete attribute having no effect in binary mode?
If dataPolicy is “lenient”, an endless loop results.
This still leaves the endless loop problem when the dataPolicy is LENIENT.
Either don’t permit LENIENT for the FixedDataReader, or treat an incomplete record as an EOF condition to exit the loop. It would seem reasonable to incorporate the enableIncomplete attribute to either report the incomplete record as an error or treat it as EOF.
This issue arose from trying to parse a fixed length, binary flat file. Unfortunately, there were extra bits of data at the end of the file.
With the dataPolicy set to lenient, when reading near the end of the file, an exception is raised because there is not the expected length of data
available to read. Due to the dataPolicy, the exception is ignored, and the parsing continues, and another attempt to read occurs, fails … repeat forever.
Originally, to solve the problem I activated the isEnableIncomplete attribute for the FixLenByteDataParser. Additionally, prior to throwing a BadDataFormatException because not enough data is available to read, I check the isEnableIncomplete attribute. If it is false, the exception is thrown, otherwise, null is returned
which ends the reading from the data source.
However, the endless loop condition remains if dataPolicy is lenient and isEnableIncomplete is false.
Later I opted to mark the parser “finished” prior to throwing the BadDataFormatException, and subsequently replacing the “while (true)” conditional in the parser’s getNext() method with a “while (notFinished)” conditional. This
eliminated the need to activate the isEnableIncomplete attribute.
However, I question whether keeping the isEnableIncomplete disabled is good design.
It seems to me that if the user indicates the isEnableIncomplete is false, but the dataPolicy is lenient, the the user wishes the parsing to fail if the data file is not a perfect fixed-length file, and any other error condition is permissable.
The enableIncomplete attribute on FixleDataReader has no effect in binary mode. Reason is in binary mode of reading input file and imposibility to search in this binary stream a record delimiter (some kind of string!) necessary to recover reading after incomplete record.
Martin
Sorry I responded to the wrong message. My thoughts were posted elsewhere in this thread.
Additionally, as I have further examined the code, it looks like the while(true) in the getNext() method of the parser should be while(!eof).
That should take care of the endless loop condition.
Hello !
Your suggestion sounds interesting. Could you elaborate on it more ?
Thnx.
David.