Delimited Reader's encoding issue

CloverDX Platform
1

Hi,
I ahve one file, encoding of that file is UTF8.
and it has data like this := 2910,“Université du Québec à Chicoutimi”

when i load this data in DB it is giving exception:-

java.lang.RuntimeException: java.io.IOException:MALFORMED[1] when converting from UTF-8 when parsing record #1 field institution_id
at org.jetel.data.parser.DelimitedDataParser.parseNext(DelimitedDataParser.java:449)
at org.jetel.data.parser.DelimitedDataParser.getNext0(DelimitedDataParser.java:173)
at org.jetel.data.parser.DelimitedDataParser.skip(DelimitedDataParser.java:595)
at org.jetel.util.MultiFileReader.nextSource(MultiFileReader.java:185)
at org.jetel.util.MultiFileReader.init(MultiFileReader.java:120)
at org.jetel.component.DelimitedDataReader.init(DelimitedDataReader.java:180)
at org.jetel.component.DelimitedDataReader.checkConfig(DelimitedDataReader.java:304)
at org.jetel.graph.Phase.checkConfig(Phase.java:204)
at org.jetel.graph.TransformationGraph.checkConfig(TransformationGraph.java:874)
at org.jetel.graph.runtime.GraphExecutor.initGraph(GraphExecutor.java:122)
at com.mps.pubstats.tool.FeedLoadGraph.runGraph(FeedLoadGraph.java:269)
at com.mps.pubstats.tool.FeedController.LoadAccountFeed(FeedController.java:514)

How to handle this problem…
and kindly tell me that how can i cahnge encding of the DelimitedReader dynamically…

I used this code…
EncodingController encoderController=new EncodingController();
DelimitedDataReader deLimitedReader= new DelimitedDataReader(“DataParser”,
feedController.inputFileName,encoderController.getEncoding(feedController.inputFileName));
deLimitedReader.setSkipFirstLine(true);
deLimitedReader.setPolicyType(“Controlled”);
encoderController.closeFileObject();

//////////////
public String getEncoding(String fileName){

String encodeName=“”;
try
{
obj = new InputStreamReader(new FileInputStream(new File(fileName)));
encodeName = obj.getEncoding();
System.out.println(“File Name---->”+fileName);
System.out.println(“Encoding---->”+encodeName);
}
catch(Exception ex)
{
ex.printStackTrace();
}
return encodeName;
}

////

but it doesn’t work…

Thanks,
Hanuman

2

Even if i set UTF-8 then it is giving same error…now got that getEncoding is not currect…it is giving the system encoding…

But UTF-8 is the super in all the encding (charset) so i want to use to read and write but still it is giving the same error…

java.lang.RuntimeException: java.io.IOException:MALFORMED[1] when converting from UTF-8 when parsing record #1 field institution_id
at org.jetel.data.parser.DelimitedDataParser.parseNext(DelimitedDataParser.java:449)
at org.jetel.data.parser.DelimitedDataParser.getNext0(DelimitedDataParser.java:173)

Please help me to short out thsi problem

3

Hi ,

please help me out how to resolve this problemm…

4

Hello,
which CloverETL version do you use? With 2.6 ver. DataReader as well as DelimitedDataReder seem to work properly.