Using try_convert in embedded clover

fschnack · May 6, 2011, 12:00am

Hello There!

I am using CloverETL 2.9.6 embedded in my application and trying to use the try_convert function in a reformat node. I am initializing the component with the following call:

Reformat fmt = new Reformat("testreformat, transform, null, null);

The “transform” variable contains the following string:

//#TL
$0.QUALIFIED_URL := try_convert($0.QUALIFIED_URL, string);
$0.URL_PROTOCOL := try_convert($0.URL_PROTOCOL, string);
$0.URL_USER := try_convert($0.URL_USER, string);
$0.URL_PASSWORD := try_convert($0.URL_PASSWORD, string);
$0.URL_SUBDOMAIN := try_convert($0.URL_SUBDOMAIN, string);
$0.URL_DOMAIN := try_convert($0.URL_DOMAIN, string);
$0.URL_PORT := try_convert($0.URL_PORT, string);
$0.URL_PATH := try_convert($0.URL_PATH, string);
$0.URL_QUERY := try_convert($0.URL_QUERY, string);
$0.URL_FRAGMENT := try_convert($0.URL_FRAGMENT, string);
$0.URL_DOMAIN_COUNTRY := try_convert($0.URL_DOMAIN_COUNTRY, string);
$0.URL_DOMAIN_CATEGORY := try_convert($0.URL_DOMAIN_CATEGORY, string);
$0.URL_IP_ADDRESS := try_convert($0.URL_IP_ADDRESS, string);
$0.URL_QUALIFICATION_FLAG_BRIEF := try_convert($0.URL_QUALIFICATION_FLAG_BRIEF, string);
$0.URL_QUALIFICATION_TEXT_BRIEF := try_convert($0.URL_QUALIFICATION_TEXT_BRIEF, string);
$0.URL_QUALIFICATION_FLAG := try_convert($0.URL_QUALIFICATION_FLAG, string);
$0.URL_QUALIFICATION_TEXT := try_convert($0.URL_QUALIFICATION_TEXT, string);
$0.QUALIFIED_EMAIL := try_convert($0.QUALIFIED_EMAIL, string);
$0.EMAIL_NAME := try_convert($0.EMAIL_NAME, string);
$0.EMAIL_USERNAME := try_convert($0.EMAIL_USERNAME, string);
$0.EMAIL_DOMAIN := try_convert($0.EMAIL_DOMAIN, string);
$0.EMAIL_DOMAIN_COUNTRY := try_convert($0.EMAIL_DOMAIN_COUNTRY, string);
$0.EMAIL_DOMAIN_CATEGORY := try_convert($0.EMAIL_DOMAIN_CATEGORY, string);
$0.EMAIL_QUALIFICATION_FLAG_BRIEF := try_convert($0.EMAIL_QUALIFICATION_FLAG_BRIEF, string);
$0.EMAIL_QUALIFICATION_TEXT_BRIEF := try_convert($0.EMAIL_QUALIFICATION_TEXT_BRIEF, string);
$0.EMAIL_QUALIFICATION_FLAG := try_convert($0.EMAIL_QUALIFICATION_FLAG, string);
$0.EMAIL_QUALIFICATION_TEXT := try_convert($0.EMAIL_QUALIFICATION_TEXT, string);
$0.CPF := try_convert($0.CPF, string);
$0.NOME := try_convert($0.NOME, string);
$0.ENDERECO := try_convert($0.ENDERECO, string);
$0.BAIRRO := try_convert($0.BAIRRO, string);
$0.CEP := try_convert($0.CEP, string);
$0.CIDADE := try_convert($0.CIDADE, string);
$0.UF := try_convert($0.UF, string);
$0.DDD := try_convert($0.DDD, decimal);
$0.TELEFONE := try_convert($0.TELEFONE, string);
$0.DDD1 := try_convert($0.DDD1, string);
$0.TELEFONE1 := try_convert($0.TELEFONE1, string);
$0.DDD2 := try_convert($0.DDD2, string);
$0.TELEFONE2 := try_convert($0.TELEFONE2, string);
$0.DDD3 := try_convert($0.DDD3, string);
$0.TELEFONE3 := try_convert($0.TELEFONE3, string);
$0.DATANASCIMENTO := try_convert($0.DATANASCIMENTO, string);
$0.SEXO := try_convert($0.SEXO, string);
$0.FXRENDA := try_convert($0.FXRENDA, string);
$0.MAE := try_convert($0.MAE, string);
$0.Q_TIPO_LOGRADOURO := try_convert($0.Q_TIPO_LOGRADOURO, string);
$0.Q_NOME_LOGRADOURO := try_convert($0.Q_NOME_LOGRADOURO, string);
$0.Q_TIPO_LOC_HORIZONTAL := try_convert($0.Q_TIPO_LOC_HORIZONTAL, string);
$0.Q_LOC_HORIZONTAL := try_convert($0.Q_LOC_HORIZONTAL, string);
$0.Q_TIPO_LOC_VERTICAL := try_convert($0.Q_TIPO_LOC_VERTICAL, string);
$0.Q_LOC_VERTICAL := try_convert($0.Q_LOC_VERTICAL, string);
$0.Q_COMPLEMENTO := try_convert($0.Q_COMPLEMENTO, string);
$0.Q_BAIRRO_INFERIDO := try_convert($0.Q_BAIRRO_INFERIDO, string);
$0.Q_UF := try_convert($0.Q_UF, string);
$0.Q_LOCALIDADE := try_convert($0.Q_LOCALIDADE, string);
$0.Q_CEP := try_convert($0.Q_CEP, string);
$0.Q_PREPOSICAO := try_convert($0.Q_PREPOSICAO, string);
$0.Q_TITULO_LOGRADOURO := try_convert($0.Q_TITULO_LOGRADOURO, string);
$0.Q_ENDERECO_FLAGS := try_convert($0.Q_ENDERECO_FLAGS, string);
$0.Q_TIPO_LOCALIDADE := try_convert($0.Q_TIPO_LOCALIDADE, string);
$0.Q_MUNICIPIO := try_convert($0.Q_MUNICIPIO, string);
$0.Q_END_COMPLETO := try_convert($0.Q_END_COMPLETO, string);
$0.Q_LONGITUDE := try_convert($0.Q_LONGITUDE, string);
$0.Q_LATITUDE := try_convert($0.Q_LATITUDE, string);
$0.Q_ZONA_CENS_COD := try_convert($0.Q_ZONA_CENS_COD, string);
$0.Q_DENSIDADE_DEMOGRAFICA := try_convert($0.Q_DENSIDADE_DEMOGRAFICA, string);
$0.Q_GEO_PRECISAO := try_convert($0.Q_GEO_PRECISAO, string);
$0.MATCH_ENDERECO := try_convert($0.MATCH_ENDERECO, string);
$0.MATCH_PF_CPF_ENDERECO := try_convert($0.MATCH_PF_CPF_ENDERECO, string);
$0.EXISTE_LB07 := try_convert($0.EXISTE_LB07, string);
$0.DT_NASC := try_convert($0.DT_NASC, string);
$0.QTD1 := try_convert($0.QTD1, string);
$0.ID := try_convert($0.ID, string);

The problem is that when I run the graph, the following error occurs:

[2011-05-06 17:35:13,207] [ERROR] [STDERR] [org.jetel.interpreter.ParseException: function "try_convert" not declared]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.FunctionCallStatement(TransformLangParser.java:3314)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.FunctionCall(TransformLangParser.java:1980)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.PrimaryExpression(TransformLangParser.java:1889)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.PostfixExpression(TransformLangParser.java:1703)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.UnaryExpression(TransformLangParser.java:1849)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.MultiplicativeExpression(TransformLangParser.java:1556)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.AdditiveExpression(TransformLangParser.java:1476)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.RelationalExpression(TransformLangParser.java:1356)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.EqualityExpression(TransformLangParser.java:1276)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.ConditionalAndExpression(TransformLangParser.java:1234)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.ConditionalOrExpression(TransformLangParser.java:1192)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.DirectMapping(TransformLangParser.java:1091)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.Mapping(TransformLangParser.java:1039)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.CompilationUnit(TransformLangParser.java:516)]
[2011-05-06 17:35:13,208] [ERROR] [STDERR] [	at org.jetel.interpreter.TransformLangParser.Start(TransformLangParser.java:297)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at org.jetel.component.WrapperTL.init(WrapperTL.java:158)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at org.jetel.component.RecordTransformTL.init(RecordTransformTL.java:64)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at org.jetel.component.RecordTransformFactory.createTransform(RecordTransformFactory.java:256)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at org.jetel.component.Reformat.checkConfig(Reformat.java:500)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at org.jetel.graph.Phase.checkConfig(Phase.java:208)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at org.jetel.graph.TransformationGraph.checkConfig(TransformationGraph.java:905)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at br.com.godigital.goquality3g.service.impl.ExecutorBean.execute(ExecutorBean.java:379)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at br.com.godigital.goquality3g.service.impl.ExecutorBean.execute(ExecutorBean.java:173)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at br.com.godigital.goquality3g.service.impl.ExecutorBean.execute(ExecutorBean.java:133)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)]
[2011-05-06 17:35:13,209] [ERROR] [STDERR] [	at java.lang.reflect.Method.invoke(Method.java:597)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:112)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.ejb3.interceptor.InvocationContextImpl.proceed(InvocationContextImpl.java:166)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.ejb3.interceptor.EJB3InterceptorsInterceptor.invoke(EJB3InterceptorsInterceptor.java:63)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.ejb3.entity.TransactionScopedEntityManagerInterceptor.invoke(TransactionScopedEntityManagerInterceptor.java:54)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.ejb3.AllowedOperationsInterceptor.invoke(AllowedOperationsInterceptor.java:47)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aspects.tx.TxPolicy.invokeInOurTx(TxPolicy.java:79)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aspects.tx.TxInterceptor$Required.invoke(TxInterceptor.java:191)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aspects.tx.TxPropagationInterceptor.invoke(TxPropagationInterceptor.java:95)]
[2011-05-06 17:35:13,210] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.ejb3.stateless.StatelessInstanceInterceptor.invoke(StatelessInstanceInterceptor.java:62)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.aspects.security.AuthenticationInterceptor.invoke(AuthenticationInterceptor.java:77)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.ejb3.security.Ejb3AuthenticationInterceptor.invoke(Ejb3AuthenticationInterceptor.java:110)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.ejb3.ENCPropagationInterceptor.invoke(ENCPropagationInterceptor.java:46)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.ejb3.asynchronous.AsynchronousInterceptor.invoke(AsynchronousInterceptor.java:106)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.ejb3.stateless.StatelessContainer.dynamicInvoke(StatelessContainer.java:304)]
[2011-05-06 17:35:13,211] [ERROR] [STDERR] [	at org.jboss.ejb3.remoting.IsLocalInterceptor.invokeLocal(IsLocalInterceptor.java:81)]
[2011-05-06 17:35:13,212] [ERROR] [STDERR] [	at org.jboss.ejb3.remoting.IsLocalInterceptor.invoke(IsLocalInterceptor.java:72)]
[2011-05-06 17:35:13,212] [ERROR] [STDERR] [	at org.jboss.aop.joinpoint.MethodInvocation.invokeNext(MethodInvocation.java:101)]
[2011-05-06 17:35:13,212] [ERROR] [STDERR] [	at org.jboss.ejb3.stateless.StatelessRemoteProxy.invoke(StatelessRemoteProxy.java:107)]

What I am doing wrong? I am having trouble finding good examples of my specific use.

Thanks in advance!

avackova · May 9, 2011, 9:14am

Hello,
it seems, that the function is not loaded during the CloverETL Engine initialization. There is list of the loaded plug-ins on the begging on the graph log. It should contain something like:

DEBUG [main] - Plugin org.jetel.tlfunction loaded.
	id - org.jetel.tlfunction
	version - 3.1.0.devel20110506143324avackova
	provider-name - null
		tlfunction { libraryName = math; className = org.jetel.interpreter.extensions.MathLib; function = sqrt,log,log10,exp,round,pow,pi,e,random,random_gaussian,random_boolean,random_int,random_long,abs,bit_and,bit_or,bit_xor,bit_set,bit_invert,bit_is_set,bit_lshift,bit_rshift;  }
		tlfunction { libraryName = date; className = org.jetel.interpreter.extensions.DateLib; function = today,dateadd,datediff,trunc,trunc_date,random_date;  }
		tlfunction { libraryName = string; className = org.jetel.interpreter.extensions.StringLib; function = concat,uppercase,lowercase,substring,left,right,trim,length,replace,split,char_at,is_blank,is_ascii,is_number,is_integer,is_long,is_date,remove_diacritic,remove_blank_space,get_alphanumeric_chars,translate,join,index_of,count_char,chop,remove_nonprintable,remove_nonascii,find,cut,random_string;  }
		tlfunction { libraryName = convert; className = org.jetel.interpreter.extensions.ConvertLib; function = num2str,date2str,str2date,date2num,str2num,try_convert,base64byte,byte2base64,bits2str,str2bits,hex2byte,byte2hex,num2num,num2bool,bool2num,str2bool,long2date,date2long,to_string,md5,sha,long2pacdecimal,pacdecimal2long,get_field_name,get_field_type;  }
		tlfunction { libraryName = container; className = org.jetel.interpreter.extensions.ContainerLib; function = remove_all,push,pop,poll,remove,insert,sort,copy,reverse,dict_put_str,dict_get_str;  }

If you don’t have the similar fragment in your log (note the DEBUG log level) no function will work.

By the way, if you need to convert something to the string, you would rather use to_string or proper type2str (num2str, date2str, …) function instead of try_convert. If you want to assign string input field to the string output field use simple mapping:


$0.QUALIFIED_URL := t$0.QUALIFIED_URL;
$0.URL_PROTOCOL := $0.URL_PROTOCOL;
$0.URL_USER := $0.URL_USER;
$0.URL_PASSWORD := $0.URL_PASSWORD;
.
.

Or, if the names and types of input and output fields are the same, you can use a wild card mapping:

$0.* := $0.*;

The above statement assigns the values from input port to the output port for all fields with same names and types. Then you need to assign only the fields, which can’t be guessed.

fschnack · May 9, 2011, 1:55pm

Hello there! Thanks for the great tip, I really wasn’t loading the appropriate plugin.xml. I’ve added it to my engine initialization code and that error is gone.

Regarding the string conversion you mention, that sample TL code I sent before wasn’t very well written. In most cases, I want to convert a String to another type, mainly number. But is that code’s syntax correct? For some reason, now I get a NPE error, but I can’t understand why. Here is the CTL I am using now:


//#TL
$0.QUALIFIED_URL :=  $0.QUALIFIED_URL;
$0.URL_PROTOCOL :=  $0.URL_PROTOCOL;
$0.URL_USER :=  $0.URL_USER;
$0.URL_PASSWORD :=  $0.URL_PASSWORD;
$0.URL_SUBDOMAIN :=  $0.URL_SUBDOMAIN;
$0.URL_DOMAIN :=  $0.URL_DOMAIN;
$0.URL_PORT :=  $0.URL_PORT;
$0.URL_PATH :=  $0.URL_PATH;
$0.URL_QUERY :=  $0.URL_QUERY;
$0.URL_FRAGMENT :=  $0.URL_FRAGMENT;
$0.URL_DOMAIN_COUNTRY :=  $0.URL_DOMAIN_COUNTRY;
$0.URL_DOMAIN_CATEGORY :=  $0.URL_DOMAIN_CATEGORY;
$0.URL_IP_ADDRESS :=  $0.URL_IP_ADDRESS;
$0.URL_QUALIFICATION_FLAG_BRIEF :=  $0.URL_QUALIFICATION_FLAG_BRIEF;
$0.URL_QUALIFICATION_TEXT_BRIEF :=  $0.URL_QUALIFICATION_TEXT_BRIEF;
$0.URL_QUALIFICATION_FLAG :=  $0.URL_QUALIFICATION_FLAG;
$0.URL_QUALIFICATION_TEXT :=  $0.URL_QUALIFICATION_TEXT;
$0.QUALIFIED_EMAIL :=  $0.QUALIFIED_EMAIL;
$0.EMAIL_NAME :=  $0.EMAIL_NAME;
$0.EMAIL_USERNAME :=  $0.EMAIL_USERNAME;
$0.EMAIL_DOMAIN :=  $0.EMAIL_DOMAIN;
$0.EMAIL_DOMAIN_COUNTRY :=  $0.EMAIL_DOMAIN_COUNTRY;
$0.EMAIL_DOMAIN_CATEGORY :=  $0.EMAIL_DOMAIN_CATEGORY;
$0.EMAIL_QUALIFICATION_FLAG_BRIEF :=  $0.EMAIL_QUALIFICATION_FLAG_BRIEF;
$0.EMAIL_QUALIFICATION_TEXT_BRIEF :=  $0.EMAIL_QUALIFICATION_TEXT_BRIEF;
$0.EMAIL_QUALIFICATION_FLAG :=  $0.EMAIL_QUALIFICATION_FLAG;
$0.EMAIL_QUALIFICATION_TEXT :=  $0.EMAIL_QUALIFICATION_TEXT;
$0.CPF :=  $0.CPF;
$0.NOME :=  $0.NOME;
$0.ENDERECO :=  $0.ENDERECO;
$0.BAIRRO :=  $0.BAIRRO;
$0.CEP :=  $0.CEP;
$0.CIDADE :=  $0.CIDADE;
$0.UF :=  $0.UF;
$0.DDD :=  try_convert( $0.DDD, decimal);
$0.TELEFONE :=  $0.TELEFONE;
$0.DDD1 :=  $0.DDD1;
$0.TELEFONE1 :=  $0.TELEFONE1;
$0.DDD2 :=  $0.DDD2;
$0.TELEFONE2 :=  $0.TELEFONE2;
$0.DDD3 :=  $0.DDD3;
$0.TELEFONE3 :=  $0.TELEFONE3;
$0.DATANASCIMENTO :=  $0.DATANASCIMENTO;
$0.SEXO :=  $0.SEXO;
$0.FXRENDA :=  $0.FXRENDA;
$0.MAE :=  $0.MAE;
$0.Q_TIPO_LOGRADOURO :=  $0.Q_TIPO_LOGRADOURO;
$0.Q_NOME_LOGRADOURO :=  $0.Q_NOME_LOGRADOURO;
$0.Q_TIPO_LOC_HORIZONTAL :=  $0.Q_TIPO_LOC_HORIZONTAL;
$0.Q_LOC_HORIZONTAL :=  $0.Q_LOC_HORIZONTAL;
$0.Q_TIPO_LOC_VERTICAL :=  $0.Q_TIPO_LOC_VERTICAL;
$0.Q_LOC_VERTICAL :=  $0.Q_LOC_VERTICAL;
$0.Q_COMPLEMENTO :=  $0.Q_COMPLEMENTO;
$0.Q_BAIRRO_INFERIDO :=  $0.Q_BAIRRO_INFERIDO;
$0.Q_UF :=  $0.Q_UF;
$0.Q_LOCALIDADE :=  $0.Q_LOCALIDADE;
$0.Q_CEP :=  $0.Q_CEP;
$0.Q_PREPOSICAO :=  $0.Q_PREPOSICAO;
$0.Q_TITULO_LOGRADOURO :=  $0.Q_TITULO_LOGRADOURO;
$0.Q_ENDERECO_FLAGS :=  $0.Q_ENDERECO_FLAGS;
$0.Q_TIPO_LOCALIDADE :=  $0.Q_TIPO_LOCALIDADE;
$0.Q_MUNICIPIO :=  $0.Q_MUNICIPIO;
$0.Q_END_COMPLETO :=  $0.Q_END_COMPLETO;
$0.Q_LONGITUDE :=  $0.Q_LONGITUDE;
$0.Q_LATITUDE :=  $0.Q_LATITUDE;
$0.Q_ZONA_CENS_COD :=  $0.Q_ZONA_CENS_COD;
$0.Q_DENSIDADE_DEMOGRAFICA :=  $0.Q_DENSIDADE_DEMOGRAFICA;
$0.Q_GEO_PRECISAO :=  $0.Q_GEO_PRECISAO;
$0.MATCH_ENDERECO :=  $0.MATCH_ENDERECO;
$0.MATCH_PF_CPF_ENDERECO :=  $0.MATCH_PF_CPF_ENDERECO;
$0.EXISTE_LB07 :=  $0.EXISTE_LB07;
$0.DT_NASC :=  $0.DT_NASC;
$0.QTD1 :=  $0.QTD1;
$0.ID :=  $0.ID;

And the error log:


[2011-05-09 10:52:42,781] [ERROR] [Reformat] [java.lang.NullPointerException]
[2011-05-09 10:52:42,782] [ERROR] [WatchDog] [Phase initialization failed with reason: TratamentoInternetEmail2_refmt_dbwriter ...FAILED ! 
Reason: null]
TratamentoInternetEmail2_refmt_dbwriter ...FAILED ! 
Reason: null
	at org.jetel.graph.Phase.init(Phase.java:160)
	at org.jetel.graph.runtime.WatchDog.executePhase(WatchDog.java:449)
	at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:184)
	at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:62)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.lang.Thread.run(Thread.java:619)
Caused by: null
	at org.jetel.component.WrapperTL.init(WrapperTL.java:198)
	at org.jetel.component.RecordTransformTL.init(RecordTransformTL.java:64)
	at org.jetel.component.RecordTransformFactory.createTransform(RecordTransformFactory.java:256)
	at org.jetel.component.Reformat.init(Reformat.java:332)
	at org.jetel.graph.Phase.init(Phase.java:155)
	... 6 more

Thanks a lot!!!

avackova · May 9, 2011, 2:21pm

Is it a full stack trace?
I’m afraid I have too little information to find the reason. Can you show the graph?

fschnack · May 9, 2011, 3:06pm

Unfortunately that’s the full stack. Here is my full log:



[2011-05-09 12:03:40,240] [INFO ] [Defaults] [Loading default properties from: defaultProperties]
[2011-05-09 12:03:40,256] [INFO ] [Defaults] [Loading default properties from: defaultProperties]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'DB2'.]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'INFORMIX'.]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'MSSQL'.]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'MSSQL2005'.]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'MYSQL'.]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'ORACLE'.]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'POSTGRE'.]
[2011-05-09 12:03:40,456] [WARN ] [JdbcSpecificFactory] [Some of the plugin tried to register already registered JDBC specific under same database name: 'SYBASE'.]
[2011-05-09 12:03:47,054] [ERROR] [Reformat] [java.lang.NullPointerException]
[2011-05-09 12:03:47,082] [INFO ] [EngineInitializer] [Checking graph configuration...]
[2011-05-09 12:03:47,115] [ERROR] [Reformat] [java.lang.NullPointerException]
[2011-05-09 12:03:47,116] [INFO ] [EngineInitializer] [Graph configuration is valid.]
[2011-05-09 12:03:47,117] [WARN ] [ConfigurationStatus] [[EscritaBancoDados3] - Unsupported feature]
[2011-05-09 12:03:47,117] [WARN ] [ConfigurationStatus] [[TratamentoInternetEmail2_refmt_dbwriter].transform - null]
[2011-05-09 12:03:47,117] [INFO ] [EngineInitializer] [Graph initialization (FLOW_2213107)]
[2011-05-09 12:03:47,121] [INFO ] [TransformationGraph] [Initializing connection: ]
[2011-05-09 12:03:47,122] [INFO ] [TransformationGraph] [DBConnection driver[org.jetel.connection.jdbc.driver.JdbcDriver@19a145d1]:jndi[null]:url[jdbc:oracle:thin:@172.16.6.202:1521:lisa]:user[consulta] ... OK]
[2011-05-09 12:03:47,231] [INFO ] [WatchDog] [register MBean with name:org.jetel.graph.runtime:type=CLOVERJMX_2229829_0]
[2011-05-09 12:03:47,244] [INFO ] [TransformationGraph] [*** TRANSFORMATION GRAPH CONFIGURATION ***
]
[2011-05-09 12:03:47,276] [INFO ] [TransformationGraph] [--- Phase [0] ---]
[2011-05-09 12:03:47,276] [INFO ] [TransformationGraph] [	... nodes ...]
[2011-05-09 12:03:47,276] [INFO ] [TransformationGraph] [	EscritaBancoDados3 :  phase: 0]
[2011-05-09 12:03:47,276] [INFO ] [TransformationGraph] [	LeituraArquivoGenerico1 :  phase: 0]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [	TratamentoInternetEmail2 :  phase: 0]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [	TratamentoInternetEmail2_refmt_dbwriter :  phase: 0]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [	... edges ...]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [	TratamentoInternetEmail2_0_EscritaBancoDados3_0_dbwriter type: direct]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [	TratamentoInternetEmail2_0_EscritaBancoDados3_0 type: direct]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [	LeituraArquivoGenerico1_0_TratamentoInternetEmail2_0 type: direct]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [--- end phase ---]
[2011-05-09 12:03:47,277] [INFO ] [TransformationGraph] [*** END OF GRAPH LIST ***]
[2011-05-09 12:03:47,288] [INFO ] [GraphExecutionListener] [Received notification from JMX.]
[2011-05-09 12:03:47,288] [INFO ] [GraphExecutionListener] [Received notification from JMX.]
[2011-05-09 12:03:47,288] [INFO ] [Phase] [[Clover] Initializing phase: 0]
[2011-05-09 12:03:47,443] [INFO ] [RemoteBeanFactory] [EJB lookup for jnp://sgq01:1099/quality-ear/InternetValidatorBean/remote returning jboss.j2ee:ear=quality-ear.ear,jar=quality-ejb-1.0.13.jar,name=InternetValidatorBean,service=EJB3.]
[2011-05-09 12:03:47,466] [ERROR] [Reformat] [java.lang.NullPointerException]
[2011-05-09 12:03:47,473] [ERROR] [WatchDog] [Phase initialization failed with reason: TratamentoInternetEmail2_refmt_dbwriter ...FAILED ! 
Reason: null]
TratamentoInternetEmail2_refmt_dbwriter ...FAILED ! 
Reason: null
	at org.jetel.graph.Phase.init(Phase.java:160)
	at org.jetel.graph.runtime.WatchDog.executePhase(WatchDog.java:449)
	at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:184)
	at org.jetel.graph.runtime.WatchDog.call(WatchDog.java:62)
	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
	at java.lang.Thread.run(Thread.java:619)
Caused by: null
	at org.jetel.component.WrapperTL.init(WrapperTL.java:198)
	at org.jetel.component.RecordTransformTL.init(RecordTransformTL.java:64)
	at org.jetel.component.RecordTransformFactory.createTransform(RecordTransformFactory.java:256)
	at org.jetel.component.Reformat.init(Reformat.java:332)
	at org.jetel.graph.Phase.init(Phase.java:155)
	... 6 more
[2011-05-09 12:03:47,498] [ERROR] [WatchDog] [!!! Phase finished with error - stopping graph run !!!]
[2011-05-09 12:03:47,499] [INFO ] [GraphExecutionListener] [Received notification from JMX.]

avackova · May 9, 2011, 3:13pm

Unfortunately, the error in log is thrown from the catch statement. So we know, that this is NullPointerException (probably), but we don’t know where the null occurs exactly.
Try to reduce the transformation to find what is causing the problem:

Does the graph with only one direct mapping fail?
Does the graph with only one mapping with try_convert function fail?

fschnack · May 9, 2011, 4:25pm

It failed in the same way by using this TL expression:

//#TL
$0.QUALIFIED_URL := $0.QUALIFIED_URL;

fschnack · May 9, 2011, 8:28pm

I found the problem. The problem was that I wasn’t declaring the transform() function to put the TL code inside of it. Is that mandatory, right?

I guess clover’s validation didn’t get this issue.

Thanks a lot!

avackova · May 10, 2011, 7:06am

Yes, transform function must be defined and other functions might be defined.

This NPE is known problem, when using input/output records in global scope and, I believe, is resolved in CloverETL 3.0

Topic		Views
Try/catch block in TL CloverDX Platform	1	June 9, 2011
How to skip transformation errors in the reformat component? CloverDX Platform	3	July 2, 2009
If Statement in Reformat Component CloverDX Platform	1	April 21, 2017
Transformation Language for Reformat CloverDX Platform	2	January 7, 2008
Pls a example of a use "If"in a REFORMAT CloverDX Platform	7	April 26, 2013

Using try_convert in embedded clover

Related topics