Reformat Mapping

CloverDX Platform
1

Hi,

I am new to CloverETL tool and having issue. I am trying to count no of errors from a logfile. The log file have dates,type and messages.

I have used Universal Data Reader and loaded the source file. In the Messages i want find if there a word “error” exists then count by date or if we can replace it with 1 0r 0 whenever error word exists in the messages.

The Messages are like this below

httpclient.wire.content - >> “<?xml version='1.0' encoding='UTF-8'?><soapenv:Envelope xmlns:soapenv=”[http://www.w3.org/2003/05/soap-envelope">soapenv:Bodyns2:retrieveUsersList](http://www.w3.org/2003/05/soap-envelope"soapenv:Body<ns2:retrieveUsersList) xmlns:ns2=“[http://service.ws.bsapi.ra.services.gxs.com”>ns2:request](http://service.ws.bsapi.ra.services.gxs.com"<ns2:request) xmlns:ns1=“http://entity.ws.bsapi.ra.services.gxs.com/xsd” xmlns:xsi=“http://www.w3.org/2001/XMLSchema-instance” xsi:type=“ns1:RetrieveUsersListRequest”>ns1:clientReferencens1:clientTypegxs</ns1:clientType>ns1:clientVersion1.0</ns1:clientVersion>ns1:idBMO-FTF</ns1:id></ns1:clientReference>ns1:userIdentifierns1:userIdGCP464V1FI799</ns1:userId></ns1:userIdentifier></ns2:request></ns2:retrieveUsersList></soapenv:Body></soapenv:Envelope>"

I have tired Contains,replace, find funtions in Reformat Transformation but it says incompatible with String ,string.

Any Ideas will be really appreciated

Thanks
Sri

2

Hi,

Could you please rephrase your question and use an example containing some error? From the provided example and description, I am not sure what exactly you are trying to do.

You could also use XMLExtract and avoid any parsing in Reformat this way.

Thanks,

3

Sure I will.

I have Error Log file generated by the system and which is source in txt format.Sample content of the file data is show below

2014-03-31 04:25:16,274 [TP-Processor33] [2CEC4D065611321FE087ECD0C766808F.jvm2] DEBUG httpclient.wire.header - << “Server: Apache-Coyote/1.1[\r][\n]”
2014-03-31 04:25:16,274 [TP-Processor33] [2CEC4D065611321FE087ECD0C766808F.jvm2] DEBUG httpclient.wire.header - << “X-HP-CAM-COLOR: V=1;ServerAddr=L+dXR13ZQ5/BHQN6DaCjUg==;GUID=1|KccQLvo21nBHcCB5lpH908p5gJjoLX0orvihFXyTAW9dKpla8iaaxTSzFZxZHFU3YF6R-H-tZ0P4r4LeePj6_A…|L1JBQlNBUEkyQlVTZXJ2aWNlL3NlcnZpY2VzL0JTQVBJQlVTZXJ2aWNl[\r][\n]”
2014-03-31 04:25:16,274 [TP-Processor33] [2CEC4D065611321FE087ECD0C766808F.jvm2] DEBUG httpclient.wire.header - << “Content-Type: application/soap+xml; action=“urn:retrievePermissionsResponse”;charset=UTF-8[\r][\n]”
2014-03-31 04:25:16,274 [TP-Processor33] [2CEC4D065611321FE087ECD0C766808F.jvm2] DEBUG httpclient.wire.header - << “Transfer-Encoding: chunked[\r][\n]”
2014-03-31 04:25:16,274 [TP-Processor33] [2CEC4D065611321FE087ECD0C766808F.jvm2] DEBUG httpclient.wire.header - << “Date: Mon, 31 Mar 2014 04:25:15 GMT[\r][\n]”
2014-03-31 04:25:16,274 [TP-Processor33] [2CEC4D065611321FE087ECD0C766808F.jvm2] DEBUG httpclient.wire.header - << “[\r][\n]”

so from I have used UnviersalDataRedear to load the file with delimiter as Space which gives me Date,TP Processor,Code,Type & Msg as columns.

Now I want to see if a “error” word exists in Msg column then Count No of Errors in Day.So that I can report no of errors in day and if I want msg too.

So any suggestions how would be my be my graph.If you can give me in detail that would better as I am new to CloverETL

My apologize for not using technical words.

Thanks
Sri

4

Sure I will.

I have system log file need to identify error’s which is txt format and sample data is shown below

2014-03-31 04:08:17,044 [TP-Processor28] [53A8D42682134E3017F3C1E70C5AF3CD.jvm2] DEBUG alertLog - Starting the process of com.gxs.bmo.ftf.portlet.TransmissionSummaryPortlet
2014-03-31 04:08:17,044 [TP-Processor28] [53A8D42682134E3017F3C1E70C5AF3CD.jvm2] DEBUG com.gxs.bmo.ftf.portlet.FTFBasePortlet - Locale: en_CA
2014-03-31 04:08:17,053 [TP-Processor28] [53A8D42682134E3017F3C1E70C5AF3CD.jvm2] DEBUG com.gxs.bmo.ftf.portlet.FTFBasePortlet - Ticket generated: d1mHDy0CtN1xRn42mZ1qn6XvMDuWYOr5
2014-03-31 04:08:17,054 [TP-Processor28] [53A8D42682134E3017F3C1E70C5AF3CD.jvm2] DEBUG com.gxs.bmo.ftf.portlet.FTFBasePortlet - Check Login

by using Space delimiter i have loaded the data into Universal Data Reader.So it has Date,Processor,JVMCode,Braces,Type,Msg.

Now I want to search if “error” word exists in Msg column then count no of errors per day.

what would like my graph look like?

Please let me know if it is not still clear.

Thanks
Sridhar

5

Thank you, I think I understand now.

You can use for example indexOf(string arg, string substring);

If the string contains the substring, a non-negative index number is returned. -1 is returned otherwise.

FYI, CloverETL Server is able to detect errors in much smarter ways. For example with Graph Event Listeners or in ExecuteGraph component.

Regards,

6

Thanks Lubos.

Sorry to ask again. I tired indexOf(string $in.0.Msg2, string error) but gives me error Variable ‘indexOf’ is not declared. I have looked at the examples how to use Indexof but was not able succeed. If you don’t mind can you elaborate how to use the function.

I really appreciate your help.

Thanks
Sridhar

7

Hi Sridhar,

Correct usage should be:

indexOf($in.0.Msg2, "error")

You can learn about CTL2 here: http://doc.cloveretl.com/documentation/ … /ctl2.html

I hope this helps.