Q: Stripping newline chars from an XML document

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

Posts: 2
Joined: Tue Nov 22, 2011 11:15 pm

Q: Stripping newline chars from an XML document

Postby Crooner » Wed Nov 23, 2011 8:39 pm

I am trying to 'serialize' an XML document to pass it's contents as an arg to an external web service. I thought it would be straightforward with CTL2 using a RegEx, but have not been successful thus far. Here is how I am attempting to do it:

XMLdoc >> UniversalDataReader-->Reformatter (CTL2 Regex pattern) --> Reformatter (Transform=ExternalAPI( arg0 ))

Is there a simple way to strip newline characters from an XML document? That's really all I need.


Posts: 8
Joined: Tue Nov 15, 2011 2:53 pm

Re: Q: Stripping newline chars from an XML document

Postby tkramolis » Mon Nov 28, 2011 1:42 pm

Hi Tony,

Your solution seems to be correct, there has to be only a minor change required. Here's my suggestion:

I would use the UniversalDataReader to read the whole imput XML into single field of single record. This requires the output edge metadata to be properly configured: Create metadata with single field and clear value of the "Record delimiter" record metadata property. Also set the "EOF as delimiter" property of the single metadata field to true.

Then use the Reformatter with replace function to strip the new line characters...

Attached is a demo graph which strips the new lines of the read XML and prints the result into the log using the Trash component.


(2.67 KiB) Downloaded 310 times

Posts: 163
Joined: Fri Jul 20, 2007 9:25 am

Re: Q: Stripping newline chars from an XML document

Postby jurban » Mon Nov 28, 2011 3:10 pm

Hi Tony,

just one note - you need to be careful about the size of the XML. The whole XML content needs to fit into the edges, so the size is limited by the Record.MAX_RECORD_SIZE property, see documentation here. The current default of MAX_RECORD_SIZE is 64KB, so you might need to modify it depending on your data - however, increasing the value increases memory consumption of the graph.

In CloverETL 3.2 we'll have edges that can "grow" dynamically as needed, so they'll be more suitable to transport large data in each record. They'll have a default maximum size of 32MB.

Best regards,
Jaroslav Urban
CloverCARE Support
CloverETL | Rapid Data Integration

Visit us online at http://www.cloveretl.com