XSLT support

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

mgrove
Posts: 2
Joined: Mon Sep 08, 2008 9:02 pm

XSLT support

Postby mgrove » Mon Sep 08, 2008 9:18 pm

I see in clover 2.5 there is support for XSLT transformations, at least, I see a node called XSLDataTransformer that leads me to believe i *could* do something w/ XSLT.

my use case is I'm converting some XML and stuffing it into a database. The XML I'm given is in the most horrendous format possible and is really difficult to use.

So I made an XSLT to transform the bad XML input into something more reasonable to handle. the specifics of why its bad are not important, just that it needs to be transformed.

what I would like to have is a node read in the bad xml, pipe that to another node which will transform that xml into the "good" format, and then send that XML blob to the XmlXPathReader where I can pull out all the relevant bits into a data record which will eventually end up in the database.

reading the XSLDataTransformer, well, half the javadoc looks like it was copied and pasted from the Sort node, so I dont know what to think. But i'm left with the impression that you can only perform transformations on fields in a data record, and not an arbitrary blob of xml. what i'm looking for is:

1) can XSLDataTransformer be used for my use case?
2) if XSLDataTransformer cannot be used for my use case, is there another out of the box component than can be?
3) if there is not another out of the box component, what's the best way to put an xslt transformation in front of an xml file being read in?

If all else fails, I can do this xslt pre-processing before the ETL phase, but this is a use case that clover should support (in my opinion) so i'm hoping this can be done within the existing framework.

ideas?

Thanks.

jausperger
Posts: 19
Joined: Wed Apr 04, 2007 2:44 pm

Postby jausperger » Tue Sep 09, 2008 8:32 am

Hello, I believe you can use the first use case. The first component reads a xml file (last matadata field must use eofDelimiter if you use UniversalDataReader..), the second component is XSLTTranformer and the last component is XMLXpathReader.

There is one condition you must set up appropriate field/record size in 'defaultProperties' file for the xml data.

mgrove
Posts: 2
Joined: Mon Sep 08, 2008 9:02 pm

Postby mgrove » Tue Sep 09, 2008 12:59 pm

jausperger wrote:The first component reads a xml file (last matadata field must use eofDelimiter if you use UniversalDataReader..)


which component reads a raw XML file? i've only been able to do that in the past with the XmlXPathReader. could you give me an idea of components to use and the metadata setup?

jausperger
Posts: 19
Joined: Wed Apr 04, 2007 2:44 pm

Postby jausperger » Tue Sep 09, 2008 3:00 pm

You can use whatever reader (JMS, Universal, DBInputTable,...) and get whole xml into a field. I don't know how big is your xml file and it can cause memory problem. If you have simple file, you can use ie UniversalDataReader and metadata:

<Metadata id="Metadata0">
<Record name="recordName1" type="delimited">
<Field eofAsDelimiter="true" name="field1" type="byte"/>
</Record>
</Metadata>

Or you can also use SystemExecute/JavaExecute component the xslt tranformation.
Or you can derive XSLTransformer component for your purpose but this is not easy for common user.

mzatopek
Posts: 76
Joined: Fri May 11, 2007 9:49 am

Postby mzatopek » Sat Sep 27, 2008 11:24 am

Suggested solution is reasonable only for really small xml files, which are able to fit in a single field. This XSLT component should be able transform xml directly from file as well. I have committed new feature request to our bug system.