XML writing from the two source

CloverDX Platform
1

I have two phases in the graph. First phase - it is DBInputTable to the XMLWriter and second phase the same.
Data to write in the same XML file. But when first phase completed, second phase rewrite this XML file and I lost data from first table in the XML file.

The variant with one XMLWriter and two DBInputTable is not suitable

Could you please help to resolve this problem?

2

Hi Sergii,

Described behavior is correct, second XmlWriter really overrides file created by first.

I am not sure why

The variant with one XMLWriter and two DBInputTable is not suitable

. Can you elaborate more on reasons? The only reason I can think of is really huge data set.

3

Hi Jaroslav,

Thank for your reply.

I have attached Image with sample of structure of the graph. File URL, the same, on the both XMLWriter components (XMLWriter_1 and XMLWriter_2).
In this time I don’t have huge data set.
I know that I can use two source and send them to one XMLWriter. In this case, I will be have one data stream, and file created one time. In the future I can have situation when I should separate data stream on the two phase (as described in the Image) and send data from this sources in the one XML file. Only in this case second phase rewrite this file and I lost data set from first source.

Maybe XMLWriter component has some additional property that allow us to append to the file or something like that.

Best Regards
Segrii

4

OK, now I see. The way how to do it in CloverETL is following:

  1. read the previous version of file using XmlExtract, XmlReader, XmlXPathReader
  2. merge read records with new using SimpleGather (or another join component)
  3. store all records using XmlWriter

Because this use case sounds interesting I have created feature request - https://bug.javlin.eu/browse/CLO-3147

5

Jaroslav,
thanks for your help and created request. I believe that developers will add some appropriate property for this component.