Partition data

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

mohamedosharif
Posts: 17
Joined: Tue Dec 11, 2018 9:19 pm

Partition data

Postby mohamedosharif » Wed Feb 06, 2019 8:38 pm

is there a way to separate data in to different outputs?

i want something that works similar to a router that sorts data based on a row value in a column. I see that there's a parallelPartition component, in the expression, i defined 3 different outputs 0, 1, 2. but i'm not sure how to set the edge to different writers.

bartonv
Posts: 125
Joined: Wed May 03, 2017 12:10 pm

Re: Partition data

Postby bartonv » Fri Feb 08, 2019 1:10 pm

Hi Mohamad,
yes, there are a couple of different options on how to separate data into different outputs. Choosing the right one depends on some of the specific use case details, namely, whether there is a known number of the possible column values and then, also the number itself. In other words, if there is a known number of values, you can often use a single component. For example, if there are 2 possible values (true/false), you can use a Filter component with simple filter expression ($in.0.field1 == true). If there are 3 or more, you can use Partition or Reformat, for instance. However, if the number of values (based on which you decide the number of output ports) is not known before the graph runs, it would require a more complex approach possibly using more components. Could you provide sample input and demonstrate the desired result?
Note: ParallelPartition is not a good candidate in this case as it is designed to distribute incoming data records in the CloverDX Cluster among different cluster nodes.
Kind regards,
---
Vladimir Barton
CloverCARE Support
CloverDX

Visit us online at http://www.cloverdx.com