How to deduplicate information?

Support/help with CloverETL (4.9) and CloverDX (5.0 or newer) implementation problems

nsoria
Posts: 1
Joined: Tue Mar 06, 2018 2:41 pm

How to deduplicate information?

Postby nsoria » Wed Apr 04, 2018 1:55 pm

Hi everyone,
I am new to CloverETL and we have a scenario that we are still not sure how to address it.

Scenario:
We have to upload to data on a weekly basis to a Oracle database.
Data is related to orders, which some of them can have status open the first week and have status closed the second week.

Approach:
Currently, we are inserting records with the timestamp to prevent conflict with primary key.

One of the ideas that came up was to have one more box in CloverETL to purge duplicated records with old timestamp, but I am not quite sure if that is possible.

If it is possible, could someone guide me on how to accomplish this?

Regards,
Nicolás.

cholastal
Posts: 137
Joined: Tue Sep 01, 2015 1:22 pm

Re: How to deduplicate information?

Postby cholastal » Mon Apr 09, 2018 8:46 am

Hi Nicolás,

There is a component called Dedup in CloverETL software which is able to remove duplicates from the input data based on a key - specific column/field within the input data. If this is not what you are looking for, please provide me with more detailed example with some sample input data.

Best regards,

---
Lukas Cholasta
CloverCARE Support
CloverDX

Visit us online at http://www.cloverdx.com


cron