How Clover support Incremental Load(CDC, SCD, ...)?

CloverDX Platform
1

Does Clover support this crucial feature? And How?

2

Hi,
You can see the incremental support in the DBInputTable and the UniversalDataReader components. The database support is a bit primitive in that it just lets you set a max value and include that max value your query. You may be thinking of deltas directly from a database log files. The file incremental is a bit interesting in that it allows you to read files that are still being written to. I haven’t tried out this feature yet, as it isn’t something I would do.

I also didn’t see any SCD transformers out of the box, but the features are there for you to create your own.

Hope this helps.

3

Yes, I’m thinking of “deltas directly from a database log files”, but it’s too expensive for us to read log. As I know, informatica supports CDC by analyse database transaction log, they could support oracle, mssql, etc.
I took a look at clover etl but didn’t find that. If none, is clover going to support the transaction-log-based incremental load? I guess it should be in the roadmap

4

Can someone from Clover comment on whether there is an out-of-the-box SCD support?

5

Clover has templates for SCD (both type I & II). There is no single component as it can be handled by connecting some of the existing - like Data Intersection.