Performance tuning of endeca_delete_records component

CloverDX Platform
1

Hi,

For deleting/purging records from the data domain we were using the delete records component specifying the “record set specifier attribute”.

Now we are using the delete records in the standalone mode which just requires an “Eql Record Set Specifier” according to which it will delete data from the data domain.

{Ideally it should take less time compared to the conventional way because it doesn’t interact with source data}

But while handling huge amount of data like 4-5 million records, the standalone mode is taking a huge amount of time. The data load for 4-5 million records takes approximately 5 mins. Ideally the deletion of records should take less time than loading but using the delete records in standalone is taking about 30 - 40 mins, which is unacceptable in real-time scenario.

I have tried increasing the Java heap size (256MB - 1024MB), still it doesnt make much of a difference.

We also tried running the Graph in Server Mode, which doesn’t suffice as well.

Could you help us to find a way to reduce the time for deleting records using standalone mode ??

Thanks

2

Dear Sonamganapathy,

As this issue relates to Endeca Data Integrator please do contact Oracle Endeca support in this matter directly. They need to be informed about this issue and will work with us directly if needed.

Thank you for your cooperation!