Process each file separately

CloverDX Platform
1

Hi,
I am fairly new to Clover ETL and I would like to ask if it is possible to read each file separately and then execute sequence of steps\activities per file. Currently the situation is that I am using UniversalDataReader , which read all the files in specific directory and then execute the next step while all the files data already loaded.

Thanks Ilan,

2

Hi Ilan,

There are basically two ways how to do that and you can choose which one fits your use case the best:

  • run graph for each file separately providing a filename as parameter

  • use metadata autofilling functions to populate source filename to each record. That way you can process all files at once and just partition the output by filename.