Validating timestamps to find missing rows

CloverDX Platform
1

Let’s say I have a csv with the following format

Timestamp, Amount_1, Amount_2
3/12/2015 0:00, 5, 10
3/12/2015 0:15, 10, 0
3/12/2015 0:30, 1, 2

This then repeats for 15 minute intervals for the rest of the data. However, there is 15 minute intervals missing from the data. These should be inserted and given 0’s as their amounts.

I understand how to approach parsing the data, and doing what I need to after it’s validated. However, I’m at a loss in regards to what approach I should even be taking to ensure there are no missing times. If it helps I have the paid version with all functionality.

Thanks

2

Hi,

I would recommend that you use a Normalizer, since it can produce more records than are being read into the input port. You will need to sort the data first using ExtSort or FastSort depending on the amount of data you have, and pass it into the Normalizer.

I have taken the liberty of adding a sample graph with this process, if you have any doubts please let me know.