How to use incremental reading?

How-to(s), solutions to common problems, pearls of wisdom

Posts: 19
Joined: Wed Apr 04, 2007 2:44 pm

How to use incremental reading?

Postby jausperger » Wed Oct 22, 2008 10:37 am

What is and how to use incremental reading?

If you have a file (url) that you want to process but at once you suppose the file is not complete (additional data will be appended at the end of file) then it is suitable to use the incremental reading.

Most of readers have incrementalFile and incrementalKey attribute.
The incrementalFile defines a file where the clover engine writes a file position.
The incrementalKey is arbitrary name you can define. That key holds the value of the file position.

The clover writes the position into incrementalFile when the phase of graph happens successfully.