It is possible to create graph, that reads only “new” (added from the last run of graph) data from database.
Look at following DBInputTable definition:
<Node dbConnection="Connection0" id="INPUT" incrementalFile="dbInc.txt" incrementalKey="key=max(employee_id)" sqlQuery="select * from employee where employee_id>#key" type="DB_INPUT_TABLE"/>
In the above node, new records are defined by clause where employee_id>#key and key is stored as maximum value from the last database reading. You have also to define the file, that stores the incremental key value. For the first reading you must define the key by hand (create file, where are keys and theirs values eg.
key = -1
).
The incremental key can consist of more then one field (sqlQuery=“select * from employee where hire_date > #key1 or employee_id > #key2”, incrementalKey=“key1=first(hire_date);key2=max(employee_id)”) and available value in key definition are: first,last,max and min.