Heya,
I am running some tests where I am running 5 graphs simultaneously and some are failing due to running out of temp space (in /var/tmp).
1. Is there a reccomended size for temp space?
2. There are a lot of .fbuf* files in /var/tmp, I assume those are buffers for sorting, etc. Does Clover clean them up as they are no longer needed or does it wait until the very end? It would be nice if these files were removes as they are no longer needed…
I trying to do some research before passing any conclusions on to my IT department… 
Thanks,
Anna
Hello Ann,
thees files are temporary files for sorting, joining and for phase edge’s data and they should be removed after requested phase has finished. If the files exist after graph execution it is a bug, but we have never been able to reproduce the issue.
In components you can change temporary directory by setting proper attribute, but it is unfortunately impossible for phase edges (http://bug.cloveretl.org/view.php?id=2541).
Heya,
Going back through the posts on this subject, it looks like I can divert most of the temporary files (the .fb*) by adding the following flag to the JVM upon startup:
-Djava.io.tmpdir=/wherever_I_want
I plan to do some more testing, but this should not do any harm, right?
I’ve found that our IT group usually allocates about 6G to /var. My 5 graph test using a small-to-large dataset (for our data) caused about 3G worth of temp files, so it’s not going to be enough when we run really big datasets.
Thanks,
Anna