I have a 25GB Clover file that I’m trying to read using the CloverDataReader and keep getting hit with Java heap space issues. It appears the component is putting all of the records in memory as it reads??? I tried using a filter right after the CloverDataReader in hopes or reducing the JVM foot print but that doesn’t seem to work either. I’ve increased the JVM size as much as I can (~7GB) but still no dice. Also tried change the edge types and some defaultProperties. Any other ideas or advise are certainly welcomed.
ERROR [WatchDog] - Node CLOVER_READER0 finished with status: ERROR caused by: Java heap space
ERROR [WatchDog] - Node CLOVER_READER0 error details:
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.(HeapByteBuffer.java:39)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
Thanks,
Chris