Page 1 of 1

Graph executed many times leaks memory

Posted: Sun Jun 22, 2014 6:42 pm
by esegal
Another thing please - my application run Clover Graph multiple times - mainly for aggregation purpose.
As I see by JConsole the application consume more and more memory until I get:
java.lang.OutOfMemoryError: Direct buffer memory
at java.nio.Bits.reserveMemory(Bits.java:633)
at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
at org.jetel.graph.DirectEdge.init(DirectEdge.java:107)
at org.jetel.graph.Edge.init(Edge.java:265)
at org.jetel.graph.Phase.init(Phase.java:147)
at org.jetel.graph.TransformationGraph.init(TransformationGraph.java:463)
at org.jetel.graph.runtime.EngineInitializer.initGraph(EngineInitializer.java:176)

When I set the JVM for enlarge the Direct buffer memory I get the same result - but after more time.
I took dump file of the application after running a while and I see 41,040 instances of org.jetel.component.DataReader (!!!)
- Well something is malfunction here - I think I use Clover as needed - mybe I'm wrong ?
my code call EngineInitializer.initEngine() only once, and after each graph running call also:
Map<String,Node> nodes= graph.getNodes();
for (Node node : nodes.values())
node.free();
graph.free();

Is any one know about memory-leak in CloverETL?? or am I doing something wrong ??

Thanks in advance !
Segal

Re: Graph executed many times leaks memory

Posted: Thu Jun 26, 2014 10:48 am
by jurban
Hi esegal,

can you please post the graph you're using? Then we'll analyse it.

Regards,
Jaro

Re: Graph executed many times leaks memory

Posted: Thu Jun 26, 2014 3:10 pm
by esegal
Hi,
Please find attached the Graph I use.
Thanks !
Segal

Re: Graph executed many times leaks memory

Posted: Thu Jun 26, 2014 4:07 pm
by jurban
I didn't find anything suspicious in the graph. However, you're running the graph multiple times - how are you doing that? Are you running it from your own Java code? You might be not calling proper cleanup after the graph execution (e.g. you should call graph.free()).

Re: Graph executed many times leaks memory

Posted: Mon Jun 30, 2014 10:53 am
by esegal
The graph is running multiple times by Java code.
I mentioned before that I call graph.free(), and even free() for each node in graph...
I'm trying to compile Clover code in order to try to debug what's happening .
I'll update what's going on.
Thanks !

Re: Graph executed many times leaks memory

Posted: Tue Dec 29, 2015 11:53 am
by mahendran
Hi esegal,

Can you update us on what issue you found and how you fixed it.

Re: Graph executed many times leaks memory

Posted: Sun Nov 25, 2018 9:28 pm
by Stella365
How did you fix the issue, esegal?