Memory/Space Estimating Tool?

CloverDX Platform
1

Does anyone have a memory and/or space estimating tool? I’m looking to estimate how much temp space and memory space I’ll need to process my graphs once I take some of the record filters off of them that I’ve used for development to limit the counts going through.

Thanks in advance for any help

2

Hi pintail,

Sorry but we have no such estimation tool. I think it is impossible for the computer to estimate those things without running the graph first. I can give you a few tips though.

  1. Memory space depends on the number of input records and their size and the way you process them. (the components used, CTL functions used, efficiency of processing, number of graph phases, …) Especially components like sorters, joiners or some others which keep all records in memory can have high influence on the overall memory requirements.
  2. Temp space depends on the number of debug edges, buffered edges and size of the buffers, the number of temp files you create and the number of records stored in them.
  3. Graph run log shows the amount of memory used. Exploring a graph log from a run in development environment, you can get a hint about the memory requirements for production.
  4. You can also use tools like jvisualvm to see and examine JVM behavior during the graph run. If you see the results for a graph with 100 records processed and the other one for 200 records, you should be able to estimate the results for 300 records.

I know I provided you only with a high level view on the topic so do not hesitate to ask specific things you are interested in.

3

How to overcome the following issue “Temp space E:\CloverDXServer.5.2\temp\clover_temp_node01 is almost full”.Can you please sort me out this problem.

4

Hi,

Since it looks like you are using CloverDX Server then TEMP space configuration options are available in our documentation.