Forum: Managing VoltDB

Post: what is the 10MB heap memory for each table used for?

what is the 10MB heap memory for each table used for?
matrix
Oct 25, 2014
Hi, all!

According to the panning guide book, the Java heap size for VoltDB consists of a base Java heap size of 384MB plus 10MB for every table in the catalog. When I run a VoltDB server, the 10MB memory is already allocated for each table, which is quite a consumption of memory resource when there are a lot of tables in the scheme.

I want to know what the 10MB heap memory for each table is used for. And is it possible to configure this to a smaller value?

Thanks!
jhugg
Oct 27, 2014
First of all, it would be great for us at VoltDB to get a sense of what you're trying to do. How many tables do you have? What kind of hardware are you planning to run on?

The 10mb per table is a guideline for the Java heap size, but it's not a hard number that represents some specific objects. It does cover metadata for the table and its indexes. It covers metadata for any default procedures created for those tables. It also allows for any buffers that may be needed during export, database replication elastic rebalancing or other operational procedures. It's very unlikely that this memory would be needed under normal circumstances; the guideline is there to avoid a heap exhaustion under any conceivable circumstance.

So one thing we're doing is trying to make this number smaller or nonexistent. We'll add items to release notes as we start shipping some of that work.

Another thing you can do is make the number smaller. It's very likely that you won't run into problems. Still, I would suggest telling us as much as you can about what you're doing and we might have some insight on how safe you would be.
GopalS
Feb 11, 2015
Java heap memory size:

What will an under sized Java heap memory (VOLTDB_HEAPMAX) lead to ?
pzhao
Feb 11, 2015
GopalS,

Ultimately, an undersized Java heap memory (VOLTDB_HEAPMAX) leads to VoltDB crashing due to not enough heap memory or VoltDB performance will suffer due to java garbage collection being triggered until its completion. As Jhugg mentions, a lot of factors goes into planning an adequate size. We recommend to avoid heap exhaustion. Please look here for more information on planning heap size.

Peter
pzhao
Feb 11, 2015
Gopals,

Added, you could define row limits on tables to prevent over utilizing heap size.

Peter
GopalS
Feb 12, 2015
Thanks Folks.

We suffered VoltDB crash and I noted the following error message:

volt-console.log:
WARN: The configuration of 12 tables, 8 sites-per-host, and k-factor of 1 requires at least 1528 MB of Java heap memory. The maximum amount of heap memory available to the JVM is 985 MB. Please increase the maximum heap size using the VOLTDB_HEAPMAX environment variable and then restart VoltDB.
pzhao
Feb 12, 2015
Gopals,

I suspect there is more to the crash then just this warning message. Within your working directory, there should be a log/volt.log file present you can check.
Alternatively, you can run 'voltdb collect' and this command will generate a zip file. It will contain all logs and crash files you can use to troubleshoot why VoltDB crashed.

Please let me know if you have any other questions.