Forum: Building VoltDB Applications

Post: Memory taken by VoltDB

Memory taken by VoltDB
sanket
Dec 1, 2016
Hi,
I am measuring the memory taken by VoltDB after loading the database of 10GB. I am measuring it from VoltDB Management Center and also from command line using 'free' command. I also take precaution that no other heavy process is running on my server while measuring the memory usage from command line. The memory usage shown at VMC is 21.4 GB while at Command line, it is 25 GB.

Why does this difference occur? Is it normal that from 10 GB voltDB consumes 21.4 GB?
bballard
Dec 5, 2016
Hi,
The "free" command shows the total memory used by all applications and the operating system. The memory chart in VMC is based on the RSS (Resident Set Size) number from @Statistics MEMORY, which is the same value that would be reported as the set size in "top".

You can call @Statistics TABLE 0; to get a report per table per partition. It is not uncommon for a 10GB CSV file to need 21.4 GB in memory, as there would be empty spaces for any fixed-width VARCHAR columns, plus the pointers, reference object, and sizes for larger VARCHAR columns. Plus if you have any indexes, those use additional memory. Depending in your data, you might be able to optimize the VARCHAR() sizes to use less space. The Planning Guide has a chapter that describes the formulas for memory use.

If you'd like, contact me (ben) on our public Slack channel (chat.voltdb.com) and I can send you a script which provides a detailed report of memory usage by table.
sanket
Dec 6, 2016
Thanks, Ben.