Forum: VoltDB Architecture

Post: Does VoltDB use many small objects (1 or more per row) or few large objects (columns)

Does VoltDB use many small objects (1 or more per row) or few large objects (columns)
monster
Oct 3, 2010
The point of my question is this: It is my experience, that a typical Java application with may smaller objects will be about 20% bigger when run in 64Bit Java; in other words, if your data just about fits in the heap of the 32Bit JVM of say 1000MB, then to store that same data in a 64Bit JVM you need 1200MB, even with the pointer-compression thing.


If the biggest machines my provider offer are 24GB RAM, and I take 18GB for VoltDB, leaving 6GB for the rest. Should I have one 18GB 64Bit VoltDB JVM, or divide it into the biggest 32Bit JVM I can have (3GB?) ? I save the 64Bit overhead, but have the Java code duplicated, so I doubt I'm gaining anything, maybe even wasting more RAM.


What is the general opinion on the subject?
Not much of an issue...
jhugg
Oct 4, 2010
VoltDB is written in a mix of Java and C++. We do networking, transaction bookkeeping, stored procedure execution and a few other tasks in Java. In C++, we execute SQL statements and store data.


So your tables are not being stored in Java at all, and the larger pointer size is less of an issue. We do move some data through Java on it's way to other machines or to the client, but this is almost entirely done with ByteBuffers. Even in C++, your data is largely stored in big buffers of memory.
C++ !
monster
Oct 4, 2010
VoltDB is written in a mix of Java and C++. We do networking, transaction bookkeeping, stored procedure execution and a few other tasks in Java. In C++, we execute SQL statements and store data.


So your tables are not being stored in Java at all, and the larger pointer size is less of an issue. We do move some data through Java on it's way to other machines or to the client, but this is almost entirely done with ByteBuffers. Even in C++, your data is largely stored in big buffers of memory.


Wow, I haven't done any C++ in ... let me think ... a very long time (I haven't got enough fingers to count that far). Those were the days ... :D


I could probably read the C++ code without too much trouble, but I wouldn't know where to start to setup a development environment for C++. Would Eclipse be OK? I use it for my Java stuff, although it's pretty slow for the size of corporate project I work on.
I'll chime in and add some
aweisberg
Oct 4, 2010
I'll chime in and add some more detail about pointers and storage.


Tuples as stored in the database are allocated in fixed size 2 megabytes blocks. Tuples are always fixed size. Variable length columns like VARCHARs are stored on the heap with a pointer stored in the tuple which allows the tuple to remain fixed size. If a string is declared to be 63 characters or less in the DDL then that amount of storage is allocated in the tuple (still fixed size) and the string is stored there.


The wire protocol and Java VoltTable work differently. There the tuples are variable length because the expectation is that every row will be iterated over linearly.


Indexes also contain pointers to tuples right now. I have proposed using 4-byte integer indices instead of 8-byte pointers in the indexes. It is easy to do the math to go from an index to a block and offset because tuples are fixed size. See https://issues.voltdb.com/browse/ENG-726


We could also start using 6-byte pointers for out of band strings, but that would make valgrind mad. We could make some #ifdefs to satisfy valgrind, but it would be one more moving part to worry about.


Regarding memory sizing for your application. There are a lot of things you need to take into account. Online snapshots will suck up quite a bit of RAM due to the page cache and the COW data. Theoretical worst case is 2x RAM for COW data, but you will have a hard time reaching the worst case with a typical CRUD workload. Data snapshots much faster then you can change it via typical stored procedures. On a system with a of 48 gigabytes of RAM you will pay a large amount in page table overhead, I can't recall the exact number but I think it might be a gigabyte. Huge page support would solve that (https://issues.voltdb.com/browse/ENG-735).


You only want a 1-2 gigabyte Java heap specified via -Xmx. Larger Java heaps can cause serious GC issues which is a case of more not being better.
That sounds good
monster
Oct 4, 2010
I'll chime in and add some more detail about pointers and storage.


Tuples as stored in the database are allocated in fixed size 2 megabytes blocks. Tuples are always fixed size. Variable length columns like VARCHARs are stored on the heap with a pointer stored in the tuple which allows the tuple to remain fixed size. If a string is declared to be 63 characters or less in the DDL then that amount of storage is allocated in the tuple (still fixed size) and the string is stored there.


The wire protocol and Java VoltTable work differently. There the tuples are variable length because the expectation is that every row will be iterated over linearly.


Indexes also contain pointers to tuples right now. I have proposed using 4-byte integer indices instead of 8-byte pointers in the indexes. It is easy to do the math to go from an index to a block and offset because tuples are fixed size. See https://issues.voltdb.com/browse/ENG-726


We could also start using 6-byte pointers for out of band strings, but that would make valgrind mad. We could make some #ifdefs to satisfy valgrind, but it would be one more moving part to worry about.


Regarding memory sizing for your application. There are a lot of things you need to take into account. Online snapshots will suck up quite a bit of RAM due to the page cache and the COW data. Theoretical worst case is 2x RAM for COW data, but you will have a hard time reaching the worst case with a typical CRUD workload. Data snapshots much faster then you can change it via typical stored procedures. On a system with a of 48 gigabytes of RAM you will pay a large amount in page table overhead, I can't recall the exact number but I think it might be a gigabyte. Huge page support would solve that (https://issues.voltdb.com/browse/ENG-735).


You only want a 1-2 gigabyte Java heap specified via -Xmx. Larger Java heaps can cause serious GC issues which is a case of more not being better.


That sounds like my kind of thinking. :) But I don't want to go back to doing C++ if I can help it. Life is too short, and most of mine is behind me. Scala seems like the way to go for me.


I haven't done any native development inside Java. Is it so that the native part can use as much memory as it wants/can, because it is not constrained like the Java heap by a -Xmx ? If so, then it really doesn't make much of a difference if you have only one, or several VoltDB instances in a 64Bit machine.


Any chance of real BLOBs in the near future? (Without having to use base64 encoding to VARCHARs?)


How about compressing large BLOBs and VARCHARs, like Terracotta does with Strings?


Java really doesn't like to be swapped, but what about VoltDB, if most of the storage is in "native memory"? Can I just create really large partitions, and let the OS swap some of in onto disk? Could I somehow sort the tuples according to last access time, so that "unused" tuples (or entire tables) are stored together, and then swapped to disk? This would greatly alleviate the need to implement inside Java some "manual" exporting of BLOBs to persistent storage, as I was discussing in other threads. I've read that Varnish work along the same principle; the OS knows better then me what is not needed, and swapping to a swap partition is faster then swapping "manually".
I think we would have done it
aweisberg
Oct 5, 2010
That sounds like my kind of thinking. :) But I don't want to go back to doing C++ if I can help it. Life is too short, and most of mine is behind me. Scala seems like the way to go for me.


I haven't done any native development inside Java. Is it so that the native part can use as much memory as it wants/can, because it is not constrained like the Java heap by a -Xmx ? If so, then it really doesn't make much of a difference if you have only one, or several VoltDB instances in a 64Bit machine.


Any chance of real BLOBs in the near future? (Without having to use base64 encoding to VARCHARs?)


How about compressing large BLOBs and VARCHARs, like Terracotta does with Strings?


Java really doesn't like to be swapped, but what about VoltDB, if most of the storage is in "native memory"? Can I just create really large partitions, and let the OS swap some of in onto disk? Could I somehow sort the tuples according to last access time, so that "unused" tuples (or entire tables) are stored together, and then swapped to disk? This would greatly alleviate the need to implement inside Java some "manual" exporting of BLOBs to persistent storage, as I was discussing in other threads. I've read that Varnish work along the same principle; the OS knows better then me what is not needed, and swapping to a swap partition is faster then swapping "manually".


I think we would have done it all in C++ if we didn't have to host stored procedures. The native side allocates memory on the libc heap and not the Java heap so it is not constrained by -Xmx.


Blob support is planned but not scheduled. See https://issues.voltdb.com/browse/ENG-703
You can vote on issues that are important to you.


If you want compression you should do it at the client. Compression via GZipOutputStream is quite slow. When we implemented it at a client it pegged the CPU with full parallelism and wouldn't saturate gig-E.


Java does not like being swapped out. This ticket (https://issues.voltdb.com/browse/ENG-722) would address that by using mlock. The problem with growing beyond physical memory is that you need a lot of threads so that waiting on page faults doesn't hurt performance. This does not mesh well with the shared nothing threading model that makes VoltDB so good at OLTP. VM paging is also not very granular. If your records are significantly smaller then the VM page size you will rapidly start to lose efficiency as the hot data set eclipses main memory.


Why is it important to store your large blobs in VoltDB? Are you looking for a one size fits all solution?


-Ariel
one size fits all
monster
Oct 6, 2010
I think we would have done it all in C++ if we didn't have to host stored procedures. The native side allocates memory on the libc heap and not the Java heap so it is not constrained by -Xmx.


Blob support is planned but not scheduled. See https://issues.voltdb.com/browse/ENG-703
You can vote on issues that are important to you.


If you want compression you should do it at the client. Compression via GZipOutputStream is quite slow. When we implemented it at a client it pegged the CPU with full parallelism and wouldn't saturate gig-E.


Java does not like being swapped out. This ticket (https://issues.voltdb.com/browse/ENG-722) would address that by using mlock. The problem with growing beyond physical memory is that you need a lot of threads so that waiting on page faults doesn't hurt performance. This does not mesh well with the shared nothing threading model that makes VoltDB so good at OLTP. VM paging is also not very granular. If your records are significantly smaller then the VM page size you will rapidly start to lose efficiency as the hot data set eclipses main memory.


Why is it important to store your large blobs in VoltDB? Are you looking for a one size fits all solution?


-Ariel


I understand that with the compression. What you mean is that the goal of VoltDB is do get the absolute best performance possible, using the least possible number of machines, so everything that can be done efficiently in the client should not be done on the server. I can relate to that.


Why is it important to store your large blobs in VoltDB? Are you looking for a one size fits all solution?


Well, yes. I mean, that would be nice, but that's not a pre-requisite. I'm just starting, so I haven't got a heavy load yet, and a one-size-fits-all makes things easier in that case. I just need to understand where you are heading, so that I can work-out what makes sense and what doesn't, from your point of view.


It looks like something I wrote got lost... I was wondering if the native part of VoltDB would lend itself well to running inside a "graphic card" (assuming the data would fit in)? Massive parallelism seems ideal when working with tabular data.
Graphics cards
jhugg
Oct 6, 2010
I understand that with the compression. What you mean is that the goal of VoltDB is do get the absolute best performance possible, using the least possible number of machines, so everything that can be done efficiently in the client should not be done on the server. I can relate to that.


Why is it important to store your large blobs in VoltDB? Are you looking for a one size fits all solution?


Well, yes. I mean, that would be nice, but that's not a pre-requisite. I'm just starting, so I haven't got a heavy load yet, and a one-size-fits-all makes things easier in that case. I just need to understand where you are heading, so that I can work-out what makes sense and what doesn't, from your point of view.


It looks like something I wrote got lost... I was wondering if the native part of VoltDB would lend itself well to running inside a "graphic card" (assuming the data would fit in)? Massive parallelism seems ideal when working with tabular data.


We've been asked about using GPUs before and have done a bit of thinking about it. In my view, there are two big reasons not to get into GPUs for a while:


1. Our memory access pattern is random across gigabytes of data, and is often bottlenecked on memory access. Since most GPUs have less than 2gb for the foreseeable future, a large number of accesses will need to be fetched from main memory. It would be difficult to keep a GPU from being quite starved for data. In certain cases that involved scans or aggregates, it might make sense, but that's only a very small portion of our workload.


2. Developing for GPUs is much more work than developing for CPUs. If VoltDB has X developer-weeks for each release, we still get a lot more bang for the buck making general improvements to VoltDB, rather than specialized performance increases. We don't believe performance is holding back much VoltDB adoption.