Forum: VoltDB Architecture

Post: Could BLOBs/large varchar be stored externally?

Could BLOBs/large varchar be stored externally?
monster
Sep 25, 2010
If an application would be "agreeable" to VoltDB "limitations", but would also require BLOBs/large varchar which would be too expensive to be stored in memory (for example images), how difficult would it be to stored them externally? Either in files or some other type of database. The problem with handling this in the application logic, is that you then get distributed transactions, and I don't think VoltDB support them yet. Having some kind of transparent swap to external storage would simplify the client code.
VoltDB is strictly designed
rbetts
Sep 28, 2010
VoltDB is strictly designed as a high-throughput OLTP transactional store. We haven't developed provisions for external storage. Of course, nothing is preventing a user from building or re-using existing out-of-band storage for images and tracking locations of those associated images in their VoltDB store. In fact, VoltDB could be an interesting solution to indexing a large mass-storage system in some applications.


In VoltDB, all transactions are exactly one stored procedure invocation. VoltDB supports procedure invocations (transactions) that involve multiple VoltDB hosts. In these terms, VoltDB supports distributed transactions.


Transparent swap to external storage sounds a lot like a buffer cache. Part of VoltDB's throughput advantage over other transactional OLTP engines is its elimination of a buffer cache. It was suggested in early discussions about VoltDB that building in some kind of storage pre-fetch function / MRU style caching would be an interesting extension of capability. But in its current form, VoltDB is an in-memory store.