⬅ Forum: VoltDB Architecture

Post: command logging in community edition

command logging in community edition

rk1

Apr 8, 2012

Can we enable command logging in community edition of voltdb ? or any other feature that captures transactions between snapshots.

In our case we will have memory foot print of at least 2 TB with k-safety of 1. This would take good time for the snapshots as it has to write to disks. I am guessing that minimum interval between snapshots would be around 30 minutes or so.

How do we recover the transactions that happened after the last snapshot upto the time voltdb crashed.

Thanks
RK

Command Logging only in Enterprise Edition

jpiekos

Apr 8, 2012

Hi RK,

Command Logging is an Enterprise Edition feature and isn't available in the Open Source edition.

If full durability between snapshots is important to you (it sounds like it is) consider upgrading to the Enterprise Edition.

Thanks,
John Piekos

Some more information on snapshots

aweisberg

Apr 8, 2012

Hi,

+1 to what John said.

It's unlikely that the minimum interval between snapshots will be 30 minutes.

Snapshots get faster as you add nodes because each node only snapshots the data it has locally so the thing you have measure is how long it takes to snapshot a single node that is loaded to capacity. How much IO throughput is available also matters. Before compression we were able to write 240 megabytes/sec of raw data and we can probably do that amount in compressed data as well.

By default snapshots are throttled so that they don't impact latency on a heavily loaded cluster. See "Automated Snapshots" under HostConfigDBOpts. Set snapshot priority 0 to find out what the max throughput is for the disk you have available. How you have configured command logging and whether snapshots and command logging share a disk will effect the throughput you see.

You also have to keep in mind that snapshots are compressed and don't contain secondary indexes as well as indirection overhead from storing data in memory so they will usually be smaller then the RSS of the node although by how much depends on the schema and data.

-Ariel

Thanks John & Ariel.

rk1

Apr 9, 2012

Thanks John & Ariel.