Results 1 to 9 of 9

Thread: Command log

  1. #1
    New Member
    Join Date
    Sep 2013
    Posts
    5

    Command log

    I am new to VoltDB. I have read http://forum.voltdb.com/showthread.p...light=Recovery which describe the protocol of snapshot. I want to know whether VoltDB keeps the consistency if it uses command log. If it is, does the command log also written by a sync protocol or it just flushes to disk locally? I also want to know how it keep the consistency in tech details.

    Thanks

  2. #2
    Super Moderator
    Join Date
    Dec 2011
    Posts
    72
    VoltDB maintains consistency when using command logging. It is written locally at each node based on the partitioning scheme and the level of k-safety. So, for example, if you are running command log in synchronous mode on a cluster with k=1, if you issue a command to insert a record into a partitioned table, the transaction is routed to the 2 servers that store the affected partition, where the command will also be written to the command log. When the client receives a successful (committed) response, you are guaranteed that the command was executed and logged on both servers, so it is fully consistent and durable. You can read more technical details about command logging here (http://voltdb.com/docs/UsingVoltDB/ChapCmdLog.php).

  3. #3
    Super Moderator
    Join Date
    Feb 2010
    Posts
    186
    Quote Originally Posted by shadowdragon View Post
    I am new to VoltDB. I have read http://forum.voltdb.com/showthread.p...light=Recovery which describe the protocol of snapshot. I want to know whether VoltDB keeps the consistency if it uses command log. If it is, does the command log also written by a sync protocol or it just flushes to disk locally? I also want to know how it keep the consistency in tech details.

    Thanks
    In addition to Ben's reply, you can find some additional detail about command log implementation on the blog:http://blog.voltdb.com/tag/command-logging/. Please refer to the documentation as the official source of configuration help, though.

    Thanks,
    Ryan.

  4. #4
    New Member
    Join Date
    Sep 2013
    Posts
    5
    Thanks for the reply. Based on the reply, I want to know the following case can happen or not. Let's say we use command logging in asynchronous way. There are two transactions, transaction 1 and transaction 2. Transaction 1 is depended on transaction 2. Transaction 1's command log and Transaction 2's command log are stored in a different group of nodes. Because the command log is flushed to the disk asynchronously and locally, Transaction 2's command log may go to disk first. At this time. Then the system crash before Transaction 1's command log go to disk. When recovery, there is only Transaction 2's command log. Is it possible?

  5. #5
    New Member
    Join Date
    Sep 2013
    Posts
    5
    Thanks. These pages are useful for me. But I still have some confusion as I described above.

  6. #6
    Super Moderator
    Join Date
    Feb 2010
    Posts
    186
    Quote Originally Posted by shadowdragon View Post
    Thanks for the reply. Based on the reply, I want to know the following case can happen or not. Let's say we use command logging in asynchronous way. There are two transactions, transaction 1 and transaction 2. Transaction 1 is depended on transaction 2. Transaction 1's command log and Transaction 2's command log are stored in a different group of nodes. Because the command log is flushed to the disk asynchronously and locally, Transaction 2's command log may go to disk first. At this time. Then the system crash before Transaction 1's command log go to disk. When recovery, there is only Transaction 2's command log. Is it possible?
    First - you can run synchronous command logging to guarantee that every transaction is fsync'd to each replica's disk before a reply is generated to the client. That avoids this question entirely.

    Re: async command logging:

    1. In the case that Transaction 1 (Tx1) and Transaction 2 (Tx2) are single partition transactions for the same partition, they will replay in the order Tx1, Tx2. Either may be lost (not fsync'd), but their order will never be commuted.

    2. In the case that Tx1 and Tx2 are both multi-partition transactions, same as above.

    3. In the case that Tx1 and Tx2 are for different partitions, or that one is a single partition transaction and the other is a multi-partition transaction, it would be possible to replay Tx2 without Tx1. But see my first point about using synchronous command logging..

    Ryan.

  7. #7
    New Member
    Join Date
    Sep 2013
    Posts
    5
    Hi Ryan, one more question. I am curious where VoltDB store the command log. Is it store in all the partitions one transaction touches? I ask the question because it is not clear to me for the second case of async command logging. In this case, do you mean Tx1 and Tx2 are both multi-partition transactions and touch the same partitions also? Because if they touch different partitions, I don't know why they can be the same as the first case.

  8. #8
    Super Moderator
    Join Date
    Feb 2010
    Posts
    186
    Quote Originally Posted by shadowdragon View Post
    Hi Ryan, one more question. I am curious where VoltDB store the command log. Is it store in all the partitions one transaction touches? I ask the question because it is not clear to me for the second case of async command logging. In this case, do you mean Tx1 and Tx2 are both multi-partition transactions and touch the same partitions also? Because if they touch different partitions, I don't know why they can be the same as the first case.
    For single partition work, VoltDB writes orders the log by partition and then writes it to disk (with some added complexity to be fast). A mental model of thinking of it as per-partition at every replica is probably sufficient.

    For multi-partition work, VoltDB nominates one partition to be the logger of all MP work. All replicas of that partition log the multi-partition work. There is some additional logging at each partition to capture the serializable point of the multi-part work (like a bookmark, basically).

    Ryan.

  9. #9
    New Member
    Join Date
    Sep 2013
    Posts
    5
    Thanks. I see.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •