Forum: VoltDB Architecture

Post: Two phase commit has no ack from participant to coordinator?

Two phase commit has no ack from participant to coordinator?
Aug 9, 2014
I' study the distributed commit protocol of voltdb.
After tracing the system, I find that the MpSite would coordinate the distributed transaction. For a multi-partition transaction, it would be done as the following picture, is that? Here, I don't find an ack message from the participant to coordinator at the end of the transaction's lifetime. What if the participant site did not receive the CompleteTransactionMessage, the participant site would never get the transaction done. ?
Aug 11, 2014
The picture looks basically correct. First, we're talking about cross-partition or multi-partition transactions, and we're talking about those that may modify state. There are different semantics for reads and for single-partition transactions.

Essentially, the participant site waits (blocks) for the receipt of the CompleteTransactionMessage (which may mean commit or rollback). If it's not received, that implies one of three things:
1. The MpSite has failed.
2. The transaction is running, but is very slow at one or more participants.
3. A bug in VoltDB was hit.

In case 1, the participant will block until the failure detection subsystem notifies it that the node containing the active MpSite is no longer part of the cluster. At that point the participant will go through another process to figure out if some participants got the CompleteTransactionMessage or not. It will then either commit or rollback in concert will all surviving participants. Note we rely on ordering guarantees in our messaging layer that we won't drop an individual message in a stream.

In case 2, there isn't a ton we can do today. An example of this is a transaction that does complex processing on billions or rows, or worse, an infinite loop in stored procedure logic. We are working on some things to mitigate these cases, but they illustrate why testing out your app in a preproduction environment is valuable.

In case 3, we try to test our commit protocols using complex fuzzed tests. For one example, see This one app is designed to be as intertwined as possible. We run it for hours while killing and restarting nodes as well as killing and restoring the cluster. Still, if you find a bug, let us know; we take commit protocol bugs very seriously. We tout our ACID-serializable semantics as a VoltDB advantage, so we care a lot to get it right.