Forum: Building VoltDB Applications

Post: Monitoring task

Monitoring task
hayatoa
Dec 6, 2011
Hi,

How do I monitor which tasks/sqls are being processed by "each partition" and which tasks/sqls are in waiting list for "each partition" in VoltDB?

Is there command to monitor?
Hi Hayatoa, VoltDB currently
nshi
Dec 7, 2011
Hi Hayatoa,

VoltDB currently doesn't have a way for the user to monitor the transaction queue depth for each partition. But there are other statistics that may help.

You can monitor the outstanding transactions for live clients by calling the @Statistics system procedure with the selector "liveclients". You can also look at the initiator and the procedure statistics for the number of completed transactions. These are in the Using VoltDB manual, which can be found here sysprocstatistics .

If you are using the Java client library, there are also some methods that help you get related statistics. The getIOStats() and the getProcedureStats() method in the Java client library will give you the number of completed transactions for this client.
Any write miss?
hayatoa
Dec 7, 2011
Hi Hayatoa,

VoltDB currently doesn't have a way for the user to monitor the transaction queue depth for each partition. But there are other statistics that may help...


Thank you!

Ok, I see that currently voltdb does not have a way to monitor transaction queue for each partition.

If I understand correctly, then we cannot check exact transaction queue for voltdb as a whole or each partition.

Using @statistics(LIVECLIENTS), we can see how much work is left in the execution queues.

Then, let me ask in different way.

If I tries to insert data much more than voltdb can handle, what is going to happen? (voltdb may be busy with other query or maybe insert is just too much for voltdb)

Database will crash, or output message to log saying "voltdb was not able to handle this write and that write"?

Will voltdb ignore some insert so that write miss will occurs?
Monitoring and Capacity planning
rbetts
Dec 8, 2011
Thank you!

Ok, I see that currently voltdb does not have a way to monitor transaction queue for each partition.

If I understand correctly, then we cannot check exact transaction queue for voltdb as a whole or each partition...


Ok, I see that currently voltdb does not have a way to monitor transaction queue for each partition.

We expose the pipeline depth per client. We also expose a starvation statistic that roughly measures what percentage of time an execution site is running SQL. (You can also infer this from general CPU utilization). Additionally, we track how many microseconds a procedure takes to execute at a partition.

We could perhaps add an average-per-partition-pipeline depth when each new transaction arrives. In normal cases, though, that would be a fixed proportion of the live clients statistic.

Monitoring the queue more directly is difficult - it changes much faster than the network latency of a monitoring poll.

If I tries to insert data much more than voltdb can handle, what is going to happen? (voltdb may be busy with other query or maybe insert is just too much for voltdb)

I don't understand the parenthetical, "busy with other query." However, if you exceed the memory capacity of a node, you will crash out of memory and will have to restore from the command log and/or snapshots the system creates. We suggest that production users disable OS swap when running VoltDB. VoltDB will not function appropriately if the OS starts swapping it to disk. You get a much clearer / faster fault with swap disabled.

This is an application sizing problem - all databases must be correctly sized whether using disk or memory.

Ryan