Forum: Other

Post: Voter benchmark

Voter benchmark
Nasser
Nov 26, 2012
Hello everyone..

I'm benchmarking voltdb using the voter example that comes with it. However, I was analyzing the results when scaling voltdb up (increase the number of partitions) on my single node machine with 4 cores and got the following results.

Throuput:
# of partitions / Throughput(TPS)
1 / 23170
2 / 38662
3 / 49575
4 / 67412
5 / 76551
6 / 74724
7 / 72389
8 / 72339
9 / 71978
10 / 71001
Latency:
# of partitions / ms
1 / 6.08
2 / 7.17
3 / 6.34
4 / 8.09
5 / 210
6 / 218
7 / 226
8 / 221
9 / 226
10 / 229

my question is: why did I reach the peak in the throughput with 5 partitions while I reached the peak with latency with 4 partitions?

According to voltdb documentations "the optimal number being approximately 3/4 of the number of CPUs" so with my 4 cores machine I should get the best results with 3 partitions but the best was with 5!!!

P.S No hyper threading in my machine
Thank you...
It's not exact
jhugg
Nov 26, 2012
That line about the optimal number is a rough guide, and could be off by a bit, depending on the workload and the specific machine. For a different benchmark, the ideal number could be 3 partitions on the same hardware. Voter in 2.8.x is usually bottlenecked on transaction ordering, so adding more partitions may not slow you down, as you've noticed. You might see different results with the 3.0 beta (or maybe not). The best thing to do (if you have time) is the exact thing you did, which is try some different numbers.

As for the latency, this could simply be an imperfection in the auto-tuning mechanism the benchmark code uses. It tries to find the highest throughput possible with internal cluster latency less than 6ms. The internal latency is measured time from accepting a request to sending off a response, and doesn't include the client round trip time. The auto-tune settings are all configurable in the run.sh file under the async-benchmark function. You can even turn the auto-tune off and set a fixed limit. Setting a fixed limit below a known max should give decent latency measurements.
Thanks for the reply. "You
Nasser
Nov 30, 2012
That line about the optimal number is a rough guide, and could be off by a bit, depending on the workload and the specific machine. For a different benchmark, the ideal number could be 3 partitions on the same hardware. Voter in 2.8.x is usually bottlenecked on transaction ordering, so adding more partitions may not slow you down, as you've noticed. You might see different results with the 3.0 beta (or maybe not). The best thing to do (if you have time) is the exact thing you did, which is try some different numbers...


Thanks for the reply.

"You can even turn the auto-tune off and set a fixed limit" what do you mean by a fixed limit and where can i set that?

Thanks again.
In run.sh
jhugg
Dec 3, 2012
"You can even turn the auto-tune off and set a fixed limit" what do you mean by a fixed limit and where can i set that?


In the run.sh file, look for the function called "async-benchmark". Set "autotune" to "false", and set "ratelimit" to whatever number you want (in transactions per second).