VoltDB tpc-c-like Benchmark Comparison-Benchmark Description
May 26, 2010
Benchmark Description
The benchmark described here is an OLTP system performance test. It simulates an order-entry environment for a business with multiple warehouses. The transactions include entering and delivering orders, recording payments, checking the status of orders, and monitoring the level of stock at the warehouses.Hardware
Node Hardware
All nodes used for this test had the following identical configuration.- Dell R610 1-Unit Chassis
- 2x 2.66Ghz Quad-Core Xeon 5550.
- 12x 4GB (48GB) DDR3-1333 Registered ECC DIMMs
- PERC6/i SAS Controller with 256mb Battery-Backed Cache (set to "Write Back")
- 3x 72gb 15K RPM 2.5in Enterprise SAS 6Gbps Drives
- 2x in RAID0 for table/index data
- 1x in standalone for log data
- 2x in RAID0 for table/index data
- Broadcom Gigabit NIC
Node Software
- Centos Linux 5.4
- Sun Java JDK/JRE 1.6u19
- RDBMS1
- MySQL 5.5m3
- VoltDB 1.0.01
Switch
- Dell PowerConnect 6248 48 port Gigabit Ethernet Switch
RDBMS1 Configuration
- 4k block size
- 8GB database cache
- Table/Index data on 2x RAID0
- Log Data on 1x standalone
- results were recorded by checking timestamps of rows in the “orders” table (excluding orders from initial warehouse loading)
- first 10 seconds of the run were ignored
- client tested through LAN and directly on server switch
MySQL Configuration
- sync_binlog = 1
- innodb_buffer_pool_size = 8589934592
- innodb_log_file_size = 2040109465
- innodb_flush_method = O_DIRECT
- innodb_thread_concurrency = 0
- innodb_doublewrite = 1
- innodb_flush_lot_at_trx_commit = 1
- Table/Index data on 2x RAID0
- Log Data (binary and InnoDB) on 1x standalone
- results were recorded by checking number of new orders created over two minutes
- client tested through LAN and directly on server switch
RDBMS1 and MySQL
- Wait times were set to 0.
- Transaction type percentages:
- new order 43.48%
- payment 43.48%
- delivery 4.35%
- stock level 4.35%
- order status 4.35%
- new order 43.48%
VoltDB Benchmark
The VoltDB benchmark differs from the official tpc-c benchmark in two significant ways. Operationally, the VoltDB benchmark does not include any wait times, which we feel are no longer relevant. It also does not include fulfillment of orders submitted to one warehouse, with items from another warehouse (approximately 10% of the new order transactions in the official benchmark). Each benchmark was run with 12 warehouses (partitions) per node.The code for the VoltDB benchmark is available here:http://svnmirror.voltdb.com/eng/trunk/tests/frontend/org/voltdb/benchmark/tpcc/
Results
Each benchmark was run several times to ensure system tuning/parameters.RDBMS1 - Total Transactions/second number shown is derived from peak new orders created for a 60 second interval during the run.
Nodes | Warehouses | Clients | Total Transactions / Second |
---|---|---|---|
1 | 3 | 1 | 483 |
1 | 3 | 3 | 1,134 |
1 | 3 | 5 | 1,155 |
1 | 10 | 5 | 911 |
1 | 10 | 10 | 697 |
MySQL - Total Transactions/second number shown derived from total new orders over 2 minute run.
Nodes | Warehouses | Clients | Total Transactions / Second | MySQL Configuration Changes |
---|---|---|---|---|
1 | 1 | 3 | 232 | Base configuration |
1 | 5 | 5 | 208 | Base configuration |
1 | 3 | 3 | 229 | Base configuration |
1 | 3 | 3 | 232 | innodb_doublewrite=0 (potential for partial page writes) |
1 | 3 | 3 | 373 | above plus innodb_flush_log_at_trx_commit=0 (write to log 1x per second, flush to disk 1x per second) |
1 | 3 | 3 | 816 | above plus sync_binlog=0 (number of binary log writes before flushing to disk, 0 = OS control) |
VoltDB - Total Transactions/second number shown is averaged over a three minute run.
Nodes | Warehouses | Clients | Total Transactions / Second |
---|---|---|---|
1 | 12 | 1 | 53,000 |
3 | 36 | 2 | 150,000 |
5 | 60 | 3 | 250,000 |
6 | 72 | 3 | 300,000 |
12 | 144 | 5 | 560,000 |