Forum: VoltDB Architecture

Post: Performance limiting around 100Mbps

Performance limiting around 100Mbps
MichaelGG
Mar 9, 2011
Hi,


I'm running some simple trials of VoltDB and running into an odd performance problem.


All the server nodes are CentOS 5.5 x64, generic Intel server boxes (Intel 1000/PRO EB NICs) with 2x Xeon 5150 (4 cores total per box). I've turned off all offload features, disabled I/OAT in the BIOS, and turned off TCP window scaling. This is 3 node VoltDB cluster. Clock skew is reported as 9ms.


The test is very simple. A single stored procedure that executes these two SQL statements:
INSERT INTO Calls (AccountId, FromNum, ToNum, CreatedOn, EndedOn, Duration, Price) VALUES (?, ?, ?, ?, ?, ?, ?)
UPDATE Accounts SET Balance = Balance - ? WHERE AccountId = ?


Both tables are partitioned on Accounts.AccountId, so it's a single-sited transaction. I restart VoltDB each test, so there's no data in either table (yes, the second SQL statement updates nothing).


No matter what configuration I try or number of clients I can't seem to get above 100Kps. It maxes out around 3 clients; I've run up to 8 clients with little improvement (.NET lib, round-robin LB to all nodes). Kfactor 0 versus 1 doesn't seem to make much of a difference in tps (maybe 10K), just in intra-node network transfer and CPU usage. With lots of clients and kfactor 1, CPU in Java is around 220%; with kfactor 0 its around 180%.


What bothers me is that in all the configurations I've tried, I never see network traffic pass 100Mbps. It's always under 11.7MB/sec. All clients and servers are on the same gig switch (C3560G). SCP'ing a file from one to the other hits 50MB/sec (~400Mbps), so I know the NICs and switch are functioning at 1G.


Should I expect VoltDB to scale up beyond these numbers? Is the problem likely to be in my NICs somehow, or am I falsely seeing a bottleneck on bandwidth? How should I go about inspecting this?


Thanks,
Michael
Actually, this might not be
MichaelGG
Mar 10, 2011
Actually, this might not be an issue after all. The NICs are definitely working and can sustain 950Mbps. Comparing my numbers to the SGI benchmark, they rather line up, considering The Xeon 5150 is rather outdated now.
re: Performance limiting around 100Mbps
tcallaghan
Mar 14, 2011
Actually, this might not be an issue after all. The NICs are definitely working and can sustain 950Mbps. Comparing my numbers to the SGI benchmark, they rather line up, considering The Xeon 5150 is rather outdated now.


Michael,


Thanks for following up on your own post. The Xeon 5150 CPUs have been around for a while. You'll see a significant performance boost with any Nehalem based Xeons (55xx or 56xx).


-Tim