Forum: Other

Post: High Latency & Database Freezing

High Latency & Database Freezing
jadent
Aug 11, 2011
I've put a up an enterprise voltdb server
* EC2 XL (4 cores)
* CentOS 5.4
* Single server in cluster 3 sites / host

Put i receive very slows inserts averaging around 300 / sec and never over 600

when running through the ent manager i can see latency go through the roof (500 ms and then up to 400,000 ms) and CPU looked taxed so i upgraded to a EC2 High Compute server w/ 8 cores and got the same results

any ideas on where to start trying to figure out what is going on?
More Info
jadent
Aug 11, 2011
i'm sending traffic through separate Apache web server cluster using a PHP script. I'm stressing it using multiple server and Apache AB with 100 concurrent users per server.
I've also tried setting the Insert SP to single & mult-partition w/ the same results
Possible reasons
sebc
Aug 11, 2011
In a non-virtual environment, I would suspect you might be firehosing your server (trying to post too many transactions at the same time; could also be a problem with your partitioning), causing such high latencies. Things are a bit different on EC2 though, but it's hard to tell you what exactly the problem is without a peek at your code. I suspect this is your own application, right? Not one of the samples?

We do have a support process to help you review and tune up your application - reach out to Lyn Vercellone so she can arrange a call with one of our senior technical consultants:

lvercellone@voltdb.com

Cheers,
Seb
i'm using the Helloworld
jadent
Aug 12, 2011
In a non-virtual environment, I would suspect you might be firehosing your server (trying to post too many transactions at the same time; could also be a problem with your partitioning), causing such high latencies. Things are a bit different on EC2 though, but it's hard to tell you what exactly the problem is without a peek at your code. I suspect this is your own application, right? Not one of the samples? ..


i'm using the Helloworld sample app with the helloworld.php script. after getting poor performance i modified the script so it just does 1 insert and thats it (before did 3 inserts, 1 read)

i even ran the voter example as well and got terrible performance as well. i've got a support call setup via lyn so hopefully we'll iron it out


thanks!
Well, Helloworld is not
sebc
Aug 12, 2011
i'm using the Helloworld sample app with the helloworld.php script...


Well, Helloworld is not really a very good example since it's synchronous and just... well, a bit dumb. Voter is usually what I point users to.

We do have a VMWare instance and an EC2 AMI that you could use to run tests on that should give you very reasonable results, even on a laptop (or small AMI instance, respectively). But let's catch up during our support call.

Cheers,
Seb