Forum: Building VoltDB Clients

Post: Load Balancing

Load Balancing
steves
Apr 30, 2011
I'm new to VoltDB and very interested in using the system for a new application that will require heavy writes (many client connections small data), and heavy reads (many connections small data).
In the documentation it recommends "opening connections on all nodes to balance the load." Can you clarify how this helps balance load/ how load balancing works?
I hope the desired stored procedure isn't called N times where N is the number of nodes in the cluster.
Why wouldn't a (hardware) load balancer work? -- where a single IP receives the request and selects the least busy VoltDB node which then decides which node(s)/partitions are needed to respond to the query. I'm also hoping that the requesting client code doesn't need to know where the data is (which node/partition). From what I've read the system automatically handles query routing as long as we call the appropriate procedure. Please confirm.
Thanks!
Steve.
Load balancing
rbetts
Apr 30, 2011
Hi Steve,
Why wouldn't a (hardware) load balancer work? -- where a single IP receives the request and selects the least busy VoltDB node which then decides which node(s)/partitions are needed to respond to the query.
This will work as expected. The important part is to spread the transaction initiation load across the servers in the VoltDB cluster. Also, you want to multiplex lots of transactions across the same client-server TCP/IP connection to avoid TCP/IP connection overhead.
From what I've read the system automatically handles query routing as long as we call the appropriate procedure. Please confirm.
Yes - the system routes internally transparently to the end user.
Ryan.
Load Balancing.
steves
Apr 30, 2011
Hi Steve,
Why wouldn't a (hardware) load balancer work? -- where a single IP receives the request and selects the least busy VoltDB node which then decides which node(s)/partitions are needed to respond to the query.
This will work as expected. The important part is to spread the transaction initiation load across the servers in the VoltDB cluster. Also, you want to multiplex lots of transactions across the same client-server TCP/IP connection to avoid TCP/IP connection overhead.
From what I've read the system automatically handles query routing as long as we call the appropriate procedure. Please confirm.
Yes - the system routes internally transparently to the end user.
Ryan.


Thanks Ryan.
Good to hear that spreading initiation will be easy if I use a load balancer.
Are there any code examples on the site for multiplexing transactions on the same TCP/IP connection?
Thanks.
Steve.
Load Balancing
rbetts
Apr 30, 2011
Thanks Ryan.
Good to hear that spreading initiation will be easy if I use a load balancer.
Are there any code examples on the site for multiplexing transactions on the same TCP/IP connection?
Thanks.
Steve.


The examples that use the VoltDB Java client (or the CPP client) all maintain persistent connections to the server and round-robin procedure invocations across that connection pool.
We don't have example configuration for maintaining a persistent connection pool from a LB to the cluster, unfortunately.
Client accessing a VoltDB cluster
smurtyvol
Jul 29, 2011
Thanks Ryan.
Good to hear that spreading initiation will be easy if I use a load balancer.
Are there any code examples on the site for multiplexing transactions on the same TCP/IP connection?
Thanks.
Steve.


Hi,
I have been searching the docs for how a client should access the cluster and how load balancing of requests works - to better understand the parallelism benefits.
Specifically here's my question:
Assume a VoltDB cluster with several nodes. The best parallelism is achieved if a client "knows" the partition that can handle the query and submits it directly there. (While this may seem ridiculous - no location transparency - there are systems that do this w/o the client programmer having to code anything and achieve a great level of parallelism). The next best thing may be to send the query to any node and have the node figure out where the query can be handled (since it has access to the server project & deployment config) and dispatch the query there. Is this the automatic routing you referred to ?
Any light you can throw on the following?
1. In a cluster, how does a client decide which node to send the query to?
2. How are requests dispatched to the right node in a cluster?
thanks for your help
regards
sudarshan
Leave it to the server
sebc
Jul 29, 2011
Hi,
I have been searching the docs for how a client should access the cluster and how load balancing of requests works - to better understand the parallelism benefits.
Specifically here's my question:
Assume a VoltDB cluster with several nodes. The best parallelism is achieved if a client "knows" the partition that can handle the query and submits it directly there. (While this may seem ridiculous - no location transparency - there are systems that do this w/o the client programmer having to code anything and achieve a great level of parallelism). The next best thing may be to send the query to any node and have the node figure out where the query can be handled (since it has access to the server project & deployment config) and dispatch the query there. Is this the automatic routing you referred to ?
Any light you can throw on the following?
1. In a cluster, how does a client decide which node to send the query to?
2. How are requests dispatched to the right node in a cluster?
thanks for your help
regards
sudarshan


Hey,
We tried a few things - ultimately the servers will communicate with each other to decide which exact node should process a specific transaction, so bringing that logic the the client to try to preempt the decision will not help you as far as performance goes, or so minimally it certainly isn't worth the hair-splitting effort.
So... If you have a multi-node cluster, simply connect to all nodes and let the client library round-robin your requests (this is the answer to #1: the client library will simply round-robin requests to each node). Once the requests reaches a node, that node will communicate with the others to decide (based on K-safety and the partition target deduced from your partitioning key when you called the procedure), which specific sites (partitions on a specific node/list of nodes) will have to execute the transaction (that's the answer to #2).
Hope that helps,
Cheers,
Seb
Leave it to the server
sebc
Jul 29, 2011
Hi,
I have been searching the docs for how a client should access the cluster and how load balancing of requests works - to better understand the parallelism benefits.
Specifically here's my question:
Assume a VoltDB cluster with several nodes. The best parallelism is achieved if a client "knows" the partition that can handle the query and submits it directly there. (While this may seem ridiculous - no location transparency - there are systems that do this w/o the client programmer having to code anything and achieve a great level of parallelism). The next best thing may be to send the query to any node and have the node figure out where the query can be handled (since it has access to the server project & deployment config) and dispatch the query there. Is this the automatic routing you referred to ?
Any light you can throw on the following?
1. In a cluster, how does a client decide which node to send the query to?
2. How are requests dispatched to the right node in a cluster?
thanks for your help
regards
sudarshan


Hey,
We tried a few things - ultimately the servers will communicate with each other to decide which exact node should process a specific transaction, so bringing that logic the the client to try to preempt the decision will not help you as far as performance goes, or so minimally it certainly isn't worth the hair-splitting effort.
So... If you have a multi-node cluster, simply connect to all nodes and let the client library round-robin your requests (this is the answer to #1: the client library will simply round-robin requests to each node). Once the requests reaches a node, that node will communicate with the others to decide (based on K-safety and the partition target deduced from your partitioning key when you called the procedure), which specific sites (partitions on a specific node/list of nodes) will have to execute the transaction (that's the answer to #2).
Hope that helps,
Cheers,
Seb
Interesting suggestion
sebc
Aug 2, 2011
Hi Seb,
Thanks for the reply.
#3. What happens in this scenario ?
A server node picks up the request and routes it to the appropriate node that will do the processing.
Say the response to a SELECT query returns 'several' rows.
Is the response routed back to the client through the messenger node?
Why am I asking this?
In a multi node configuration, the request will almost always land on a messenger node (~ wrong node).
If the response to #3 is yes, I assume there is a benefit to skipping the intermediate 'messenger' network hop.
If no, you can ignore the lines below.
Can you comment on this alternative approach?
The same logic you use on the server to dispatch a request can be used by the client library.
If no connection to a node exists, fall back to messenger node approach.
Then most requests will land on the correct node.
Yes this simplistic approach assumes that each client will open connections to almost all nodes - which is bad.
For deployments with large # nodes and/or clients, multiple client concentrators can funnel clients requests.
Each client concentrator is 'partition aware' (alternative client lib).
While this adds the extra hop back, there are advantages to it.
thanks
sudarshan


Sudarshan,
Well, the response will come back to you on the same TCP socket, right (protocol would be a fair bit more complex otherwise), but it is certainly an interesting approach that could limit inter-node communication; bit like with Torrent where, while you request the file from the seeder, you'd get a fragment response from a client much closer to you/faster.
I'll suggest it to our team for consideration.
Cheers,
Seb
Client partition affinity
rbetts
Aug 2, 2011
Hi Seb,
Thanks for the reply.
#3. What happens in this scenario ?
A server node picks up the request and routes it to the appropriate node that will do the processing.
Say the response to a SELECT query returns 'several' rows.
Is the response routed back to the client through the messenger node?
Why am I asking this?
In a multi node configuration, the request will almost always land on a messenger node (~ wrong node).
If the response to #3 is yes, I assume there is a benefit to skipping the intermediate 'messenger' network hop.
If no, you can ignore the lines below.
Can you comment on this alternative approach?
The same logic you use on the server to dispatch a request can be used by the client library.
If no connection to a node exists, fall back to messenger node approach.
Then most requests will land on the correct node.
Yes this simplistic approach assumes that each client will open connections to almost all nodes - which is bad.
For deployments with large # nodes and/or clients, multiple client concentrators can funnel clients requests.
Each client concentrator is 'partition aware' (alternative client lib).
While this adds the extra hop back, there are advantages to it.
thanks
sudarshan


We have considered client affinity at some length - the client can hypothetically calculate the partition-key hash and route directly to a server that has that partition locally.
However, in a replicated cluster (the common case), that receiving server still must broadcast the initiation to and receive responses from the k replicas. There are other internal messages relayed for initiation ordering as well.
The messaging savings isn't as great as it might initially appear to be once replication is considered.