Forum: Building VoltDB Applications

Post: Cluster with VoltDB community Edition

Cluster with VoltDB community Edition
Raveendra
Aug 12, 2011
Hi,
I am new to the VoltDB. I am very much interested in this. I am able to run the sample example of voltdb by downloading community edition.

Now I want to know is cluster going to work on VoltDB Community Edition?

I know that we need to configure the number of hosts in the deployment.xml.
But where do I mention the all the nodes' IP Address which are in cluster.

Please provide one sample for this.

Thanks and regards
Ravee
No need: you only need to indicate the "leader" for startup
sebc
Aug 12, 2011
Hi Ravee,

There is no need for you to list out all your nodes anywhere: you only need to point to a single "leader" node that's used at startup to connect all your nodes together. The leader isn't a "master" or even much of a "leader" in truth: in VoltDB, all nodes are peers, for purposes of reliability (you wouldn't want a single point of failure by having a critical server node in your data center!). What the leader does is, at startup, serve as a central dispatcher/communicator to connect all the nodes together. After the startup has completed, all nodes communicate with all other nodes in your cluster and the notion of "leader" loses its meaning.

The only place you would want to list the DNS/IP addresses of your nodes is the client application, so that your client can round-robin requests between the nodes (and gain some resilience in case one of the nodes should fail). For the examples, you can modify this in the build.xml, generally in the ant target named "client" (or some such name) - the reference that indicates "localhost" is what you want to change: simply list your servers in CSV, for instance: 192.168.1.201,192.168.1.202

Oh, and yes, the Community Edition will let you create clusters and has just about all the features of the Enterprise edition (except the visual manager application, and some disaster recovery options for critical shops coming up with the next major release of VoltDB).

Keep in mind that we also have a full team of technical consultants whose sole purpose is to help companies get started quickly by providing architecture and coding support. In a very short time, they can create a solid proof of concept application to demonstrate how VoltDB could deal with your specific problems. Learning VoltDB is like learning how to drive a racecar, so it might take a little time - ifm you just want to win the race, you'll want to grab one of our professional drivers (if you'll forgive me the funny imagery! ;-))

Hope this helps,

Cheers,
Seb
Hi Seb, Thanks for your
Raveendra
Aug 15, 2011
Hi Ravee,

There is no need for you to list out all your nodes anywhere: you only need to point to a single "leader" node that's used at startup to connect all your nodes together. The leader isn't a "master" or even much of a "leader" in truth: in VoltDB, all nodes are peers, for purposes of reliability (you wouldn't want a single point of failure by having a critical server node in your data center!). What the leader does is, at startup, serve as a central dispatcher/communicator to connect all the nodes together. After the startup has completed, all nodes communicate with all other nodes in your cluster and the notion of "leader" loses its meaning.
..


Hi Seb,

Thanks for your reply.
Let me try. If I face problem, will come back to you.

Ravee
Certainly - keep me posted as
sebc
Aug 16, 2011
Hi Seb,

Thanks for your reply.
Let me try. If I face problem, will come back to you.

Ravee


Certainly - keep me posted as to your progress!
Cheers,
Seb
Hi Seb, Still I am not able
Raveendra
Aug 17, 2011
Hi Seb,

Thanks for your reply.
Let me try. If I face problem, will come back to you.

Ravee


Hi Seb,

Still I am not able to run VoltDB in the cluster environment. Do I need to mention the IP address of nodes in deployment.xml like below:

deployment
cluster hostcount="1" sitesperhost="1" leader="localhost, 10.20.30.41, 10.20.30.51"
or else is there any other configuration?


Waiting for your reply.

Thanks and regards
Ravee
Hi Seb, I was looking at your
Raveendra
Aug 17, 2011
Hi Seb,

Still I am not able to run VoltDB in the cluster environment. Do I need to mention the IP address of nodes in deployment.xml like below:

deployment
cluster hostcount="1" sitesperhost="1" leader="localhost, 10.20.30.41, 10.20.30.51"
or else is there any other configuration?



Hi Seb,

I was looking at your reply in the link http://community.voltdb.com/node/472
In this post you are mentioned that

"Keep in mind that your catalog and deployment files need to be exactly identical on all your nodes for a cluster deployment. Indeed, one easy way to go about it is to use a NFS mount, or have a replication system in place to make sure this is always the case)."

So in my client application I need to create connection for each node like:

ClientConfig clientConfig = new ClientConfig("program", "none");
org.voltdb.client.Client voltclient=org.voltdb.client.ClientFactory.createClient(clientConfig);
voltclient.createConnection("leaderIP");
voltclient.createConnection("node1IP");
voltclient.createConnection("node2IP");


or else need to put into deployment.xml with comma separated IP addresses?
And also while staring the cluster do I need to start all the nodes first then start the leader.

Help me in this.

Thanks and regards
Ravee
Deployment: same on all nodes; client connecting to all nodes (b
sebc
Aug 17, 2011
Hi Seb,

I was looking at your reply in the link http://community.voltdb.com/node/472
In this post you are mentioned that


Hey Ravee,
Yes, the deployment should be the same on all node and in all cases only have ONE IP or Name in the "leader" attribute.

Your client can connect to one or several nodes, then, either as you indeed describe, or, if you look at the samples in /example, by providing a comma-separated list of names and/or IPs in build.xml (not deployment.xml)
Does that make more sense?
Seb
Leader: 1 Name/IP only
sebc
Aug 17, 2011
Hey,
You should only have 1 name or IP address in that leader specification - not a comma separated list of servers. For instance:

<cluster hostcount="1" sitesperhost="1" leader="10.20.30.51" ... />

Notes:
- As soon as you have more than 1 node, you can't use "locahost" for the leader (each node would believe itself to be the leader!)
- You have to be able to access all nodes by both name and IP address. For instance, suppose 10.20.30.51 is serverA, then you need to get an answer whether you do 'ping serverA' or 'ping 10.20.30.51'. More to the point: each node needs to be able to ping all nodes in the cluster by both name and IP. If you do not have DNS support, you can remedy this by maintaining an IP/Name mape in your nodes 'hosts' file

Hope this helps,
Seb
Cluster and Skew error
Raveendra
Aug 19, 2011
Hey,
You should only have 1 name or IP address in that leader specification - not a comma separated list of servers. For instance:

<cluster hostcount="1" sitesperhost="1" leader="10.20.30.51" ... />

..


Hi Seb,


I am able to configure the cluster enviroment. What I did is, built the one catalog file (only one IP in deployment.xml's leader attribute) and put this catalog file on 2 nodes. In one text file I put all my nodes' IP and in my client application I coded like below:

ClientConfig clientConfig = new ClientConfig("program", "none");
org.voltdb.client.Client voltclient = org.voltdb.client.ClientFactory.createClient(clientConfig);
String[] nodeIPArray = nodeIPs.split(",");
for (String thisServer : nodeIPArray) {
try {
thisServer = thisServer.trim();
voltclient.createConnection(thisServer);
} catch (IOException e) {
System.exit(-1);
}
}

Let me know is this the correct way of clustering in VoltDB?

And now while starting the VoltDB on 2 nodes it is giving me the skew error:

[java] 11/08/19 10:22:23 INFO HOST: Maximum clock/network skew is 23300 milliseconds (according to leader)
[java] 11/08/19 10:22:23 ERROR HOST: Maximum clock/network is 23300% higher than allowable limit

I tried to synchronize the time between two nodes using ntpdate -ud node1IP in node2 machine. But I am not successful.

May be it is out of VoltDB question, can you help me how to synchronize the time between two different machines.

We are using machine with fedora (Linux LICDR2INFOBLOX02 2.6.35.6-45.fc14.x86_64 #1 SMP Mon Oct 18 23:57:44 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux)

Waiting for your reply

Thanks and regards
Ravee
Ntp
rbetts
Aug 19, 2011
Hi Seb,

I am able to configure the cluster enviroment. What I did is, built the one catalog file (only one IP in deployment.xml's leader attribute) and put this catalog file on 2 nodes. In one text file I put all my nodes' IP and in my client application I coded like below:


Volt does require that the clocks on your cluster nodes are roughly in sync. 23 seconds of clock drift is pretty severe!

I think your ntpdate command will work if you omit the "-d". My ntpdate man page says, "-d Enable the debugging mode, in which ntpdate will go through all the steps, but not adjust the local clock."

Ariel wrote a nice blog post about the ntp config we use here: http://www.afewmoreamps.com/2011/07/configuring-ntp-for-voltdb.html

Or, you might this EC2 post interesting if you are running virtualized nodes: http://support.rightscale.com/06-FAQs/FAQ_0025_-_Can_I_use_Network_Time_Protocol_(NTP)_on_my_RightScale_servers%3F

Ryan.
Shanky
Aug 30, 2013
Hi ,

I am running application in cluster.

If i shutdown the leader which i have taken then the other nodes are also getting shutdown

Reply ASAP
jpiekos
Aug 30, 2013
Hi, yes, if you do not have k-safety enabled (high availability - replica copies of each partition) enabled, VoltDB will shut down if it does not have a full copy of data available (due to node failure). If you have k-safety enabled, you can lose at least k nodes from the cluster and still have a complete copy of the data in memory (and thus continue to process transactions). VoltDB is strongly consistent and will not return incorrect/incomplete answers.

I suggest you review our training material for a great overview of VoltDB, found here: http://voltdb.com/volt-university/training.php
It covers these concepts and more.

John
Shanky
Aug 30, 2013
Hi K safety is enabled in my case .

Server 1 is the leader . i start server 1 and then after the server 2.

Now if i shutdown server2 server1 keeps on taking the request but if i shutdown server1 then server2 also goes off.

Can you tell me what am i doing wrong.

cluster host="2" sitesperhost="2" kfactor=1
jpiekos
Aug 30, 2013
Check out the second half of this documentation page on network partition detection: http://voltdb.com/docs/UsingVoltDB/KsafeNetPart.php
You are hitting the 50/50 case, given a 2 node cluster. When one node fails, VoltDB doesn't know if it failed due to a network partition or not, so it has to make a decision to continue, and with an even node count cluster, it has a 50/50 chance of making the correct decision, as your testing has shown. You can either turn off network partition detection (and possibly hit a split-brain scenario), or add another node to your cluster (make it 3 nodes) to avoid this situation. We recommend using odd-node count clusters for small clusters.

John