Forum: Managing VoltDB

Post: WARN: Joining primary failed: Connection refused retrying..

WARN: Joining primary failed: Connection refused retrying..
pavangadiya
May 26, 2015
Hello,

I am trying to make a cluster of 2 machines(nodes) in voltdb. I am using the community edition of Voltdb. Here is the process which I followed:

1. I have put "2nodedeploy.xml(i.e.my customized deployment.xml)" file in /opt/voltdb-master/bin/voltdbroot location in both the machines(nodes).
The structure of my deployment.xml is like that:

<?xml version="1.0"?>
<deployment>
<cluster hostcount="2"
sitesperhost="4"
/>
<httpd enabled="true">
<jsonapi enabled="true" />
</httpd>
</deployment>


2. After that, I have given the command on the master node(say leader or node1) like that:

voltdb create --deployment=/opt/voltdb-master/bin/voltdbroot/2nodedeploy.xml

After this command the following process happened.

Initializing VoltDB...

_ __ ____ ____ ____
| | / /___ / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __ |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/

--------------------------------

Build: 5.1 This is not from a known repository Community Edition
Connecting to VoltDB cluster as the leader...
Host id of this node is: 0



3. Then after I gave the command on the other machine (say slave or just peer)in which like that:

voltdb create --deployment=/opt/voltdb-master/bin/voltdbroot/2nodedeploy.xml --host=node1 host name

Then,I am getting the following error :

Initializing VoltDB...

_ __ ____ ____ ____
| | / /___ / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __ |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/

--------------------------------

Build: 5.3 This is not from a known repository Community Edition
Connecting to the VoltDB cluster leader "node1 name"/"ip-address":"port"
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..
WARN: Joining primary failed: Connection refused retrying..


So, Please tell me how to resolve this "Connection refused" problem ??

Basically, I want to connect the "cluster of 2 machines" with my java program to perform some database related operations. But, before that I am getting the problem like that.

So, Please guide me to resolve this problem.

Thank You!

Regards,
Pavan
pzhao
May 26, 2015
Pavan,

Generally, a connection refused problem occurs when target port is not open/active. I just want to clarify that your hostname should be one word/ip and not space separated as you've indicated:
voltdb create --deployment=/opt/voltdb-master/bin/voltdbroot/2nodedeploy.xml --host=peters-machine

Added, if you are running in AWS, you may need to specify internal/external hosts. Please see here for more information.

Let me know if this helps you.

Peter Zhao
pzhao
May 26, 2015
Pavan,

Can you try specifying the host on the first server as well? Let me know if this worked.

Peter Zhao
pavangadiya
May 27, 2015
Hello Pzhao,

Thank you for the quick reply.

Yes, I am giving the host name as a one word only (not separately,I indicated the hostname wrongly..sorry for the typos)

Now, I have specified the hostname on the first server as well like that.

But, the problem of node version incompatibility is coming..

Here is the following process:

On NODE1 (leader)

voltdb create --deployment=/opt/voltdb-master/bin/voltdbroot/2nodedeploy.xml --host=node1hostname
Initializing VoltDB...

_ __ ____ ____ ____
| | / /___ / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __ |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/

--------------------------------

Build: 5.3 This is not from a known repository Community Edition
Connecting to VoltDB cluster as the leader...
Host id of this node is: 0


Then on NODE2

voltdb create --deployment=/opt/voltdb-master/bin/voltdbroot/2nodedeploy.xml --host=node1hostname
Initializing VoltDB...

_ __ ____ ____ ____
| | / /___ / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __ |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/

--------------------------------

Build: 5.1 This is not from a known repository Community Edition
Connecting to the VoltDB cluster leader "node1hostname"/"ip-address":"port"
FATAL: Cluster contains nodes running VoltDB version 5.3 which is incompatbile with local version 5.1.

VoltDB has encountered an unrecoverable error and is exiting.
The log may contain additional information.


& then on NODE1 , these exceptions are coming.

ERROR: Error joining new node
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:197)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at org.voltcore.messaging.HostMessenger.requestJoin(HostMessenger.java:538)
at org.voltcore.messaging.SocketJoiner.processSSC(SocketJoiner.java:379)
at org.voltcore.messaging.SocketJoiner.runPrimary(SocketJoiner.java:405)
at org.voltcore.messaging.SocketJoiner.access$000(SocketJoiner.java:64)
at org.voltcore.messaging.SocketJoiner$2.run(SocketJoiner.java:191)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.voltcore.utils.CoreUtils$7$1.run(CoreUtils.java:735)
at java.lang.Thread.run(Thread.java:745)
FATAL:
FATAL: Fatal exception
java.lang.UnsupportedOperationException
at com.google_voltpatches.common.collect.ImmutableCollection.add(ImmutableCollection.java:96)
at org.voltcore.messaging.HostMessenger.requestJoin(HostMessenger.java:555)
at org.voltcore.messaging.SocketJoiner.processSSC(SocketJoiner.java:379)
at org.voltcore.messaging.SocketJoiner.runPrimary(SocketJoiner.java:405)
at org.voltcore.messaging.SocketJoiner.access$000(SocketJoiner.java:64)
at org.voltcore.messaging.SocketJoiner$2.run(SocketJoiner.java:191)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at org.voltcore.utils.CoreUtils$7$1.run(CoreUtils.java:735)
at java.lang.Thread.run(Thread.java:745)
VoltDB has encountered an unrecoverable error and is exiting.
The log may contain additional information.


So, problem is now not having the same version is on both the nodes.
Now, I am trying to update the version on the other node. Please tell me the quickest way to update the version :) ? I am using the community edition.

Thank you!

Regards,
Pavan
pzhao
May 27, 2015
Pavan,

You can always start over on both servers:
remove both all voltdb directories
git clone https://github.com/VoltDB/voltdb.git
cd voltdb; ant

Once you clean up and use the same version on all servers, you should be all set.

Peter Zhao
pavangadiya
May 28, 2015
Hello Peter,

Thanks for resolving my queries! :)

Anyways, I have already installed the latest Voltdb community edition on the other node also(Without removing the previous directories & clean up). Now, I have the same versions on both the nodes & it's working fine.

Well, Could you please suggest me the example of "Java program using native(VoltDB) API for Insert Query (Insert Statement to insert data into table) Using Cluster of 2 nodes" ?

Your help will be highly appreciated.

Thank You.


Regards,
Pavan
bballard
May 28, 2015
Hi Pavan,

The Tutorial is a good place to start for an example. You can also run the examples in the "examples" folder in the kit, or some of the example apps we have on Github, which are listed on our web site here:
http://voltdb.com/community/applications

I recently wrote a very simple application that I use as a template, you might try this too:
https://github.com/benjaminballard/app-template

Regards,
Ben
pavangadiya
May 29, 2015
Hi Ben,

Thanks a lot!

I will use your recommended links to start.

Regards,
Pavan