Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: C++ client with multiple nodes

  1. #1

    Question C++ client with multiple nodes

    Hi,

    In the readme file of the cpp client the V4 release notes says: 'When connected to multiple nodes in the cluster, the client library will enable client affinity and can auto-reconnect to restarted nodes.'.

    I wanted to try out these features but I kinda stuck. I was using VoltDB v5.6 and cpp client v5.2. My problems are:

    • If I use the createConnection() method it is establishing the connections to the nodes. Then if a set the client affinity true, it is working, but I have to set it manually. From the release notes I assumed that it will happen automatically. I was trying to find the actual code in the client's source which do some automatic client affinity settings but I probably was not thorough. Maybe I misunderstood the feature?
    • The createConnection() method needs a running/initialized VoltDB server when it is called and based on my observations it does not do any auto-reconnecting. If I use the createPendingConnection() method it will wait until the VoltDB starts up at the given host and can auto-reconnect to it but if I call it e.g. twice for two nodes then my program either aborts or the client throws an OverflowUnderflowException. What am I missing?


    Any advice would be greatly appreciated,
    Thanks,
    Roland
    Last edited by RolandM; 10-06-2015 at 01:46 PM.

  2. #2
    New Member
    Join Date
    Apr 2015
    Posts
    13
    Hi.

    I'm Bill White, an engineer at VoltDB. I'd like to clarify your first point, and ask for details about the second point.

    1. You appear to be right about the default setting for client affinity. It seems as if client affinity is disabled by default. There was a lot of churn in this area going from version 4 to version 5, so perhaps this got lost. It doesn't seem to be in the test suite, but I may have missed it.
    2. I'm looking into the differences between Client::createConnection and Client::createPendingConnection. In particular, I'm trying to figure out how these two do reconnection. I had thought the former is exactly like the latter, except that it waits for the connection to be established, but I'm not completely certain now.
    3. I want to make sure about your second question. You try to create two pending connections to two separate hosts with no database running, then start the database on those hosts. Sometime after the databases are initialized the client applications crash, sometimes with an OverflowUnderflowException and sometimes with an abort. Are the connections in different threads? The C++ client is not thread safe, so if they are in different threads of the same Unix process hilarious things, like exceptions, may easily occur. Are the connections in the same or different Unix processes? Do you have a stack frame? I will try to reproduce this with a couple different scenarios.

      I think your expectation is that two client connections in two different Unix processes, or two client connections in the same thread should be able to connect to two different hosts reliably. Is that right? That sounds completely reasonable to me.


    Thanks for the note. I'll generate a trouble ticket about the documentation.

  3. #3
    Hi Bill,

    Thank you for the quick response.

    Looking at the code again you are probably right. The createConnection() method is also using the PendingConnection object for the initialization. It seems I made some hasty assumptions about their behavior. I will run my tests again and I will get back with the results.

    Note: You get my scenario right. I am having a VoltDB cluster with two nodes and my single threaded application is establishing the (right now not pending) connections to each of these nodes. Then I run test cases like:
    • No database running at the given addresses when the application starts
    • The given addressess are not accessible
    • Only one of the given address is accessible
    • One of the node restarts after the connection is established


    Thanks again,
    Roland

  4. #4
    Hi,

    I managed to re-run some tests and make some debugging.

    So in my code I wrote the following:

    client.createConnection(host1,21212);client.create Connection(host2,21212); //hostx is an IP address (std::string)
    If VoltDB is initialized it is working fine.

    If I replace that with:
    client.createPendingConnection(host1,21212);client .createPendingConnection(host2,21212); then I got a 'voltdb::OverflowUnderflowException' even if VoltDB is initialized.

    #0 0x00007ffff505b5d7 in raise () from /lib64/libc.so.6
    #1 0x00007ffff505ccc8 in abort () from /lib64/libc.so.6
    #2 0x00007ffff595f9b5 in __gnu_cxx::__verbose_terminate_handler() () from /lib64/libstdc++.so.6
    #3 0x00007ffff595d926 in ?? () from /lib64/libstdc++.so.6
    #4 0x00007ffff595d953 in std::terminate() () from /lib64/libstdc++.so.6
    #5 0x00007ffff595db73 in __cxa_throw () from /lib64/libstdc++.so.6
    #6 0x00007ffff6cc0c95 in voltdb::ByteBuffer::checkGetPutIndex (this=<optimized out>, this=<optimized out>, length=<optimized out>) at include/ByteBuffer.hpp:104
    #7 0x00007ffff6cc9d17 in checkGetPutIndex (this=<optimized out>, this=<optimized out>, length=<optimized out>) at include/ByteBuffer.hpp:266
    #8 getByReference (length=<optimized out>, this=0x7fffffffe0c0) at include/ByteBuffer.hpp:435
    #9 voltdb::ByteBuffer::getString (this=this@entry=0x7fffffffe0c0, wasNull=@0x7fffffffe0a0: false) at include/ByteBuffer.hpp:264
    #10 0x00007ffff6cc842e in AuthenticationResponse (buf=..., this=0x7fffffffe0f0) at include/AuthenticationResponse.hpp:43
    #11 voltdb::authenticationReadCallback (bev=0x65c7a0, ctx=0x6531c0) at src/ClientImpl.cpp:139
    #12 0x00007ffff6cfd662 in bufferevent_readcb (fd=23, event=<optimized out>, arg=0x65c7a0) at bufferevent_sock.c:183
    #13 0x00007ffff6cf557d in event_process_active_single_queue (activeq=<optimized out>, base=<optimized out>) at event.c:1340
    #14 event_process_active (base=<optimized out>) at event.c:1407
    #15 event_base_loop (base=0x652880, flags=2) at event.c:1604
    #16 0x00007ffff6cc3916 in voltdb::ClientImpl::runOnce (this=0x652540) at src/ClientImpl.cpp:742
    #17 0x00007ffff6cc1651 in voltdb::Client::runOnce (this=<optimized out>) at src/Client.cpp:83

    Thanks,
    Roland

  5. #5
    New Member
    Join Date
    Apr 2015
    Posts
    13

    Sounds like a bug to me.

    Thanks. That certainly sounds like a bug. The client should not crash like this, or any way really.

    I'll submit a ticket and we can track progress on this.

    I'll also try to reproduce it here and see if I can shed any more light on it.

    Thanks again.

  6. #6
    New Member
    Join Date
    Apr 2015
    Posts
    13
    Can you please tell me the versions of Voltdb and the C++ client you are using? Also, is it possible for you to send me your reproducer code?

    We have one particular customer who uses the C++ client quite extensively, and has done what you are trying to do. It would be interesting to us to see what you are doing differently.

    Thanks.

  7. #7
    New Member
    Join Date
    Apr 2015
    Posts
    13
    Sorry, I just saw the version numbers in the original posting.

    It would still help us if you could post the code and the VDB application code which is causing the error.

  8. #8
    My actual code is:

    int main(int argc, char* argv[])
    {
    std::vector<std::string> hosts;
    for (auto i = 1; i < argc; ++i)
    hosts.push_back(argv[i]);

    auto client=voltdb::Client::create();

    for (const auto& host : hosts)
    client.createPendingConnection(host,21212);

    while (!_terminate) {
    /*
    ...
    ...
    handling some incoming traffic
    ...
    ...
    */
    client.runOnce();
    }

    while (!client.drain()) {}

    return 0;
    }


    Thanks in advance.
    Last edited by RolandM; 10-15-2015 at 10:07 AM.

  9. #9
    Note: The code above is flawless with Client::createConnection

  10. #10
    New Member
    Join Date
    Apr 2015
    Posts
    13
    Thanks. I'll try this out.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •