Announcement

Collapse
No announcement yet.

Passing VoltTable to custom stored procedures

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Passing VoltTable to custom stored procedures

    Hi,

    I'm currently performing lots of insertions to a volt table (partitioned on a game id column) - the rate of insertions is 60k - 100k per sec
    I noticed you can pass a VoltTable into a stored procedure as an input parameter
    I've got several questions regarding this:
    - Whats the use case of passing a VoltTable to a stored procedure (as an input param)?
    - Since i'm inserting data in batches to VoltDB I was wondering if I can convert my batches into Volt tables and then pass the table as an input param for my custom stored procedures - will this improve my insertion rate? If yes, could you explain why?

    I've already looked at this thread regarding passing VoltTables to custom stored procedures but didn't find the answer I was looking for

    Thanks in advance :)

  • #2
    Passing tabular data into VoltDB adds flexibility and enables multiple uses.

    You're right that one of them is batch insertion. For example, our bulk loading low-level procedures, @LoadSinglepartitionTable and @LoadMultipartitionTable, accept VoltTables as input, along with a table name.

    It is possible (often) to achieve a higher insertion rate, because there is less overhead for managing transactions and networking per row. The benefits will vary depending on your workload, schema and hardware, but are sometimes dramatic.

    The trick is to be aware of partitioning. Loading using single-partition procedures is usually faster, but if you send a whole chunk of tuples that don't partition together, you will need to use a multi-partition procedure.

    So you can manually partition on the client side, or you can use our bulk-loading interface which will handle that for you. See: https://docs.voltdb.com/javadoc/java-client-api/ and the getNewBulkLoader method (we actually need to include the right JavaDoc in that link).

    You can also just use a CSV or Kafta loader or something like that, depending on where your data is coming from.

    Comment


    • #3
      I noticed that by using LoadSinglePartition Table, the speed is lower than what I am writing a procedure with Async call procedure. could you tell me why ?

      Comment


      • #4
        Hi Simon, I will answer your question about LoadSinglePartition table on the other thread. Thanks!

        Comment

        Working...
        X