Forum: Building VoltDB Applications

Post: Return type of stored procedures

Return type of stored procedures
chbussler
Apr 13, 2010
Hi,

when I started implementing stored procedures, I felt that the return type of either long or VoltTable[] is rather limiting from an invoker's viewpoint when you want to distinguish between an ok result and an error result. Right now, anything I'd like to return I have to do in terms of either a long, or as data inside a Volt table.


I was thinking that a return type of a VoltResult, that internally has a VoltTable[] and maybe in addition a dynamic structure (like e.g. a hashmap) would allow me to distinguish returning results in terms of result data from returning error codes due to error situations in the stored procedure's logic. The invoker of the stored procedure could then check for an error code in the hashmap (or whatever the data structure is), and if ok, then can inspect the VoltTable (or not). I suggested hashmap to make it extensible/flexible; there are other options, of course.


This is just a thought. What I could do with such a VoltResult structure is that I could make the result processing uniform in the sense that in my application I could establish the convention that every time the same check is executed to determine if the stored procedure was successful or not (I could factor that out to a single test function). A second function could retrieve the error reason from VoltResult, and a third function could retrieve the data from VoltTable, if necessary.


A whole alternative could be to allow me to add my own exceptions to stored procedures. Then I could throw those from within the stored procedure and the caller could act accordingly. But I am not sure if this would be possible at all.


Thanks,

Christoph
Return type of stored procedures
aweisberg
Apr 13, 2010
Hi Christoph,

Currently a ClientResponse contains an error code set by the system. It is set to SUCCESS (1) if the procedure successfully returns. It is set to USER_ABORT (-1) if the procedure throws a VoltAbortException. Procedure authors are free to instantiate and throw VoltAbortExceptions . It is set to GRACEFUL_FAILURE (-2) if Volt throws an exception for a graceful failure like a constraint violation and UNEXPECTED_FAILURE (-3) for an error in Volt. If a procedure throws an exception it does not have the opportunity to return a long or VoltTable array. Client responses also contain an "extra" string containing human readable error messages generated by the system and mangled formatting of exceptions. Unfortunately the message in thrown exception is mangled before being returned so it is not suitable for passing an error string or code.

I think that dynamic structures like Maps that accept arbitrary objects as well as extensible exceptions are not likely to happen because of the serialization challenges.

I agree that the mechanism for returning results and failures needs to be more flexible. I have created a ticket and scheduled it for V1. Some additions I propose:
1. Ability to return a human readable string (separate from existing extra string) with successful invocations
2. Ability to return an result code with each result table as well as a result code for the procedure invocation
3. Ability to throw an exception with an unmangled message, result code, and table set with codes

-Ariel
ClientResponse available in async call only
chbussler
Apr 13, 2010
Hi Christoph,

Currently a ClientResponse contains an error code set by the system. It is set to SUCCESS (1) if the procedure successfully returns. It is set to USER_ABORT (-1) if the procedure throws a VoltAbortException. Procedure authors are free to instantiate and throw VoltAbortExceptions . It is set to GRACEFUL_FAILURE (-2) if Volt throws an exception for a graceful failure like a constraint violation and UNEXPECTED_FAILURE (-3) for an error in Volt. If a procedure throws an exception it does not have the opportunity to return a long or VoltTable array. Client responses also contain an "extra" string containing human readable error messages generated by the system and mangled formatting of exceptions. Unfortunately the message in thrown exception is mangled before being returned so it is not suitable for passing an error string or code....




Hi,

thanks! It seems that ClientResponse is only available in the asynchronous mode. That's fine - I did not 'discover' that as I am not yet working with asynchronous invocations.

I am not sure if it is possible, but it would be very nice if the interface would be the same (aside from the direct vs. the callback approach). Meaning, the data structure given back from VoltDB to the caller would be the same - in the synchronous call it would be the return type, in the asynchronous case it would be a parameter.

Again, just thinking aloud.

Thanks!
Christoph
Hi, With the synchronous
aweisberg
Apr 14, 2010
Hi,

thanks! It seems that ClientResponse is only available in the asynchronous mode. That's fine - I did not 'discover' that as I am not yet working with asynchronous invocations.

I am not sure if it is possible, but it would be very nice if the interface would be the same (aside from the direct vs. the callback approach). Meaning, the data structure given back from VoltDB to the caller would be the same - in the synchronous call it would be the return type, in the asynchronous case it would be a parameter.

Again, just thinking aloud.

Thanks!
Christoph


Hi,

With the synchronous interface you will get a ProcCallException with the message containing the extra string as well as a serialized exception (if the exception was serializable) as the cause. Unfortunately VoltAbortException is not serializable so it won't be sent back.

We might change them to be consistent at some point, but it is one of those break the world kind of changes since all our test code and example apps tend to use the synchronous interface.

It is a little kludgy, but the SynchronousCallback class is public and that is what the implementation uses to do synchronous invocations. It has a getResponse() method that returns the ClientResponse.

-Ariel
To clarify
aweisberg
Apr 16, 2010
Hi,

With the synchronous interface you will get a ProcCallException with the message containing the extra string as well as a serialized exception (if the exception was serializable) as the cause. Unfortunately VoltAbortException is not serializable so it won't be sent back.

We might change them to be consistent at some point, but it is one of those break the world kind of changes since all our test code and example apps tend to use the synchronous interface.

It is a little kludgy, but the SynchronousCallback class is public and that is what the implementation uses to do synchronous invocations. It has a getResponse() method that returns the ClientResponse.

-Ariel


Hi,

The tests and sample apps I refer to are the units tests of which there are hundreds and sample apps like hello world and auction (which has Thread.sleep in it). Benchmarks all use the asynchronous interface to generate load.

-Ariel
When glancing over the API
eribeiro
Apr 25, 2010
When glancing over the API javadocs, I was puzzled by the fact that many methods return arrays. Why don't you return an immutable collection (list, set) instead?

I know an array has the same effect as a collection when you are interacting over the result items in a loop, but with a collection you may:

1) have nice high level methods like .isEmpty(), first(), last(); and
2) make the api contract pretty clear about the immutability of the results;
3) allow for ready-to-use and high quality libs built on top of collections like google-collections (now guava libs).

A customized immutable set can be quite performatic, and have minimal memory impact. If you feel interested then take a look at google-collections' ImmutableSet and ImmutableList.

Another thing that made me a bit uncomfortable was the VoltTableRow having a dual nature (represents both a row in a table and an iterator for all the rows in the table). When VoltDB goes 1.0, this may cause misunderstandings among application devs and make it harder to debug client apps, in my humble opinion.

My naive suggestion would be to create a VoltRowRange class that implements Iterable and holds an immutable collection of VoltTableRows. I think this class could retrieve the rows eagerly or lazying according to VoltDB inner workings. Then you could move methods like resetRowPosition(), getRowCount(), advanceRow() for this new class. Furthermore, this would allow for cool method renaming like the following:

resetRowPosition() --> reset()
getRowCount() --> size()
advanceRow() --> next()
advanceToRow(int rowIndex) --> seek(int rowIndex)
getRowStart() --> first();

This new class would be retrieve from VoltTable class too. By dividing concerns, if the dev wants a single row she calls fetchRow(int index) and gets an VoltTableRow, that represents an unique row, but if she wants a set of rows then she should call fetchRange() or fetchRange(int from, int to) and get VoltRowRange instead.

Best regards,
Edward
Thanks for the feedback!
jhugg
Apr 26, 2010
When glancing over the API javadocs, I was puzzled by the fact that many methods return arrays. Why don't you return an immutable collection (list, set) instead?

I know an array has the same effect as a collection when you are interacting over the result items in a loop, but with a collection you may:

1) have nice high level methods like .isEmpty(), first(), last(); and
2) make the api contract pretty clear about the immutability of the results;
3) allow for ready-to-use and high quality libs built on top of collections like google-collections (now guava libs).

A customized immutable set can be quite performatic, and have minimal memory impact. If you feel interested then take a look at google-collections' ImmutableSet and ImmutableList...

Best regards,
Edward


Hi Edward,

It's great to get your perspective on our API. While the API will probably only receive minor changes before our 1.0 release, we agree there's room for improvement and we're thinking about how it might be improved for subsequent releases. Community feedback is a welcome part of that process.

Customized immutable sets sound like something to explore. I think arrays were chosen as they offer a nice mix of performance and familiarity to all, but there may be places where immutable sets make more sense.

The VoltTable is a VoltTableRow confusion is something we acknowledge. Since VoltTable is used ubiquitously throughout performance critical paths in VoltDB, the API must be performance-oriented. Given that, we opted to try to make the API as compact as possible, both in terms of numbers of methods and in its usage, even if it's less intuitively put together than we'd like. I think in future revisions, there's room for much improvement. In the meantime, we're waiting for more thoughtful feedback like yours from developers really using the API to determine what works and what doesn't. Thanks again.

-John Hugg
VoltDB Engineering