Forum: Building VoltDB Applications

Post: Can voltDB accept Multiple Catalogs ?

Can voltDB accept Multiple Catalogs ?
shetty_ritesh
Jul 20, 2010
Hi,

Usualy in any major project the distribution of projects in form of jars is highly cohesive and loosely coupled. All like minded classes are assenbled in a jar. Hence there can be mutiple catalog jars for a large project which would like to mantain this king of modularity.

It would be great if VoltDB can accept multiple catalogs during startup.

Ritesh
Would love more info
jhugg
Jul 20, 2010
Hi Ritesh,

VoltDB application catalog jars are a little different than regular Java jarfiles, though there is some overlap of use. Some good things to know:

1. They contain lots of information that wouldn't normally go in a jar, like SQL schema, SQL compiled plans and much of the metadata required to run a VoltDB application.
2. They are never added to the server's classpath (at least they shouldn't be since 1.0). Classes inside them are loaded manually and carefully managed by the catalog versioning system we've developed (and are continuing to develop) for VoltDB.
3. You can put other jarfiles in the server's classpath and call code inside them from your stored procedures.

So the primary limitations of application catalog jars are:
1. All of your stored procedure classes need to be in the same jar. Since VoltDB is designed to know your workload in advance, it makes sense that the VoltDB compiler would require the complete set of stored procedures.
2. If you are using supplemental jars containing code referenced by your stored procedures, we don't presently have a good way to replace/upgrade/modify those without restarting VoltDB.
If that makes sense, can you explain which of the limitations is causing issues for you, or if there is a third limitation I'm missing? If you could describe a use case with more detail where multiple jars would be beneficial that would help us as well. We'll take a look and see if we can figure out a way to solve your problem, or if there is something we need to adjust in the product.

Thanks Ritesh,
-John
mutiple Catalog
shetty_ritesh
Jul 21, 2010
Hi Ritesh,

VoltDB application catalog jars are a little different than regular Java jarfiles, though there is some overlap of use. Some good things to know:...

Thanks Ritesh,
-John


The jar i am talking about has just the catalog info like all procedures/sql's project.xml. It has just the right set of components needed by voltdb.

Now comming back to why i need mutiple catalogs to be accepted.

Picture this. Lets say i am in a huge project having multiple features. I would definitely club the project.xml , .sql files of like minded functionalities in a seperate jar. Now lets say 1 need to deploy these onto voltdb. To do that i will have to club all these unrelated jars together.

So instead of accepting a argument catalog mycatalog.jar you can have something like catalog mycatalog1.jar ,mycatalog2.jar ,mycatalog3.jar etc

This way i can isolate my features.
Interesting
jhugg
Jul 21, 2010
The jar i am talking about has just the catalog info like all procedures/sql's project.xml. It has just the right set of components needed by voltdb.

Now comming back to why i need mutiple catalogs to be accepted...

This way i can isolate my features.


I'm not sure what the right answer is to solve this problem, but I think I understand what you're asking for in terms of functionality.

I've created a ticket so we can address the issue in the future: https://issues.voltdb.com/browse/ENG-658. I don't think it's something we plan to tackle in 2010. I hope that's ok.
Mutiple catalog a real requirement
shetty_ritesh
Oct 20, 2010
I'm not sure what the right answer is to solve this problem, but I think I understand what you're asking for in terms of functionality.

I've created a ticket so we can address the issue in the future: https://issues.voltdb.com/browse/ENG-658. I don't think it's something we plan to tackle in 2010. I hope that's ok.


Hi,

As i predicted we now have a situation where 2 projects (going forward it would have mutiple projects) are working on voltdb.
The code bases are totally different. But as a company we need to share the infrastructure(cluster of Voldb servers)

My team has catalog A and other team has catalog B. Currently i have to manually merge the catalog components like classes and project.xml to make it work. If the deployment.xml can accept the names of mutiple catalogs then both teams can work independently.

I would request you to take this up in your next release.

Ritesh
re: Can voltDB accept Multiple Catalogs ?
tcallaghan
Oct 21, 2010
Hi,

As i predicted we now have a situation where 2 projects (going forward it would have mutiple projects) are working on voltdb.
The code bases are totally different. But as a company we need to share the infrastructure(cluster of Voldb servers)...

Ritesh
Ritesh,

I'm going to contact you via email to discuss. I want to fully understand the request and will post the results here.

-Tim
bindas45
Oct 20, 2014
Greetings,

I know this is an old thread but just following up to see if we have any update on this?

Thanks in advance
jpiekos
Oct 20, 2014
In v5.0 we're eliminating the catalog. You will be able to start an empty database and create schema interactively, alter table, etc. Look for a beta by end of November.

John