Forum: Managing VoltDB

Post: House Cleaning

House Cleaning
tony7889
Aug 5, 2011
Hi,
I am planning to do the house cleaning for my voltdb, there are some ways to do it with traditional RDBMS, such as truncate partition for historical data.

But for voltdb, there is no transaction log or journal right? so I have to just delete records from the table? but I am wondering there are billions records across physical servers with partitions, it shall cause blocking to the entire VoltDB cluster right? What can I do to avoid this? any suggested practice?

Thanks,
Tony
A strategy
jhugg
Aug 7, 2011
This is perhaps less straightforward in VoltDB as in traditional RDBMSs, but there are some things that can be done.
While restarting the database can be a quick way to reset all tables, that not usually what people want to do.

Usually what we do is write a stored procedure that removes a chunk of tuples from a table, say, the oldest 10000 tuples. Then we call that procedure repeatedly until the desired number of tuples is achieved. Some users have a external trigger that calls this "nibbler" procedure whenever the tuple counts or memory usage gets to high. It usually calls it repeatedly until the condition is remedied.

This can be done with a multi-partition procedure, but a more performant way to do this is to write a single partition procedure and send to all partitions. The way to do this is to determine the number of partitions (hosts * sites-per-host / redundancy-level), then make a partition parameter to the proc that has the values 0 through N-1, where N is the number of partitions. This will allow you to target stored procedures at specific partitions.

We're aware this is more complicated than it should be and this is an area we are working hard to streamline in future versions of VoltDB.
clean up on 2.0
tony7889
Sep 15, 2011
This is perhaps less straightforward in VoltDB as in traditional RDBMSs, but there are some things that can be done.


Hi jhugg,

I just installed the Voltdb 2.0, I found that restarting the voltdb cannot reset the tables, all records stay the same (actually it is good).

but the problem is when I issue "TRUNCATE TABLE" it cost me unreasonable time.
I got 3 site with 3 partition per site and the table is around 200k records only. and it cost me 20 mins to do the truncate. it seems 2 much for in memory db.

is there any way I can fasten the clean up?

Thanks,
Tony
Startup options
rbetts
Sep 15, 2011
Hi jhugg,

I just installed the Voltdb 2.0, I found that restarting the voltdb cannot reset the tables, all records stay the same (actually it is good).

but the problem is when I issue "TRUNCATE TABLE" it cost me unreasonable time.
I got 3 site with 3 partition per site and the table is around 200k records only. and it cost me 20 mins to do the truncate. it seems 2 much for in memory db.

is there any way I can fasten the clean up?

Thanks,
Tony


Tony,

There are actually a few options when starting the cluster - you can instruct the database to start with empty tables by using the "create" startup action. This is a bit new - and is described in the documentation here:

http://community.voltdb.com/docs/UsingVoltDB/RunRestart

Does this answer your question?

*--Ryan.
Hi Ryan, OK it is half of
tony7889
Sep 17, 2011
Tony,

There are actually a few options when starting the cluster - you can instruct the database to start with empty tables by using the "create" startup action. This is a bit new - and is described in the documentation here:

http://community.voltdb.com/docs/UsingVoltDB/RunRestart

Does this answer your question?

*--Ryan.


Hi Ryan,

OK it is half of what I need to know thanks.

But I am thinking why TRUNCATE TABLE takes so long. and when it is truncating the table, all transactions is locked out from the Voltdb.

Thanks,
Tony
Tony, Can you tell me
rbetts
Sep 29, 2011
Hi Ryan,

OK it is half of what I need to know thanks.

But I am thinking why TRUNCATE TABLE takes so long. and when it is truncating the table, all transactions is locked out from the Voltdb.

Thanks,
Tony


Tony,

Can you tell me specifically how you are truncating the table?

Thank you,
Ryan.
Тематические статейки делают
racogalra
Aug 14, 2012
Тематические статейки делают этот портальчик разнообразным, где можно много чего почерпнуть.