⬅ Forum: VoltDB Architecture

Post: Will the partition adapt to the change of workload?

Will the partition adapt to the change of workload?

beilei sun

Jan 13, 2017

Hi, I have seen the discussion of https://forum.voltdb.com/showthread.php?301-How-does-partitioning-work and checked the source code of TheHashinator.java.

It seems that each new coming row is written to a partition that is determined by a given hash algorithm.

However, if some partition is overloaded, e.g., some some rows of this partition are read frequently in the last interval, will VoltDB migrate some hot data to the other partitions dynamically?

I have read the paper presented in VLDB 2016, "Clay: FineGrained Adaptive Partitioning for General Database Schemas", which propose an adaptive partitioning strategy. Will Clay be used in VoltDB?

Thanks.

rmorgenstein

Jan 14, 2017

VoltDB does not have any ability to migrate hot data. The enterprise product does contain elastic add which allows you to add new nodes. This will migrate data from existing nodes to the new nodes, but is not based on trying to balance hot keys.

That paper is definitely interesting. Rebecca came by and gave a talk on it, but I don't know of any plans to implement this.

Ruth