Volt vs Postgres
Jun 10, 2013
I've been tasked with exploring VoltDB for work, where we hope to use it to help us do ad-hoc queries on the daily logs from our applications.
To see if it was worth further investigation I loaded both VoltDB and Postgres up with a fraction (approximately 2.7GB) of our log data, which was in CSV format.
All tests done on my laptop, a quad-core macbook pro with 8GB of RAM.
I used the csvloader provided with VoltDB, it took a little over 50 minutes to load :(
I used the COPY functionality of Postgres, it took less than 3 minutes.
"SELECT COUNT (*) from logs"
VoltDB: 14,960,452 rows, couldn't figure out how to time query execution (seemed fast, less than a second)
Postgres : 14,960,452 rows, ~2000ms
"SELECT SUM(price) from logs"
Postgres: 4342552.21367791, ~5000ms
VoltDB: VOLTDB ERROR: SQL ERROR
More than 100 MB of temp table memory used while executing SQL. Aborting.
at org.voltdb.sysprocs.AdHoc_RO_SP.run(AdHoc_RO_SP.java:53)
:(
1) Is there a way to load the data into VoltDB faster? csvloader is an order of magnitude away from good enough.
2) What do I have to do to make summing a column work? For a 2.7GB dataset on a machine with 8GB of RAM, I expect this to work out of the box.
We were very excited when first researching VoltDB, but this first hands-on experience with it has been very disappointing. Am I doing things the worst way possible or what?
To see if it was worth further investigation I loaded both VoltDB and Postgres up with a fraction (approximately 2.7GB) of our log data, which was in CSV format.
All tests done on my laptop, a quad-core macbook pro with 8GB of RAM.
I used the csvloader provided with VoltDB, it took a little over 50 minutes to load :(
I used the COPY functionality of Postgres, it took less than 3 minutes.
"SELECT COUNT (*) from logs"
VoltDB: 14,960,452 rows, couldn't figure out how to time query execution (seemed fast, less than a second)
Postgres : 14,960,452 rows, ~2000ms
"SELECT SUM(price) from logs"
Postgres: 4342552.21367791, ~5000ms
VoltDB: VOLTDB ERROR: SQL ERROR
More than 100 MB of temp table memory used while executing SQL. Aborting.
at org.voltdb.sysprocs.AdHoc_RO_SP.run(AdHoc_RO_SP.java:53)
:(
1) Is there a way to load the data into VoltDB faster? csvloader is an order of magnitude away from good enough.
2) What do I have to do to make summing a column work? For a 2.7GB dataset on a machine with 8GB of RAM, I expect this to work out of the box.
We were very excited when first researching VoltDB, but this first hands-on experience with it has been very disappointing. Am I doing things the worst way possible or what?