Forum: Building VoltDB Applications

Post: Is there a limit on number of columns and size of column name

Is there a limit on number of columns and size of column name
bindas45
Oct 28, 2014
I have to copy an existing data dump into voltdb and as per requirement it has 300+ columns and over 4gb of data.
And some of the columns names are 45 characters.

Whenever I try to compile the ddl I get this exception.
I tried multiple names and thought there are some special characters but always getting this exception.

Please advise and Thanks in advance

 voltdb compile ddl.sql
ERROR: [null:15]: DDL Error: "unknown token" in statement starting on lineno: 15


Line 15 is just
CREATE TABLE TEST (
bballard
Oct 28, 2014
Can you provide the rest of the SQL statement? Line number 15 is just the line that begins the statement, but the error could be anywhere in the statement.
bindas45
Oct 29, 2014
Can you provide the rest of the SQL statement? Line number 15 is just the line that begins the statement, but the error could be anywhere in the statement.


You are right, the problem is elsewhere. I was able to resolve.

However I have to upload a 5gb file using csvloader. But the date is in different format than volt acceptable yyyy-mm-dd format.

Please advise a workaround as opening the 5gb file in excel is not an option.

Thanks in advance
vtkstef
Oct 29, 2014
Hi,

if you could include a few of the lines of the CSV file you need to load, then we may be able to whip something up to make your dates acceptable to VoltDB

Ciao
Stefano
bindas45
Oct 30, 2014
Hi,

if you could include a few of the lines of the CSV file you need to load, then we may be able to whip something up to make your dates acceptable to VoltDB

Ciao
Stefano


Below is the format for my date fileds in csv. Please let me know if you need additional details.

Thanks in advance

Provider date Last Update Date
5/23/2005 10/13/2014
5/23/2005 10/13/2014
6/2/2005 10/13/2014
6/10/2005 10/13/2014
5/31/2005 10/13/2014
6/1/2005 10/13/2014
vtkstef
Oct 30, 2014
Hi,

is it TAB, space, or comma separated? I also need samples of full loader files rows.

Ciao
Stefano
bindas45
Oct 30, 2014
Hi,

is it TAB, space, or comma separated? I also need samples of full loader files rows.

Ciao.

Stefano


Its a comma separated csv file. I have made a sample of it.
Please let me know if any additional details needed

Thanks
vtkstef
Oct 30, 2014
try the following command


perl -i.orig -F, -lane 'map {$_ .= " 00:00:00.000"} @F[2,3] if($.>1); print join(",",@F);' sample.csv


it will save the original csv file to sample.csv.orig

Ciao
Stefano
rmorgenstein
Oct 30, 2014
By the way, if you end up needing more complicated transforms in the future, you can write a stored procedure and call csvloader with the '-p procname' switch. The only downside, is that calling a procedure turns off the batching.

Ruth