Forum: Building VoltDB Applications

Post: BUG: soft lockup - CPU#3 stuck for 580s![pdflush:46]

BUG: soft lockup - CPU#3 stuck for 580s![pdflush:46]
shetty_ritesh
Apr 23, 2010
I was trying to upload a huge file of around 5 million records.
After around 1 million records my python instance crashed.

Also the client aborted with this error

BUG: soft lockup - CPU#3 stuck for 580s![pdflush:46] [644537.584975]

BUG: soft lockup - CPU#1 stuck for 580s![pdflush:46] [java:29743]

I am not sure if it is a memory issue or not. I will investigate this tommorow. Till then if you have already encountered this problem let me know
re: soft lockup
tcallaghan
Apr 23, 2010
Shetty,

This is not something we've experienced. Can you tar/gz your entire application and input file(s) and email it to

ersupport@voltdb.com?

-Tim
Tim, I just mailed you the
shetty_ritesh
Apr 23, 2010
Shetty,

This is not something we've experienced. Can you tar/gz your entire application and input file(s) and email it to

ersupport@voltdb.com?

-Tim


Tim,
I just mailed you the entire project with a sample file. Let me know where can i upload the huge 5 million record file

Also one more observation. For the first few minutes i could see the records getting inserted into the table(using browser). Then all of a sudden something happened and i saw count(*) =0. The client was still running. And then later it aborted . a

When i check the browser with select count(*) from the table it returns 0 rows . Now i thought due to some reason it rolled back.

Now i tried to run the same file again . This time it gave an error(VOLTDB ERROR CONSTRAINT VIOLATION) that the row already existed.

Regards
Ritesh
This is the error in the logs
shetty_ritesh
Apr 23, 2010
Tim,
I just mailed you the entire project with a sample file. Let me know where can i upload the huge 5 million record file

Also one more observation. For the first few minutes i could see the records getting inserted into the table(using browser). Then all of a sudden something happened and i saw count(*) =0. The client was still running. And then later it aborted .....


This is the error in the logs ..*************
I am firing a simple select count(*) from table on a table having minimum 500K records
*****************************
[Network Thread - 0] TRACE NETWORK - Backpressure started for client org.voltdb.network.VoltPort@53f64158:/192.168.160.129:56221
[Network Thread - 0] TRACE NETWORK - Backpressure ended for client org.voltdb.network.VoltPort@53f64158:/192.168.160.129:56221
[main] INFO HOST - Loading...

_ __ ____ ____ ____
| | / /___ / / /_/ __ \/ __ )
| | / / __ \/ / __/ / / / __ |
| |/ / /_/ / / /_/ /_/ / /_/ /
|___/\____/_/\__/_____/_____/
Initialization Log Output:
--------------------------------
[main] INFO HOST - Build: 0.6.02 https://svn.voltdb.com/eng/branches/voltdb-0.6?revision=212
[main] INFO HOST - HTTP admin console listening on port 8080
[main] INFO HOST - Loading application catalog jarfile from /home/shettyr/amitabh/VoltDBMapper/subject.jar
[main] INFO HOST - Creating host manager for 1 hosts using leader localhost/127.0.0.1
[Thread-4] INFO HOST - Connecting to VoltDB cluster as the leader...
[Thread-4] DEBUG org.voltdb.messaging.impl.SocketJoiner - Starting Coordinator
[Thread-4] INFO HOST - Maximum clock/network skew is 0 milliseconds (according to leader)
[main] TRACE HOST - ExecutionSite initializing (siteid 1)
[main] INFO HOST - Attempting to load native VoltDB library voltdb-0.6.02. Expect to see a confirmation following this upon success. If none appears then you may need to compile VoltDB for your platform
[main] INFO HOST - Successfully loaded native VoltDB library voltdb-0.6.02.
[main] INFO HOST - Starting the network
[main] INFO HOST - --------------------------------
Server completed initialization.

[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[ExecutionSite:0001] TRACE HOST ExecutionSite - 1 index 0 - Unexpected exception while executing plan fragment 31
Output from SQL stmt overflowed output/network buffer of 10mb. Try a "limit" clause or a stronger predicate.
[main] INFO HOST - Loading...
re: issue
tcallaghan
Apr 27, 2010
This is the error in the logs ..*************
I am firing a simple select count(*) from table on a table having minimum 500K records
*****************************
[Network Thread - 0] TRACE NETWORK - Backpressure started for client org.voltdb.network.VoltPort@53f64158:/192.168.160.129:56221
[Network Thread - 0] TRACE NETWORK - Backpressure ended for client org.voltdb.network.VoltPort@53f64158:/192.168.160.129:56221
[main] INFO HOST - Loading...
...


Shetty,

Since you are running in a VM I am concerned that you don't have enough physical memory to be running this server with the amount of data you are loading. Please rerun your test with "top" running in another terminal window to check your available memory.

Let me know your findings.

-Tim
I guess you are right. This
shetty_ritesh
May 10, 2010
Shetty,

Since you are running in a VM I am concerned that you don't have enough physical memory to be running this server with the amount of data you are loading. Please rerun your test with "top" running in another terminal window to check your available memory.

Let me know your findings.

-Tim


I guess you are right. This might be a VM ware issue. But count(*) is definitely a problem.