Trace number 3472377

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 160046 5125.99

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.025-NOTKNOWN.cnf
MD5SUM732b9c9b0d44d6a9e75489b2f5004165
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark50676.1
Satisfiable
(Un)Satisfiability was proved
Number of variables95456
Number of clauses477186
Sum of the clauses size1137161
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2470925
Number of clauses of size 31500
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 54550

Solver Data 

0.00/0.01	c clasp version 2.0.0-RC2
0.00/0.01	c Reading from ...576359/instance-3472377-1307576359.cnf
0.79/0.85	c Solving...
160004.05/5041.35	
160004.05/5041.35	*** INTERRUPTED! ***
160004.05/5041.35	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472377-1307576359/watcher-3472377-1307576359 -o /tmp/evaluation-result-3472377-1307576359/solver-3472377-1307576359 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472377-1307576359.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.08 0.11 5.08 1/413 24421
/proc/meminfo: memFree=262723724/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=32992 CPUtime=0.02 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4202496 2542 0 0 0 2 0 0 0 20 0 1 0 23022855 33783808 2471 270582939648 4194304 4964800 140734952226944 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/24421/statm: 8248 2476 373 189 0 3122 0

[startup+0.124307 s]
/proc/loadavg: 0.08 0.11 5.08 1/413 24421
/proc/meminfo: memFree=262723724/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=41368 CPUtime=0.12 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4202496 6380 0 0 0 10 2 0 0 21 0 1 0 23022855 42360832 5474 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/24421/statm: 10342 5475 386 189 0 5216 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 41368

[startup+0.201318 s]
/proc/loadavg: 0.08 0.11 5.08 1/413 24421
/proc/meminfo: memFree=262723724/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=44768 CPUtime=0.19 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4202496 7589 0 0 0 17 2 0 0 21 0 1 0 23022855 45842432 6300 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/24421/statm: 11192 6301 386 189 0 6066 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 44768

[startup+0.30131 s]
/proc/loadavg: 0.08 0.11 5.08 1/413 24421
/proc/meminfo: memFree=262723724/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=50604 CPUtime=0.29 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4202496 9674 0 0 0 26 3 0 0 21 0 1 0 23022855 51818496 7708 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/24421/statm: 12651 7709 386 189 0 7525 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 50604

[startup+0.700328 s]
/proc/loadavg: 0.08 0.11 5.08 1/413 24421
/proc/meminfo: memFree=262723724/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=62536 CPUtime=0.69 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4202496 13520 0 0 0 65 4 0 0 22 0 1 0 23022855 64036864 10622 270582939648 4194304 4964800 140734952226944 18446744073709551615 4622574 0 0 4096 18946 0 0 0 17 12 0 0 0
/proc/24421/statm: 15634 10622 401 189 0 10508 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 62536

[startup+1.50331 s]
/proc/loadavg: 0.08 0.11 5.08 32/445 24453
/proc/meminfo: memFree=262558036/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=1158464 CPUtime=11.06 cores=0-31
/proc/24421/stat : 24421 (claspmt) S 24419 24421 24395 0 -1 4202496 102252 0 0 0 263 843 0 0 18 0 32 0 23022855 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 24 0 0 0
/proc/24421/statm: 289616 74498 440 189 0 283965 0
[pid=24421/tid=24423] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24423/stat : 24423 (claspmt) S 24419 24421 24395 0 -1 4202560 2142 0 0 0 2 30 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505797639648 0 0 -1 1 0 0 0
[pid=24421/tid=24424] ppid=24419 vsize=1158464 CPUtime=0.34 cores=0-31
/proc/24421/task/24424/stat : 24424 (claspmt) S 24419 24421 24395 0 -1 4202560 3231 0 0 0 4 30 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604497845476832 0 0 -1 2 0 0 0
[pid=24421/tid=24425] ppid=24419 vsize=1158464 CPUtime=0.28 cores=0-31
/proc/24421/task/24425/stat : 24425 (claspmt) S 24419 24421 24395 0 -1 4202560 3079 0 0 0 4 24 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487467621856 0 0 -1 15 0 0 0
[pid=24421/tid=24426] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24426/stat : 24426 (claspmt) S 24419 24421 24395 0 -1 4202560 2509 0 0 0 2 27 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505806347744 0 0 -1 12 0 0 0
[pid=24421/tid=24427] ppid=24419 vsize=1158464 CPUtime=0.44 cores=0-31
/proc/24421/task/24427/stat : 24427 (claspmt) R 24419 24421 24395 0 -1 4202560 2804 0 0 0 16 28 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 4399559 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=24421/tid=24428] ppid=24419 vsize=1158464 CPUtime=0.3 cores=0-31
/proc/24421/task/24428/stat : 24428 (claspmt) S 24419 24421 24395 0 -1 4202560 3457 0 0 0 4 26 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487927741920 0 0 -1 25 0 0 0
[pid=24421/tid=24429] ppid=24419 vsize=1158464 CPUtime=0.33 cores=0-31
/proc/24421/task/24429/stat : 24429 (claspmt) S 24419 24421 24395 0 -1 4202560 3060 0 0 0 5 28 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604486539558368 0 0 -1 13 0 0 0
[pid=24421/tid=24430] ppid=24419 vsize=1158464 CPUtime=0.31 cores=0-31
/proc/24421/task/24430/stat : 24430 (claspmt) S 24419 24421 24395 0 -1 4202560 2445 0 0 0 2 29 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604497490263520 0 0 -1 5 0 0 0
[pid=24421/tid=24431] ppid=24419 vsize=1158464 CPUtime=0.28 cores=0-31
/proc/24421/task/24431/stat : 24431 (claspmt) S 24419 24421 24395 0 -1 4202560 2551 0 0 0 2 26 0 0 18 0 32 0 23022943 1186267136 74498 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604500781252064 0 0 -1 9 0 0 0
[pid=24421/tid=24432] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24432/stat : 24432 (claspmt) S 24419 24421 24395 0 -1 4202560 3256 0 0 0 4 28 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604489928424928 0 0 -1 18 0 0 0
[pid=24421/tid=24433] ppid=24419 vsize=1158464 CPUtime=0.33 cores=0-31
/proc/24421/task/24433/stat : 24433 (claspmt) S 24419 24421 24395 0 -1 4202560 3222 0 0 0 4 29 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604497490255328 0 0 -1 14 0 0 0
[pid=24421/tid=24434] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24434/stat : 24434 (claspmt) S 24419 24421 24395 0 -1 4202560 2460 0 0 0 2 27 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505806355936 0 0 -1 22 0 0 0
[pid=24421/tid=24435] ppid=24419 vsize=1158464 CPUtime=0.5 cores=0-31
/proc/24421/task/24435/stat : 24435 (claspmt) R 24419 24421 24395 0 -1 4202560 2601 0 0 0 25 25 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=24421/tid=24436] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24436/stat : 24436 (claspmt) S 24419 24421 24395 0 -1 4202560 3194 0 0 0 3 26 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604485658467808 0 0 -1 20 0 0 0
[pid=24421/tid=24437] ppid=24419 vsize=1158464 CPUtime=0.31 cores=0-31
/proc/24421/task/24437/stat : 24437 (claspmt) S 24419 24421 24395 0 -1 4202560 2961 0 0 0 4 27 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604493153680864 0 0 -1 16 0 0 0
[pid=24421/tid=24438] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24438/stat : 24438 (claspmt) S 24419 24421 24395 0 -1 4202560 2484 0 0 0 1 28 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487260708320 0 0 -1 28 0 0 0
[pid=24421/tid=24439] ppid=24419 vsize=1158464 CPUtime=0.23 cores=0-31
/proc/24421/task/24439/stat : 24439 (claspmt) S 24419 24421 24395 0 -1 4202560 2332 0 0 0 2 21 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604556381126112 0 0 -1 7 0 0 0
[pid=24421/tid=24440] ppid=24419 vsize=1158464 CPUtime=0.22 cores=0-31
/proc/24421/task/24440/stat : 24440 (claspmt) S 24419 24421 24395 0 -1 4202560 3190 0 0 0 5 17 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604496886136288 0 0 -1 11 0 0 0
[pid=24421/tid=24441] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24441/stat : 24441 (claspmt) S 24419 24421 24395 0 -1 4202560 3320 0 0 0 4 28 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604486539550176 0 0 -1 29 0 0 0
[pid=24421/tid=24442] ppid=24419 vsize=1158464 CPUtime=0.31 cores=0-31
/proc/24421/task/24442/stat : 24442 (claspmt) S 24419 24421 24395 0 -1 4202560 2558 0 0 0 3 28 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604499994893792 0 0 -1 4 0 0 0
[pid=24421/tid=24443] ppid=24419 vsize=1158464 CPUtime=0.44 cores=0-31
/proc/24421/task/24443/stat : 24443 (claspmt) R 24419 24421 24395 0 -1 4202560 2851 0 0 0 16 28 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24421/tid=24444] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24444/stat : 24444 (claspmt) S 24419 24421 24395 0 -1 4202560 3378 0 0 0 4 28 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487927733728 0 0 -1 3 0 0 0
[pid=24421/tid=24445] ppid=24419 vsize=1158464 CPUtime=0.24 cores=0-31
/proc/24421/task/24445/stat : 24445 (claspmt) S 24419 24421 24395 0 -1 4202560 3144 0 0 0 3 21 0 0 18 0 32 0 23022943 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604501540109792 0 0 -1 19 0 0 0
[pid=24421/tid=24446] ppid=24419 vsize=1158464 CPUtime=0.27 cores=0-31
/proc/24421/task/24446/stat : 24446 (claspmt) S 24419 24421 24395 0 -1 4202560 2415 0 0 0 2 25 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487429373408 0 0 -1 21 0 0 0
[pid=24421/tid=24447] ppid=24419 vsize=1158464 CPUtime=0.3 cores=0-31
/proc/24421/task/24447/stat : 24447 (claspmt) S 24419 24421 24395 0 -1 4202560 2490 0 0 0 1 29 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604496886128096 0 0 -1 30 0 0 0
[pid=24421/tid=24448] ppid=24419 vsize=1158464 CPUtime=0.34 cores=0-31
/proc/24421/task/24448/stat : 24448 (claspmt) S 24419 24421 24395 0 -1 4202560 3160 0 0 0 4 30 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505797631456 0 0 -1 10 0 0 0
[pid=24421/tid=24449] ppid=24419 vsize=1158464 CPUtime=0.3 cores=0-31
/proc/24421/task/24449/stat : 24449 (claspmt) S 24419 24421 24395 0 -1 4202560 3193 0 0 0 3 27 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604485254905312 0 0 -1 31 0 0 0
[pid=24421/tid=24450] ppid=24419 vsize=1158464 CPUtime=0.28 cores=0-31
/proc/24421/task/24450/stat : 24450 (claspmt) S 24419 24421 24395 0 -1 4202560 2433 0 0 0 2 26 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604504806391264 0 0 -1 26 0 0 0
[pid=24421/tid=24451] ppid=24419 vsize=1158464 CPUtime=0.44 cores=0-31
/proc/24421/task/24451/stat : 24451 (claspmt) R 24419 24421 24395 0 -1 4202560 2623 0 0 0 22 22 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 4402731 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=24421/tid=24452] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24452/stat : 24452 (claspmt) S 24419 24421 24395 0 -1 4202560 3210 0 0 0 4 25 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604493235019232 0 0 -1 17 0 0 0
[pid=24421/tid=24453] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24453/stat : 24453 (claspmt) S 24419 24421 24395 0 -1 4202560 2841 0 0 0 3 29 0 0 18 0 32 0 23022944 1186267136 74499 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604489473121760 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.06
Current children cumulated vsize (KiB) 1158464

[startup+3.10137 s]
/proc/loadavg: 0.08 0.11 5.08 5/445 24453
/proc/meminfo: memFree=262428624/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=1158464 CPUtime=17.45 cores=0-31
/proc/24421/stat : 24421 (claspmt) S 24419 24421 24395 0 -1 4202496 102434 0 0 0 902 843 0 0 18 0 32 0 23022855 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 24 0 0 0
/proc/24421/statm: 289616 74680 440 189 0 283965 0
[pid=24421/tid=24423] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24423/stat : 24423 (claspmt) S 24419 24421 24395 0 -1 4202560 2142 0 0 0 2 30 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505797639648 0 0 -1 1 0 0 0
[pid=24421/tid=24424] ppid=24419 vsize=1158464 CPUtime=0.34 cores=0-31
/proc/24421/task/24424/stat : 24424 (claspmt) S 24419 24421 24395 0 -1 4202560 3231 0 0 0 4 30 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604497845476832 0 0 -1 2 0 0 0
[pid=24421/tid=24425] ppid=24419 vsize=1158464 CPUtime=0.28 cores=0-31
/proc/24421/task/24425/stat : 24425 (claspmt) S 24419 24421 24395 0 -1 4202560 3079 0 0 0 4 24 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487467621856 0 0 -1 15 0 0 0
[pid=24421/tid=24426] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24426/stat : 24426 (claspmt) S 24419 24421 24395 0 -1 4202560 2509 0 0 0 2 27 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505806347744 0 0 -1 12 0 0 0
[pid=24421/tid=24427] ppid=24419 vsize=1158464 CPUtime=2.04 cores=0-31
/proc/24421/task/24427/stat : 24427 (claspmt) R 24419 24421 24395 0 -1 4202560 2866 0 0 0 176 28 0 0 20 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 4399280 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=24421/tid=24428] ppid=24419 vsize=1158464 CPUtime=0.3 cores=0-31
/proc/24421/task/24428/stat : 24428 (claspmt) S 24419 24421 24395 0 -1 4202560 3457 0 0 0 4 26 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487927741920 0 0 -1 25 0 0 0
[pid=24421/tid=24429] ppid=24419 vsize=1158464 CPUtime=0.33 cores=0-31
/proc/24421/task/24429/stat : 24429 (claspmt) S 24419 24421 24395 0 -1 4202560 3060 0 0 0 5 28 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604486539558368 0 0 -1 13 0 0 0
[pid=24421/tid=24430] ppid=24419 vsize=1158464 CPUtime=0.31 cores=0-31
/proc/24421/task/24430/stat : 24430 (claspmt) S 24419 24421 24395 0 -1 4202560 2445 0 0 0 2 29 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604497490263520 0 0 -1 5 0 0 0
[pid=24421/tid=24431] ppid=24419 vsize=1158464 CPUtime=0.28 cores=0-31
/proc/24421/task/24431/stat : 24431 (claspmt) S 24419 24421 24395 0 -1 4202560 2551 0 0 0 2 26 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604500781252064 0 0 -1 9 0 0 0
[pid=24421/tid=24432] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24432/stat : 24432 (claspmt) S 24419 24421 24395 0 -1 4202560 3256 0 0 0 4 28 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604489928424928 0 0 -1 18 0 0 0
[pid=24421/tid=24433] ppid=24419 vsize=1158464 CPUtime=0.33 cores=0-31
/proc/24421/task/24433/stat : 24433 (claspmt) S 24419 24421 24395 0 -1 4202560 3222 0 0 0 4 29 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604497490255328 0 0 -1 14 0 0 0
[pid=24421/tid=24434] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24434/stat : 24434 (claspmt) S 24419 24421 24395 0 -1 4202560 2460 0 0 0 2 27 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505806355936 0 0 -1 22 0 0 0
[pid=24421/tid=24435] ppid=24419 vsize=1158464 CPUtime=2.1 cores=0-31
/proc/24421/task/24435/stat : 24435 (claspmt) R 24419 24421 24395 0 -1 4202560 2665 0 0 0 185 25 0 0 20 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=24421/tid=24436] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24436/stat : 24436 (claspmt) S 24419 24421 24395 0 -1 4202560 3194 0 0 0 3 26 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604485658467808 0 0 -1 20 0 0 0
[pid=24421/tid=24437] ppid=24419 vsize=1158464 CPUtime=0.31 cores=0-31
/proc/24421/task/24437/stat : 24437 (claspmt) S 24419 24421 24395 0 -1 4202560 2961 0 0 0 4 27 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604493153680864 0 0 -1 16 0 0 0
[pid=24421/tid=24438] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31
/proc/24421/task/24438/stat : 24438 (claspmt) S 24419 24421 24395 0 -1 4202560 2484 0 0 0 1 28 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487260708320 0 0 -1 28 0 0 0
[pid=24421/tid=24439] ppid=24419 vsize=1158464 CPUtime=0.23 cores=0-31
/proc/24421/task/24439/stat : 24439 (claspmt) S 24419 24421 24395 0 -1 4202560 2332 0 0 0 2 21 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604556381126112 0 0 -1 7 0 0 0
[pid=24421/tid=24440] ppid=24419 vsize=1158464 CPUtime=0.22 cores=0-31
/proc/24421/task/24440/stat : 24440 (claspmt) S 24419 24421 24395 0 -1 4202560 3190 0 0 0 5 17 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604496886136288 0 0 -1 11 0 0 0
[pid=24421/tid=24441] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24441/stat : 24441 (claspmt) S 24419 24421 24395 0 -1 4202560 3320 0 0 0 4 28 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604486539550176 0 0 -1 29 0 0 0
[pid=24421/tid=24442] ppid=24419 vsize=1158464 CPUtime=0.31 cores=0-31
/proc/24421/task/24442/stat : 24442 (claspmt) S 24419 24421 24395 0 -1 4202560 2558 0 0 0 3 28 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604499994893792 0 0 -1 4 0 0 0
[pid=24421/tid=24443] ppid=24419 vsize=1158464 CPUtime=2.04 cores=0-31
/proc/24421/task/24443/stat : 24443 (claspmt) R 24419 24421 24395 0 -1 4202560 2902 0 0 0 176 28 0 0 20 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24421/tid=24444] ppid=24419 vsize=1158464 CPUtime=0.32 cores=0-31
/proc/24421/task/24444/stat : 24444 (claspmt) S 24419 24421 24395 0 -1 4202560 3378 0 0 0 4 28 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487927733728 0 0 -1 3 0 0 0
[pid=24421/tid=24445] ppid=24419 vsize=1158464 CPUtime=0.24 cores=0-31
/proc/24421/task/24445/stat : 24445 (claspmt) S 24419 24421 24395 0 -1 4202560 3144 0 0 0 3 21 0 0 18 0 32 0 23022943 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604501540109792 0 0 -1 19 0 0 0
[pid=24421/tid=24446] ppid=24419 vsize=1158464 CPUtime=0.27 cores=0-31
/proc/24421/task/24446/stat : 24446 (claspmt) S 24419 24421 24395 0 -1 4202560 2415 0 0 0 2 25 0 0 18 0 32 0 23022944 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604487429373408 0 0 -1 21 0 0 0
[pid=24421/tid=24447] ppid=24419 vsize=1158464 CPUtime=0.3 cores=0-31
/proc/24421/task/24447/stat : 24447 (claspmt) S 24419 24421 24395 0 -1 4202560 2490 0 0 0 1 29 0 0 18 0 32 0 23022944 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604496886128096 0 0 -1 30 0 0 0
[pid=24421/tid=24448] ppid=24419 vsize=1158464 CPUtime=0.34 cores=0-31
/proc/24421/task/24448/stat : 24448 (claspmt) S 24419 24421 24395 0 -1 4202560 3160 0 0 0 4 30 0 0 18 0 32 0 23022944 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604505797631456 0 0 -1 10 0 0 0
[pid=24421/tid=24449] ppid=24419 vsize=1158464 CPUtime=0.3 cores=0-31
/proc/24421/task/24449/stat : 24449 (claspmt) S 24419 24421 24395 0 -1 4202560 3193 0 0 0 3 27 0 0 18 0 32 0 23022944 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604485254905312 0 0 -1 31 0 0 0
[pid=24421/tid=24450] ppid=24419 vsize=1158464 CPUtime=0.28 cores=0-31
/proc/24421/task/24450/stat : 24450 (claspmt) S 24419 24421 24395 0 -1 4202560 2433 0 0 0 2 26 0 0 18 0 32 0 23022944 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 258662772457 0 0 4096 18946 18446604504806391264 0 0 -1 26 0 0 0
[pid=24421/tid=24451] ppid=24419 vsize=1158464 CPUtime=2.04 cores=0-31
/proc/24421/task/24451/stat : 24451 (claspmt) R 24419 24421 24395 0 -1 4202560 2627 0 0 0 182 22 0 0 20 0 32 0 23022944 1186267136 74680 270582939648 4194304 4964800 140734952226944 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=24421/tid=24452] ppid=24419 vsize=1158464 CPUtime=0.29 cores=0-31

################
# More data... #
################

[pid=24421/tid=24453] ppid=24419 vsize=14772656 CPUtime=4963.87 cores=0-31
/proc/24421/task/24453/stat : 24453 (claspmt) R 24419 24421 24395 0 -1 4202560 357353 0 0 0 495484 903 0 0 25 0 32 0 23022944 15127199744 3472762 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 31 0 0 0
Current children cumulated CPU time (s) 159395
Current children cumulated vsize (KiB) 14772656



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5041.3 s]
/proc/loadavg: 32.01 32.03 31.90 34/445 24611
/proc/meminfo: memFree=248805720/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14772656 CPUtime=160004 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4202496 3525890 0 0 0 15989598 10806 0 0 25 0 32 0 23022855 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4669509 0 0 4096 18946 18446744073709551615 0 0 17 24 0 0 0
/proc/24421/statm: 3693164 3472824 463 189 0 3687513 0
[pid=24421/tid=24423] ppid=24419 vsize=14772656 CPUtime=4992.29 cores=0-31
/proc/24421/task/24423/stat : 24423 (claspmt) R 24419 24421 24395 0 -1 4202560 35223 0 0 0 499071 158 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=24421/tid=24424] ppid=24419 vsize=14772656 CPUtime=4984.76 cores=0-31
/proc/24421/task/24424/stat : 24424 (claspmt) R 24419 24421 24395 0 -1 4202560 70218 0 0 0 498251 225 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=24421/tid=24425] ppid=24419 vsize=14772656 CPUtime=5000.3 cores=0-31
/proc/24421/task/24425/stat : 24425 (claspmt) R 24419 24421 24395 0 -1 4202560 40886 0 0 0 499868 162 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=24421/tid=24426] ppid=24419 vsize=14772656 CPUtime=4999.2 cores=0-31
/proc/24421/task/24426/stat : 24426 (claspmt) R 24419 24421 24395 0 -1 4202560 133956 0 0 0 499461 459 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4668544 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=24421/tid=24427] ppid=24419 vsize=14772656 CPUtime=5036.31 cores=0-31
/proc/24421/task/24427/stat : 24427 (claspmt) R 24419 24421 24395 0 -1 4202560 72374 0 0 0 503354 277 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=24421/tid=24428] ppid=24419 vsize=14772656 CPUtime=4991.04 cores=0-31
/proc/24421/task/24428/stat : 24428 (claspmt) R 24419 24421 24395 0 -1 4202560 46795 0 0 0 498919 185 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4361114 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=24421/tid=24429] ppid=24419 vsize=14772656 CPUtime=4995.19 cores=0-31
/proc/24421/task/24429/stat : 24429 (claspmt) R 24419 24421 24395 0 -1 4202560 452554 0 0 0 498432 1087 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4320958 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=24421/tid=24430] ppid=24419 vsize=14772656 CPUtime=4993.27 cores=0-31
/proc/24421/task/24430/stat : 24430 (claspmt) R 24419 24421 24395 0 -1 4202560 70953 0 0 0 499028 299 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670328 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=24421/tid=24431] ppid=24419 vsize=14772656 CPUtime=5000.26 cores=0-31
/proc/24421/task/24431/stat : 24431 (claspmt) R 24419 24421 24395 0 -1 4202560 34338 0 0 0 499872 154 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=24421/tid=24432] ppid=24419 vsize=14772656 CPUtime=5000.18 cores=0-31
/proc/24421/task/24432/stat : 24432 (claspmt) R 24419 24421 24395 0 -1 4202560 74259 0 0 0 499802 216 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4660155 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=24421/tid=24433] ppid=24419 vsize=14772656 CPUtime=4990.88 cores=0-31
/proc/24421/task/24433/stat : 24433 (claspmt) R 24419 24421 24395 0 -1 4202560 39425 0 0 0 498928 160 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=24421/tid=24434] ppid=24419 vsize=14772656 CPUtime=4995.75 cores=0-31
/proc/24421/task/24434/stat : 24434 (claspmt) R 24419 24421 24395 0 -1 4202560 159994 0 0 0 499040 535 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=24421/tid=24435] ppid=24419 vsize=14772656 CPUtime=5027.23 cores=0-31
/proc/24421/task/24435/stat : 24435 (claspmt) R 24419 24421 24395 0 -1 4202560 61242 0 0 0 502499 224 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=24421/tid=24436] ppid=24419 vsize=14772656 CPUtime=4999.67 cores=0-31
/proc/24421/task/24436/stat : 24436 (claspmt) R 24419 24421 24395 0 -1 4202560 47074 0 0 0 499779 188 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670321 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=24421/tid=24437] ppid=24419 vsize=14772656 CPUtime=4998.06 cores=0-31
/proc/24421/task/24437/stat : 24437 (claspmt) R 24419 24421 24395 0 -1 4202560 445529 0 0 0 498771 1035 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=24421/tid=24438] ppid=24419 vsize=14772656 CPUtime=5000.08 cores=0-31
/proc/24421/task/24438/stat : 24438 (claspmt) R 24419 24421 24395 0 -1 4202560 71829 0 0 0 499722 286 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=24421/tid=24439] ppid=24419 vsize=14772656 CPUtime=4999.66 cores=0-31
/proc/24421/task/24439/stat : 24439 (claspmt) R 24419 24421 24395 0 -1 4202560 39280 0 0 0 499797 169 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=24421/tid=24440] ppid=24419 vsize=14772656 CPUtime=5000.41 cores=0-31
/proc/24421/task/24440/stat : 24440 (claspmt) R 24419 24421 24395 0 -1 4202560 66099 0 0 0 499844 197 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=24421/tid=24441] ppid=24419 vsize=14772656 CPUtime=5000.39 cores=0-31
/proc/24421/task/24441/stat : 24441 (claspmt) R 24419 24421 24395 0 -1 4202560 45823 0 0 0 499846 193 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=24421/tid=24442] ppid=24419 vsize=14772656 CPUtime=4998.14 cores=0-31
/proc/24421/task/24442/stat : 24442 (claspmt) R 24419 24421 24395 0 -1 4202560 135665 0 0 0 499374 440 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=24421/tid=24443] ppid=24419 vsize=14772656 CPUtime=5038.35 cores=0-31
/proc/24421/task/24443/stat : 24443 (claspmt) R 24419 24421 24395 0 -1 4202560 61156 0 0 0 503612 223 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670323 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=24421/tid=24444] ppid=24419 vsize=14772656 CPUtime=4975.09 cores=0-31
/proc/24421/task/24444/stat : 24444 (claspmt) R 24419 24421 24395 0 -1 4202560 46801 0 0 0 497326 183 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=24421/tid=24445] ppid=24419 vsize=14772656 CPUtime=4992.65 cores=0-31
/proc/24421/task/24445/stat : 24445 (claspmt) R 24419 24421 24395 0 -1 4202560 381898 0 0 0 498391 874 0 0 25 0 32 0 23022943 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=24421/tid=24446] ppid=24419 vsize=14772656 CPUtime=4999.97 cores=0-31
/proc/24421/task/24446/stat : 24446 (claspmt) R 24419 24421 24395 0 -1 4202560 64569 0 0 0 499732 265 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=24421/tid=24447] ppid=24419 vsize=14772656 CPUtime=5000.02 cores=0-31
/proc/24421/task/24447/stat : 24447 (claspmt) R 24419 24421 24395 0 -1 4202560 42116 0 0 0 499813 189 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=24421/tid=24448] ppid=24419 vsize=14772656 CPUtime=5000.47 cores=0-31
/proc/24421/task/24448/stat : 24448 (claspmt) R 24419 24421 24395 0 -1 4202560 58536 0 0 0 499860 187 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=24421/tid=24449] ppid=24419 vsize=14772656 CPUtime=4999.89 cores=0-31
/proc/24421/task/24449/stat : 24449 (claspmt) R 24419 24421 24395 0 -1 4202560 47549 0 0 0 499801 188 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=24421/tid=24450] ppid=24419 vsize=14772656 CPUtime=4999.15 cores=0-31
/proc/24421/task/24450/stat : 24450 (claspmt) R 24419 24421 24395 0 -1 4202560 152697 0 0 0 499385 530 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=24421/tid=24451] ppid=24419 vsize=14772656 CPUtime=5022.84 cores=0-31
/proc/24421/task/24451/stat : 24451 (claspmt) R 24419 24421 24395 0 -1 4202560 44891 0 0 0 502114 170 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=24421/tid=24452] ppid=24419 vsize=14772656 CPUtime=4989.44 cores=0-31
/proc/24421/task/24452/stat : 24452 (claspmt) R 24419 24421 24395 0 -1 4202560 46478 0 0 0 498761 183 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=24421/tid=24453] ppid=24419 vsize=14772656 CPUtime=4982.33 cores=0-31
/proc/24421/task/24453/stat : 24453 (claspmt) R 24419 24421 24395 0 -1 4202560 357353 0 0 0 497330 903 0 0 25 0 32 0 23022944 15127199744 3472824 270582939648 4194304 4964800 140734952226944 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 31 0 0 0
Current children cumulated CPU time (s) 160004
Current children cumulated vsize (KiB) 14772656

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -24421

[startup+5082.3 s]
/proc/loadavg: 17.74 28.34 30.66 2/414 24614
/proc/meminfo: memFree=241436192/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14624764 CPUtime=160025 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4204032 3525902 0 0 0 15989760 12750 0 0 18 0 1 0 23022855 14975758336 3458396 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650612750 0 0 4096 18946 0 0 0 17 29 0 0 2015
/proc/24421/statm: 3656191 3458396 474 189 0 3650540 0
Current children cumulated CPU time (s) 160025
Current children cumulated vsize (KiB) 14624764

Solver just ended. Dumping a history of the last processes samples

[startup+5094.3 s]
/proc/loadavg: 15.47 27.51 30.37 1/414 24614
/proc/meminfo: memFree=239779404/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14624764 CPUtime=160030 cores=0-31
/proc/24421/stat : 24421 (claspmt) D 24419 24421 24395 0 -1 4204032 3525902 0 0 0 15989760 13211 0 0 18 0 1 0 23022855 14975758336 3458396 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650612750 0 0 4096 18946 18446744071562155273 0 0 17 29 0 0 2672
/proc/24421/statm: 3656191 3458396 474 189 0 3650540 0
Current children cumulated CPU time (s) 160030
Current children cumulated vsize (KiB) 14624764

[startup+5107.1 s]
/proc/loadavg: 12.71 26.31 29.93 2/414 24614
/proc/meminfo: memFree=237828248/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14624764 CPUtime=160036 cores=0-31
/proc/24421/stat : 24421 (claspmt) D 24419 24421 24395 0 -1 4204032 3525902 0 0 0 15989760 13830 0 0 19 0 1 0 23022855 14975758336 3458396 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650612750 0 0 4096 18946 18446744071562155273 0 0 17 29 0 0 3386
/proc/24421/statm: 3656191 3458396 474 189 0 3650540 0
Current children cumulated CPU time (s) 160036
Current children cumulated vsize (KiB) 14624764

[startup+5113.5 s]
/proc/loadavg: 11.93 25.92 29.78 2/414 24614
/proc/meminfo: memFree=236320872/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14624764 CPUtime=160040 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4204032 3525902 0 0 0 15989760 14212 0 0 19 0 1 0 23022855 14975758336 3458396 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650612750 0 0 4096 18946 0 0 0 17 1 0 0 3670
/proc/24421/statm: 3656191 3458396 474 189 0 3650540 0
Current children cumulated CPU time (s) 160040
Current children cumulated vsize (KiB) 14624764

[startup+5119.9 s]
/proc/loadavg: 11.22 25.54 29.64 1/414 24614
/proc/meminfo: memFree=235380288/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14624764 CPUtime=160042 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4204032 3525902 0 0 0 15989760 14457 0 0 18 0 1 0 23022855 14975758336 3458396 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650612750 0 0 4096 18946 0 0 0 17 1 0 0 4064
/proc/24421/statm: 3656191 3458396 474 189 0 3650540 0
Current children cumulated CPU time (s) 160042
Current children cumulated vsize (KiB) 14624764

[startup+5123.1 s]
/proc/loadavg: 10.56 25.16 29.49 2/414 24614
/proc/meminfo: memFree=234737876/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14624764 CPUtime=160044 cores=0-31
/proc/24421/stat : 24421 (claspmt) D 24419 24421 24395 0 -1 4204032 3525902 0 0 0 15989760 14601 0 0 18 0 1 0 23022855 14975758336 3458396 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650612750 0 0 4096 18946 18446744071562155273 0 0 17 1 0 0 4207
/proc/24421/statm: 3656191 3458396 474 189 0 3650540 0
Current children cumulated CPU time (s) 160044
Current children cumulated vsize (KiB) 14624764

[startup+5124.7 s]
/proc/loadavg: 10.56 25.16 29.49 2/414 24614
/proc/meminfo: memFree=234695748/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=14624764 CPUtime=160044 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4204032 3525902 0 0 0 15989760 14689 0 0 18 0 1 0 23022855 14975758336 3458396 270582939648 4194304 4964800 140734952226944 18446744073709551615 258650612750 0 0 4096 18946 0 0 0 17 29 0 0 4312
/proc/24421/statm: 3656191 3458396 474 189 0 3650540 0
Current children cumulated CPU time (s) 160044
Current children cumulated vsize (KiB) 14624764

[startup+5125.5 s]
/proc/loadavg: 9.95 24.79 29.35 2/414 24614
/proc/meminfo: memFree=234540500/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=0 CPUtime=160045 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4204036 3525902 0 0 0 15989760 14769 0 0 19 0 1 0 23022855 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 29 0 0 4312
/proc/24421/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160045
Current children cumulated vsize (KiB) 0

[startup+5125.9 s]
/proc/loadavg: 9.95 24.79 29.35 2/414 24614
/proc/meminfo: memFree=234540500/264257624 swapFree=67111528/67111528
[pid=24421] ppid=24419 vsize=0 CPUtime=160046 cores=0-31
/proc/24421/stat : 24421 (claspmt) R 24419 24421 24395 0 -1 4204036 3525902 0 0 0 15989760 14809 0 0 20 0 1 0 23022855 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 29 0 0 4312
/proc/24421/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 160046
Current children cumulated vsize (KiB) 0

Child ended because it received signal 24 (SIGXCPU)
Child dumped core
Real time (s): 5125.99
CPU time (s): 160046
CPU user time (s): 159898
CPU system time (s): 148.178
CPU usage (%): 3122.24
Max. virtual memory (cumulated for all children) (KiB): 14772656

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 159898
system time used= 148.178
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3525902
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1341231
involuntary context switches= 397127

runsolver used 29.2076 second user time and 115.14 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-09 01:39:20
IDJOB=3472377
IDBENCH=82390
IDSOLVER=1961
FILE ID=node200/3472377-1307576359
RUNJOBID= node200-1307576358-24409
PBS_JOBID= 13547687
Free space on /tmp= 61736 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.025-NOTKNOWN.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472377-1307576359/watcher-3472377-1307576359 -o /tmp/evaluation-result-3472377-1307576359/solver-3472377-1307576359 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472377-1307576359.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= 732b9c9b0d44d6a9e75489b2f5004165
RANDOM SEED=10733940

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262724508 kB
Buffers:        190328 kB
Cached:        1082912 kB
SwapCached:          0 kB
Active:         216780 kB
Inactive:      1119884 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262724508 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:            8760 kB
Writeback:           0 kB
AnonPages:       63744 kB
Mapped:          14476 kB
Slab:           121044 kB
PageTables:       4400 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   163036 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 48212 MiB
End job on node200 at 2011-06-09 03:04:46