Trace number 3472387

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 58689.8 5104.02

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
MD5SUMc6d3480970bb2819448dd0a800a3cf1c
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark98266.1
Satisfiable
(Un)Satisfiability was proved
Number of variables1134832
Number of clauses6175026
Sum of the clauses size14647926
Maximum clause length1178
Minimum clause length1
Number of clauses of size 1846
Number of clauses of size 26124260
Number of clauses of size 3960
Number of clauses of size 41200
Number of clauses of size 51200
Number of clauses of size over 546560

Solver Data 

0.00/0.01	c clasp version 2.0.0-RC2
0.00/0.01	c Reading from ...602923/instance-3472387-1307602923.cnf
2.79/2.84	c Solving...
58683.61/5100.02	
58683.61/5100.02	*** INTERRUPTED! ***
58683.61/5100.02	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472387-1307602923/watcher-3472387-1307602923 -o /tmp/evaluation-result-3472387-1307602923/solver-3472387-1307602923 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472387-1307602923.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.55 4.79 4.51 1/413 26433
/proc/meminfo: memFree=262410128/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=164956 CPUtime=0.02 cores=0-31
/proc/26433/stat : 26433 (claspmt) R 26431 26433 26258 0 -1 4202496 1344 0 0 0 2 0 0 0 20 0 1 0 25679725 168914944 1273 270582939648 4194304 4964800 140736775818528 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 29 0 0 0
/proc/26433/statm: 41239 1277 373 189 0 36113 0

[startup+0.127237 s]
/proc/loadavg: 6.55 4.79 4.51 1/413 26433
/proc/meminfo: memFree=262410128/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=164956 CPUtime=0.11 cores=0-31
/proc/26433/stat : 26433 (claspmt) R 26431 26433 26258 0 -1 4202496 8275 0 0 0 10 1 0 0 20 0 1 0 25679725 168914944 8204 270582939648 4194304 4964800 140736775818528 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 29 0 0 0
/proc/26433/statm: 41239 8208 373 189 0 36113 0
Current children cumulated CPU time (s) 0.11
Current children cumulated vsize (KiB) 164956

[startup+0.201236 s]
/proc/loadavg: 6.55 4.79 4.51 1/413 26433
/proc/meminfo: memFree=262410128/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=164956 CPUtime=0.18 cores=0-31
/proc/26433/stat : 26433 (claspmt) R 26431 26433 26258 0 -1 4202496 13928 0 0 0 16 2 0 0 20 0 1 0 25679725 168914944 13857 270582939648 4194304 4964800 140736775818528 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 29 0 0 0
/proc/26433/statm: 41239 13861 373 189 0 36113 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 164956

[startup+0.301227 s]
/proc/loadavg: 6.55 4.79 4.51 1/413 26433
/proc/meminfo: memFree=262410128/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=164956 CPUtime=0.29 cores=0-31
/proc/26433/stat : 26433 (claspmt) R 26431 26433 26258 0 -1 4202496 21579 0 0 0 25 4 0 0 21 0 1 0 25679725 168914944 21508 270582939648 4194304 4964800 140736775818528 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 29 0 0 0
/proc/26433/statm: 41239 21513 373 189 0 36113 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 164956

[startup+0.701236 s]
/proc/loadavg: 6.55 4.79 4.51 1/413 26433
/proc/meminfo: memFree=262410128/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=244760 CPUtime=0.69 cores=0-31
/proc/26433/stat : 26433 (claspmt) R 26431 26433 26258 0 -1 4202496 64264 0 0 0 56 13 0 0 22 0 1 0 25679725 250634240 55328 270582939648 4194304 4964800 140736775818528 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 29 0 0 0
/proc/26433/statm: 61190 55328 385 189 0 56064 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 244760

[startup+1.50128 s]
/proc/loadavg: 6.55 4.79 4.51 2/414 26434
/proc/meminfo: memFree=262187536/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=261792 CPUtime=1.49 cores=0-31
/proc/26433/stat : 26433 (claspmt) R 26431 26433 26258 0 -1 4202496 68609 0 0 0 132 17 0 0 25 0 1 0 25679725 268075008 59673 270582939648 4194304 4964800 140736775818528 18446744073709551615 258650955760 0 0 4096 18946 0 0 0 17 29 0 0 0
/proc/26433/statm: 65448 59674 387 189 0 60322 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 261792

[startup+3.10125 s]
/proc/loadavg: 6.10 4.72 4.49 2/414 26434
/proc/meminfo: memFree=262142796/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=981084 CPUtime=9.64 cores=0-31
/proc/26433/stat : 26433 (claspmt) S 26431 26433 26258 0 -1 4202496 192681 0 0 0 280 684 0 0 19 0 32 0 25679725 1004630016 161147 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 14 0 0 0
/proc/26433/statm: 245271 161268 424 189 0 239620 0
Current children cumulated CPU time (s) 9.64
Current children cumulated vsize (KiB) 981084

[startup+6.30229 s]
/proc/loadavg: 6.10 4.72 4.49 32/445 26465
/proc/meminfo: memFree=260566600/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=2798644 CPUtime=98.62 cores=0-31
/proc/26433/stat : 26433 (claspmt) S 26431 26433 26258 0 -1 4202496 829548 0 0 0 490 9372 0 0 19 0 32 0 25679725 2865811456 559099 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 14 0 0 0
/proc/26433/statm: 699661 559170 427 189 0 694010 0
[pid=26433/tid=26435] ppid=26431 vsize=2798644 CPUtime=3.08 cores=0-31
/proc/26433/task/26435/stat : 26435 (claspmt) R 26431 26433 26258 0 -1 4202560 25390 0 0 0 7 301 0 0 18 0 32 0 25680011 2865811456 559198 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=26433/tid=26436] ppid=26431 vsize=2798644 CPUtime=2.98 cores=0-31
/proc/26433/task/26436/stat : 26436 (claspmt) R 26431 26433 26258 0 -1 4202560 28781 0 0 0 8 290 0 0 18 0 32 0 25680011 2865811456 559236 270582939648 4194304 4964800 140736775818528 18446744073709551615 4377995 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=26433/tid=26437] ppid=26431 vsize=2798644 CPUtime=3.04 cores=0-31
/proc/26433/task/26437/stat : 26437 (claspmt) R 26431 26433 26258 0 -1 4202560 30029 0 0 0 9 295 0 0 18 0 32 0 25680011 2865811456 559267 270582939648 4194304 4964800 140736775818528 18446744073709551615 4697492 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=26433/tid=26438] ppid=26431 vsize=2798644 CPUtime=3.07 cores=0-31
/proc/26433/task/26438/stat : 26438 (claspmt) R 26431 26433 26258 0 -1 4202560 11119 0 0 0 3 304 0 0 18 0 32 0 25680011 2865811456 559298 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=26433/tid=26439] ppid=26431 vsize=2798644 CPUtime=3.15 cores=0-31
/proc/26433/task/26439/stat : 26439 (claspmt) R 26431 26433 26258 0 -1 4202560 15509 0 0 0 5 310 0 0 18 0 32 0 25680011 2865811456 559330 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=26433/tid=26440] ppid=26431 vsize=2798644 CPUtime=3.06 cores=0-31
/proc/26433/task/26440/stat : 26440 (claspmt) R 26431 26433 26258 0 -1 4202560 29402 0 0 0 9 297 0 0 18 0 32 0 25680011 2865811456 559359 270582939648 4194304 4964800 140736775818528 18446744073709551615 4377995 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=26433/tid=26441] ppid=26431 vsize=2798644 CPUtime=3.11 cores=0-31
/proc/26433/task/26441/stat : 26441 (claspmt) R 26431 26433 26258 0 -1 4202560 27282 0 0 0 9 302 0 0 18 0 32 0 25680011 2865811456 559395 270582939648 4194304 4964800 140736775818528 18446744073709551615 4380986 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=26433/tid=26442] ppid=26431 vsize=2798644 CPUtime=3.1 cores=0-31
/proc/26433/task/26442/stat : 26442 (claspmt) R 26431 26433 26258 0 -1 4202560 12364 0 0 0 6 304 0 0 18 0 32 0 25680011 2865811456 559424 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26433/tid=26443] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26443/stat : 26443 (claspmt) R 26431 26433 26258 0 -1 4202560 22124 0 0 0 9 300 0 0 18 0 32 0 25680011 2865811456 559458 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=26433/tid=26444] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) R 26431 26433 26258 0 -1 4202560 22804 0 0 0 7 302 0 0 18 0 32 0 25680011 2865811456 559484 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26433/tid=26445] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26445/stat : 26445 (claspmt) R 26431 26433 26258 0 -1 4202560 22844 0 0 0 7 302 0 0 18 0 32 0 25680011 2865811456 559515 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=26433/tid=26446] ppid=26431 vsize=2798644 CPUtime=3.07 cores=0-31
/proc/26433/task/26446/stat : 26446 (claspmt) R 26431 26433 26258 0 -1 4202560 22525 0 0 0 7 300 0 0 18 0 32 0 25680011 2865811456 559541 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=26433/tid=26447] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26447/stat : 26447 (claspmt) R 26431 26433 26258 0 -1 4202560 24977 0 0 0 7 302 0 0 18 0 32 0 25680011 2865811456 559569 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=26433/tid=26448] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26448/stat : 26448 (claspmt) R 26431 26433 26258 0 -1 4202560 20460 0 0 0 5 304 0 0 18 0 32 0 25680011 2865811456 559602 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679151 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=26433/tid=26449] ppid=26431 vsize=2798644 CPUtime=3.03 cores=0-31
/proc/26433/task/26449/stat : 26449 (claspmt) R 26431 26433 26258 0 -1 4202560 25332 0 0 0 7 296 0 0 18 0 32 0 25680011 2865811456 559630 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=26433/tid=26450] ppid=26431 vsize=2798644 CPUtime=3.1 cores=0-31
/proc/26433/task/26450/stat : 26450 (claspmt) R 26431 26433 26258 0 -1 4202560 21824 0 0 0 6 304 0 0 18 0 32 0 25680011 2865811456 559665 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=26433/tid=26451] ppid=26431 vsize=2798644 CPUtime=3.1 cores=0-31
/proc/26433/task/26451/stat : 26451 (claspmt) R 26431 26433 26258 0 -1 4202560 26989 0 0 0 6 304 0 0 18 0 32 0 25680011 2865811456 559693 270582939648 4194304 4964800 140736775818528 18446744073709551615 4367521 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=26433/tid=26452] ppid=26431 vsize=2798644 CPUtime=3.1 cores=0-31
/proc/26433/task/26452/stat : 26452 (claspmt) R 26431 26433 26258 0 -1 4202560 15744 0 0 0 5 305 0 0 18 0 32 0 25680011 2865811456 559723 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=26433/tid=26453] ppid=26431 vsize=2798644 CPUtime=3.11 cores=0-31
/proc/26433/task/26453/stat : 26453 (claspmt) R 26431 26433 26258 0 -1 4202560 23062 0 0 0 6 305 0 0 18 0 32 0 25680011 2865811456 559758 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=26433/tid=26454] ppid=26431 vsize=2798644 CPUtime=3.1 cores=0-31
/proc/26433/task/26454/stat : 26454 (claspmt) R 26431 26433 26258 0 -1 4202560 27118 0 0 0 8 302 0 0 18 0 32 0 25680011 2865811456 559792 270582939648 4194304 4964800 140736775818528 18446744073709551615 4367521 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=26433/tid=26455] ppid=26431 vsize=2798644 CPUtime=3.06 cores=0-31
/proc/26433/task/26455/stat : 26455 (claspmt) R 26431 26433 26258 0 -1 4202560 22614 0 0 0 7 299 0 0 18 0 32 0 25680012 2865811456 559823 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=26433/tid=26456] ppid=26431 vsize=2798644 CPUtime=3.06 cores=0-31
/proc/26433/task/26456/stat : 26456 (claspmt) R 26431 26433 26258 0 -1 4202560 27923 0 0 0 6 300 0 0 18 0 32 0 25680012 2865811456 559850 270582939648 4194304 4964800 140736775818528 18446744073709551615 4377995 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=26433/tid=26457] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26457/stat : 26457 (claspmt) R 26431 26433 26258 0 -1 4202560 23972 0 0 0 5 304 0 0 18 0 32 0 25680012 2865811456 559878 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=26433/tid=26458] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26458/stat : 26458 (claspmt) R 26431 26433 26258 0 -1 4202560 21787 0 0 0 8 301 0 0 18 0 32 0 25680012 2865811456 559907 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=26433/tid=26459] ppid=26431 vsize=2798644 CPUtime=3.07 cores=0-31
/proc/26433/task/26459/stat : 26459 (claspmt) R 26431 26433 26258 0 -1 4202560 24432 0 0 0 6 301 0 0 18 0 32 0 25680012 2865811456 559940 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=26433/tid=26460] ppid=26431 vsize=2798644 CPUtime=3.05 cores=0-31
/proc/26433/task/26460/stat : 26460 (claspmt) R 26431 26433 26258 0 -1 4202560 22807 0 0 0 8 297 0 0 18 0 32 0 25680013 2865811456 559972 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=26433/tid=26461] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26461/stat : 26461 (claspmt) R 26431 26433 26258 0 -1 4202560 22639 0 0 0 7 302 0 0 18 0 32 0 25680013 2865811456 560001 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679094 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=26433/tid=26462] ppid=26431 vsize=2798644 CPUtime=3.12 cores=0-31
/proc/26433/task/26462/stat : 26462 (claspmt) R 26431 26433 26258 0 -1 4202560 24281 0 0 0 6 306 0 0 18 0 32 0 25680013 2865811456 560035 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679151 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=26433/tid=26463] ppid=26431 vsize=2798644 CPUtime=3.09 cores=0-31
/proc/26433/task/26463/stat : 26463 (claspmt) R 26431 26433 26258 0 -1 4202560 22897 0 0 0 6 303 0 0 18 0 32 0 25680013 2865811456 560061 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=26433/tid=26464] ppid=26431 vsize=2798644 CPUtime=2.99 cores=0-31
/proc/26433/task/26464/stat : 26464 (claspmt) R 26431 26433 26258 0 -1 4202560 32011 0 0 0 13 286 0 0 18 0 32 0 25680013 2865811456 560094 270582939648 4194304 4964800 140736775818528 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=26433/tid=26465] ppid=26431 vsize=2798644 CPUtime=3.06 cores=0-31
/proc/26433/task/26465/stat : 26465 (claspmt) R 26431 26433 26258 0 -1 4202560 25197 0 0 0 6 300 0 0 18 0 32 0 25680013 2865811456 560120 270582939648 4194304 4964800 140736775818528 18446744073709551615 4679106 0 0 4096 18946 0 0 0 -1 24 0 0 0
Current children cumulated CPU time (s) 98.62
Current children cumulated vsize (KiB) 2798644

[startup+12.7016 s]
/proc/loadavg: 7.77 5.14 4.63 5/445 26465
/proc/meminfo: memFree=258843224/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=4270828 CPUtime=146.99 cores=0-31
/proc/26433/stat : 26433 (claspmt) S 26431 26433 26258 0 -1 4202496 1232919 0 0 0 2939 11760 0 0 19 0 32 0 25679725 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 14 0 0 0
/proc/26433/statm: 1067707 891883 434 189 0 1062056 0
[pid=26433/tid=26435] ppid=26431 vsize=4270828 CPUtime=3.51 cores=0-31
/proc/26433/task/26435/stat : 26435 (claspmt) S 26431 26433 26258 0 -1 4202560 29555 0 0 0 16 335 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604572176383456 0 0 -1 18 0 0 0
[pid=26433/tid=26436] ppid=26431 vsize=4270828 CPUtime=4.31 cores=0-31
/proc/26433/task/26436/stat : 26436 (claspmt) S 26431 26433 26258 0 -1 4202560 43112 0 0 0 73 358 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604559136210400 0 0 -1 23 0 0 0
[pid=26433/tid=26437] ppid=26431 vsize=4270828 CPUtime=4.41 cores=0-31
/proc/26433/task/26437/stat : 26437 (claspmt) S 26431 26433 26258 0 -1 4202560 43627 0 0 0 75 366 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604561659821536 0 0 -1 7 0 0 0
[pid=26433/tid=26438] ppid=26431 vsize=4270828 CPUtime=4.33 cores=0-31
/proc/26433/task/26438/stat : 26438 (claspmt) S 26431 26433 26258 0 -1 4202560 29888 0 0 0 22 411 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604563661184480 0 0 -1 0 0 0 0
[pid=26433/tid=26439] ppid=26431 vsize=4270828 CPUtime=7.02 cores=0-31
/proc/26433/task/26439/stat : 26439 (claspmt) R 26431 26433 26258 0 -1 4202560 37181 0 0 0 275 427 0 0 22 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670097 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=26433/tid=26440] ppid=26431 vsize=4270828 CPUtime=4.4 cores=0-31
/proc/26433/task/26440/stat : 26440 (claspmt) S 26431 26433 26258 0 -1 4202560 42371 0 0 0 77 363 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604643668401632 0 0 -1 8 0 0 0
[pid=26433/tid=26441] ppid=26431 vsize=4270828 CPUtime=4.96 cores=0-31
/proc/26433/task/26441/stat : 26441 (claspmt) S 26431 26433 26258 0 -1 4202560 39469 0 0 0 111 385 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604566186606048 0 0 -1 19 0 0 0
[pid=26433/tid=26442] ppid=26431 vsize=4270828 CPUtime=4.23 cores=0-31
/proc/26433/task/26442/stat : 26442 (claspmt) S 26431 26433 26258 0 -1 4202560 29792 0 0 0 22 401 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604575314591200 0 0 -1 2 0 0 0
[pid=26433/tid=26443] ppid=26431 vsize=4270828 CPUtime=3.79 cores=0-31
/proc/26433/task/26443/stat : 26443 (claspmt) S 26431 26433 26258 0 -1 4202560 28685 0 0 0 19 360 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604572947865056 0 0 -1 4 0 0 0
[pid=26433/tid=26444] ppid=26431 vsize=4270828 CPUtime=4.57 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) S 26431 26433 26258 0 -1 4202560 37821 0 0 0 67 390 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604638453177824 0 0 -1 6 0 0 0
[pid=26433/tid=26445] ppid=26431 vsize=4270828 CPUtime=4.66 cores=0-31
/proc/26433/task/26445/stat : 26445 (claspmt) S 26431 26433 26258 0 -1 4202560 39955 0 0 0 71 395 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604640436510176 0 0 -1 29 0 0 0
[pid=26433/tid=26446] ppid=26431 vsize=4270828 CPUtime=3.74 cores=0-31
/proc/26433/task/26446/stat : 26446 (claspmt) S 26431 26433 26258 0 -1 4202560 28425 0 0 0 16 358 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604642877521376 0 0 -1 13 0 0 0
[pid=26433/tid=26447] ppid=26431 vsize=4270828 CPUtime=6.95 cores=0-31
/proc/26433/task/26447/stat : 26447 (claspmt) R 26431 26433 26258 0 -1 4202560 38748 0 0 0 297 398 0 0 22 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=26433/tid=26448] ppid=26431 vsize=4270828 CPUtime=4.7 cores=0-31
/proc/26433/task/26448/stat : 26448 (claspmt) S 26431 26433 26258 0 -1 4202560 39508 0 0 0 69 401 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604568705736160 0 0 -1 30 0 0 0
[pid=26433/tid=26449] ppid=26431 vsize=4270828 CPUtime=4.93 cores=0-31
/proc/26433/task/26449/stat : 26449 (claspmt) S 26431 26433 26258 0 -1 4202560 39704 0 0 0 100 393 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604639916604896 0 0 -1 11 0 0 0
[pid=26433/tid=26450] ppid=26431 vsize=4270828 CPUtime=3.94 cores=0-31
/proc/26433/task/26450/stat : 26450 (claspmt) S 26431 26433 26258 0 -1 4202560 29802 0 0 0 17 377 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604642041109984 0 0 -1 9 0 0 0
[pid=26433/tid=26451] ppid=26431 vsize=4270828 CPUtime=3.39 cores=0-31
/proc/26433/task/26451/stat : 26451 (claspmt) S 26431 26433 26258 0 -1 4202560 29542 0 0 0 14 325 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604569640263136 0 0 -1 26 0 0 0
[pid=26433/tid=26452] ppid=26431 vsize=4270828 CPUtime=4.76 cores=0-31
/proc/26433/task/26452/stat : 26452 (claspmt) S 26431 26433 26258 0 -1 4202560 38351 0 0 0 68 408 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604573425556960 0 0 -1 21 0 0 0
[pid=26433/tid=26453] ppid=26431 vsize=4270828 CPUtime=4.65 cores=0-31
/proc/26433/task/26453/stat : 26453 (claspmt) S 26431 26433 26258 0 -1 4202560 42750 0 0 0 68 397 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604643800563168 0 0 -1 22 0 0 0
[pid=26433/tid=26454] ppid=26431 vsize=4270828 CPUtime=3.34 cores=0-31
/proc/26433/task/26454/stat : 26454 (claspmt) S 26431 26433 26258 0 -1 4202560 29557 0 0 0 16 318 0 0 18 0 32 0 25680011 4373327872 891883 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662772457 0 0 4096 18946 18446604640543874528 0 0 -1 10 0 0 0
[pid=26433/tid=26455] ppid=26431 vsize=4270828 CPUtime=7.2 cores=0-31

################
# More data... #
################

[pid=26433/tid=26457] ppid=26431 vsize=41244404 CPUtime=1356.65 cores=0-31
/proc/26433/task/26457/stat : 26457 (claspmt) R 26431 26433 26258 0 -1 4202560 483420 0 0 0 134756 909 0 0 25 0 32 0 25680012 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4660444 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=26433/tid=26458] ppid=26431 vsize=41244404 CPUtime=1355.93 cores=0-31
/proc/26433/task/26458/stat : 26458 (claspmt) R 26431 26433 26258 0 -1 4202560 169442 0 0 0 134956 637 0 0 25 0 32 0 25680012 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=26433/tid=26459] ppid=26431 vsize=41244404 CPUtime=1355.87 cores=0-31
/proc/26433/task/26459/stat : 26459 (claspmt) R 26431 26433 26258 0 -1 4202560 112854 0 0 0 135053 534 0 0 25 0 32 0 25680012 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=26433/tid=26460] ppid=26431 vsize=41244404 CPUtime=1352.73 cores=0-31
/proc/26433/task/26460/stat : 26460 (claspmt) R 26431 26433 26258 0 -1 4202560 756312 0 0 0 134133 1140 0 0 25 0 32 0 25680013 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4627252 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=26433/tid=26461] ppid=26431 vsize=41244404 CPUtime=1352.9 cores=0-31
/proc/26433/task/26461/stat : 26461 (claspmt) R 26431 26433 26258 0 -1 4202560 410071 0 0 0 134423 867 0 0 25 0 32 0 25680013 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=26433/tid=26462] ppid=26431 vsize=41244404 CPUtime=1355.64 cores=0-31
/proc/26433/task/26462/stat : 26462 (claspmt) R 26431 26433 26258 0 -1 4202560 177159 0 0 0 134950 614 0 0 25 0 32 0 25680013 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=26433/tid=26463] ppid=26431 vsize=41244404 CPUtime=5035.24 cores=0-31
/proc/26433/task/26463/stat : 26463 (claspmt) R 26431 26433 26258 0 -1 4202560 47978 0 0 0 503106 418 0 0 25 0 32 0 25680013 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=26433/tid=26464] ppid=26431 vsize=41244404 CPUtime=1352.35 cores=0-31
/proc/26433/task/26464/stat : 26464 (claspmt) R 26431 26433 26258 0 -1 4202560 437146 0 0 0 134369 866 0 0 25 0 32 0 25680013 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26433/tid=26465] ppid=26431 vsize=41244404 CPUtime=1355.42 cores=0-31
/proc/26433/task/26465/stat : 26465 (claspmt) R 26431 26433 26258 0 -1 4202560 493425 0 0 0 134649 893 0 0 22 0 32 0 25680013 42234269696 10114026 270582939648 4194304 4964800 140736775818528 18446744073709551615 47336995192442 0 0 4096 18946 0 0 0 -1 19 0 0 0
Current children cumulated CPU time (s) 58117.4
Current children cumulated vsize (KiB) 41244404



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 32.06 31.83 25.65 33/445 26613
/proc/meminfo: memFree=221353668/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=41933556 CPUtime=58683.6 cores=0-31
/proc/26433/stat : 26433 (claspmt) R 26431 26433 26258 0 -1 4202496 10591541 0 0 0 5844327 24034 0 0 21 0 32 0 25679725 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670299 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/26433/statm: 10483389 10245969 457 189 0 10477738 0
[pid=26433/tid=26435] ppid=26431 vsize=41933556 CPUtime=1373.37 cores=0-31
/proc/26433/task/26435/stat : 26435 (claspmt) R 26431 26433 26258 0 -1 4202560 157863 0 0 0 136770 567 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4627536 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=26433/tid=26436] ppid=26431 vsize=41933556 CPUtime=1369.13 cores=0-31
/proc/26433/task/26436/stat : 26436 (claspmt) R 26431 26433 26258 0 -1 4202560 858783 0 0 0 135681 1232 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=26433/tid=26437] ppid=26431 vsize=41933556 CPUtime=1373.28 cores=0-31
/proc/26433/task/26437/stat : 26437 (claspmt) R 26431 26433 26258 0 -1 4202560 413037 0 0 0 136397 931 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=26433/tid=26438] ppid=26431 vsize=41933556 CPUtime=1374.16 cores=0-31
/proc/26433/task/26438/stat : 26438 (claspmt) R 26431 26433 26258 0 -1 4202560 170273 0 0 0 136771 645 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4380234 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=26433/tid=26439] ppid=26431 vsize=41933556 CPUtime=5093.37 cores=0-31
/proc/26433/task/26439/stat : 26439 (claspmt) R 26431 26433 26258 0 -1 4202560 45927 0 0 0 508887 450 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4627204 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=26433/tid=26440] ppid=26431 vsize=41933556 CPUtime=1373.41 cores=0-31
/proc/26433/task/26440/stat : 26440 (claspmt) R 26431 26433 26258 0 -1 4202560 418164 0 0 0 136433 908 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=26433/tid=26441] ppid=26431 vsize=41933556 CPUtime=1370.43 cores=0-31
/proc/26433/task/26441/stat : 26441 (claspmt) R 26431 26433 26258 0 -1 4202560 486831 0 0 0 136203 840 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=26433/tid=26442] ppid=26431 vsize=41933556 CPUtime=1373.96 cores=0-31
/proc/26433/task/26442/stat : 26442 (claspmt) R 26431 26433 26258 0 -1 4202560 169118 0 0 0 136761 635 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 258650616986 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=26433/tid=26443] ppid=26431 vsize=41933556 CPUtime=1373.58 cores=0-31
/proc/26433/task/26443/stat : 26443 (claspmt) R 26431 26433 26258 0 -1 4202560 163898 0 0 0 136764 594 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=26433/tid=26444] ppid=26431 vsize=41933556 CPUtime=1367.66 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) R 26431 26433 26258 0 -1 4202560 819601 0 0 0 135532 1234 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4313764 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=26433/tid=26445] ppid=26431 vsize=41933556 CPUtime=1371.65 cores=0-31
/proc/26433/task/26445/stat : 26445 (claspmt) R 26431 26433 26258 0 -1 4202560 385398 0 0 0 136309 856 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4363677 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=26433/tid=26446] ppid=26431 vsize=41933556 CPUtime=1373.54 cores=0-31
/proc/26433/task/26446/stat : 26446 (claspmt) R 26431 26433 26258 0 -1 4202560 186108 0 0 0 136720 634 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=26433/tid=26447] ppid=26431 vsize=41933556 CPUtime=5046.29 cores=0-31
/proc/26433/task/26447/stat : 26447 (claspmt) R 26431 26433 26258 0 -1 4202560 47768 0 0 0 504208 421 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=26433/tid=26448] ppid=26431 vsize=41933556 CPUtime=1370.99 cores=0-31
/proc/26433/task/26448/stat : 26448 (claspmt) R 26431 26433 26258 0 -1 4202560 403023 0 0 0 136206 893 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4320909 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=26433/tid=26449] ppid=26431 vsize=41933556 CPUtime=1372.2 cores=0-31
/proc/26433/task/26449/stat : 26449 (claspmt) R 26431 26433 26258 0 -1 4202560 407152 0 0 0 136417 803 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4627338 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=26433/tid=26450] ppid=26431 vsize=41933556 CPUtime=1373.7 cores=0-31
/proc/26433/task/26450/stat : 26450 (claspmt) R 26431 26433 26258 0 -1 4202560 163375 0 0 0 136722 648 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4684732 0 0 4096 18946 0 0 0 -1 1 0 0 0
[pid=26433/tid=26451] ppid=26431 vsize=41933556 CPUtime=1373.21 cores=0-31
/proc/26433/task/26451/stat : 26451 (claspmt) R 26431 26433 26258 0 -1 4202560 152136 0 0 0 136779 542 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670361 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=26433/tid=26452] ppid=26431 vsize=41933556 CPUtime=1371.46 cores=0-31
/proc/26433/task/26452/stat : 26452 (claspmt) R 26431 26433 26258 0 -1 4202560 709368 0 0 0 136013 1133 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=26433/tid=26453] ppid=26431 vsize=41933556 CPUtime=1371.27 cores=0-31
/proc/26433/task/26453/stat : 26453 (claspmt) R 26431 26433 26258 0 -1 4202560 424015 0 0 0 136177 950 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=26433/tid=26454] ppid=26431 vsize=41933556 CPUtime=1372.49 cores=0-31
/proc/26433/task/26454/stat : 26454 (claspmt) R 26431 26433 26258 0 -1 4202560 166108 0 0 0 136671 578 0 0 25 0 32 0 25680011 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=26433/tid=26455] ppid=26431 vsize=41933556 CPUtime=5069.37 cores=0-31
/proc/26433/task/26455/stat : 26455 (claspmt) R 26431 26433 26258 0 -1 4202560 44480 0 0 0 506525 412 0 0 25 0 32 0 25680012 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=26433/tid=26456] ppid=26431 vsize=41933556 CPUtime=1373.33 cores=0-31
/proc/26433/task/26456/stat : 26456 (claspmt) R 26431 26433 26258 0 -1 4202560 438211 0 0 0 136411 922 0 0 25 0 32 0 25680012 42939961344 10245969 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=26433/tid=26457] ppid=26431 vsize=41933556 CPUtime=1374.35 cores=0-31
/proc/26433/task/26457/stat : 26457 (claspmt) R 26431 26433 26258 0 -1 4202560 493536 0 0 0 136512 923 0 0 25 0 32 0 25680012 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4668900 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=26433/tid=26458] ppid=26431 vsize=41933556 CPUtime=1373.65 cores=0-31
/proc/26433/task/26458/stat : 26458 (claspmt) R 26431 26433 26258 0 -1 4202560 171324 0 0 0 136725 640 0 0 25 0 32 0 25680012 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=26433/tid=26459] ppid=26431 vsize=41933556 CPUtime=1373.59 cores=0-31
/proc/26433/task/26459/stat : 26459 (claspmt) R 26431 26433 26258 0 -1 4202560 114130 0 0 0 136822 537 0 0 25 0 32 0 25680012 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4370266 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=26433/tid=26460] ppid=26431 vsize=41933556 CPUtime=1370.45 cores=0-31
/proc/26433/task/26460/stat : 26460 (claspmt) R 26431 26433 26258 0 -1 4202560 760288 0 0 0 135900 1145 0 0 25 0 32 0 25680013 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4313736 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=26433/tid=26461] ppid=26431 vsize=41933556 CPUtime=1370.61 cores=0-31
/proc/26433/task/26461/stat : 26461 (claspmt) R 26431 26433 26258 0 -1 4202560 414088 0 0 0 136191 870 0 0 25 0 32 0 25680013 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=26433/tid=26462] ppid=26431 vsize=41933556 CPUtime=1373.36 cores=0-31
/proc/26433/task/26462/stat : 26462 (claspmt) R 26431 26433 26258 0 -1 4202560 178964 0 0 0 136718 618 0 0 21 0 32 0 25680013 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 9 0 0 0
[pid=26433/tid=26463] ppid=26431 vsize=41933556 CPUtime=5052.96 cores=0-31
/proc/26433/task/26463/stat : 26463 (claspmt) R 26431 26433 26258 0 -1 4202560 47978 0 0 0 504878 418 0 0 25 0 32 0 25680013 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4669011 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=26433/tid=26464] ppid=26431 vsize=41933556 CPUtime=1370.05 cores=0-31
/proc/26433/task/26464/stat : 26464 (claspmt) R 26431 26433 26258 0 -1 4202560 442528 0 0 0 136130 875 0 0 25 0 32 0 25680013 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=26433/tid=26465] ppid=26431 vsize=41933556 CPUtime=1373.14 cores=0-31
/proc/26433/task/26465/stat : 26465 (claspmt) R 26431 26433 26258 0 -1 4202560 496602 0 0 0 136414 900 0 0 25 0 32 0 25680013 42939961344 10245970 270582939648 4194304 4964800 140736775818528 18446744073709551615 4365094 0 0 4096 18946 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 58683.6
Current children cumulated vsize (KiB) 41933556

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -26433

Solver just ended. Dumping a history of the last processes samples

[startup+5100.7 s]
/proc/loadavg: 32.06 31.83 25.65 3/417 26614
/proc/meminfo: memFree=221352404/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=41933556 CPUtime=58685.4 cores=0-31
/proc/26433/stat : 26433 (claspmt) S 26431 26433 26258 0 -1 4202496 10591714 0 0 0 5844502 24034 0 0 21 0 3 0 25679725 42939961344 10246142 270582939648 4194304 4964800 140736775818528 18446744073709551615 258662759221 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/26433/statm: 10483389 10246142 459 189 0 10477738 0
[pid=26433/tid=26444] ppid=26431 vsize=41933556 CPUtime=1368.36 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) R 26431 26433 26258 0 -1 4202560 819601 0 0 0 135602 1234 0 0 25 0 3 0 25680011 42939961344 10246142 270582939648 4194304 4964800 140736775818528 18446744073709551615 4313761 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=26433/tid=26460] ppid=26431 vsize=41933556 CPUtime=1371.14 cores=0-31
/proc/26433/task/26460/stat : 26460 (claspmt) R 26431 26433 26258 0 -1 4202560 760288 0 0 0 135969 1145 0 0 25 0 3 0 25680013 42939961344 10246142 270582939648 4194304 4964800 140736775818528 18446744073709551615 4313736 0 0 4096 18946 0 0 0 -1 24 0 0 0
Current children cumulated CPU time (s) 58685.4
Current children cumulated vsize (KiB) 41933556

[startup+5102.3 s]
/proc/loadavg: 32.06 31.83 25.65 3/417 26614
/proc/meminfo: memFree=221352404/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=0 CPUtime=58688.1 cores=0-31
/proc/26433/stat : 26433 (claspmt) Z 26431 26433 26258 0 -1 4203548 10591714 0 0 0 5844745 24060 0 0 18 0 2 0 25679725 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/26433/statm: 0 0 0 0 0 0 0
[pid=26433/tid=26444] ppid=26431 vsize=0 CPUtime=1369.94 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) R 26431 26433 26258 0 -1 4203588 819601 0 0 0 135733 1261 0 0 25 0 2 0 25680011 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 -1 11 0 0 0
Current children cumulated CPU time (s) 58688.1
Current children cumulated vsize (KiB) 0

[startup+5103.1 s]
/proc/loadavg: 29.57 31.32 25.52 2/415 26614
/proc/meminfo: memFree=238027824/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=0 CPUtime=58688.9 cores=0-31
/proc/26433/stat : 26433 (claspmt) Z 26431 26433 26258 0 -1 4203548 10591714 0 0 0 5844745 24140 0 0 18 0 2 0 25679725 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/26433/statm: 0 0 0 0 0 0 0
[pid=26433/tid=26444] ppid=26431 vsize=0 CPUtime=1370.74 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) R 26431 26433 26258 0 -1 4203588 819601 0 0 0 135733 1341 0 0 25 0 2 0 25680011 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 -1 11 0 0 0
Current children cumulated CPU time (s) 58688.8
Current children cumulated vsize (KiB) 0

[startup+5103.5 s]
/proc/loadavg: 29.57 31.32 25.52 2/415 26614
/proc/meminfo: memFree=238027824/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=0 CPUtime=58689.2 cores=0-31
/proc/26433/stat : 26433 (claspmt) Z 26431 26433 26258 0 -1 4203548 10591714 0 0 0 5844745 24180 0 0 18 0 2 0 25679725 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/26433/statm: 0 0 0 0 0 0 0
[pid=26433/tid=26444] ppid=26431 vsize=0 CPUtime=1371.14 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) R 26431 26433 26258 0 -1 4203588 819601 0 0 0 135733 1381 0 0 25 0 2 0 25680011 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 -1 11 0 0 0
Current children cumulated CPU time (s) 58689.2
Current children cumulated vsize (KiB) 0

[startup+5103.9 s]
/proc/loadavg: 29.57 31.32 25.52 2/415 26614
/proc/meminfo: memFree=238027824/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=0 CPUtime=58689.6 cores=0-31
/proc/26433/stat : 26433 (claspmt) Z 26431 26433 26258 0 -1 4203548 10591714 0 0 0 5844745 24220 0 0 18 0 2 0 25679725 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/26433/statm: 0 0 0 0 0 0 0
[pid=26433/tid=26444] ppid=26431 vsize=0 CPUtime=1371.54 cores=0-31
/proc/26433/task/26444/stat : 26444 (claspmt) R 26431 26433 26258 0 -1 4203588 819601 0 0 0 135733 1421 0 0 25 0 2 0 25680011 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 -1 11 0 0 0
Current children cumulated CPU time (s) 58689.6
Current children cumulated vsize (KiB) 0

[startup+5104 s]
/proc/loadavg: 29.57 31.32 25.52 4/415 26614
/proc/meminfo: memFree=262374992/264257624 swapFree=67111528/67111528
[pid=26433] ppid=26431 vsize=0 CPUtime=58689.8 cores=0-31
/proc/26433/stat : 26433 (claspmt) Z 26431 26433 26258 0 -1 4203548 10591714 0 0 0 5844745 24232 0 0 18 0 1 0 25679725 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 11 0 0 0
/proc/26433/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 58689.8
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5104.02
CPU time (s): 58689.8
CPU user time (s): 58447.5
CPU system time (s): 242.321
CPU usage (%): 1149.87
Max. virtual memory (cumulated for all children) (KiB): 41933556

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 58447.5
system time used= 242.321
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 10591714
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1066467
involuntary context switches= 105328

runsolver used 26.9439 second user time and 112.368 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-09 09:02:08
IDJOB=3472387
IDBENCH=82413
IDSOLVER=1961
FILE ID=node200/3472387-1307602923
RUNJOBID= node200-1307602005-26272
PBS_JOBID= 13547727
Free space on /tmp= 61628 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.030-NOTKNOWN.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472387-1307602923/watcher-3472387-1307602923 -o /tmp/evaluation-result-3472387-1307602923/solver-3472387-1307602923 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472387-1307602923.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= c6d3480970bb2819448dd0a800a3cf1c
RANDOM SEED=55222095

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262410772 kB
Buffers:        214328 kB
Cached:        1343216 kB
SwapCached:          0 kB
Active:         355552 kB
Inactive:      1268420 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262410772 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:          121524 kB
Writeback:           0 kB
AnonPages:       66048 kB
Mapped:          14468 kB
Slab:           146388 kB
PageTables:       4400 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   161808 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61624 MiB
End job on node200 at 2011-06-09 10:27:13