Trace number 3472364

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
claspmt 2.0-R4095-patched (fixed)? (TO) 156488 5105.31

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.045-NOTKNOWN.cnf
MD5SUM0385ba51f44fb47d0b0726e26312816f
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables171676
Number of clauses858766
Sum of the clauses size2046721
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2847665
Number of clauses of size 32700
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 58190

Solver Data 

0.00/0.00	c clasp version 2.0.0-RC2
0.00/0.00	c Reading from ...525970/instance-3472364-1307525970.cnf
1.29/1.35	c Solving...
156482.56/5100.03	
156482.56/5100.03	*** INTERRUPTED! ***
156482.56/5100.03	*** Shutting down threads...

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3472364-1307525970/watcher-3472364-1307525970 -o /tmp/evaluation-result-3472364-1307525970/solver-3472364-1307525970 -C 160000 -W 5100 -M 256000 HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472364-1307525970.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 262144000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 262195200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.40 5.21 8.18 1/413 21885
/proc/meminfo: memFree=262926536/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=42672 CPUtime=0.03 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4202496 4833 0 0 0 2 1 0 0 21 0 1 0 17984048 43696128 4762 270582939648 4194304 4964800 140737034341616 18446744073709551615 4700412 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/21885/statm: 10668 4770 373 189 0 5542 0

[startup+0.0562471 s]
/proc/loadavg: 5.40 5.21 8.18 1/413 21885
/proc/meminfo: memFree=262926536/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=54752 CPUtime=0.04 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4202496 10039 0 0 0 3 1 0 0 21 0 1 0 17984048 56066048 8627 270582939648 4194304 4964800 140737034341616 18446744073709551615 4367404 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/21885/statm: 13688 8642 377 189 0 8562 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 54752

[startup+0.101257 s]
/proc/loadavg: 5.40 5.21 8.18 1/413 21885
/proc/meminfo: memFree=262926536/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=57184 CPUtime=0.09 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4202496 10943 0 0 0 7 2 0 0 21 0 1 0 17984048 58556416 9360 270582939648 4194304 4964800 140737034341616 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/21885/statm: 14296 9360 386 189 0 9170 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 57184

[startup+0.301293 s]
/proc/loadavg: 5.40 5.21 8.18 1/413 21885
/proc/meminfo: memFree=262926536/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=79728 CPUtime=0.29 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4202496 18916 0 0 0 25 4 0 0 22 0 1 0 17984048 81641472 14834 270582939648 4194304 4964800 140737034341616 18446744073709551615 258650616632 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/21885/statm: 19965 14836 386 189 0 14839 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 79728

[startup+0.70126 s]
/proc/loadavg: 5.40 5.21 8.18 1/413 21885
/proc/meminfo: memFree=262926536/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=108812 CPUtime=0.69 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4202496 27654 0 0 0 63 6 0 0 23 0 1 0 17984048 111423488 21854 270582939648 4194304 4964800 140737034341616 18446744073709551615 4612197 0 0 4096 18946 0 0 0 17 8 0 0 0
/proc/21885/statm: 27203 21854 401 189 0 22077 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 108812

[startup+1.50433 s]
/proc/loadavg: 5.05 5.14 8.14 2/414 21886
/proc/meminfo: memFree=262838472/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=754424 CPUtime=5.07 cores=0-31
/proc/21885/stat : 21885 (claspmt) S 21883 21885 21777 0 -1 4202496 63914 0 0 0 143 364 0 0 18 0 32 0 17984048 772530176 46860 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 20 0 0 0
/proc/21885/statm: 188606 46870 428 189 0 182955 0
Current children cumulated CPU time (s) 5.07
Current children cumulated vsize (KiB) 754424

[startup+3.10231 s]
/proc/loadavg: 5.05 5.14 8.14 4/445 21917
/proc/meminfo: memFree=262486148/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=1239328 CPUtime=23.81 cores=0-31
/proc/21885/stat : 21885 (claspmt) S 21883 21885 21777 0 -1 4202496 187828 0 0 0 710 1671 0 0 18 0 32 0 17984048 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 20 0 0 0
/proc/21885/statm: 309832 137217 440 189 0 304181 0
[pid=21885/tid=21887] ppid=21883 vsize=1239328 CPUtime=0.63 cores=0-31
/proc/21885/task/21887/stat : 21887 (claspmt) S 21883 21885 21777 0 -1 4202560 4331 0 0 0 4 59 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604643549822432 0 0 -1 1 0 0 0
[pid=21885/tid=21888] ppid=21883 vsize=1239328 CPUtime=0.66 cores=0-31
/proc/21885/task/21888/stat : 21888 (claspmt) S 21883 21885 21777 0 -1 4202560 5981 0 0 0 11 55 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604637053033952 0 0 -1 14 0 0 0
[pid=21885/tid=21889] ppid=21883 vsize=1239328 CPUtime=0.69 cores=0-31
/proc/21885/task/21889/stat : 21889 (claspmt) S 21883 21885 21777 0 -1 4202560 5308 0 0 0 10 59 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604641246510560 0 0 -1 3 0 0 0
[pid=21885/tid=21890] ppid=21883 vsize=1239328 CPUtime=0.47 cores=0-31
/proc/21885/task/21890/stat : 21890 (claspmt) S 21883 21885 21777 0 -1 4202560 4644 0 0 0 2 45 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604499278921184 0 0 -1 28 0 0 0
[pid=21885/tid=21891] ppid=21883 vsize=1239328 CPUtime=1.44 cores=0-31
/proc/21885/task/21891/stat : 21891 (claspmt) R 21883 21885 21777 0 -1 4202560 4300 0 0 0 86 58 0 0 19 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 4399292 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=21885/tid=21892] ppid=21883 vsize=1239328 CPUtime=0.64 cores=0-31
/proc/21885/task/21892/stat : 21892 (claspmt) S 21883 21885 21777 0 -1 4202560 6031 0 0 0 10 54 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502155394528 0 0 -1 29 0 0 0
[pid=21885/tid=21893] ppid=21883 vsize=1239328 CPUtime=0.69 cores=0-31
/proc/21885/task/21893/stat : 21893 (claspmt) S 21883 21885 21777 0 -1 4202560 5751 0 0 0 13 56 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604505587744224 0 0 -1 28 0 0 0
[pid=21885/tid=21894] ppid=21883 vsize=1239328 CPUtime=0.48 cores=0-31
/proc/21885/task/21894/stat : 21894 (claspmt) S 21883 21885 21777 0 -1 4202560 4646 0 0 0 4 44 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604564120133088 0 0 -1 20 0 0 0
[pid=21885/tid=21895] ppid=21883 vsize=1239328 CPUtime=0.53 cores=0-31
/proc/21885/task/21895/stat : 21895 (claspmt) S 21883 21885 21777 0 -1 4202560 4566 0 0 0 4 49 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604504547073504 0 0 -1 11 0 0 0
[pid=21885/tid=21896] ppid=21883 vsize=1239328 CPUtime=0.62 cores=0-31
/proc/21885/task/21896/stat : 21896 (claspmt) S 21883 21885 21777 0 -1 4202560 6045 0 0 0 7 55 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604642724208096 0 0 -1 16 0 0 0
[pid=21885/tid=21897] ppid=21883 vsize=1239328 CPUtime=0.63 cores=0-31
/proc/21885/task/21897/stat : 21897 (claspmt) S 21883 21885 21777 0 -1 4202560 5961 0 0 0 8 55 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502229286368 0 0 -1 24 0 0 0
[pid=21885/tid=21898] ppid=21883 vsize=1239328 CPUtime=0.52 cores=0-31
/proc/21885/task/21898/stat : 21898 (claspmt) S 21883 21885 21777 0 -1 4202560 4389 0 0 0 2 50 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604498127953376 0 0 -1 8 0 0 0
[pid=21885/tid=21899] ppid=21883 vsize=1239328 CPUtime=1.42 cores=0-31
/proc/21885/task/21899/stat : 21899 (claspmt) R 21883 21885 21777 0 -1 4202560 4794 0 0 0 83 59 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 4399504 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=21885/tid=21900] ppid=21883 vsize=1239328 CPUtime=0.64 cores=0-31
/proc/21885/task/21900/stat : 21900 (claspmt) S 21883 21885 21777 0 -1 4202560 6165 0 0 0 9 55 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604643642555872 0 0 -1 19 0 0 0
[pid=21885/tid=21901] ppid=21883 vsize=1239328 CPUtime=0.72 cores=0-31
/proc/21885/task/21901/stat : 21901 (claspmt) S 21883 21885 21777 0 -1 4202560 5562 0 0 0 13 59 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604639174548960 0 0 -1 2 0 0 0
[pid=21885/tid=21902] ppid=21883 vsize=1239328 CPUtime=0.52 cores=0-31
/proc/21885/task/21902/stat : 21902 (claspmt) S 21883 21885 21777 0 -1 4202560 4301 0 0 0 5 47 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604642283240928 0 0 -1 7 0 0 0
[pid=21885/tid=21903] ppid=21883 vsize=1239328 CPUtime=0.51 cores=0-31
/proc/21885/task/21903/stat : 21903 (claspmt) S 21883 21885 21777 0 -1 4202560 4344 0 0 0 4 47 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604505877986784 0 0 -1 27 0 0 0
[pid=21885/tid=21904] ppid=21883 vsize=1239328 CPUtime=0.66 cores=0-31
/proc/21885/task/21904/stat : 21904 (claspmt) S 21883 21885 21777 0 -1 4202560 5430 0 0 0 9 57 0 0 18 0 32 0 17984184 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604639708274144 0 0 -1 26 0 0 0
[pid=21885/tid=21905] ppid=21883 vsize=1239328 CPUtime=0.67 cores=0-31
/proc/21885/task/21905/stat : 21905 (claspmt) S 21883 21885 21777 0 -1 4202560 5962 0 0 0 10 57 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604499278912992 0 0 -1 10 0 0 0
[pid=21885/tid=21906] ppid=21883 vsize=1239328 CPUtime=0.58 cores=0-31
/proc/21885/task/21906/stat : 21906 (claspmt) S 21883 21885 21777 0 -1 4202560 4343 0 0 0 4 54 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604642897378784 0 0 -1 6 0 0 0
[pid=21885/tid=21907] ppid=21883 vsize=1239328 CPUtime=1.44 cores=0-31
/proc/21885/task/21907/stat : 21907 (claspmt) R 21883 21885 21777 0 -1 4202560 4858 0 0 0 87 57 0 0 19 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669119 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=21885/tid=21908] ppid=21883 vsize=1239328 CPUtime=0.6 cores=0-31
/proc/21885/task/21908/stat : 21908 (claspmt) S 21883 21885 21777 0 -1 4202560 5742 0 0 0 8 52 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604505877978592 0 0 -1 4 0 0 0
[pid=21885/tid=21909] ppid=21883 vsize=1239328 CPUtime=0.67 cores=0-31
/proc/21885/task/21909/stat : 21909 (claspmt) S 21883 21885 21777 0 -1 4202560 4978 0 0 0 13 54 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604643458792928 0 0 -1 21 0 0 0
[pid=21885/tid=21910] ppid=21883 vsize=1239328 CPUtime=0.55 cores=0-31
/proc/21885/task/21910/stat : 21910 (claspmt) S 21883 21885 21777 0 -1 4202560 4449 0 0 0 3 52 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502229278176 0 0 -1 9 0 0 0
[pid=21885/tid=21911] ppid=21883 vsize=1239328 CPUtime=0.5 cores=0-31
/proc/21885/task/21911/stat : 21911 (claspmt) S 21883 21885 21777 0 -1 4202560 4320 0 0 0 5 45 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502229269984 0 0 -1 15 0 0 0
[pid=21885/tid=21912] ppid=21883 vsize=1239328 CPUtime=0.68 cores=0-31
/proc/21885/task/21912/stat : 21912 (claspmt) S 21883 21885 21777 0 -1 4202560 5881 0 0 0 9 59 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604499278896608 0 0 -1 27 0 0 0
[pid=21885/tid=21913] ppid=21883 vsize=1239328 CPUtime=0.66 cores=0-31
/proc/21885/task/21913/stat : 21913 (claspmt) S 21883 21885 21777 0 -1 4202560 5962 0 0 0 11 55 0 0 18 0 32 0 17984185 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604640593608160 0 0 -1 18 0 0 0
[pid=21885/tid=21914] ppid=21883 vsize=1239328 CPUtime=0.55 cores=0-31
/proc/21885/task/21914/stat : 21914 (claspmt) S 21883 21885 21777 0 -1 4202560 4436 0 0 0 3 52 0 0 18 0 32 0 17984186 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604642283281888 0 0 -1 22 0 0 0
[pid=21885/tid=21915] ppid=21883 vsize=1239328 CPUtime=1.51 cores=0-31
/proc/21885/task/21915/stat : 21915 (claspmt) R 21883 21885 21777 0 -1 4202560 4592 0 0 0 104 47 0 0 18 0 32 0 17984186 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 4668920 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=21885/tid=21916] ppid=21883 vsize=1239328 CPUtime=0.62 cores=0-31
/proc/21885/task/21916/stat : 21916 (claspmt) S 21883 21885 21777 0 -1 4202560 5916 0 0 0 12 50 0 0 18 0 32 0 17984186 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604500743986656 0 0 -1 23 0 0 0
[pid=21885/tid=21917] ppid=21883 vsize=1239328 CPUtime=0.68 cores=0-31
/proc/21885/task/21917/stat : 21917 (claspmt) S 21883 21885 21777 0 -1 4202560 5652 0 0 0 13 55 0 0 18 0 32 0 17984186 1269071872 137217 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502155378144 0 0 -1 13 0 0 0
Current children cumulated CPU time (s) 23.81
Current children cumulated vsize (KiB) 1239328

[startup+6.30144 s]
/proc/loadavg: 5.05 5.14 8.14 5/445 21917
/proc/meminfo: memFree=262379112/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=1239328 CPUtime=36.6 cores=0-31
/proc/21885/stat : 21885 (claspmt) S 21883 21885 21777 0 -1 4202496 187899 0 0 0 1989 1671 0 0 18 0 32 0 17984048 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446744073709551615 0 0 17 20 0 0 0
/proc/21885/statm: 309832 137288 440 189 0 304181 0
[pid=21885/tid=21887] ppid=21883 vsize=1239328 CPUtime=0.63 cores=0-31
/proc/21885/task/21887/stat : 21887 (claspmt) S 21883 21885 21777 0 -1 4202560 4331 0 0 0 4 59 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604643549822432 0 0 -1 1 0 0 0
[pid=21885/tid=21888] ppid=21883 vsize=1239328 CPUtime=0.66 cores=0-31
/proc/21885/task/21888/stat : 21888 (claspmt) S 21883 21885 21777 0 -1 4202560 5981 0 0 0 11 55 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604637053033952 0 0 -1 14 0 0 0
[pid=21885/tid=21889] ppid=21883 vsize=1239328 CPUtime=0.69 cores=0-31
/proc/21885/task/21889/stat : 21889 (claspmt) S 21883 21885 21777 0 -1 4202560 5308 0 0 0 10 59 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604641246510560 0 0 -1 3 0 0 0
[pid=21885/tid=21890] ppid=21883 vsize=1239328 CPUtime=0.47 cores=0-31
/proc/21885/task/21890/stat : 21890 (claspmt) S 21883 21885 21777 0 -1 4202560 4644 0 0 0 2 45 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604499278921184 0 0 -1 28 0 0 0
[pid=21885/tid=21891] ppid=21883 vsize=1239328 CPUtime=4.64 cores=0-31
/proc/21885/task/21891/stat : 21891 (claspmt) R 21883 21885 21777 0 -1 4202560 4317 0 0 0 406 58 0 0 25 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 4678036 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=21885/tid=21892] ppid=21883 vsize=1239328 CPUtime=0.64 cores=0-31
/proc/21885/task/21892/stat : 21892 (claspmt) S 21883 21885 21777 0 -1 4202560 6031 0 0 0 10 54 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502155394528 0 0 -1 29 0 0 0
[pid=21885/tid=21893] ppid=21883 vsize=1239328 CPUtime=0.69 cores=0-31
/proc/21885/task/21893/stat : 21893 (claspmt) S 21883 21885 21777 0 -1 4202560 5751 0 0 0 13 56 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604505587744224 0 0 -1 28 0 0 0
[pid=21885/tid=21894] ppid=21883 vsize=1239328 CPUtime=0.48 cores=0-31
/proc/21885/task/21894/stat : 21894 (claspmt) S 21883 21885 21777 0 -1 4202560 4646 0 0 0 4 44 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604564120133088 0 0 -1 20 0 0 0
[pid=21885/tid=21895] ppid=21883 vsize=1239328 CPUtime=0.53 cores=0-31
/proc/21885/task/21895/stat : 21895 (claspmt) S 21883 21885 21777 0 -1 4202560 4566 0 0 0 4 49 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604504547073504 0 0 -1 11 0 0 0
[pid=21885/tid=21896] ppid=21883 vsize=1239328 CPUtime=0.62 cores=0-31
/proc/21885/task/21896/stat : 21896 (claspmt) S 21883 21885 21777 0 -1 4202560 6045 0 0 0 7 55 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604642724208096 0 0 -1 16 0 0 0
[pid=21885/tid=21897] ppid=21883 vsize=1239328 CPUtime=0.63 cores=0-31
/proc/21885/task/21897/stat : 21897 (claspmt) S 21883 21885 21777 0 -1 4202560 5961 0 0 0 8 55 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502229286368 0 0 -1 24 0 0 0
[pid=21885/tid=21898] ppid=21883 vsize=1239328 CPUtime=0.52 cores=0-31
/proc/21885/task/21898/stat : 21898 (claspmt) S 21883 21885 21777 0 -1 4202560 4389 0 0 0 2 50 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604498127953376 0 0 -1 8 0 0 0
[pid=21885/tid=21899] ppid=21883 vsize=1239328 CPUtime=4.62 cores=0-31
/proc/21885/task/21899/stat : 21899 (claspmt) R 21883 21885 21777 0 -1 4202560 4803 0 0 0 403 59 0 0 25 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=21885/tid=21900] ppid=21883 vsize=1239328 CPUtime=0.64 cores=0-31
/proc/21885/task/21900/stat : 21900 (claspmt) S 21883 21885 21777 0 -1 4202560 6165 0 0 0 9 55 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604643642555872 0 0 -1 19 0 0 0
[pid=21885/tid=21901] ppid=21883 vsize=1239328 CPUtime=0.72 cores=0-31
/proc/21885/task/21901/stat : 21901 (claspmt) S 21883 21885 21777 0 -1 4202560 5562 0 0 0 13 59 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604639174548960 0 0 -1 2 0 0 0
[pid=21885/tid=21902] ppid=21883 vsize=1239328 CPUtime=0.52 cores=0-31
/proc/21885/task/21902/stat : 21902 (claspmt) S 21883 21885 21777 0 -1 4202560 4301 0 0 0 5 47 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604642283240928 0 0 -1 7 0 0 0
[pid=21885/tid=21903] ppid=21883 vsize=1239328 CPUtime=0.51 cores=0-31
/proc/21885/task/21903/stat : 21903 (claspmt) S 21883 21885 21777 0 -1 4202560 4344 0 0 0 4 47 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604505877986784 0 0 -1 27 0 0 0
[pid=21885/tid=21904] ppid=21883 vsize=1239328 CPUtime=0.66 cores=0-31
/proc/21885/task/21904/stat : 21904 (claspmt) S 21883 21885 21777 0 -1 4202560 5430 0 0 0 9 57 0 0 18 0 32 0 17984184 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604639708274144 0 0 -1 26 0 0 0
[pid=21885/tid=21905] ppid=21883 vsize=1239328 CPUtime=0.67 cores=0-31
/proc/21885/task/21905/stat : 21905 (claspmt) S 21883 21885 21777 0 -1 4202560 5962 0 0 0 10 57 0 0 18 0 32 0 17984185 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604499278912992 0 0 -1 10 0 0 0
[pid=21885/tid=21906] ppid=21883 vsize=1239328 CPUtime=0.58 cores=0-31
/proc/21885/task/21906/stat : 21906 (claspmt) S 21883 21885 21777 0 -1 4202560 4343 0 0 0 4 54 0 0 18 0 32 0 17984185 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604642897378784 0 0 -1 6 0 0 0
[pid=21885/tid=21907] ppid=21883 vsize=1239328 CPUtime=4.64 cores=0-31
/proc/21885/task/21907/stat : 21907 (claspmt) R 21883 21885 21777 0 -1 4202560 4895 0 0 0 407 57 0 0 25 0 32 0 17984185 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 4342193 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=21885/tid=21908] ppid=21883 vsize=1239328 CPUtime=0.6 cores=0-31
/proc/21885/task/21908/stat : 21908 (claspmt) S 21883 21885 21777 0 -1 4202560 5742 0 0 0 8 52 0 0 18 0 32 0 17984185 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604505877978592 0 0 -1 4 0 0 0
[pid=21885/tid=21909] ppid=21883 vsize=1239328 CPUtime=0.67 cores=0-31
/proc/21885/task/21909/stat : 21909 (claspmt) S 21883 21885 21777 0 -1 4202560 4978 0 0 0 13 54 0 0 18 0 32 0 17984185 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604643458792928 0 0 -1 21 0 0 0
[pid=21885/tid=21910] ppid=21883 vsize=1239328 CPUtime=0.55 cores=0-31
/proc/21885/task/21910/stat : 21910 (claspmt) S 21883 21885 21777 0 -1 4202560 4449 0 0 0 3 52 0 0 18 0 32 0 17984185 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502229278176 0 0 -1 9 0 0 0
[pid=21885/tid=21911] ppid=21883 vsize=1239328 CPUtime=0.5 cores=0-31
/proc/21885/task/21911/stat : 21911 (claspmt) S 21883 21885 21777 0 -1 4202560 4320 0 0 0 5 45 0 0 18 0 32 0 17984185 1269071872 137288 270582939648 4194304 4964800 140737034341616 18446744073709551615 258662772457 0 0 4096 18946 18446604502229269984 0 0 -1 15 0 0 0

################
# More data... #
################

[pid=21885/tid=21903] ppid=21883 vsize=66926324 CPUtime=4841.6 cores=0-31
/proc/21885/task/21903/stat : 21903 (claspmt) R 21883 21885 21777 0 -1 4202560 216803 0 0 0 483567 593 0 0 25 0 32 0 17984184 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21885/tid=21904] ppid=21883 vsize=66926324 CPUtime=4844.27 cores=0-31
/proc/21885/task/21904/stat : 21904 (claspmt) R 21883 21885 21777 0 -1 4202560 385344 0 0 0 483848 579 0 0 25 0 32 0 17984184 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21885/tid=21905] ppid=21883 vsize=66926324 CPUtime=4845.87 cores=0-31
/proc/21885/task/21905/stat : 21905 (claspmt) R 21883 21885 21777 0 -1 4202560 323153 0 0 0 483891 696 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=21885/tid=21906] ppid=21883 vsize=66926324 CPUtime=4842.33 cores=0-31
/proc/21885/task/21906/stat : 21906 (claspmt) R 21883 21885 21777 0 -1 4202560 604573 0 0 0 483061 1172 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4362377 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=21885/tid=21907] ppid=21883 vsize=66926324 CPUtime=5079.08 cores=0-31
/proc/21885/task/21907/stat : 21907 (claspmt) R 21883 21885 21777 0 -1 4202560 5667 0 0 0 507850 58 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=21885/tid=21908] ppid=21883 vsize=66926324 CPUtime=4835.89 cores=0-31
/proc/21885/task/21908/stat : 21908 (claspmt) R 21883 21885 21777 0 -1 4202560 693385 0 0 0 482672 917 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4632355 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21885/tid=21909] ppid=21883 vsize=66926324 CPUtime=4843.5 cores=0-31
/proc/21885/task/21909/stat : 21909 (claspmt) R 21883 21885 21777 0 -1 4202560 1338585 0 0 0 482713 1637 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=21885/tid=21910] ppid=21883 vsize=66926324 CPUtime=4833.36 cores=0-31
/proc/21885/task/21910/stat : 21910 (claspmt) R 21883 21885 21777 0 -1 4202560 634489 0 0 0 482133 1203 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669136 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=21885/tid=21911] ppid=21883 vsize=66926324 CPUtime=4845.35 cores=0-31
/proc/21885/task/21911/stat : 21911 (claspmt) R 21883 21885 21777 0 -1 4202560 197147 0 0 0 484001 534 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=21885/tid=21912] ppid=21883 vsize=66926324 CPUtime=4850.19 cores=0-31
/proc/21885/task/21912/stat : 21912 (claspmt) R 21883 21885 21777 0 -1 4202560 391653 0 0 0 484399 620 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670346 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21885/tid=21913] ppid=21883 vsize=66926324 CPUtime=4846.51 cores=0-31
/proc/21885/task/21913/stat : 21913 (claspmt) R 21883 21885 21777 0 -1 4202560 372306 0 0 0 483913 738 0 0 25 0 32 0 17984185 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21885/tid=21914] ppid=21883 vsize=66926324 CPUtime=4846.56 cores=0-31
/proc/21885/task/21914/stat : 21914 (claspmt) R 21883 21885 21777 0 -1 4202560 538289 0 0 0 483623 1033 0 0 25 0 32 0 17984186 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=21885/tid=21915] ppid=21883 vsize=66926324 CPUtime=5079.76 cores=0-31
/proc/21885/task/21915/stat : 21915 (claspmt) R 21883 21885 21777 0 -1 4202560 5084 0 0 0 507928 48 0 0 25 0 32 0 17984186 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=21885/tid=21916] ppid=21883 vsize=66926324 CPUtime=4845.1 cores=0-31
/proc/21885/task/21916/stat : 21916 (claspmt) R 21883 21885 21777 0 -1 4202560 609248 0 0 0 483562 948 0 0 25 0 32 0 17984186 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=21885/tid=21917] ppid=21883 vsize=66926324 CPUtime=4844.02 cores=0-31
/proc/21885/task/21917/stat : 21917 (claspmt) R 21883 21885 21777 0 -1 4202560 1401571 0 0 0 482702 1700 0 0 25 0 32 0 17984186 68532555776 16431220 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 9 0 0 0
Current children cumulated CPU time (s) 155916
Current children cumulated vsize (KiB) 66926324



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 32.05 32.01 31.88 33/445 22077
/proc/meminfo: memFree=197065112/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=66926324 CPUtime=156483 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4202496 16563496 0 0 0 15621300 26956 0 0 25 0 32 0 17984048 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670328 0 0 4096 18946 18446744073709551615 0 0 17 1 0 0 0
/proc/21885/statm: 16731581 16433926 463 189 0 16725930 0
[pid=21885/tid=21887] ppid=21883 vsize=66926324 CPUtime=4852.28 cores=0-31
/proc/21885/task/21887/stat : 21887 (claspmt) R 21883 21885 21777 0 -1 4202560 212927 0 0 0 484665 563 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 4 0 0 0
[pid=21885/tid=21888] ppid=21883 vsize=66926324 CPUtime=4867.92 cores=0-31
/proc/21885/task/21888/stat : 21888 (claspmt) R 21883 21885 21777 0 -1 4202560 450518 0 0 0 486176 616 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 14 0 0 0
[pid=21885/tid=21889] ppid=21883 vsize=66926324 CPUtime=4866.46 cores=0-31
/proc/21885/task/21889/stat : 21889 (claspmt) R 21883 21885 21777 0 -1 4202560 337796 0 0 0 485932 714 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 16 0 0 0
[pid=21885/tid=21890] ppid=21883 vsize=66926324 CPUtime=4860.03 cores=0-31
/proc/21885/task/21890/stat : 21890 (claspmt) R 21883 21885 21777 0 -1 4202560 539325 0 0 0 485000 1003 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669131 0 0 4096 18946 0 0 0 -1 17 0 0 0
[pid=21885/tid=21891] ppid=21883 vsize=66926324 CPUtime=5096.8 cores=0-31
/proc/21885/task/21891/stat : 21891 (claspmt) R 21883 21885 21777 0 -1 4202560 5089 0 0 0 509621 59 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669089 0 0 4096 18946 0 0 0 -1 19 0 0 0
[pid=21885/tid=21892] ppid=21883 vsize=66926324 CPUtime=4858.12 cores=0-31
/proc/21885/task/21892/stat : 21892 (claspmt) R 21883 21885 21777 0 -1 4202560 535586 0 0 0 484998 814 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4668568 0 0 4096 18946 0 0 0 -1 13 0 0 0
[pid=21885/tid=21893] ppid=21883 vsize=66926324 CPUtime=4862.51 cores=0-31
/proc/21885/task/21893/stat : 21893 (claspmt) R 21883 21885 21777 0 -1 4202560 1397496 0 0 0 484562 1689 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670299 0 0 4096 18946 0 0 0 -1 20 0 0 0
[pid=21885/tid=21894] ppid=21883 vsize=66926324 CPUtime=4856.69 cores=0-31
/proc/21885/task/21894/stat : 21894 (claspmt) R 21883 21885 21777 0 -1 4202560 708020 0 0 0 484465 1204 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670180 0 0 4096 18946 0 0 0 -1 28 0 0 0
[pid=21885/tid=21895] ppid=21883 vsize=66926324 CPUtime=4862.65 cores=0-31
/proc/21885/task/21895/stat : 21895 (claspmt) R 21883 21885 21777 0 -1 4202560 218517 0 0 0 485676 589 0 0 25 0 32 0 17984184 68532555776 16433926 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 23 0 0 0
[pid=21885/tid=21896] ppid=21883 vsize=66926324 CPUtime=4864.51 cores=0-31
/proc/21885/task/21896/stat : 21896 (claspmt) R 21883 21885 21777 0 -1 4202560 324927 0 0 0 485928 523 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4320944 0 0 4096 18946 0 0 0 -1 22 0 0 0
[pid=21885/tid=21897] ppid=21883 vsize=66926324 CPUtime=4864.41 cores=0-31
/proc/21885/task/21897/stat : 21897 (claspmt) R 21883 21885 21777 0 -1 4202560 392314 0 0 0 485709 732 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 26 0 0 0
[pid=21885/tid=21898] ppid=21883 vsize=66926324 CPUtime=4856.22 cores=0-31
/proc/21885/task/21898/stat : 21898 (claspmt) R 21883 21885 21777 0 -1 4202560 523767 0 0 0 484700 922 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 29 0 0 0
[pid=21885/tid=21899] ppid=21883 vsize=66926324 CPUtime=5095.33 cores=0-31
/proc/21885/task/21899/stat : 21899 (claspmt) R 21883 21885 21777 0 -1 4202560 5567 0 0 0 509472 61 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 24 0 0 0
[pid=21885/tid=21900] ppid=21883 vsize=66926324 CPUtime=4858.12 cores=0-31
/proc/21885/task/21900/stat : 21900 (claspmt) R 21883 21885 21777 0 -1 4202560 607804 0 0 0 484938 874 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4363327 0 0 4096 18946 0 0 0 -1 18 0 0 0
[pid=21885/tid=21901] ppid=21883 vsize=66926324 CPUtime=4862.5 cores=0-31
/proc/21885/task/21901/stat : 21901 (claspmt) R 21883 21885 21777 0 -1 4202560 1303455 0 0 0 484620 1630 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4627263 0 0 4096 18946 0 0 0 -1 3 0 0 0
[pid=21885/tid=21902] ppid=21883 vsize=66926324 CPUtime=4845.86 cores=0-31
/proc/21885/task/21902/stat : 21902 (claspmt) R 21883 21885 21777 0 -1 4202560 648247 0 0 0 483338 1248 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4669350 0 0 4096 18946 0 0 0 -1 27 0 0 0
[pid=21885/tid=21903] ppid=21883 vsize=66926324 CPUtime=4859.3 cores=0-31
/proc/21885/task/21903/stat : 21903 (claspmt) R 21883 21885 21777 0 -1 4202560 216803 0 0 0 485337 593 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4325367 0 0 4096 18946 0 0 0 -1 0 0 0 0
[pid=21885/tid=21904] ppid=21883 vsize=66926324 CPUtime=4861.99 cores=0-31
/proc/21885/task/21904/stat : 21904 (claspmt) R 21883 21885 21777 0 -1 4202560 385344 0 0 0 485620 579 0 0 25 0 32 0 17984184 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 5 0 0 0
[pid=21885/tid=21905] ppid=21883 vsize=66926324 CPUtime=4863.59 cores=0-31
/proc/21885/task/21905/stat : 21905 (claspmt) R 21883 21885 21777 0 -1 4202560 323153 0 0 0 485663 696 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 21 0 0 0
[pid=21885/tid=21906] ppid=21883 vsize=66926324 CPUtime=4859.53 cores=0-31
/proc/21885/task/21906/stat : 21906 (claspmt) R 21883 21885 21777 0 -1 4202560 604614 0 0 0 484781 1172 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4320951 0 0 4096 18946 0 0 0 -1 31 0 0 0
[pid=21885/tid=21907] ppid=21883 vsize=66926324 CPUtime=5096.8 cores=0-31
/proc/21885/task/21907/stat : 21907 (claspmt) R 21883 21885 21777 0 -1 4202560 5667 0 0 0 509622 58 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4342199 0 0 4096 18946 0 0 0 -1 30 0 0 0
[pid=21885/tid=21908] ppid=21883 vsize=66926324 CPUtime=4853.61 cores=0-31
/proc/21885/task/21908/stat : 21908 (claspmt) R 21883 21885 21777 0 -1 4202560 693385 0 0 0 484444 917 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4363327 0 0 4096 18946 0 0 0 -1 7 0 0 0
[pid=21885/tid=21909] ppid=21883 vsize=66926324 CPUtime=4861.21 cores=0-31
/proc/21885/task/21909/stat : 21909 (claspmt) R 21883 21885 21777 0 -1 4202560 1338585 0 0 0 484484 1637 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 8 0 0 0
[pid=21885/tid=21910] ppid=21883 vsize=66926324 CPUtime=4851.08 cores=0-31
/proc/21885/task/21910/stat : 21910 (claspmt) R 21883 21885 21777 0 -1 4202560 634489 0 0 0 483905 1203 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 10 0 0 0
[pid=21885/tid=21911] ppid=21883 vsize=66926324 CPUtime=4863.06 cores=0-31
/proc/21885/task/21911/stat : 21911 (claspmt) R 21883 21885 21777 0 -1 4202560 197147 0 0 0 485772 534 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 12 0 0 0
[pid=21885/tid=21912] ppid=21883 vsize=66926324 CPUtime=4867.9 cores=0-31
/proc/21885/task/21912/stat : 21912 (claspmt) R 21883 21885 21777 0 -1 4202560 391656 0 0 0 486170 620 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4320970 0 0 4096 18946 0 0 0 -1 6 0 0 0
[pid=21885/tid=21913] ppid=21883 vsize=66926324 CPUtime=4864.23 cores=0-31
/proc/21885/task/21913/stat : 21913 (claspmt) R 21883 21885 21777 0 -1 4202560 372306 0 0 0 485685 738 0 0 25 0 32 0 17984185 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 2 0 0 0
[pid=21885/tid=21914] ppid=21883 vsize=66926324 CPUtime=4864.28 cores=0-31
/proc/21885/task/21914/stat : 21914 (claspmt) R 21883 21885 21777 0 -1 4202560 538289 0 0 0 485395 1033 0 0 25 0 32 0 17984186 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670308 0 0 4096 18946 0 0 0 -1 15 0 0 0
[pid=21885/tid=21915] ppid=21883 vsize=66926324 CPUtime=5097.48 cores=0-31
/proc/21885/task/21915/stat : 21915 (claspmt) R 21883 21885 21777 0 -1 4202560 5084 0 0 0 509700 48 0 0 25 0 32 0 17984186 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4670155 0 0 4096 18946 0 0 0 -1 25 0 0 0
[pid=21885/tid=21916] ppid=21883 vsize=66926324 CPUtime=4862.82 cores=0-31
/proc/21885/task/21916/stat : 21916 (claspmt) R 21883 21885 21777 0 -1 4202560 609752 0 0 0 485333 949 0 0 25 0 32 0 17984186 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4332557 0 0 4096 18946 0 0 0 -1 11 0 0 0
[pid=21885/tid=21917] ppid=21883 vsize=66926324 CPUtime=4861.74 cores=0-31
/proc/21885/task/21917/stat : 21917 (claspmt) R 21883 21885 21777 0 -1 4202560 1401571 0 0 0 484474 1700 0 0 25 0 32 0 17984186 68532555776 16433928 270582939648 4194304 4964800 140737034341616 18446744073709551615 4668987 0 0 4096 18946 0 0 0 -1 9 0 0 0
Current children cumulated CPU time (s) 156483
Current children cumulated vsize (KiB) 66926324

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -21885

Solver just ended. Dumping a history of the last processes samples

[startup+5100.6 s]
/proc/loadavg: 32.05 32.01 31.88 2/415 22078
/proc/meminfo: memFree=197093828/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=66806448 CPUtime=156483 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4202496 16563517 0 0 0 15621392 26956 0 0 25 0 1 0 17984048 68409802752 16424287 270582939648 4194304 4964800 140737034341616 18446744073709551615 258650611056 0 0 4096 18946 0 0 0 17 1 0 0 0
/proc/21885/statm: 16701612 16424287 474 189 0 16695961 0
Current children cumulated CPU time (s) 156483
Current children cumulated vsize (KiB) 66806448

[startup+5103.8 s]
/proc/loadavg: 29.56 31.50 31.71 2/414 22078
/proc/meminfo: memFree=205458752/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=0 CPUtime=156487 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4203524 16563517 0 0 0 15621532 27133 0 0 25 0 1 0 17984048 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 19 0 0 0
/proc/21885/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 156487
Current children cumulated vsize (KiB) 0

[startup+5104.6 s]
/proc/loadavg: 29.56 31.50 31.71 2/414 22078
/proc/meminfo: memFree=229985312/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=0 CPUtime=156487 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4203524 16563517 0 0 0 15621532 27213 0 0 25 0 1 0 17984048 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 19 0 0 0
/proc/21885/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 156487
Current children cumulated vsize (KiB) 0

[startup+5105 s]
/proc/loadavg: 29.56 31.50 31.71 2/414 22078
/proc/meminfo: memFree=256175360/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=0 CPUtime=156488 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4203524 16563517 0 0 0 15621532 27255 0 0 25 0 1 0 17984048 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 19 0 0 0
/proc/21885/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 156488
Current children cumulated vsize (KiB) 0

[startup+5105.2 s]
/proc/loadavg: 29.56 31.50 31.71 2/414 22078
/proc/meminfo: memFree=256175360/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=0 CPUtime=156488 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4203524 16563517 0 0 0 15621532 27273 0 0 25 0 1 0 17984048 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 19 0 0 0
/proc/21885/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 156488
Current children cumulated vsize (KiB) 0

[startup+5105.3 s]
/proc/loadavg: 29.56 31.50 31.71 2/414 22078
/proc/meminfo: memFree=256175360/264257624 swapFree=67111528/67111528
[pid=21885] ppid=21883 vsize=0 CPUtime=156488 cores=0-31
/proc/21885/stat : 21885 (claspmt) R 21883 21885 21777 0 -1 4203524 16563517 0 0 0 15621532 27283 0 0 25 0 1 0 17984048 0 0 270582939648 0 0 0 0 0 0 0 4096 18946 18446744073709551615 0 0 17 19 0 0 0
/proc/21885/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 156488
Current children cumulated vsize (KiB) 0

Child ended because it received signal 9 (SIGKILL)
Real time (s): 5105.31
CPU time (s): 156488
CPU user time (s): 156215
CPU system time (s): 272.848
CPU usage (%): 3065.2
Max. virtual memory (cumulated for all children) (KiB): 66926324

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 156215
system time used= 272.848
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 16563517
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3607260
involuntary context switches= 401942

runsolver used 29.7895 second user time and 113.706 second system time

The end

Launcher Data 

Begin job on node200 at 2011-06-08 11:39:32
IDJOB=3472364
IDBENCH=82369
IDSOLVER=1961
FILE ID=node200/3472364-1307525970
RUNJOBID= node200-1307525069-21791
PBS_JOBID= 13514240
Free space on /tmp= 61728 MiB

SOLVER NAME= claspmt 2.0-R4095-patched (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_1.045-NOTKNOWN.cnf
COMMAND LINE= HOME/claspmt -tNBCORE -p HOME/sat11-port.txt --distribute=all,2 BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3472364-1307525970/watcher-3472364-1307525970 -o /tmp/evaluation-result-3472364-1307525970/solver-3472364-1307525970 -C 160000 -W 5100 -M 256000  HOME/claspmt -t32 -p HOME/sat11-port.txt --distribute=all,2 HOME/instance-3472364-1307525970.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 256000 MiB
NBCORE= 32

MD5SUM BENCH= 0385ba51f44fb47d0b0726e26312816f
RANDOM SEED=1533332535

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.99
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.03
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.97
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.10
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.18
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.045
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      262926816 kB
Buffers:        172516 kB
Cached:         897072 kB
SwapCached:          0 kB
Active:         208144 kB
Inactive:       925200 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      262926816 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           15972 kB
Writeback:           0 kB
AnonPages:       63900 kB
Mapped:          14532 kB
Slab:           121364 kB
PageTables:       4952 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   160532 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61728 MiB
End job on node200 at 2011-06-08 13:04:37