Trace number 3347373

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 160003 5010.14

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.025-NOTKNOWN.cnf
MD5SUM3c5c73741ba21d5f39c7f436ba834743
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark56366.2
Satisfiable
(Un)Satisfiability was proved
Number of variables95456
Number of clauses483561
Sum of the clauses size1149911
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2477300
Number of clauses of size 31500
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 54550

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347373-1305085703/watcher-3347373-1305085703 -o /tmp/evaluation-result-3347373-1305085703/solver-3347373-1305085703 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347373-1305085703.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.08 30.98 31.71 1/413 7112
/proc/meminfo: memFree=256172704/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 21 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0

[startup+0.0579961 s]
/proc/loadavg: 27.08 30.98 31.71 1/413 7112
/proc/meminfo: memFree=256172704/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 21 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100974 s]
/proc/loadavg: 27.08 30.98 31.71 1/413 7112
/proc/meminfo: memFree=256172704/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 21 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.301003 s]
/proc/loadavg: 27.08 30.98 31.71 1/413 7112
/proc/meminfo: memFree=256172704/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 20 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700999 s]
/proc/loadavg: 27.08 30.98 31.71 1/413 7112
/proc/meminfo: memFree=256172704/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 20 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50107 s]
/proc/loadavg: 27.08 30.98 31.71 6/419 7123
/proc/meminfo: memFree=255944200/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
[pid=7113] ppid=7112 vsize=71192 CPUtime=1.49 cores=0
/proc/7113/stat : 7113 (cryptominisat) R 7112 7112 7086 0 -1 4202496 11703 0 0 0 146 3 0 0 25 0 1 0 620056741 72900608 11488 268435456000 4194304 4587354 140737407696080 18446744073709551615 4248764 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/7113/statm: 17798 11488 310 96 0 14360 0
[pid=7114] ppid=7112 vsize=43632 CPUtime=1.49 cores=1
/proc/7114/stat : 7114 (clasp) R 7112 7112 7086 0 -1 4202496 11628 0 0 0 145 4 0 0 20 0 1 0 620056741 44679168 8009 268435456000 4194304 4785740 140734494827616 18446744073709551615 4279309 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/7114/statm: 10908 8009 338 145 0 7953 0
[pid=7115] ppid=7112 vsize=478928 CPUtime=1.48 cores=2
/proc/7115/stat : 7115 (TNM) R 7112 7112 7086 0 -1 4202496 9358 0 0 0 144 4 0 0 20 0 1 0 620056741 490422272 9256 268435456000 134512640 134534564 4287142016 18446744073709551615 134524552 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/7115/statm: 119732 9256 91 6 0 118397 0
[pid=7116] ppid=7112 vsize=73740 CPUtime=1.48 cores=3
/proc/7116/stat : 7116 (march_hi) R 7112 7112 7086 0 -1 4194304 25550 0 0 0 141 7 0 0 20 0 1 0 620056741 75509760 12245 268435456000 4194304 4331988 140737161079792 18446744073709551615 4252174 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7116/statm: 18435 12245 127 34 0 16877 0
[pid=7117] ppid=7112 vsize=79772 CPUtime=1.48 cores=4-31
/proc/7117/stat : 7117 (plingeling) R 7112 7112 7086 0 -1 4202496 18717 1238 0 0 142 5 0 1 21 0 1 0 620056741 81686528 18602 268435456000 4194304 4356188 140736763407456 18446744073709551615 4278302 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/7117/statm: 19943 18602 129 40 0 18487 0
Current children cumulated CPU time (s) 7.42
Current children cumulated vsize (KiB) 765588

[startup+3.10106 s]
/proc/loadavg: 27.08 30.98 31.71 6/419 7123
/proc/meminfo: memFree=255831980/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
[pid=7113] ppid=7112 vsize=71192 CPUtime=3.09 cores=0
/proc/7113/stat : 7113 (cryptominisat) R 7112 7112 7086 0 -1 4202496 11707 0 0 0 306 3 0 0 25 0 1 0 620056741 72900608 11492 268435456000 4194304 4587354 140737407696080 18446744073709551615 4367369 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/7113/statm: 17798 11492 310 96 0 14360 0
[pid=7114] ppid=7112 vsize=62604 CPUtime=3.08 cores=1
/proc/7114/stat : 7114 (clasp) R 7112 7112 7086 0 -1 4202496 16408 0 0 0 303 5 0 0 25 0 1 0 620056741 64106496 12789 268435456000 4194304 4785740 140734494827616 18446744073709551615 258650616632 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/7114/statm: 15651 12789 338 145 0 12696 0
[pid=7115] ppid=7112 vsize=488960 CPUtime=3.09 cores=2
/proc/7115/stat : 7115 (TNM) R 7112 7112 7086 0 -1 4202496 15295 0 0 0 303 6 0 0 25 0 1 0 620056741 500695040 15193 268435456000 134512640 134534564 4287142016 18446744073709551615 134528497 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/7115/statm: 122240 15193 95 6 0 120905 0
[pid=7116] ppid=7112 vsize=67828 CPUtime=3.08 cores=3
/proc/7116/stat : 7116 (march_hi) R 7112 7112 7086 0 -1 4194304 50943 0 0 0 296 12 0 0 25 0 1 0 620056741 69455872 10797 268435456000 4194304 4331988 140737161079792 18446744073709551615 4251688 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7116/statm: 16957 10797 127 34 0 15399 0
[pid=7117] ppid=7112 vsize=186980 CPUtime=3.09 cores=4-31
/proc/7117/stat : 7117 (plingeling) R 7112 7112 7086 0 -1 4202496 45567 1238 0 0 297 11 0 1 25 0 1 0 620056741 191467520 45404 268435456000 4194304 4356188 140736763407456 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/7117/statm: 46745 45404 129 40 0 45289 0
Current children cumulated CPU time (s) 15.43
Current children cumulated vsize (KiB) 895888

[startup+6.30212 s]
/proc/loadavg: 25.32 30.55 31.56 34/447 7151
/proc/meminfo: memFree=255628312/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
[pid=7113] ppid=7112 vsize=82704 CPUtime=6.29 cores=0
/proc/7113/stat : 7113 (cryptominisat) R 7112 7112 7086 0 -1 4202496 19980 0 0 0 624 5 0 0 25 0 1 0 620056741 84688896 14954 268435456000 4194304 4587354 140737407696080 18446744073709551615 4236573 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/7113/statm: 20676 14954 328 96 0 17238 0
[pid=7114] ppid=7112 vsize=91868 CPUtime=6.29 cores=1
/proc/7114/stat : 7114 (clasp) R 7112 7112 7086 0 -1 4202496 23721 0 0 0 621 8 0 0 25 0 1 0 620056741 94072832 20102 268435456000 4194304 4785740 140734494827616 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/7114/statm: 22967 20102 338 145 0 20012 0
[pid=7115] ppid=7112 vsize=488960 CPUtime=6.29 cores=2
/proc/7115/stat : 7115 (TNM) R 7112 7112 7086 0 -1 4202496 15295 0 0 0 623 6 0 0 25 0 1 0 620056741 500695040 15193 268435456000 134512640 134534564 4287142016 18446744073709551615 134530897 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/7115/statm: 122240 15193 95 6 0 120905 0
[pid=7116] ppid=7112 vsize=72700 CPUtime=6.28 cores=3
/proc/7116/stat : 7116 (march_hi) R 7112 7112 7086 0 -1 4194304 89895 0 0 0 607 21 0 0 25 0 1 0 620056741 74444800 11986 268435456000 4194304 4331988 140737161079792 18446744073709551615 258650604544 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7116/statm: 18175 11986 127 34 0 16617 0
[pid=7117] ppid=7112 vsize=820316 CPUtime=44.13 cores=4-31
/proc/7117/stat : 7117 (plingeling) S 7112 7112 7086 0 -1 4202496 155033 1238 0 0 4368 44 0 1 18 0 29 0 620056741 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 31 0 0 0
/proc/7117/statm: 205079 77319 154 40 0 203623 0
[pid=7117/tid=7124] ppid=7112 vsize=820316 CPUtime=1.43 cores=4-31
/proc/7117/task/7124/stat : 7124 (plingeling) R 7112 7112 7086 0 -1 4202560 1284 1238 0 0 142 0 0 1 19 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322024 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7117/tid=7125] ppid=7112 vsize=820316 CPUtime=1.42 cores=4-31
/proc/7117/task/7125/stat : 7125 (plingeling) R 7112 7112 7086 0 -1 4202560 2136 1238 0 0 141 0 0 1 25 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321560 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=7117/tid=7126] ppid=7112 vsize=820316 CPUtime=1.39 cores=4-31
/proc/7117/task/7126/stat : 7126 (plingeling) R 7112 7112 7086 0 -1 4202560 2662 1238 0 0 138 0 0 1 22 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321566 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=7117/tid=7127] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7127/stat : 7127 (plingeling) R 7112 7112 7086 0 -1 4202560 4105 1238 0 0 138 1 0 1 18 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=7117/tid=7128] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7128/stat : 7128 (plingeling) R 7112 7112 7086 0 -1 4202560 1751 1238 0 0 140 0 0 1 19 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321963 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=7117/tid=7129] ppid=7112 vsize=820316 CPUtime=1.39 cores=4-31
/proc/7117/task/7129/stat : 7129 (plingeling) R 7112 7112 7086 0 -1 4202560 635 1238 0 0 138 0 0 1 19 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321924 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=7117/tid=7130] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7130/stat : 7130 (plingeling) R 7112 7112 7086 0 -1 4202560 2987 1238 0 0 139 0 0 1 18 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322016 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=7117/tid=7131] ppid=7112 vsize=820316 CPUtime=1.43 cores=4-31
/proc/7117/task/7131/stat : 7131 (plingeling) R 7112 7112 7086 0 -1 4202560 1875 1238 0 0 142 0 0 1 18 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4210605 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7117/tid=7132] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7132/stat : 7132 (plingeling) R 7112 7112 7086 0 -1 4202560 3219 1238 0 0 138 1 0 1 18 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322085 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=7117/tid=7133] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7133/stat : 7133 (plingeling) R 7112 7112 7086 0 -1 4202560 2874 1238 0 0 139 0 0 1 24 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322016 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=7117/tid=7134] ppid=7112 vsize=820316 CPUtime=1.42 cores=4-31
/proc/7117/task/7134/stat : 7134 (plingeling) R 7112 7112 7086 0 -1 4202560 1057 1238 0 0 141 0 0 1 19 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322016 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=7117/tid=7135] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7135/stat : 7135 (plingeling) R 7112 7112 7086 0 -1 4202560 3605 1238 0 0 139 1 0 1 25 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=7117/tid=7136] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7136/stat : 7136 (plingeling) R 7112 7112 7086 0 -1 4202560 4766 1238 0 0 139 1 0 1 19 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=7117/tid=7137] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7137/stat : 7137 (plingeling) R 7112 7112 7086 0 -1 4202560 4637 1238 0 0 140 0 0 1 18 0 29 0 620057226 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322106 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=7117/tid=7138] ppid=7112 vsize=820316 CPUtime=1.38 cores=4-31
/proc/7117/task/7138/stat : 7138 (plingeling) R 7112 7112 7086 0 -1 4202560 122 1238 0 0 137 0 0 1 22 0 29 0 620057227 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321594 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=7117/tid=7139] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7139/stat : 7139 (plingeling) R 7112 7112 7086 0 -1 4202560 2898 1238 0 0 139 0 0 1 22 0 29 0 620057227 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322106 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=7117/tid=7140] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7140/stat : 7140 (plingeling) R 7112 7112 7086 0 -1 4202560 1667 1238 0 0 140 0 0 1 19 0 29 0 620057227 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322024 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=7117/tid=7141] ppid=7112 vsize=820316 CPUtime=1.36 cores=4-31
/proc/7117/task/7141/stat : 7141 (plingeling) R 7112 7112 7086 0 -1 4202560 1635 1238 0 0 135 0 0 1 22 0 29 0 620057227 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321963 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=7117/tid=7142] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7142/stat : 7142 (plingeling) R 7112 7112 7086 0 -1 4202560 1796 1238 0 0 140 0 0 1 18 0 29 0 620057227 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321571 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=7117/tid=7143] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7143/stat : 7143 (plingeling) R 7112 7112 7086 0 -1 4202560 634 1238 0 0 140 0 0 1 22 0 29 0 620057227 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4245442 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=7117/tid=7144] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7144/stat : 7144 (plingeling) R 7112 7112 7086 0 -1 4202560 2377 1238 0 0 139 0 0 1 19 0 29 0 620057227 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321963 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=7117/tid=7145] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7145/stat : 7145 (plingeling) R 7112 7112 7086 0 -1 4202560 2666 1238 0 0 139 1 0 1 18 0 29 0 620057228 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322053 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=7117/tid=7146] ppid=7112 vsize=820316 CPUtime=1.39 cores=4-31
/proc/7117/task/7146/stat : 7146 (plingeling) R 7112 7112 7086 0 -1 4202560 1654 1238 0 0 138 0 0 1 22 0 29 0 620057228 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322151 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=7117/tid=7147] ppid=7112 vsize=820316 CPUtime=1.39 cores=4-31
/proc/7117/task/7147/stat : 7147 (plingeling) R 7112 7112 7086 0 -1 4202560 630 1238 0 0 138 0 0 1 22 0 29 0 620057228 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=7117/tid=7148] ppid=7112 vsize=820316 CPUtime=1.41 cores=4-31
/proc/7117/task/7148/stat : 7148 (plingeling) R 7112 7112 7086 0 -1 4202560 4305 1238 0 0 139 1 0 1 22 0 29 0 620057228 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4321594 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=7117/tid=7149] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7149/stat : 7149 (plingeling) R 7112 7112 7086 0 -1 4202560 2451 1238 0 0 139 0 0 1 22 0 29 0 620057228 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322000 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=7117/tid=7150] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7150/stat : 7150 (plingeling) R 7112 7112 7086 0 -1 4202560 3698 1238 0 0 138 1 0 1 19 0 29 0 620057228 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4322016 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=7117/tid=7151] ppid=7112 vsize=820316 CPUtime=1.4 cores=4-31
/proc/7117/task/7151/stat : 7151 (plingeling) R 7112 7112 7086 0 -1 4202560 1639 1238 0 0 139 0 0 1 22 0 29 0 620057228 840003584 77319 268435456000 4194304 4356188 140736763407456 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 19 0 0 0
Current children cumulated CPU time (s) 69.28
Current children cumulated vsize (KiB) 1574872

[startup+12.7023 s]
/proc/loadavg: 25.85 30.58 31.57 33/447 7151
/proc/meminfo: memFree=255253136/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 0 0 0 0 15 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
[pid=7113] ppid=7112 vsize=120208 CPUtime=12.68 cores=0
/proc/7113/stat : 7113 (cryptominisat) R 7112 7112 7086 0 -1 4202496 22455 0 0 0 1263 5 0 0 25 0 1 0 620056741 123092992 17425 268435456000 4194304 4587354 140737407696080 18446744073709551615 4236561 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

/proc/7113/statm: 428011 86929 338 96 0 424573 0
[pid=7114] ppid=7112 vsize=1875628 CPUtime=5009.09 cores=1
/proc/7114/stat : 7114 (clasp) R 7112 7112 7086 0 -1 4202496 468424 0 0 0 500756 153 0 0 25 0 1 0 620056741 1920643072 464805 268435456000 4194304 4785740 140734494827616 18446744073709551615 4278093 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/7114/statm: 468907 464805 338 145 0 465952 0
[pid=7115] ppid=7112 vsize=488960 CPUtime=5009.13 cores=2
/proc/7115/stat : 7115 (TNM) R 7112 7112 7086 0 -1 4202496 15295 0 0 0 500907 6 0 0 25 0 1 0 620056741 500695040 15193 268435456000 134512640 134534564 4287142016 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/7115/statm: 122240 15193 95 6 0 120905 0
[pid=7116] ppid=7112 vsize=324096 CPUtime=5009.14 cores=3
/proc/7116/stat : 7116 (march_hi) R 7112 7112 7086 0 -1 4202496 159640 0 0 0 500858 56 0 0 25 0 1 0 620056741 331874304 42081 268435456000 4194304 4331988 140737161079792 18446744073709551615 4228287 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7116/statm: 81024 42081 174 34 0 79466 0
[pid=7117] ppid=7112 vsize=5306752 CPUtime=139966 cores=4-31
/proc/7117/stat : 7117 (plingeling) S 7112 7112 7086 0 -1 4202496 22245793 1238 0 0 13985757 10848 0 1 18 0 29 0 620056741 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 31 0 0 0
/proc/7117/statm: 1326688 1205343 159 40 0 1325232 0
[pid=7117/tid=7124] ppid=7112 vsize=5306752 CPUtime=5003.99 cores=4-31
/proc/7117/task/7124/stat : 7124 (plingeling) R 7112 7112 7086 0 -1 4202560 776815 1238 0 0 500043 355 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4330976 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=7117/tid=7125] ppid=7112 vsize=5306752 CPUtime=5003.9 cores=4-31
/proc/7117/task/7125/stat : 7125 (plingeling) R 7112 7112 7086 0 -1 4202560 547151 1238 0 0 500168 221 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=7117/tid=7126] ppid=7112 vsize=5306752 CPUtime=4979.49 cores=4-31
/proc/7117/task/7126/stat : 7126 (plingeling) R 7112 7112 7086 0 -1 4202560 794001 1238 0 0 497598 350 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4242664 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=7117/tid=7127] ppid=7112 vsize=5306752 CPUtime=4988.19 cores=4-31
/proc/7117/task/7127/stat : 7127 (plingeling) R 7112 7112 7086 0 -1 4202560 918723 1238 0 0 498323 495 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=7117/tid=7128] ppid=7112 vsize=5306752 CPUtime=5003.82 cores=4-31
/proc/7117/task/7128/stat : 7128 (plingeling) R 7112 7112 7086 0 -1 4202560 793003 1238 0 0 500057 324 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4283920 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=7117/tid=7129] ppid=7112 vsize=5306752 CPUtime=5004.05 cores=4-31
/proc/7117/task/7129/stat : 7129 (plingeling) R 7112 7112 7086 0 -1 4202560 837673 1238 0 0 500000 404 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4279490 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=7117/tid=7130] ppid=7112 vsize=5306752 CPUtime=5003.93 cores=4-31
/proc/7117/task/7130/stat : 7130 (plingeling) R 7112 7112 7086 0 -1 4202560 459249 1238 0 0 500194 198 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331035 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=7117/tid=7131] ppid=7112 vsize=5306752 CPUtime=5004.26 cores=4-31
/proc/7117/task/7131/stat : 7131 (plingeling) R 7112 7112 7086 0 -1 4202560 575635 1238 0 0 499896 529 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331352 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7117/tid=7132] ppid=7112 vsize=5306752 CPUtime=5003.7 cores=4-31
/proc/7117/task/7132/stat : 7132 (plingeling) R 7112 7112 7086 0 -1 4202560 795646 1238 0 0 500019 350 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4233966 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=7117/tid=7133] ppid=7112 vsize=5306752 CPUtime=5004.15 cores=4-31
/proc/7117/task/7133/stat : 7133 (plingeling) R 7112 7112 7086 0 -1 4202560 959019 1238 0 0 499973 441 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4284120 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=7117/tid=7134] ppid=7112 vsize=5306752 CPUtime=4993.16 cores=4-31
/proc/7117/task/7134/stat : 7134 (plingeling) R 7112 7112 7086 0 -1 4202560 812127 1238 0 0 498957 358 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=7117/tid=7135] ppid=7112 vsize=5306752 CPUtime=5003.99 cores=4-31
/proc/7117/task/7135/stat : 7135 (plingeling) R 7112 7112 7086 0 -1 4202560 1028224 1238 0 0 499982 416 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4279663 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=7117/tid=7136] ppid=7112 vsize=5306752 CPUtime=5004 cores=4-31
/proc/7117/task/7136/stat : 7136 (plingeling) R 7112 7112 7086 0 -1 4202560 733610 1238 0 0 500080 319 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4210579 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=7117/tid=7137] ppid=7112 vsize=5306752 CPUtime=5004.16 cores=4-31
/proc/7117/task/7137/stat : 7137 (plingeling) R 7112 7112 7086 0 -1 4202560 395636 1238 0 0 500264 151 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331799 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=7117/tid=7138] ppid=7112 vsize=5306752 CPUtime=5001.06 cores=4-31
/proc/7117/task/7138/stat : 7138 (plingeling) R 7112 7112 7086 0 -1 4202560 1035068 1238 0 0 499686 419 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4279478 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=7117/tid=7139] ppid=7112 vsize=5306752 CPUtime=5003.69 cores=4-31
/proc/7117/task/7139/stat : 7139 (plingeling) R 7112 7112 7086 0 -1 4202560 537385 1238 0 0 499845 523 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=7117/tid=7140] ppid=7112 vsize=5306752 CPUtime=4981.38 cores=4-31
/proc/7117/task/7140/stat : 7140 (plingeling) R 7112 7112 7086 0 -1 4202560 799085 1238 0 0 497726 411 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4281508 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7117/tid=7141] ppid=7112 vsize=5306752 CPUtime=4966.95 cores=4-31
/proc/7117/task/7141/stat : 7141 (plingeling) R 7112 7112 7086 0 -1 4202560 726256 1238 0 0 496423 271 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=7117/tid=7142] ppid=7112 vsize=5306752 CPUtime=5003.36 cores=4-31
/proc/7117/task/7142/stat : 7142 (plingeling) R 7112 7112 7086 0 -1 4202560 1057674 1238 0 0 499795 540 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4274252 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=7117/tid=7143] ppid=7112 vsize=5306752 CPUtime=5003.46 cores=4-31
/proc/7117/task/7143/stat : 7143 (plingeling) R 7112 7112 7086 0 -1 4202560 980589 1238 0 0 499882 463 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=7117/tid=7144] ppid=7112 vsize=5306752 CPUtime=5003.72 cores=4-31
/proc/7117/task/7144/stat : 7144 (plingeling) R 7112 7112 7086 0 -1 4202560 662885 1238 0 0 500115 256 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=7117/tid=7145] ppid=7112 vsize=5306752 CPUtime=5002.44 cores=4-31
/proc/7117/task/7145/stat : 7145 (plingeling) R 7112 7112 7086 0 -1 4202560 808366 1238 0 0 499946 297 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331345 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=7117/tid=7146] ppid=7112 vsize=5306752 CPUtime=4993.89 cores=4-31
/proc/7117/task/7146/stat : 7146 (plingeling) R 7112 7112 7086 0 -1 4202560 763823 1238 0 0 499062 326 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=7117/tid=7147] ppid=7112 vsize=5306752 CPUtime=5003.94 cores=4-31
/proc/7117/task/7147/stat : 7147 (plingeling) R 7112 7112 7086 0 -1 4202560 582060 1238 0 0 499695 698 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=7117/tid=7148] ppid=7112 vsize=5306752 CPUtime=5003.6 cores=4-31
/proc/7117/task/7148/stat : 7148 (plingeling) R 7112 7112 7086 0 -1 4202560 1056220 1238 0 0 499862 497 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4281686 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=7117/tid=7149] ppid=7112 vsize=5306752 CPUtime=5003.15 cores=4-31
/proc/7117/task/7149/stat : 7149 (plingeling) R 7112 7112 7086 0 -1 4202560 1017926 1238 0 0 499890 424 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4283951 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=7117/tid=7150] ppid=7112 vsize=5306752 CPUtime=5003.86 cores=4-31
/proc/7117/task/7150/stat : 7150 (plingeling) R 7112 7112 7086 0 -1 4202560 836899 1238 0 0 500037 348 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=7117/tid=7151] ppid=7112 vsize=5306752 CPUtime=4982.02 cores=4-31
/proc/7117/task/7151/stat : 7151 (plingeling) R 7112 7112 7086 0 -1 4202560 865777 1238 0 0 497770 431 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4283925 0 0 4096 17442 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 9725804

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.1 s]
/proc/loadavg: 32.12 32.11 32.04 33/447 7318
/proc/meminfo: memFree=248901016/264257624 swapFree=67111528/67111528
[pid=7112] ppid=7110 vsize=18324 CPUtime=0.07 cores=0-31
/proc/7112/stat : 7112 (ppfolio) S 7110 7112 7086 0 -1 4202496 513 0 0 0 1 6 0 0 15 0 1 0 620056741 18763776 386 268435456000 4194304 4287625 140734541590384 18446744073709551615 258650975999 0 0 4096 65536 9207012132762157055 0 0 17 20 0 0 0
/proc/7112/statm: 4581 386 337 23 0 91 0
[pid=7113] ppid=7112 vsize=1712044 CPUtime=5009.16 cores=0
/proc/7113/stat : 7113 (cryptominisat) R 7112 7112 7086 0 -1 4202496 1443973 0 0 0 500276 640 0 0 25 0 1 0 620056741 1753133056 86929 268435456000 4194304 4587354 140737407696080 18446744073709551615 258650652552 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/7113/statm: 428011 86929 338 96 0 424573 0
[pid=7114] ppid=7112 vsize=1875628 CPUtime=5009.09 cores=1
/proc/7114/stat : 7114 (clasp) R 7112 7112 7086 0 -1 4202496 468424 0 0 0 500756 153 0 0 25 0 1 0 620056741 1920643072 464805 268435456000 4194304 4785740 140734494827616 18446744073709551615 4278093 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/7114/statm: 468907 464805 338 145 0 465952 0
[pid=7115] ppid=7112 vsize=488960 CPUtime=5009.13 cores=2
/proc/7115/stat : 7115 (TNM) R 7112 7112 7086 0 -1 4202496 15295 0 0 0 500907 6 0 0 25 0 1 0 620056741 500695040 15193 268435456000 134512640 134534564 4287142016 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/7115/statm: 122240 15193 95 6 0 120905 0
[pid=7116] ppid=7112 vsize=324096 CPUtime=5009.14 cores=3
/proc/7116/stat : 7116 (march_hi) R 7112 7112 7086 0 -1 4202496 159640 0 0 0 500858 56 0 0 25 0 1 0 620056741 331874304 42081 268435456000 4194304 4331988 140737161079792 18446744073709551615 4228287 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/7116/statm: 81024 42081 174 34 0 79466 0
[pid=7117] ppid=7112 vsize=5306752 CPUtime=139966 cores=4-31
/proc/7117/stat : 7117 (plingeling) S 7112 7112 7086 0 -1 4202496 22245793 1238 0 0 13985757 10848 0 1 18 0 29 0 620056741 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 31 0 0 0
/proc/7117/statm: 1326688 1205343 159 40 0 1325232 0
[pid=7117/tid=7124] ppid=7112 vsize=5306752 CPUtime=5003.99 cores=4-31
/proc/7117/task/7124/stat : 7124 (plingeling) R 7112 7112 7086 0 -1 4202560 776815 1238 0 0 500043 355 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4330976 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=7117/tid=7125] ppid=7112 vsize=5306752 CPUtime=5003.9 cores=4-31
/proc/7117/task/7125/stat : 7125 (plingeling) R 7112 7112 7086 0 -1 4202560 547151 1238 0 0 500168 221 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=7117/tid=7126] ppid=7112 vsize=5306752 CPUtime=4979.49 cores=4-31
/proc/7117/task/7126/stat : 7126 (plingeling) R 7112 7112 7086 0 -1 4202560 794001 1238 0 0 497598 350 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4242664 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=7117/tid=7127] ppid=7112 vsize=5306752 CPUtime=4988.19 cores=4-31
/proc/7117/task/7127/stat : 7127 (plingeling) R 7112 7112 7086 0 -1 4202560 918723 1238 0 0 498323 495 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=7117/tid=7128] ppid=7112 vsize=5306752 CPUtime=5003.82 cores=4-31
/proc/7117/task/7128/stat : 7128 (plingeling) R 7112 7112 7086 0 -1 4202560 793003 1238 0 0 500057 324 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4283920 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=7117/tid=7129] ppid=7112 vsize=5306752 CPUtime=5004.05 cores=4-31
/proc/7117/task/7129/stat : 7129 (plingeling) R 7112 7112 7086 0 -1 4202560 837673 1238 0 0 500000 404 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4279490 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=7117/tid=7130] ppid=7112 vsize=5306752 CPUtime=5003.93 cores=4-31
/proc/7117/task/7130/stat : 7130 (plingeling) R 7112 7112 7086 0 -1 4202560 459249 1238 0 0 500194 198 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331035 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=7117/tid=7131] ppid=7112 vsize=5306752 CPUtime=5004.26 cores=4-31
/proc/7117/task/7131/stat : 7131 (plingeling) R 7112 7112 7086 0 -1 4202560 575635 1238 0 0 499896 529 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331352 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=7117/tid=7132] ppid=7112 vsize=5306752 CPUtime=5003.7 cores=4-31
/proc/7117/task/7132/stat : 7132 (plingeling) R 7112 7112 7086 0 -1 4202560 795646 1238 0 0 500019 350 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4233966 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=7117/tid=7133] ppid=7112 vsize=5306752 CPUtime=5004.15 cores=4-31
/proc/7117/task/7133/stat : 7133 (plingeling) R 7112 7112 7086 0 -1 4202560 959019 1238 0 0 499973 441 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4284120 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=7117/tid=7134] ppid=7112 vsize=5306752 CPUtime=4993.16 cores=4-31
/proc/7117/task/7134/stat : 7134 (plingeling) R 7112 7112 7086 0 -1 4202560 812127 1238 0 0 498957 358 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=7117/tid=7135] ppid=7112 vsize=5306752 CPUtime=5003.99 cores=4-31
/proc/7117/task/7135/stat : 7135 (plingeling) R 7112 7112 7086 0 -1 4202560 1028224 1238 0 0 499982 416 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4279663 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=7117/tid=7136] ppid=7112 vsize=5306752 CPUtime=5004 cores=4-31
/proc/7117/task/7136/stat : 7136 (plingeling) R 7112 7112 7086 0 -1 4202560 733610 1238 0 0 500080 319 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4210579 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=7117/tid=7137] ppid=7112 vsize=5306752 CPUtime=5004.16 cores=4-31
/proc/7117/task/7137/stat : 7137 (plingeling) R 7112 7112 7086 0 -1 4202560 395636 1238 0 0 500264 151 0 1 25 0 29 0 620057226 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331799 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=7117/tid=7138] ppid=7112 vsize=5306752 CPUtime=5001.06 cores=4-31
/proc/7117/task/7138/stat : 7138 (plingeling) R 7112 7112 7086 0 -1 4202560 1035068 1238 0 0 499686 419 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4279478 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=7117/tid=7139] ppid=7112 vsize=5306752 CPUtime=5003.69 cores=4-31
/proc/7117/task/7139/stat : 7139 (plingeling) R 7112 7112 7086 0 -1 4202560 537385 1238 0 0 499845 523 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=7117/tid=7140] ppid=7112 vsize=5306752 CPUtime=4981.38 cores=4-31
/proc/7117/task/7140/stat : 7140 (plingeling) R 7112 7112 7086 0 -1 4202560 799085 1238 0 0 497726 411 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4281508 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=7117/tid=7141] ppid=7112 vsize=5306752 CPUtime=4966.95 cores=4-31
/proc/7117/task/7141/stat : 7141 (plingeling) R 7112 7112 7086 0 -1 4202560 726256 1238 0 0 496423 271 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=7117/tid=7142] ppid=7112 vsize=5306752 CPUtime=5003.36 cores=4-31
/proc/7117/task/7142/stat : 7142 (plingeling) R 7112 7112 7086 0 -1 4202560 1057674 1238 0 0 499795 540 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4274252 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=7117/tid=7143] ppid=7112 vsize=5306752 CPUtime=5003.46 cores=4-31
/proc/7117/task/7143/stat : 7143 (plingeling) R 7112 7112 7086 0 -1 4202560 980589 1238 0 0 499882 463 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=7117/tid=7144] ppid=7112 vsize=5306752 CPUtime=5003.72 cores=4-31
/proc/7117/task/7144/stat : 7144 (plingeling) R 7112 7112 7086 0 -1 4202560 662885 1238 0 0 500115 256 0 1 25 0 29 0 620057227 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=7117/tid=7145] ppid=7112 vsize=5306752 CPUtime=5002.44 cores=4-31
/proc/7117/task/7145/stat : 7145 (plingeling) R 7112 7112 7086 0 -1 4202560 808366 1238 0 0 499946 297 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331345 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=7117/tid=7146] ppid=7112 vsize=5306752 CPUtime=4993.89 cores=4-31
/proc/7117/task/7146/stat : 7146 (plingeling) R 7112 7112 7086 0 -1 4202560 763823 1238 0 0 499062 326 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=7117/tid=7147] ppid=7112 vsize=5306752 CPUtime=5003.94 cores=4-31
/proc/7117/task/7147/stat : 7147 (plingeling) R 7112 7112 7086 0 -1 4202560 582060 1238 0 0 499695 698 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=7117/tid=7148] ppid=7112 vsize=5306752 CPUtime=5003.6 cores=4-31
/proc/7117/task/7148/stat : 7148 (plingeling) R 7112 7112 7086 0 -1 4202560 1056220 1238 0 0 499862 497 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4281686 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=7117/tid=7149] ppid=7112 vsize=5306752 CPUtime=5003.15 cores=4-31
/proc/7117/task/7149/stat : 7149 (plingeling) R 7112 7112 7086 0 -1 4202560 1017926 1238 0 0 499890 424 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4283951 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=7117/tid=7150] ppid=7112 vsize=5306752 CPUtime=5003.86 cores=4-31
/proc/7117/task/7150/stat : 7150 (plingeling) R 7112 7112 7086 0 -1 4202560 836899 1238 0 0 500037 348 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=7117/tid=7151] ppid=7112 vsize=5306752 CPUtime=4982.02 cores=4-31
/proc/7117/task/7151/stat : 7151 (plingeling) R 7112 7112 7086 0 -1 4202560 865777 1238 0 0 497770 431 0 1 25 0 29 0 620057228 5434114048 1205343 268435456000 4194304 4356188 140736763407456 18446744073709551615 4283925 0 0 4096 17442 0 0 0 -1 15 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 9725804

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7112 and gives
#  childrusage.ru_utime.tv_sec=5009
#  childrusage.ru_utime.tv_usec=95501
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=127980
# CPU time returned by wait4() is 5009.22
# while last known CPU time is 160003
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5010.14
CPU time (s): 160003
CPU user time (s): 159886
CPU system time (s): 117.1
CPU usage (%): 3193.58
Max. virtual memory (cumulated for all children) (KiB): 11964164

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5009.1
system time used= 0.12798
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 15809
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7960
involuntary context switches= 17481

runsolver used 30.3864 second user time and 111.876 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-11 05:48:23
IDJOB=3347373
IDBENCH=82381
IDSOLVER=1808
FILE ID=node200/3347373-1305085703
RUNJOBID= node200-1305085702-7100
PBS_JOBID= 13282054
Free space on /tmp= 62044 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.025-NOTKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347373-1305085703/watcher-3347373-1305085703 -o /tmp/evaluation-result-3347373-1305085703/solver-3347373-1305085703 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347373-1305085703.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 3c5c73741ba21d5f39c7f436ba834743
RANDOM SEED=921659789

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      256173108 kB
Buffers:        384612 kB
Cached:        7356428 kB
SwapCached:          0 kB
Active:        6871004 kB
Inactive:       938740 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      256173108 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           18248 kB
Writeback:           0 kB
AnonPages:       68500 kB
Mapped:          13484 kB
Slab:           195504 kB
PageTables:       3996 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   187304 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62032 MiB
End job on node200 at 2011-05-11 07:11:55