Trace number 3347381

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 160003 5014.44

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.045-NOTKNOWN.cnf
MD5SUM4ab53801f3b1a3f7169c9fb38cba4184
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables171676
Number of clauses870241
Sum of the clauses size2069671
Maximum clause length63
Minimum clause length1
Number of clauses of size 1211
Number of clauses of size 2859140
Number of clauses of size 32700
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 58190

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347381-1305105159/watcher-3347381-1305105159 -o /tmp/evaluation-result-3347381-1305105159/solver-3347381-1305105159 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347381-1305105159.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.74 22.53 28.53 2/413 8348
/proc/meminfo: memFree=243077372/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 24 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0

[startup+0.08439 s]
/proc/loadavg: 6.74 22.53 28.53 2/413 8348
/proc/meminfo: memFree=243077372/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 24 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100384 s]
/proc/loadavg: 6.74 22.53 28.53 2/413 8348
/proc/meminfo: memFree=243077372/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 24 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300404 s]
/proc/loadavg: 6.74 22.53 28.53 2/413 8348
/proc/meminfo: memFree=243077372/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 24 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700376 s]
/proc/loadavg: 6.74 22.53 28.53 2/413 8348
/proc/meminfo: memFree=243077372/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 18 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50142 s]
/proc/loadavg: 6.74 22.53 28.53 6/419 8359
/proc/meminfo: memFree=242697608/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
[pid=8349] ppid=8348 vsize=113388 CPUtime=1.48 cores=0
/proc/8349/stat : 8349 (cryptominisat) R 8348 8348 8322 0 -1 4202496 21744 0 0 0 141 7 0 0 20 0 1 0 622002394 116109312 21396 268435456000 4194304 4587354 140735097219824 18446744073709551615 258650616632 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8349/statm: 28347 21398 305 96 0 24909 0
[pid=8350] ppid=8348 vsize=94188 CPUtime=1.48 cores=1
/proc/8350/stat : 8350 (clasp) R 8348 8348 8322 0 -1 4202496 24739 0 0 0 140 8 0 0 20 0 1 0 622002394 96448512 20135 268435456000 4194304 4785740 140734302511440 18446744073709551615 4495610 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/8350/statm: 23547 20135 317 145 0 20592 0
[pid=8351] ppid=8348 vsize=500576 CPUtime=1.48 cores=2
/proc/8351/stat : 8351 (TNM) R 8348 8348 8322 0 -1 4202496 16661 0 0 0 143 5 0 0 20 0 1 0 622002394 512589824 16559 268435456000 134512640 134534564 4294344192 18446744073709551615 134524554 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/8351/statm: 125144 16559 91 6 0 123809 0
[pid=8352] ppid=8348 vsize=139040 CPUtime=1.48 cores=3
/proc/8352/stat : 8352 (march_hi) R 8348 8348 8322 0 -1 4194304 36964 0 0 0 138 10 0 0 20 0 1 0 622002394 142376960 24812 268435456000 4194304 4331988 140734280562912 18446744073709551615 4252228 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/8352/statm: 34760 24812 126 34 0 33202 0
[pid=8353] ppid=8348 vsize=150876 CPUtime=1.48 cores=4-31
/proc/8353/stat : 8353 (plingeling) R 8348 8348 8322 0 -1 4202496 36478 1237 0 0 139 9 0 0 21 0 1 0 622002394 154497024 36363 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210583 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/8353/statm: 37719 36363 129 40 0 36263 0
Current children cumulated CPU time (s) 7.4
Current children cumulated vsize (KiB) 1016392

[startup+3.10148 s]
/proc/loadavg: 6.74 22.53 28.53 6/419 8359
/proc/meminfo: memFree=242583652/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
[pid=8349] ppid=8348 vsize=121396 CPUtime=3.08 cores=0
/proc/8349/stat : 8349 (cryptominisat) R 8348 8348 8322 0 -1 4202496 23684 0 0 0 301 7 0 0 25 0 1 0 622002394 124309504 23336 268435456000 4194304 4587354 140735097219824 18446744073709551615 4367369 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8349/statm: 30349 23336 310 96 0 26911 0
[pid=8350] ppid=8348 vsize=82008 CPUtime=3.08 cores=1
/proc/8350/stat : 8350 (clasp) R 8348 8348 8322 0 -1 4202496 24864 0 0 0 300 8 0 0 25 0 1 0 622002394 83976192 17298 268435456000 4194304 4785740 140734302511440 18446744073709551615 4549020 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/8350/statm: 20502 17298 338 145 0 17547 0
[pid=8351] ppid=8348 vsize=530936 CPUtime=3.09 cores=2
/proc/8351/stat : 8351 (TNM) R 8348 8348 8322 0 -1 4202496 31038 0 0 0 301 8 0 0 25 0 1 0 622002394 543678464 30936 268435456000 134512640 134534564 4294344192 18446744073709551615 134529970 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/8351/statm: 132734 30936 95 6 0 131399 0
[pid=8352] ppid=8348 vsize=138184 CPUtime=3.08 cores=3
/proc/8352/stat : 8352 (march_hi) R 8348 8348 8322 0 -1 4194304 68339 0 0 0 291 17 0 0 25 0 1 0 622002394 141500416 24600 268435456000 4194304 4331988 140734280562912 18446744073709551615 4252197 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/8352/statm: 34546 24600 127 34 0 32988 0
[pid=8353] ppid=8348 vsize=297452 CPUtime=3.08 cores=4-31
/proc/8353/stat : 8353 (plingeling) R 8348 8348 8322 0 -1 4202496 73186 1237 0 0 292 16 0 0 25 0 1 0 622002394 304590848 73023 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210554 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/8353/statm: 74363 73023 129 40 0 72907 0
Current children cumulated CPU time (s) 15.41
Current children cumulated vsize (KiB) 1188300

[startup+6.30061 s]
/proc/loadavg: 6.76 22.27 28.41 6/419 8359
/proc/meminfo: memFree=242270136/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
[pid=8349] ppid=8348 vsize=162852 CPUtime=6.28 cores=0
/proc/8349/stat : 8349 (cryptominisat) R 8348 8348 8322 0 -1 4202496 46619 0 0 0 615 13 0 0 25 0 1 0 622002394 166760448 33347 268435456000 4194304 4587354 140735097219824 18446744073709551615 4236606 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8349/statm: 40713 33347 328 96 0 37275 0
[pid=8350] ppid=8348 vsize=128812 CPUtime=6.28 cores=1
/proc/8350/stat : 8350 (clasp) R 8348 8348 8322 0 -1 4202496 36640 0 0 0 616 12 0 0 25 0 1 0 622002394 131903488 29074 268435456000 4194304 4785740 140734302511440 18446744073709551615 4290607 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/8350/statm: 32203 29074 338 145 0 29248 0
[pid=8351] ppid=8348 vsize=530936 CPUtime=6.28 cores=2
/proc/8351/stat : 8351 (TNM) R 8348 8348 8322 0 -1 4202496 31038 0 0 0 620 8 0 0 25 0 1 0 622002394 543678464 30936 268435456000 134512640 134534564 4294344192 18446744073709551615 134530641 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/8351/statm: 132734 30936 95 6 0 131399 0
[pid=8352] ppid=8348 vsize=119308 CPUtime=6.28 cores=3
/proc/8352/stat : 8352 (march_hi) R 8348 8348 8322 0 -1 4194304 132753 0 0 0 597 31 0 0 25 0 1 0 622002394 122171392 19896 268435456000 4194304 4331988 140734280562912 18446744073709551615 4251467 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/8352/statm: 29827 19896 127 34 0 28269 0
[pid=8353] ppid=8348 vsize=491256 CPUtime=6.28 cores=4-31
/proc/8353/stat : 8353 (plingeling) R 8348 8348 8322 0 -1 4202496 121765 1237 0 0 602 26 0 0 25 0 1 0 622002394 503046144 121474 268435456000 4194304 4356188 140736902139808 18446744073709551615 258650574435 0 0 4096 17442 0 0 0 17 8 0 0 0
/proc/8353/statm: 122814 121474 129 40 0 121358 0
Current children cumulated CPU time (s) 31.4
Current children cumulated vsize (KiB) 1451488

[startup+12.7018 s]
/proc/loadavg: 6.78 22.01 28.30 33/447 8389
/proc/meminfo: memFree=241876104/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
[pid=8349] ppid=8348 vsize=162852 CPUtime=12.68 cores=0
/proc/8349/stat : 8349 (cryptominisat) R 8348 8348 8322 0 -1 4202496 47471 0 0 0 1254 14 0 0 25 0 1 0 622002394 166760448 34198 268435456000 4194304 4587354 140735097219824 18446744073709551615 4236606 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8349/statm: 40713 34198 328 96 0 37275 0
[pid=8350] ppid=8348 vsize=200540 CPUtime=12.69 cores=1
/proc/8350/stat : 8350 (clasp) R 8348 8348 8322 0 -1 4202496 54565 0 0 0 1251 18 0 0 25 0 1 0 622002394 205352960 46999 268435456000 4194304 4785740 140734302511440 18446744073709551615 258650616632 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/8350/statm: 50135 46999 338 145 0 47180 0
[pid=8351] ppid=8348 vsize=530936 CPUtime=12.68 cores=2
/proc/8351/stat : 8351 (TNM) R 8348 8348 8322 0 -1 4202496 31038 0 0 0 1260 8 0 0 25 0 1 0 622002394 543678464 30936 268435456000 134512640 134534564 4294344192 18446744073709551615 134530379 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/8351/statm: 132734 30936 95 6 0 131399 0
[pid=8352] ppid=8348 vsize=121824 CPUtime=12.68 cores=3
/proc/8352/stat : 8352 (march_hi) R 8348 8348 8322 0 -1 4194304 241661 0 0 0 1211 57 0 0 25 0 1 0 622002394 124747776 20512 268435456000 4194304 4331988 140734280562912 18446744073709551615 4261688 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/8352/statm: 30456 20512 127 34 0 28898 0
[pid=8353] ppid=8348 vsize=1322984 CPUtime=105.3 cores=4-31
/proc/8353/stat : 8353 (plingeling) S 8348 8348 8322 0 -1 4202496 323284 1237 0 0 10412 118 0 0 18 0 29 0 622002394 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 20 0 0 0
/proc/8353/statm: 330746 176017 159 40 0 329290 0
[pid=8353/tid=8360] ppid=8348 vsize=1322984 CPUtime=3.41 cores=4-31
/proc/8353/task/8360/stat : 8360 (plingeling) R 8348 8348 8322 0 -1 4202560 2349 1237 0 0 340 1 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4321963 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=8353/tid=8361] ppid=8348 vsize=1322984 CPUtime=3.48 cores=4-31
/proc/8353/task/8361/stat : 8361 (plingeling) R 8348 8348 8322 0 -1 4202560 6863 1237 0 0 346 2 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4287099 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8353/tid=8362] ppid=8348 vsize=1322984 CPUtime=3.48 cores=4-31
/proc/8353/task/8362/stat : 8362 (plingeling) R 8348 8348 8322 0 -1 4202560 7634 1237 0 0 345 3 0 0 20 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4290493 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=8353/tid=8363] ppid=8348 vsize=1322984 CPUtime=3.48 cores=4-31
/proc/8353/task/8363/stat : 8363 (plingeling) R 8348 8348 8322 0 -1 4202560 7092 1237 0 0 346 2 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210605 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=8353/tid=8364] ppid=8348 vsize=1322984 CPUtime=3.39 cores=4-31
/proc/8353/task/8364/stat : 8364 (plingeling) R 8348 8348 8322 0 -1 4202560 11017 1237 0 0 332 7 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4217704 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=8353/tid=8365] ppid=8348 vsize=1322984 CPUtime=3.46 cores=4-31
/proc/8353/task/8365/stat : 8365 (plingeling) R 8348 8348 8322 0 -1 4202560 5178 1237 0 0 345 1 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=8353/tid=8366] ppid=8348 vsize=1322984 CPUtime=3.37 cores=4-31
/proc/8353/task/8366/stat : 8366 (plingeling) R 8348 8348 8322 0 -1 4202560 3067 1237 0 0 336 1 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4290607 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=8353/tid=8367] ppid=8348 vsize=1322984 CPUtime=3.46 cores=4-31
/proc/8353/task/8367/stat : 8367 (plingeling) R 8348 8348 8322 0 -1 4202560 2428 1237 0 0 345 1 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4322106 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=8353/tid=8368] ppid=8348 vsize=1322984 CPUtime=3.47 cores=4-31
/proc/8353/task/8368/stat : 8368 (plingeling) R 8348 8348 8322 0 -1 4202560 3379 1237 0 0 346 1 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4290639 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=8353/tid=8369] ppid=8348 vsize=1322984 CPUtime=3.47 cores=4-31
/proc/8353/task/8369/stat : 8369 (plingeling) R 8348 8348 8322 0 -1 4202560 3689 1237 0 0 346 1 0 0 25 0 29 0 622003311 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4290528 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=8353/tid=8370] ppid=8348 vsize=1322984 CPUtime=3.42 cores=4-31
/proc/8353/task/8370/stat : 8370 (plingeling) R 8348 8348 8322 0 -1 4202560 9870 1237 0 0 335 7 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4300701 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8353/tid=8371] ppid=8348 vsize=1322984 CPUtime=3.45 cores=4-31
/proc/8353/task/8371/stat : 8371 (plingeling) R 8348 8348 8322 0 -1 4202560 1499 1237 0 0 345 0 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4279435 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8353/tid=8372] ppid=8348 vsize=1322984 CPUtime=3.34 cores=4-31
/proc/8353/task/8372/stat : 8372 (plingeling) R 8348 8348 8322 0 -1 4202560 11863 1237 0 0 326 8 0 0 24 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4301704 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=8353/tid=8373] ppid=8348 vsize=1322984 CPUtime=3.45 cores=4-31
/proc/8353/task/8373/stat : 8373 (plingeling) R 8348 8348 8322 0 -1 4202560 5108 1237 0 0 344 1 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8353/tid=8374] ppid=8348 vsize=1322984 CPUtime=3.24 cores=4-31
/proc/8353/task/8374/stat : 8374 (plingeling) R 8348 8348 8322 0 -1 4202560 3403 1237 0 0 324 0 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4321585 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=8353/tid=8375] ppid=8348 vsize=1322984 CPUtime=3.46 cores=4-31
/proc/8353/task/8375/stat : 8375 (plingeling) R 8348 8348 8322 0 -1 4202560 983 1237 0 0 345 1 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4290493 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=8353/tid=8376] ppid=8348 vsize=1322984 CPUtime=3.45 cores=4-31
/proc/8353/task/8376/stat : 8376 (plingeling) R 8348 8348 8322 0 -1 4202560 3479 1237 0 0 344 1 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4322041 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=8353/tid=8377] ppid=8348 vsize=1322984 CPUtime=3.45 cores=4-31
/proc/8353/task/8377/stat : 8377 (plingeling) R 8348 8348 8322 0 -1 4202560 3492 1237 0 0 344 1 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4287094 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=8353/tid=8378] ppid=8348 vsize=1322984 CPUtime=3.44 cores=4-31
/proc/8353/task/8378/stat : 8378 (plingeling) R 8348 8348 8322 0 -1 4202560 6030 1237 0 0 343 1 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4286818 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=8353/tid=8379] ppid=8348 vsize=1322984 CPUtime=3.43 cores=4-31
/proc/8353/task/8379/stat : 8379 (plingeling) R 8348 8348 8322 0 -1 4202560 7050 1237 0 0 341 2 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4283916 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=8353/tid=8380] ppid=8348 vsize=1322984 CPUtime=3.47 cores=4-31
/proc/8353/task/8380/stat : 8380 (plingeling) R 8348 8348 8322 0 -1 4202560 3763 1237 0 0 346 1 0 0 25 0 29 0 622003312 1354735616 176017 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=8353/tid=8381] ppid=8348 vsize=1322984 CPUtime=3.45 cores=4-31

################
# More data... #
################

[pid=8351] ppid=8348 vsize=530936 CPUtime=4961.26 cores=2
/proc/8351/stat : 8351 (TNM) R 8348 8348 8322 0 -1 4202496 31038 0 0 0 496118 8 0 0 25 0 1 0 622002394 543678464 30936 268435456000 134512640 134534564 4294344192 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/8351/statm: 132734 30936 95 6 0 131399 0
[pid=8352] ppid=8348 vsize=416088 CPUtime=4961.19 cores=3
/proc/8352/stat : 8352 (march_hi) R 8348 8348 8322 0 -1 4202496 445637 0 0 0 496011 108 0 0 25 0 1 0 622002394 426074112 52830 268435456000 4194304 4331988 140734280562912 18446744073709551615 4228257 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/8352/statm: 104022 52830 174 34 0 102464 0
[pid=8353] ppid=8348 vsize=9272632 CPUtime=138493 cores=4-31
/proc/8353/stat : 8353 (plingeling) S 8348 8348 8322 0 -1 4202496 21473255 1237 0 0 13833545 15735 0 0 18 0 29 0 622002394 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 20 0 0 0
/proc/8353/statm: 2318158 2162600 159 40 0 2316702 0
[pid=8353/tid=8360] ppid=8348 vsize=9272632 CPUtime=4921.04 cores=4-31
/proc/8353/task/8360/stat : 8360 (plingeling) R 8348 8348 8322 0 -1 4202560 799372 1237 0 0 491575 529 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281495 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=8353/tid=8361] ppid=8348 vsize=9272632 CPUtime=4950.72 cores=4-31
/proc/8353/task/8361/stat : 8361 (plingeling) R 8348 8348 8322 0 -1 4202560 830148 1237 0 0 494667 405 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281793 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=8353/tid=8362] ppid=8348 vsize=9272632 CPUtime=4951.17 cores=4-31
/proc/8353/task/8362/stat : 8362 (plingeling) R 8348 8348 8322 0 -1 4202560 771765 1237 0 0 494613 504 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210605 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=8353/tid=8363] ppid=8348 vsize=9272632 CPUtime=4951.12 cores=4-31
/proc/8353/task/8363/stat : 8363 (plingeling) R 8348 8348 8322 0 -1 4202560 683051 1237 0 0 494629 483 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4284253 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=8353/tid=8364] ppid=8348 vsize=9272632 CPUtime=4950.5 cores=4-31
/proc/8353/task/8364/stat : 8364 (plingeling) R 8348 8348 8322 0 -1 4202560 740673 1237 0 0 494677 373 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281766 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=8353/tid=8365] ppid=8348 vsize=9272632 CPUtime=4950.6 cores=4-31
/proc/8353/task/8365/stat : 8365 (plingeling) R 8348 8348 8322 0 -1 4202560 587387 1237 0 0 494333 727 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4334538 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=8353/tid=8366] ppid=8348 vsize=9272632 CPUtime=4951.14 cores=4-31
/proc/8353/task/8366/stat : 8366 (plingeling) R 8348 8348 8322 0 -1 4202560 862849 1237 0 0 494621 493 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4279535 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=8353/tid=8367] ppid=8348 vsize=9272632 CPUtime=4930.56 cores=4-31
/proc/8353/task/8367/stat : 8367 (plingeling) R 8348 8348 8322 0 -1 4202560 847758 1237 0 0 491815 1241 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210575 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=8353/tid=8368] ppid=8348 vsize=9272632 CPUtime=4950.3 cores=4-31
/proc/8353/task/8368/stat : 8368 (plingeling) R 8348 8348 8322 0 -1 4202560 819195 1237 0 0 494471 559 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4274252 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=8353/tid=8369] ppid=8348 vsize=9272632 CPUtime=4950.43 cores=4-31
/proc/8353/task/8369/stat : 8369 (plingeling) R 8348 8348 8322 0 -1 4202560 818266 1237 0 0 494493 550 0 0 25 0 29 0 622003311 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4334501 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=8353/tid=8370] ppid=8348 vsize=9272632 CPUtime=4932.25 cores=4-31
/proc/8353/task/8370/stat : 8370 (plingeling) R 8348 8348 8322 0 -1 4202560 674706 1237 0 0 492757 468 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4234302 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=8353/tid=8371] ppid=8348 vsize=9272632 CPUtime=4948.84 cores=4-31
/proc/8353/task/8371/stat : 8371 (plingeling) R 8348 8348 8322 0 -1 4202560 543345 1237 0 0 494598 286 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=8353/tid=8372] ppid=8348 vsize=9272632 CPUtime=4949.81 cores=4-31
/proc/8353/task/8372/stat : 8372 (plingeling) R 8348 8348 8322 0 -1 4202560 806391 1237 0 0 494442 539 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281557 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=8353/tid=8373] ppid=8348 vsize=9272632 CPUtime=4947.36 cores=4-31
/proc/8353/task/8373/stat : 8373 (plingeling) R 8348 8348 8322 0 -1 4202560 839985 1237 0 0 494182 554 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4284141 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=8353/tid=8374] ppid=8348 vsize=9272632 CPUtime=4950.27 cores=4-31
/proc/8353/task/8374/stat : 8374 (plingeling) R 8348 8348 8322 0 -1 4202560 717295 1237 0 0 494519 508 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4279490 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=8353/tid=8375] ppid=8348 vsize=9272632 CPUtime=4946.18 cores=4-31
/proc/8353/task/8375/stat : 8375 (plingeling) R 8348 8348 8322 0 -1 4202560 840538 1237 0 0 493510 1108 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4233969 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=8353/tid=8376] ppid=8348 vsize=9272632 CPUtime=4928.36 cores=4-31
/proc/8353/task/8376/stat : 8376 (plingeling) R 8348 8348 8322 0 -1 4202560 748000 1237 0 0 492336 500 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4234297 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=8353/tid=8377] ppid=8348 vsize=9272632 CPUtime=4936.02 cores=4-31
/proc/8353/task/8377/stat : 8377 (plingeling) R 8348 8348 8322 0 -1 4202560 533280 1237 0 0 493218 384 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8353/tid=8378] ppid=8348 vsize=9272632 CPUtime=4951.28 cores=4-31
/proc/8353/task/8378/stat : 8378 (plingeling) R 8348 8348 8322 0 -1 4202560 843398 1237 0 0 494530 598 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=8353/tid=8379] ppid=8348 vsize=9272632 CPUtime=4951.05 cores=4-31
/proc/8353/task/8379/stat : 8379 (plingeling) R 8348 8348 8322 0 -1 4202560 680557 1237 0 0 494688 417 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4284141 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=8353/tid=8380] ppid=8348 vsize=9272632 CPUtime=4950.49 cores=4-31
/proc/8353/task/8380/stat : 8380 (plingeling) R 8348 8348 8322 0 -1 4202560 879334 1237 0 0 494478 571 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8353/tid=8381] ppid=8348 vsize=9272632 CPUtime=4936.9 cores=4-31
/proc/8353/task/8381/stat : 8381 (plingeling) R 8348 8348 8322 0 -1 4202560 706190 1237 0 0 493203 487 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4284148 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8353/tid=8382] ppid=8348 vsize=9272632 CPUtime=4943.58 cores=4-31
/proc/8353/task/8382/stat : 8382 (plingeling) R 8348 8348 8322 0 -1 4202560 735533 1237 0 0 493785 573 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4284190 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=8353/tid=8383] ppid=8348 vsize=9272632 CPUtime=4951.43 cores=4-31
/proc/8353/task/8383/stat : 8383 (plingeling) R 8348 8348 8322 0 -1 4202560 779824 1237 0 0 494306 837 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4284248 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=8353/tid=8384] ppid=8348 vsize=9272632 CPUtime=4950.87 cores=4-31
/proc/8353/task/8384/stat : 8384 (plingeling) R 8348 8348 8322 0 -1 4202560 866553 1237 0 0 494596 491 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4335716 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8353/tid=8385] ppid=8348 vsize=9272632 CPUtime=4949.85 cores=4-31
/proc/8353/task/8385/stat : 8385 (plingeling) R 8348 8348 8322 0 -1 4202560 817753 1237 0 0 494451 534 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=8353/tid=8386] ppid=8348 vsize=9272632 CPUtime=4951.01 cores=4-31
/proc/8353/task/8386/stat : 8386 (plingeling) R 8348 8348 8322 0 -1 4202560 806371 1237 0 0 494521 580 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=8353/tid=8387] ppid=8348 vsize=9272632 CPUtime=4950.25 cores=4-31
/proc/8353/task/8387/stat : 8387 (plingeling) R 8348 8348 8322 0 -1 4202560 717336 1237 0 0 494640 385 0 0 25 0 29 0 622003312 9495175168 2162600 268435456000 4194304 4356188 140736902139808 18446744073709551615 4331039 0 0 4096 17442 0 0 0 -1 27 0 0 0
Current children cumulated CPU time (s) 158337
Current children cumulated vsize (KiB) 17326276



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5014.4 s]
/proc/loadavg: 32.04 32.05 32.02 33/447 8536
/proc/meminfo: memFree=241923516/264257624 swapFree=67111528/67111528
[pid=8348] ppid=8346 vsize=18324 CPUtime=0.14 cores=0-31
/proc/8348/stat : 8348 (ppfolio) S 8346 8348 8322 0 -1 4202496 514 0 0 0 2 12 0 0 15 0 1 0 622002394 18763776 386 268435456000 4194304 4287625 140733506691728 18446744073709551615 258650975999 0 0 4096 65536 2748533058008252415 0 0 17 30 0 0 0
/proc/8348/statm: 4581 386 337 23 0 91 0
[pid=8349] ppid=8348 vsize=1869736 CPUtime=5013.46 cores=0
/proc/8349/stat : 8349 (cryptominisat) R 8348 8348 8322 0 -1 4202496 7315602 0 0 0 499447 1899 0 0 25 0 1 0 622002394 1914609664 86259 268435456000 4194304 4587354 140735097219824 18446744073709551615 4238618 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/8349/statm: 467434 86259 338 96 0 463996 0
[pid=8350] ppid=8348 vsize=5218560 CPUtime=5012.78 cores=1
/proc/8350/stat : 8350 (clasp) R 8348 8348 8322 0 -1 4202496 1304872 0 0 0 500846 432 0 0 25 0 1 0 622002394 5343805440 1297306 268435456000 4194304 4785740 140734302511440 18446744073709551615 4278595 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/8350/statm: 1304640 1297306 338 145 0 1301685 0
[pid=8351] ppid=8348 vsize=530936 CPUtime=5013.37 cores=2
/proc/8351/stat : 8351 (TNM) R 8348 8348 8322 0 -1 4202496 31038 0 0 0 501329 8 0 0 25 0 1 0 622002394 543678464 30936 268435456000 134512640 134534564 4294344192 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/8351/statm: 132734 30936 95 6 0 131399 0
[pid=8352] ppid=8348 vsize=416088 CPUtime=5013.31 cores=3
/proc/8352/stat : 8352 (march_hi) R 8348 8348 8322 0 -1 4202496 445700 0 0 0 501222 109 0 0 25 0 1 0 622002394 426074112 52893 268435456000 4194304 4331988 140734280562912 18446744073709551615 4228270 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/8352/statm: 104022 52893 174 34 0 102464 0
[pid=8353] ppid=8348 vsize=8832292 CPUtime=139950 cores=4-31
/proc/8353/stat : 8353 (plingeling) S 8348 8348 8322 0 -1 4202496 21760999 1237 0 0 13979149 15873 0 0 18 0 29 0 622002394 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 20 0 0 0
/proc/8353/statm: 2208073 2052515 159 40 0 2206617 0
[pid=8353/tid=8360] ppid=8348 vsize=8832292 CPUtime=4972.09 cores=4-31
/proc/8353/task/8360/stat : 8360 (plingeling) R 8348 8348 8322 0 -1 4202560 805516 1237 0 0 496679 530 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281706 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=8353/tid=8361] ppid=8348 vsize=8832292 CPUtime=5002.83 cores=4-31
/proc/8353/task/8361/stat : 8361 (plingeling) R 8348 8348 8322 0 -1 4202560 851652 1237 0 0 499874 409 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=8353/tid=8362] ppid=8348 vsize=8832292 CPUtime=5003.28 cores=4-31
/proc/8353/task/8362/stat : 8362 (plingeling) R 8348 8348 8322 0 -1 4202560 771765 1237 0 0 499824 504 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4330999 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=8353/tid=8363] ppid=8348 vsize=8832292 CPUtime=5003.22 cores=4-31
/proc/8353/task/8363/stat : 8363 (plingeling) R 8348 8348 8322 0 -1 4202560 685099 1237 0 0 499839 483 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281652 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=8353/tid=8364] ppid=8348 vsize=8832292 CPUtime=5002.62 cores=4-31
/proc/8353/task/8364/stat : 8364 (plingeling) R 8348 8348 8322 0 -1 4202560 755009 1237 0 0 499886 376 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4274240 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=8353/tid=8365] ppid=8348 vsize=8832292 CPUtime=5002.71 cores=4-31
/proc/8353/task/8365/stat : 8365 (plingeling) R 8348 8348 8322 0 -1 4202560 589435 1237 0 0 499543 728 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281639 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=8353/tid=8366] ppid=8348 vsize=8832292 CPUtime=5003.25 cores=4-31
/proc/8353/task/8366/stat : 8366 (plingeling) R 8348 8348 8322 0 -1 4202560 879233 1237 0 0 499829 496 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=8353/tid=8367] ppid=8348 vsize=8832292 CPUtime=4982.67 cores=4-31
/proc/8353/task/8367/stat : 8367 (plingeling) R 8348 8348 8322 0 -1 4202560 847758 1237 0 0 497026 1241 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4236939 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=8353/tid=8368] ppid=8348 vsize=8832292 CPUtime=5002.4 cores=4-31
/proc/8353/task/8368/stat : 8368 (plingeling) R 8348 8348 8322 0 -1 4202560 823291 1237 0 0 499680 560 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=8353/tid=8369] ppid=8348 vsize=8832292 CPUtime=5002.54 cores=4-31
/proc/8353/task/8369/stat : 8369 (plingeling) R 8348 8348 8322 0 -1 4202560 822362 1237 0 0 499702 552 0 0 25 0 29 0 622003311 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=8353/tid=8370] ppid=8348 vsize=8832292 CPUtime=4984.36 cores=4-31
/proc/8353/task/8370/stat : 8370 (plingeling) R 8348 8348 8322 0 -1 4202560 689042 1237 0 0 497965 471 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281771 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=8353/tid=8371] ppid=8348 vsize=8832292 CPUtime=5000.95 cores=4-31
/proc/8353/task/8371/stat : 8371 (plingeling) R 8348 8348 8322 0 -1 4202560 553585 1237 0 0 499807 288 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4283908 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=8353/tid=8372] ppid=8348 vsize=8832292 CPUtime=5001.92 cores=4-31
/proc/8353/task/8372/stat : 8372 (plingeling) R 8348 8348 8322 0 -1 4202560 822775 1237 0 0 499650 542 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4244255 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=8353/tid=8373] ppid=8348 vsize=8832292 CPUtime=4999.46 cores=4-31
/proc/8353/task/8373/stat : 8373 (plingeling) R 8348 8348 8322 0 -1 4202560 848177 1237 0 0 499391 555 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=8353/tid=8374] ppid=8348 vsize=8832292 CPUtime=5002.39 cores=4-31
/proc/8353/task/8374/stat : 8374 (plingeling) R 8348 8348 8322 0 -1 4202560 719343 1237 0 0 499725 514 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210605 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=8353/tid=8375] ppid=8348 vsize=8832292 CPUtime=4998.29 cores=4-31
/proc/8353/task/8375/stat : 8375 (plingeling) R 8348 8348 8322 0 -1 4202560 840538 1237 0 0 498721 1108 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=8353/tid=8376] ppid=8348 vsize=8832292 CPUtime=4980.48 cores=4-31
/proc/8353/task/8376/stat : 8376 (plingeling) R 8348 8348 8322 0 -1 4202560 764384 1237 0 0 497544 504 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4236968 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=8353/tid=8377] ppid=8348 vsize=8832292 CPUtime=4987.7 cores=4-31
/proc/8353/task/8377/stat : 8377 (plingeling) R 8348 8348 8322 0 -1 4202560 553760 1237 0 0 498355 415 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4210567 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=8353/tid=8378] ppid=8348 vsize=8832292 CPUtime=5003.39 cores=4-31
/proc/8353/task/8378/stat : 8378 (plingeling) R 8348 8348 8322 0 -1 4202560 861830 1237 0 0 499736 603 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281624 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=8353/tid=8379] ppid=8348 vsize=8832292 CPUtime=5003.16 cores=4-31
/proc/8353/task/8379/stat : 8379 (plingeling) R 8348 8348 8322 0 -1 4202560 688749 1237 0 0 499878 438 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=8353/tid=8380] ppid=8348 vsize=8832292 CPUtime=5002.6 cores=4-31
/proc/8353/task/8380/stat : 8380 (plingeling) R 8348 8348 8322 0 -1 4202560 879334 1237 0 0 499689 571 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281487 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=8353/tid=8381] ppid=8348 vsize=8832292 CPUtime=4989.02 cores=4-31
/proc/8353/task/8381/stat : 8381 (plingeling) R 8348 8348 8322 0 -1 4202560 724622 1237 0 0 498410 492 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4236939 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=8353/tid=8382] ppid=8348 vsize=8832292 CPUtime=4995.59 cores=4-31
/proc/8353/task/8382/stat : 8382 (plingeling) R 8348 8348 8322 0 -1 4202560 758061 1237 0 0 498966 593 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281681 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=8353/tid=8383] ppid=8348 vsize=8832292 CPUtime=5003.54 cores=4-31
/proc/8353/task/8383/stat : 8383 (plingeling) R 8348 8348 8322 0 -1 4202560 788016 1237 0 0 499516 838 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4211015 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=8353/tid=8384] ppid=8348 vsize=8832292 CPUtime=5002.93 cores=4-31
/proc/8353/task/8384/stat : 8384 (plingeling) R 8348 8348 8322 0 -1 4202560 872697 1237 0 0 499800 493 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281624 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=8353/tid=8385] ppid=8348 vsize=8832292 CPUtime=5001.94 cores=4-31
/proc/8353/task/8385/stat : 8385 (plingeling) R 8348 8348 8322 0 -1 4202560 832089 1237 0 0 499648 546 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=8353/tid=8386] ppid=8348 vsize=8832292 CPUtime=5003.11 cores=4-31
/proc/8353/task/8386/stat : 8386 (plingeling) R 8348 8348 8322 0 -1 4202560 828899 1237 0 0 499726 585 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4234244 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=8353/tid=8387] ppid=8348 vsize=8832292 CPUtime=5002.36 cores=4-31
/proc/8353/task/8387/stat : 8387 (plingeling) R 8348 8348 8322 0 -1 4202560 727576 1237 0 0 499849 387 0 0 25 0 29 0 622003312 9044267008 2052515 268435456000 4194304 4356188 140736902139808 18446744073709551615 4211010 0 0 4096 17442 0 0 0 -1 27 0 0 0
Current children cumulated CPU time (s) 160003
Current children cumulated vsize (KiB) 16885936

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 8348 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=22996
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=125980
# CPU time returned by wait4() is 0.148976
# while last known CPU time is 160003
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5014.44
CPU time (s): 160003
CPU user time (s): 159820
CPU system time (s): 183.33
CPU usage (%): 3190.85
Max. virtual memory (cumulated for all children) (KiB): 18884960

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.022996
system time used= 0.12598
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 514
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7338
involuntary context switches= 2

runsolver used 26.384 second user time and 114.954 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-11 11:12:40
IDJOB=3347381
IDBENCH=82389
IDSOLVER=1808
FILE ID=node200/3347381-1305105159
RUNJOBID= node200-1305105159-8336
PBS_JOBID= 13282058
Free space on /tmp= 62036 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/rintanen/SATPlanning/openstacks-sequencedstrips-nonadl-nonnegated-os-sequencedstrips-p30_3.045-NOTKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347381-1305105159/watcher-3347381-1305105159 -o /tmp/evaluation-result-3347381-1305105159/solver-3347381-1305105159 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347381-1305105159.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 4ab53801f3b1a3f7169c9fb38cba4184
RANDOM SEED=1980275411

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      243077908 kB
Buffers:        412260 kB
Cached:        7421440 kB
SwapCached:          0 kB
Active:        6961020 kB
Inactive:     13253664 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      243077908 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           39600 kB
Writeback:           0 kB
AnonPages:       70072 kB
Mapped:          13484 kB
Slab:           880044 kB
PageTables:       4780 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   183192 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62024 MiB
End job on node200 at 2011-05-11 12:36:16