Trace number 3347637

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 160002 5009.64

General information on the benchmark

NameSAT03/industrial/li/li-exam/
li-exam-61.shuffled-as.sat03-366.cnf
MD5SUM3be662940f394fa876137a905247d755
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables28147
Number of clauses108436
Sum of the clauses size373110
Maximum clause length166
Minimum clause length1
Number of clauses of size 114
Number of clauses of size 218786
Number of clauses of size 351588
Number of clauses of size 437440
Number of clauses of size 53
Number of clauses of size over 5605

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347637-1305409410/watcher-3347637-1305409410 -o /tmp/evaluation-result-3347637-1305409410/solver-3347637-1305409410 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347637-1305409410.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 27.18 31.01 31.67 1/413 490
/proc/meminfo: memFree=255577704/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 21 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0

[startup+0.0883549 s]
/proc/loadavg: 27.18 31.01 31.67 1/413 490
/proc/meminfo: memFree=255577704/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 21 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100374 s]
/proc/loadavg: 27.18 31.01 31.67 1/413 490
/proc/meminfo: memFree=255577704/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 21 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300398 s]
/proc/loadavg: 27.18 31.01 31.67 1/413 490
/proc/meminfo: memFree=255577704/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 20 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700379 s]
/proc/loadavg: 27.18 31.01 31.67 1/413 490
/proc/meminfo: memFree=255577704/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50038 s]
/proc/loadavg: 27.18 31.01 31.67 6/419 501
/proc/meminfo: memFree=255427380/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
[pid=491] ppid=490 vsize=53844 CPUtime=1.48 cores=0
/proc/491/stat : 491 (cryptominisat) R 490 490 464 0 -1 4202496 9936 0 0 0 145 3 0 0 25 0 1 0 652427444 55136256 8663 268435456000 4194304 4587354 140733483058304 18446744073709551615 4243933 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/491/statm: 13461 8663 330 96 0 10023 0
[pid=492] ppid=490 vsize=33048 CPUtime=1.48 cores=1
/proc/492/stat : 492 (clasp) R 490 490 464 0 -1 4202496 6868 0 0 0 146 2 0 0 20 0 1 0 652427444 33841152 5517 268435456000 4194304 4785740 140734110014624 18446744073709551615 4545865 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/492/statm: 8262 5517 331 145 0 5307 0
[pid=493] ppid=490 vsize=464700 CPUtime=1.48 cores=2
/proc/493/stat : 493 (TNM) R 490 490 464 0 -1 4202496 5129 0 0 0 146 2 0 0 20 0 1 0 652427444 475852800 5026 268435456000 134512640 134534564 4287046928 18446744073709551615 134528710 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/493/statm: 116175 5026 95 6 0 114840 0
[pid=494] ppid=490 vsize=27788 CPUtime=1.48 cores=3
/proc/494/stat : 494 (march_hi) R 490 490 464 0 -1 4202496 17035 0 0 0 143 5 0 0 20 0 1 0 652427444 28454912 4489 268435456000 4194304 4331988 140736824348560 18446744073709551615 4217696 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/494/statm: 6947 4489 155 34 0 5389 0
[pid=495] ppid=490 vsize=99060 CPUtime=1.48 cores=4-31
/proc/495/stat : 495 (plingeling) R 490 490 464 0 -1 4202496 23540 1241 0 0 140 7 0 1 21 0 1 0 652427444 101437440 23424 268435456000 4194304 4356188 140735122061728 18446744073709551615 4277939 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/495/statm: 24765 23424 129 40 0 23309 0
Current children cumulated CPU time (s) 7.4
Current children cumulated vsize (KiB) 696764

[startup+3.10144 s]
/proc/loadavg: 27.18 31.01 31.67 6/419 501
/proc/meminfo: memFree=255352436/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
[pid=491] ppid=490 vsize=53844 CPUtime=3.08 cores=0
/proc/491/stat : 491 (cryptominisat) R 490 490 464 0 -1 4202496 9967 0 0 0 305 3 0 0 25 0 1 0 652427444 55136256 8691 268435456000 4194304 4587354 140733483058304 18446744073709551615 4251533 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/491/statm: 13461 8691 331 96 0 10023 0
[pid=492] ppid=490 vsize=33048 CPUtime=3.08 cores=1
/proc/492/stat : 492 (clasp) R 490 490 464 0 -1 4202496 6875 0 0 0 306 2 0 0 25 0 1 0 652427444 33841152 5524 268435456000 4194304 4785740 140734110014624 18446744073709551615 4318807 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/492/statm: 8262 5524 338 145 0 5307 0
[pid=493] ppid=490 vsize=464700 CPUtime=3.08 cores=2
/proc/493/stat : 493 (TNM) R 490 490 464 0 -1 4202496 5129 0 0 0 306 2 0 0 25 0 1 0 652427444 475852800 5026 268435456000 134512640 134534564 4287046928 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/493/statm: 116175 5026 95 6 0 114840 0
[pid=494] ppid=490 vsize=67844 CPUtime=3.08 cores=3
/proc/494/stat : 494 (march_hi) R 490 490 464 0 -1 4202496 27126 0 0 0 302 6 0 0 25 0 1 0 652427444 69472256 8517 268435456000 4194304 4331988 140736824348560 18446744073709551615 4226434 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/494/statm: 16961 8517 185 34 0 15403 0
[pid=495] ppid=490 vsize=150544 CPUtime=3.09 cores=4-31
/proc/495/stat : 495 (plingeling) R 490 490 464 0 -1 4202496 36539 1241 0 0 298 10 0 1 25 0 1 0 652427444 154157056 36295 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210554 0 0 4096 17442 0 0 0 17 10 0 0 0
/proc/495/statm: 37636 36295 129 40 0 36180 0
Current children cumulated CPU time (s) 15.41
Current children cumulated vsize (KiB) 788304

[startup+6.30152 s]
/proc/loadavg: 25.41 30.57 31.52 34/447 529
/proc/meminfo: memFree=255098456/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
[pid=491] ppid=490 vsize=53844 CPUtime=6.28 cores=0
/proc/491/stat : 491 (cryptominisat) R 490 490 464 0 -1 4202496 10012 0 0 0 625 3 0 0 25 0 1 0 652427444 55136256 8728 268435456000 4194304 4587354 140733483058304 18446744073709551615 4237618 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/491/statm: 13461 8728 331 96 0 10023 0
[pid=492] ppid=490 vsize=33048 CPUtime=6.28 cores=1
/proc/492/stat : 492 (clasp) R 490 490 464 0 -1 4202496 6875 0 0 0 626 2 0 0 25 0 1 0 652427444 33841152 5524 268435456000 4194304 4785740 140734110014624 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/492/statm: 8262 5524 338 145 0 5307 0
[pid=493] ppid=490 vsize=464700 CPUtime=6.28 cores=2
/proc/493/stat : 493 (TNM) R 490 490 464 0 -1 4202496 5129 0 0 0 626 2 0 0 25 0 1 0 652427444 475852800 5026 268435456000 134512640 134534564 4287046928 18446744073709551615 134528979 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/493/statm: 116175 5026 95 6 0 114840 0
[pid=494] ppid=490 vsize=68260 CPUtime=6.28 cores=3
/proc/494/stat : 494 (march_hi) R 490 490 464 0 -1 4202496 27371 0 0 0 622 6 0 0 25 0 1 0 652427444 69898240 8760 268435456000 4194304 4331988 140736824348560 18446744073709551615 4301433 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/494/statm: 17065 8760 183 34 0 15507 0
[pid=495] ppid=490 vsize=811684 CPUtime=50.42 cores=4-31
/proc/495/stat : 495 (plingeling) S 490 490 464 0 -1 4202496 120739 1241 0 0 4916 125 0 1 19 0 29 0 652427444 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 24 0 0 0
/proc/495/statm: 202921 86233 159 40 0 201465 0
[pid=495/tid=502] ppid=490 vsize=811684 CPUtime=1.66 cores=4-31
/proc/495/task/502/stat : 502 (plingeling) R 490 490 464 0 -1 4202560 2465 1241 0 0 162 3 0 1 18 0 29 0 652427899 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4211720 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=495/tid=503] ppid=490 vsize=811684 CPUtime=1.66 cores=4-31
/proc/495/task/503/stat : 503 (plingeling) R 490 490 464 0 -1 4202560 2488 1241 0 0 162 3 0 1 18 0 29 0 652427899 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4211178 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=495/tid=504] ppid=490 vsize=811684 CPUtime=1.69 cores=4-31
/proc/495/task/504/stat : 504 (plingeling) R 490 490 464 0 -1 4202560 2612 1241 0 0 164 4 0 1 18 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4211720 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=495/tid=505] ppid=490 vsize=811684 CPUtime=1.65 cores=4-31
/proc/495/task/505/stat : 505 (plingeling) R 490 490 464 0 -1 4202560 2584 1241 0 0 160 4 0 1 25 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=495/tid=506] ppid=490 vsize=811684 CPUtime=1.65 cores=4-31
/proc/495/task/506/stat : 506 (plingeling) R 490 490 464 0 -1 4202560 2569 1241 0 0 160 4 0 1 19 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4303438 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=495/tid=507] ppid=490 vsize=811684 CPUtime=1.65 cores=4-31
/proc/495/task/507/stat : 507 (plingeling) R 490 490 464 0 -1 4202560 2848 1241 0 0 161 3 0 1 25 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4232885 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=495/tid=508] ppid=490 vsize=811684 CPUtime=1.65 cores=4-31
/proc/495/task/508/stat : 508 (plingeling) R 490 490 464 0 -1 4202560 2569 1241 0 0 161 3 0 1 19 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4232971 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=495/tid=509] ppid=490 vsize=811684 CPUtime=1.64 cores=4-31
/proc/495/task/509/stat : 509 (plingeling) R 490 490 464 0 -1 4202560 2600 1241 0 0 159 4 0 1 18 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4231370 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=495/tid=510] ppid=490 vsize=811684 CPUtime=1.65 cores=4-31
/proc/495/task/510/stat : 510 (plingeling) R 490 490 464 0 -1 4202560 2665 1241 0 0 160 4 0 1 19 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4232894 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=495/tid=511] ppid=490 vsize=811684 CPUtime=1.64 cores=4-31
/proc/495/task/511/stat : 511 (plingeling) R 490 490 464 0 -1 4202560 3134 1241 0 0 159 4 0 1 20 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4232827 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=495/tid=512] ppid=490 vsize=811684 CPUtime=1.69 cores=4-31
/proc/495/task/512/stat : 512 (plingeling) R 490 490 464 0 -1 4202560 2408 1241 0 0 164 4 0 1 19 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4215966 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=495/tid=513] ppid=490 vsize=811684 CPUtime=1.66 cores=4-31
/proc/495/task/513/stat : 513 (plingeling) R 490 490 464 0 -1 4202560 2693 1241 0 0 162 3 0 1 20 0 29 0 652427900 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4332969 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=495/tid=514] ppid=490 vsize=811684 CPUtime=1.67 cores=4-31
/proc/495/task/514/stat : 514 (plingeling) R 490 490 464 0 -1 4202560 2260 1241 0 0 164 2 0 1 20 0 29 0 652427901 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4217793 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=495/tid=515] ppid=490 vsize=811684 CPUtime=1.6 cores=4-31
/proc/495/task/515/stat : 515 (plingeling) R 490 490 464 0 -1 4202560 2865 1241 0 0 154 5 0 1 25 0 29 0 652427901 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4211127 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=495/tid=516] ppid=490 vsize=811684 CPUtime=1.64 cores=4-31
/proc/495/task/516/stat : 516 (plingeling) R 490 490 464 0 -1 4202560 2873 1241 0 0 159 4 0 1 19 0 29 0 652427901 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4311336 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=495/tid=517] ppid=490 vsize=811684 CPUtime=1.63 cores=4-31
/proc/495/task/517/stat : 517 (plingeling) R 490 490 464 0 -1 4202560 3069 1241 0 0 158 4 0 1 22 0 29 0 652427901 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4211672 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=495/tid=518] ppid=490 vsize=811684 CPUtime=1.63 cores=4-31
/proc/495/task/518/stat : 518 (plingeling) R 490 490 464 0 -1 4202560 2152 1241 0 0 159 3 0 1 18 0 29 0 652427901 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4276747 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=495/tid=519] ppid=490 vsize=811684 CPUtime=1.66 cores=4-31
/proc/495/task/519/stat : 519 (plingeling) R 490 490 464 0 -1 4202560 2644 1241 0 0 161 4 0 1 23 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4208936 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=495/tid=520] ppid=490 vsize=811684 CPUtime=1.65 cores=4-31
/proc/495/task/520/stat : 520 (plingeling) R 490 490 464 0 -1 4202560 2748 1241 0 0 160 4 0 1 23 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4208763 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=495/tid=521] ppid=490 vsize=811684 CPUtime=1.67 cores=4-31
/proc/495/task/521/stat : 521 (plingeling) R 490 490 464 0 -1 4202560 1974 1241 0 0 163 3 0 1 23 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4334545 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=495/tid=522] ppid=490 vsize=811684 CPUtime=1.64 cores=4-31
/proc/495/task/522/stat : 522 (plingeling) R 490 490 464 0 -1 4202560 2463 1241 0 0 160 3 0 1 18 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4208763 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=495/tid=523] ppid=490 vsize=811684 CPUtime=1.63 cores=4-31
/proc/495/task/523/stat : 523 (plingeling) R 490 490 464 0 -1 4202560 1937 1241 0 0 159 3 0 1 18 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4232872 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=495/tid=524] ppid=490 vsize=811684 CPUtime=1.61 cores=4-31
/proc/495/task/524/stat : 524 (plingeling) R 490 490 464 0 -1 4202560 1938 1241 0 0 159 1 0 1 22 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4232894 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=495/tid=525] ppid=490 vsize=811684 CPUtime=1.66 cores=4-31
/proc/495/task/525/stat : 525 (plingeling) R 490 490 464 0 -1 4202560 2632 1241 0 0 161 4 0 1 23 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4312083 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=495/tid=526] ppid=490 vsize=811684 CPUtime=1.65 cores=4-31
/proc/495/task/526/stat : 526 (plingeling) R 490 490 464 0 -1 4202560 2419 1241 0 0 162 2 0 1 19 0 29 0 652427902 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4271040 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=495/tid=527] ppid=490 vsize=811684 CPUtime=1.57 cores=4-31
/proc/495/task/527/stat : 527 (plingeling) R 490 490 464 0 -1 4202560 1937 1241 0 0 152 4 0 1 24 0 29 0 652427906 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4232872 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=495/tid=528] ppid=490 vsize=811684 CPUtime=1.49 cores=4-31
/proc/495/task/528/stat : 528 (plingeling) R 490 490 464 0 -1 4202560 2652 1241 0 0 145 3 0 1 22 0 29 0 652427906 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4271173 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=495/tid=529] ppid=490 vsize=811684 CPUtime=1.56 cores=4-31
/proc/495/task/529/stat : 529 (plingeling) R 490 490 464 0 -1 4202560 1630 1241 0 0 151 4 0 1 24 0 29 0 652427906 831164416 86233 268435456000 4194304 4356188 140735122061728 18446744073709551615 4334386 0 0 4096 17442 0 0 0 -1 27 0 0 0
Current children cumulated CPU time (s) 75.54
Current children cumulated vsize (KiB) 1449860

[startup+12.7017 s]
/proc/loadavg: 25.93 30.60 31.52 33/447 529
/proc/meminfo: memFree=255068088/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
[pid=491] ppid=490 vsize=53844 CPUtime=12.68 cores=0
/proc/491/stat : 491 (cryptominisat) R 490 490 464 0 -1 4202496 10167 0 0 0 1265 3 0 0 25 0 1 0 652427444 55136256 8878 268435456000 4194304 4587354 140733483058304 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=493] ppid=490 vsize=464704 CPUtime=4961.35 cores=2
/proc/493/stat : 493 (TNM) R 490 490 464 0 -1 4202496 5138 0 0 0 496133 2 0 0 25 0 1 0 652427444 475856896 5035 268435456000 134512640 134534564 4287046928 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/493/statm: 116176 5035 103 6 0 114841 0
[pid=494] ppid=490 vsize=128648 CPUtime=4961.32 cores=3
/proc/494/stat : 494 (march_hi) R 490 490 464 0 -1 4202496 39453 0 0 0 496110 22 0 0 25 0 1 0 652427444 131735552 20842 268435456000 4194304 4331988 140736824348560 18446744073709551615 4228459 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/494/statm: 32162 20842 183 34 0 30604 0
[pid=495] ppid=490 vsize=2426192 CPUtime=138645 cores=4-31
/proc/495/stat : 495 (plingeling) S 490 490 464 0 -1 4202496 4341253 1241 0 0 13861455 3036 0 1 19 0 29 0 652427444 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 24 0 0 0
/proc/495/statm: 606548 504470 159 40 0 605092 0
[pid=495/tid=502] ppid=490 vsize=2426192 CPUtime=4956.73 cores=4-31
/proc/495/task/502/stat : 502 (plingeling) R 490 490 464 0 -1 4202560 156819 1241 0 0 495566 106 0 1 25 0 29 0 652427899 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210992 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=495/tid=503] ppid=490 vsize=2426192 CPUtime=4955.22 cores=4-31
/proc/495/task/503/stat : 503 (plingeling) R 490 490 464 0 -1 4202560 153829 1241 0 0 495447 74 0 1 25 0 29 0 652427899 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=495/tid=504] ppid=490 vsize=2426192 CPUtime=4956.46 cores=4-31
/proc/495/task/504/stat : 504 (plingeling) R 490 490 464 0 -1 4202560 162387 1241 0 0 495547 98 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4281672 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=495/tid=505] ppid=490 vsize=2426192 CPUtime=4956.73 cores=4-31
/proc/495/task/505/stat : 505 (plingeling) R 490 490 464 0 -1 4202560 121619 1241 0 0 495568 104 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4234281 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=495/tid=506] ppid=490 vsize=2426192 CPUtime=4956.7 cores=4-31
/proc/495/task/506/stat : 506 (plingeling) R 490 490 464 0 -1 4202560 163816 1241 0 0 495550 119 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4281793 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=495/tid=507] ppid=490 vsize=2426192 CPUtime=4874.93 cores=4-31
/proc/495/task/507/stat : 507 (plingeling) R 490 490 464 0 -1 4202560 134845 1241 0 0 487385 107 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210875 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=495/tid=508] ppid=490 vsize=2426192 CPUtime=4956.71 cores=4-31
/proc/495/task/508/stat : 508 (plingeling) R 490 490 464 0 -1 4202560 162326 1241 0 0 495570 100 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4284120 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=495/tid=509] ppid=490 vsize=2426192 CPUtime=4955.74 cores=4-31
/proc/495/task/509/stat : 509 (plingeling) R 490 490 464 0 -1 4202560 185304 1241 0 0 495400 173 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=495/tid=510] ppid=490 vsize=2426192 CPUtime=4956.71 cores=4-31
/proc/495/task/510/stat : 510 (plingeling) R 490 490 464 0 -1 4202560 134264 1241 0 0 495590 80 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=495/tid=511] ppid=490 vsize=2426192 CPUtime=4954.47 cores=4-31
/proc/495/task/511/stat : 511 (plingeling) R 490 490 464 0 -1 4202560 154687 1241 0 0 495345 101 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=495/tid=512] ppid=490 vsize=2426192 CPUtime=4956.7 cores=4-31
/proc/495/task/512/stat : 512 (plingeling) R 490 490 464 0 -1 4202560 148249 1241 0 0 495581 88 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331311 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=495/tid=513] ppid=490 vsize=2426192 CPUtime=4956.73 cores=4-31
/proc/495/task/513/stat : 513 (plingeling) R 490 490 464 0 -1 4202560 101480 1241 0 0 495612 60 0 1 25 0 29 0 652427900 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=495/tid=514] ppid=490 vsize=2426192 CPUtime=4956.73 cores=4-31
/proc/495/task/514/stat : 514 (plingeling) R 490 490 464 0 -1 4202560 172170 1241 0 0 495552 120 0 1 25 0 29 0 652427901 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=495/tid=515] ppid=490 vsize=2426192 CPUtime=4956.65 cores=4-31
/proc/495/task/515/stat : 515 (plingeling) R 490 490 464 0 -1 4202560 160481 1241 0 0 495546 118 0 1 25 0 29 0 652427901 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=495/tid=516] ppid=490 vsize=2426192 CPUtime=4956.71 cores=4-31
/proc/495/task/516/stat : 516 (plingeling) R 490 490 464 0 -1 4202560 147956 1241 0 0 495588 82 0 1 25 0 29 0 652427901 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4281624 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=495/tid=517] ppid=490 vsize=2426192 CPUtime=4956.67 cores=4-31
/proc/495/task/517/stat : 517 (plingeling) R 490 490 464 0 -1 4202560 198427 1241 0 0 495484 182 0 1 25 0 29 0 652427901 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210740 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=495/tid=518] ppid=490 vsize=2426192 CPUtime=4956.67 cores=4-31
/proc/495/task/518/stat : 518 (plingeling) R 490 490 464 0 -1 4202560 160939 1241 0 0 495567 99 0 1 25 0 29 0 652427901 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4234276 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=495/tid=519] ppid=490 vsize=2426192 CPUtime=4956.63 cores=4-31
/proc/495/task/519/stat : 519 (plingeling) R 490 490 464 0 -1 4202560 154094 1241 0 0 495551 111 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=495/tid=520] ppid=490 vsize=2426192 CPUtime=4956.48 cores=4-31
/proc/495/task/520/stat : 520 (plingeling) R 490 490 464 0 -1 4202560 177444 1241 0 0 495510 137 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=495/tid=521] ppid=490 vsize=2426192 CPUtime=4956.73 cores=4-31
/proc/495/task/521/stat : 521 (plingeling) R 490 490 464 0 -1 4202560 86676 1241 0 0 495615 57 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4279717 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=495/tid=522] ppid=490 vsize=2426192 CPUtime=4956.68 cores=4-31
/proc/495/task/522/stat : 522 (plingeling) R 490 490 464 0 -1 4202560 142835 1241 0 0 495590 77 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=495/tid=523] ppid=490 vsize=2426192 CPUtime=4956.62 cores=4-31
/proc/495/task/523/stat : 523 (plingeling) R 490 490 464 0 -1 4202560 157999 1241 0 0 495564 97 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4283920 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=495/tid=524] ppid=490 vsize=2426192 CPUtime=4956.62 cores=4-31
/proc/495/task/524/stat : 524 (plingeling) R 490 490 464 0 -1 4202560 160132 1241 0 0 495578 83 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331122 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=495/tid=525] ppid=490 vsize=2426192 CPUtime=4956.73 cores=4-31
/proc/495/task/525/stat : 525 (plingeling) R 490 490 464 0 -1 4202560 178802 1241 0 0 495505 167 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=495/tid=526] ppid=490 vsize=2426192 CPUtime=4899.38 cores=4-31
/proc/495/task/526/stat : 526 (plingeling) R 490 490 464 0 -1 4202560 170844 1241 0 0 489801 136 0 1 25 0 29 0 652427902 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=495/tid=527] ppid=490 vsize=2426192 CPUtime=4954.13 cores=4-31
/proc/495/task/527/stat : 527 (plingeling) R 490 490 464 0 -1 4202560 145002 1241 0 0 495265 147 0 1 25 0 29 0 652427906 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=495/tid=528] ppid=490 vsize=2426192 CPUtime=4956.56 cores=4-31
/proc/495/task/528/stat : 528 (plingeling) R 490 490 464 0 -1 4202560 156048 1241 0 0 495578 77 0 1 25 0 29 0 652427906 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331076 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=495/tid=529] ppid=490 vsize=2426192 CPUtime=4956.57 cores=4-31
/proc/495/task/529/stat : 529 (plingeling) R 490 490 464 0 -1 4202560 140868 1241 0 0 495548 108 0 1 25 0 29 0 652427906 2484420608 504470 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331419 0 0 4096 17442 0 0 0 -1 27 0 0 0
Current children cumulated CPU time (s) 158490
Current children cumulated vsize (KiB) 5785812



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.6 s]
/proc/loadavg: 32.11 32.08 32.04 33/447 684
/proc/meminfo: memFree=252564696/264257624 swapFree=67111528/67111528
[pid=490] ppid=488 vsize=18324 CPUtime=0 cores=0-31
/proc/490/stat : 490 (ppfolio) S 488 490 464 0 -1 4202496 514 0 0 0 0 0 0 0 15 0 1 0 652427444 18763776 386 268435456000 4194304 4287625 140733669442736 18446744073709551615 258650975999 0 0 4096 65536 8967142950924451839 0 0 17 12 0 0 0
/proc/490/statm: 4581 386 337 23 0 91 0
[pid=491] ppid=490 vsize=2202284 CPUtime=5008.63 cores=0
/proc/491/stat : 491 (cryptominisat) R 490 490 464 0 -1 4202496 2293009 0 0 0 499822 1041 0 0 25 0 1 0 652427444 2255138816 93565 268435456000 4194304 4587354 140733483058304 18446744073709551615 258650652667 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/491/statm: 550571 93565 337 96 0 547133 0
[pid=492] ppid=490 vsize=545660 CPUtime=5008.64 cores=1
/proc/492/stat : 492 (clasp) R 490 490 464 0 -1 4202496 134760 0 0 0 500795 69 0 0 25 0 1 0 652427444 558755840 132754 268435456000 4194304 4785740 140734110014624 18446744073709551615 4552628 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/492/statm: 136415 132754 339 145 0 133460 0
[pid=493] ppid=490 vsize=464704 CPUtime=5008.65 cores=2
/proc/493/stat : 493 (TNM) R 490 490 464 0 -1 4202496 5138 0 0 0 500863 2 0 0 25 0 1 0 652427444 475856896 5035 268435456000 134512640 134534564 4287046928 18446744073709551615 134528661 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/493/statm: 116176 5035 103 6 0 114841 0
[pid=494] ppid=490 vsize=128648 CPUtime=5008.62 cores=3
/proc/494/stat : 494 (march_hi) R 490 490 464 0 -1 4202496 39560 0 0 0 500840 22 0 0 25 0 1 0 652427444 131735552 20949 268435456000 4194304 4331988 140736824348560 18446744073709551615 258650351201 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/494/statm: 32162 20949 183 34 0 30604 0
[pid=495] ppid=490 vsize=2415712 CPUtime=139968 cores=4-31
/proc/495/stat : 495 (plingeling) S 490 490 464 0 -1 4202496 4379276 1241 0 0 13993698 3093 0 1 19 0 29 0 652427444 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 24 0 0 0
/proc/495/statm: 603928 501850 159 40 0 602472 0
[pid=495/tid=502] ppid=490 vsize=2415712 CPUtime=5004.05 cores=4-31
/proc/495/task/502/stat : 502 (plingeling) R 490 490 464 0 -1 4202560 157044 1241 0 0 500297 107 0 1 25 0 29 0 652427899 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210704 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=495/tid=503] ppid=490 vsize=2415712 CPUtime=5002.52 cores=4-31
/proc/495/task/503/stat : 503 (plingeling) R 490 490 464 0 -1 4202560 155078 1241 0 0 500174 77 0 1 25 0 29 0 652427899 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=495/tid=504] ppid=490 vsize=2415712 CPUtime=5003.77 cores=4-31
/proc/495/task/504/stat : 504 (plingeling) R 490 490 464 0 -1 4202560 168531 1241 0 0 500266 110 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331074 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=495/tid=505] ppid=490 vsize=2415712 CPUtime=5004.04 cores=4-31
/proc/495/task/505/stat : 505 (plingeling) R 490 490 464 0 -1 4202560 124691 1241 0 0 500294 109 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331045 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=495/tid=506] ppid=490 vsize=2415712 CPUtime=5004.02 cores=4-31
/proc/495/task/506/stat : 506 (plingeling) R 490 490 464 0 -1 4202560 163816 1241 0 0 500281 120 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4279461 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=495/tid=507] ppid=490 vsize=2415712 CPUtime=4922.25 cores=4-31
/proc/495/task/507/stat : 507 (plingeling) R 490 490 464 0 -1 4202560 134845 1241 0 0 492117 107 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=495/tid=508] ppid=490 vsize=2415712 CPUtime=5004.02 cores=4-31
/proc/495/task/508/stat : 508 (plingeling) R 490 490 464 0 -1 4202560 162326 1241 0 0 500301 100 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=495/tid=509] ppid=490 vsize=2415712 CPUtime=5003.05 cores=4-31
/proc/495/task/509/stat : 509 (plingeling) R 490 490 464 0 -1 4202560 185304 1241 0 0 500131 173 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=495/tid=510] ppid=490 vsize=2415712 CPUtime=5004.03 cores=4-31
/proc/495/task/510/stat : 510 (plingeling) R 490 490 464 0 -1 4202560 138074 1241 0 0 500313 89 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=495/tid=511] ppid=490 vsize=2415712 CPUtime=5001.79 cores=4-31
/proc/495/task/511/stat : 511 (plingeling) R 490 490 464 0 -1 4202560 156223 1241 0 0 500076 102 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=495/tid=512] ppid=490 vsize=2415712 CPUtime=5004.02 cores=4-31
/proc/495/task/512/stat : 512 (plingeling) R 490 490 464 0 -1 4202560 151065 1241 0 0 500312 89 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=495/tid=513] ppid=490 vsize=2415712 CPUtime=5004.04 cores=4-31
/proc/495/task/513/stat : 513 (plingeling) R 490 490 464 0 -1 4202560 103016 1241 0 0 500343 60 0 1 25 0 29 0 652427900 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210704 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=495/tid=514] ppid=490 vsize=2415712 CPUtime=5004.04 cores=4-31
/proc/495/task/514/stat : 514 (plingeling) R 490 490 464 0 -1 4202560 173194 1241 0 0 500283 120 0 1 25 0 29 0 652427901 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331035 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=495/tid=515] ppid=490 vsize=2415712 CPUtime=5003.97 cores=4-31
/proc/495/task/515/stat : 515 (plingeling) R 490 490 464 0 -1 4202560 160481 1241 0 0 500277 119 0 1 25 0 29 0 652427901 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=495/tid=516] ppid=490 vsize=2415712 CPUtime=5004.02 cores=4-31
/proc/495/task/516/stat : 516 (plingeling) R 490 490 464 0 -1 4202560 148182 1241 0 0 500319 82 0 1 25 0 29 0 652427901 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331305 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=495/tid=517] ppid=490 vsize=2415712 CPUtime=5003.98 cores=4-31
/proc/495/task/517/stat : 517 (plingeling) R 490 490 464 0 -1 4202560 198427 1241 0 0 500215 182 0 1 25 0 29 0 652427901 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4279789 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=495/tid=518] ppid=490 vsize=2415712 CPUtime=5003.97 cores=4-31
/proc/495/task/518/stat : 518 (plingeling) R 490 490 464 0 -1 4202560 165547 1241 0 0 500290 106 0 1 25 0 29 0 652427901 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=495/tid=519] ppid=490 vsize=2415712 CPUtime=5003.95 cores=4-31
/proc/495/task/519/stat : 519 (plingeling) R 490 490 464 0 -1 4202560 154094 1241 0 0 500283 111 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=495/tid=520] ppid=490 vsize=2415712 CPUtime=5003.79 cores=4-31
/proc/495/task/520/stat : 520 (plingeling) R 490 490 464 0 -1 4202560 177444 1241 0 0 500241 137 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=495/tid=521] ppid=490 vsize=2415712 CPUtime=5004.04 cores=4-31
/proc/495/task/521/stat : 521 (plingeling) R 490 490 464 0 -1 4202560 89748 1241 0 0 500343 60 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4332327 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=495/tid=522] ppid=490 vsize=2415712 CPUtime=5003.99 cores=4-31
/proc/495/task/522/stat : 522 (plingeling) R 490 490 464 0 -1 4202560 143860 1241 0 0 500321 77 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=495/tid=523] ppid=490 vsize=2415712 CPUtime=5003.94 cores=4-31
/proc/495/task/523/stat : 523 (plingeling) R 490 490 464 0 -1 4202560 157999 1241 0 0 500296 97 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4233969 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=495/tid=524] ppid=490 vsize=2415712 CPUtime=5003.93 cores=4-31
/proc/495/task/524/stat : 524 (plingeling) R 490 490 464 0 -1 4202560 160132 1241 0 0 500309 83 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=495/tid=525] ppid=490 vsize=2415712 CPUtime=5004.04 cores=4-31
/proc/495/task/525/stat : 525 (plingeling) R 490 490 464 0 -1 4202560 179314 1241 0 0 500236 167 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4284205 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=495/tid=526] ppid=490 vsize=2415712 CPUtime=4945.32 cores=4-31
/proc/495/task/526/stat : 526 (plingeling) R 490 490 464 0 -1 4202560 173916 1241 0 0 494388 143 0 1 25 0 29 0 652427902 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4330841 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=495/tid=527] ppid=490 vsize=2415712 CPUtime=5001.41 cores=4-31
/proc/495/task/527/stat : 527 (plingeling) R 490 490 464 0 -1 4202560 145002 1241 0 0 499993 147 0 1 25 0 29 0 652427906 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=495/tid=528] ppid=490 vsize=2415712 CPUtime=5003.87 cores=4-31
/proc/495/task/528/stat : 528 (plingeling) R 490 490 464 0 -1 4202560 156048 1241 0 0 500309 77 0 1 25 0 29 0 652427906 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4281627 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=495/tid=529] ppid=490 vsize=2415712 CPUtime=5003.88 cores=4-31
/proc/495/task/529/stat : 529 (plingeling) R 490 490 464 0 -1 4202560 144964 1241 0 0 500274 113 0 1 25 0 29 0 652427906 2473689088 501850 268435456000 4194304 4356188 140735122061728 18446744073709551615 4279711 0 0 4096 17442 0 0 0 -1 27 0 0 0
Current children cumulated CPU time (s) 160002
Current children cumulated vsize (KiB) 5775332

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 490 and gives
#  childrusage.ru_utime.tv_sec=10017
#  childrusage.ru_utime.tv_usec=79172
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=256960
# CPU time returned by wait4() is 10017.3
# while last known CPU time is 160002
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5009.64
CPU time (s): 160002
CPU user time (s): 159960
CPU system time (s): 42.28
CPU usage (%): 3193.89
Max. virtual memory (cumulated for all children) (KiB): 7707500

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 10017.1
system time used= 0.25696
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 45213
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 499
involuntary context switches= 15058

runsolver used 28.4587 second user time and 114.32 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-14 23:43:30
IDJOB=3347637
IDBENCH=84223
IDSOLVER=1808
FILE ID=node200/3347637-1305409410
RUNJOBID= node200-1305409410-478
PBS_JOBID= 13285495
Free space on /tmp= 62048 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT03/industrial/li/li-exam/li-exam-61.shuffled-as.sat03-366.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347637-1305409410/watcher-3347637-1305409410 -o /tmp/evaluation-result-3347637-1305409410/solver-3347637-1305409410 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347637-1305409410.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 3be662940f394fa876137a905247d755
RANDOM SEED=1727971234

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255578108 kB
Buffers:        438348 kB
Cached:        7837536 kB
SwapCached:          0 kB
Active:        7388304 kB
Inactive:       956540 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255578108 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           12428 kB
Writeback:           0 kB
AnonPages:       70352 kB
Mapped:          14056 kB
Slab:           250228 kB
PageTables:       5308 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   186616 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62040 MiB
End job on node200 at 2011-05-15 01:07:02