Trace number 3347537

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 156229 5100.04

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-top24.cnf
MD5SUM1f485d737caaaca49e2c467c83cbdcec
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark4562.96
Satisfiable
(Un)Satisfiability was proved
Number of variables65983
Number of clauses209942
Sum of the clauses size513194
Maximum clause length2578
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 2127592
Number of clauses of size 380970
Number of clauses of size 4299
Number of clauses of size 522
Number of clauses of size over 51058

Solver Data

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 32 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

Verifier Data

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3347537-1305298746/watcher-3347537-1305298746 -o /tmp/evaluation-result-3347537-1305298746/solver-3347537-1305298746 -C 160000 -W 5100 -M 266338304 ppfolio HOME/instance-3347537-1305298746.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 28.67 30.58 30.86 1/413 25611
/proc/meminfo: memFree=255623572/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 511 0 0 0 0 0 0 0 22 0 1 0 641361096 18763776 385 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9185487131064139775 0 0 17 4 0 0 0
/proc/25611/statm: 4581 385 337 23 0 91 0

[startup+0.11589 s]
/proc/loadavg: 28.67 30.58 30.86 1/413 25611
/proc/meminfo: memFree=255623572/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 22 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197619763380224 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 18324

[startup+0.200875 s]
/proc/loadavg: 28.67 30.58 30.86 1/413 25611
/proc/meminfo: memFree=255623572/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 22 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197619763380224 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 18324

[startup+0.300876 s]
/proc/loadavg: 28.67 30.58 30.86 1/413 25611
/proc/meminfo: memFree=255623572/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 20 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197971950698496 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 18324

[startup+0.700904 s]
/proc/loadavg: 28.67 30.58 30.86 1/413 25611
/proc/meminfo: memFree=255623572/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 20 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197971950698496 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 18324

[startup+1.5009 s]
/proc/loadavg: 28.67 30.58 30.86 5/418 25622
/proc/meminfo: memFree=255428084/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 20 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197971950698496 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
[pid=25612] ppid=25611 vsize=47080 CPUtime=1.48 cores=0
/proc/25612/stat : 25612 (cryptominisat) R 25611 25611 25371 0 -1 4202496 6526 0 0 0 146 2 0 0 20 0 1 0 641361097 48209920 6252 268435456000 4194304 4587354 140736800319312 18446744073709551615 4237674 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25612/statm: 11770 6252 303 96 0 8332 0
[pid=25613] ppid=25611 vsize=47844 CPUtime=1.48 cores=1
/proc/25613/stat : 25613 (clasp) R 25611 25611 25371 0 -1 4202496 11574 0 0 0 144 4 0 0 20 0 1 0 641361097 48992256 9036 268435456000 4194304 4785740 140736951626560 18446744073709551615 4550310 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25613/statm: 11961 9036 338 145 0 9006 0
[pid=25615] ppid=25611 vsize=38716 CPUtime=1.48 cores=3
/proc/25615/stat : 25615 (march_hi) R 25611 25611 25371 0 -1 4202496 48760 0 0 0 135 13 0 0 20 0 1 0 641361097 39645184 5967 268435456000 4194304 4331988 140734492818064 18446744073709551615 258650653376 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25615/statm: 9679 5967 148 34 0 8121 0
[pid=25616] ppid=25611 vsize=131640 CPUtime=1.48 cores=4-31
/proc/25616/stat : 25616 (plingeling) R 25611 25611 25371 0 -1 4202496 31684 1240 0 0 139 8 0 1 21 0 1 0 641361097 134799360 31569 268435456000 4194304 4356188 140736902408352 18446744073709551615 4212997 0 0 4096 17442 0 0 0 17 24 0 0 0
/proc/25616/statm: 32910 31569 129 40 0 31454 0
Current children cumulated CPU time (s) 5.97
Current children cumulated vsize (KiB) 283604

[startup+3.10095 s]
/proc/loadavg: 26.69 30.14 30.71 5/418 25622
/proc/meminfo: memFree=255235016/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 15 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197971950698496 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
[pid=25612] ppid=25611 vsize=59228 CPUtime=3.08 cores=0
/proc/25612/stat : 25612 (cryptominisat) R 25611 25611 25371 0 -1 4202496 9507 0 0 0 305 3 0 0 25 0 1 0 641361097 60649472 9233 268435456000 4194304 4587354 140736800319312 18446744073709551615 4248764 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25612/statm: 14807 9233 312 96 0 11369 0
[pid=25613] ppid=25611 vsize=47844 CPUtime=3.08 cores=1
/proc/25613/stat : 25613 (clasp) R 25611 25611 25371 0 -1 4202496 11584 0 0 0 304 4 0 0 25 0 1 0 641361097 48992256 9046 268435456000 4194304 4785740 140736951626560 18446744073709551615 4553057 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25613/statm: 11961 9046 338 145 0 9006 0
[pid=25615] ppid=25611 vsize=147036 CPUtime=3.08 cores=3
/proc/25615/stat : 25615 (march_hi) R 25611 25611 25371 0 -1 4202496 79764 0 0 0 290 18 0 0 25 0 1 0 641361097 150564864 18527 268435456000 4194304 4331988 140734492818064 18446744073709551615 4237226 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25615/statm: 36759 18527 180 34 0 35201 0
[pid=25616] ppid=25611 vsize=241940 CPUtime=3.09 cores=4-31
/proc/25616/stat : 25616 (plingeling) R 25611 25611 25371 0 -1 4202496 59243 1240 0 0 293 15 0 1 25 0 1 0 641361097 247746560 59128 268435456000 4194304 4356188 140736902408352 18446744073709551615 4277592 0 0 4096 17442 0 0 0 17 24 0 0 0
/proc/25616/statm: 60485 59128 129 40 0 59029 0
Current children cumulated CPU time (s) 12.38
Current children cumulated vsize (KiB) 514372

[startup+6.30203 s]
/proc/loadavg: 26.69 30.14 30.71 32/446 25650
/proc/meminfo: memFree=255047036/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 15 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197976245633290 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
[pid=25612] ppid=25611 vsize=120328 CPUtime=6.29 cores=0
/proc/25612/stat : 25612 (cryptominisat) R 25611 25611 25371 0 -1 4202496 19337 0 0 0 623 6 0 0 25 0 1 0 641361097 123215872 16158 268435456000 4194304 4587354 140736800319312 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25612/statm: 30082 16158 328 96 0 26644 0
[pid=25613] ppid=25611 vsize=49508 CPUtime=6.29 cores=1
/proc/25613/stat : 25613 (clasp) R 25611 25611 25371 0 -1 4202496 12015 0 0 0 624 5 0 0 25 0 1 0 641361097 50696192 9477 268435456000 4194304 4785740 140736951626560 18446744073709551615 4553032 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25613/statm: 12377 9477 338 145 0 9422 0
[pid=25615] ppid=25611 vsize=155524 CPUtime=6.28 cores=3
/proc/25615/stat : 25615 (march_hi) R 25611 25611 25371 0 -1 4202496 81711 0 0 0 609 19 0 0 25 0 1 0 641361097 159256576 20468 268435456000 4194304 4331988 140734492818064 18446744073709551615 4265456 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25615/statm: 38881 20468 174 34 0 37323 0
[pid=25616] ppid=25611 vsize=941204 CPUtime=85.25 cores=4-31
/proc/25616/stat : 25616 (plingeling) S 25611 25611 25371 0 -1 4202496 217066 1240 0 0 8410 114 0 1 18 0 29 0 641361097 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/25616/statm: 235301 113876 159 40 0 233845 0
[pid=25616/tid=25623] ppid=25611 vsize=941204 CPUtime=3.04 cores=4-31
/proc/25616/task/25623/stat : 25623 (plingeling) R 25611 25611 25371 0 -1 4202560 5048 1240 0 0 300 3 0 1 25 0 29 0 641361418 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4217742 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=25616/tid=25624] ppid=25611 vsize=941204 CPUtime=2.85 cores=4-31
/proc/25616/task/25624/stat : 25624 (plingeling) R 25611 25611 25371 0 -1 4202560 4881 1240 0 0 284 0 0 1 20 0 29 0 641361418 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4232894 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=25616/tid=25625] ppid=25611 vsize=941204 CPUtime=3.01 cores=4-31
/proc/25616/task/25625/stat : 25625 (plingeling) R 25611 25611 25371 0 -1 4202560 7706 1240 0 0 296 4 0 1 23 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4233682 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25616/tid=25626] ppid=25611 vsize=941204 CPUtime=2.89 cores=4-31
/proc/25616/task/25626/stat : 25626 (plingeling) R 25611 25611 25371 0 -1 4202560 5060 1240 0 0 284 4 0 1 20 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=25616/tid=25627] ppid=25611 vsize=941204 CPUtime=2.94 cores=4-31
/proc/25616/task/25627/stat : 25627 (plingeling) R 25611 25611 25371 0 -1 4202560 2678 1240 0 0 290 3 0 1 25 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4292813 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=25616/tid=25628] ppid=25611 vsize=941204 CPUtime=2.93 cores=4-31
/proc/25616/task/25628/stat : 25628 (plingeling) R 25611 25611 25371 0 -1 4202560 5585 1240 0 0 287 5 0 1 20 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4300648 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=25616/tid=25629] ppid=25611 vsize=941204 CPUtime=2.96 cores=4-31
/proc/25616/task/25629/stat : 25629 (plingeling) R 25611 25611 25371 0 -1 4202560 4364 1240 0 0 291 4 0 1 23 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4249298 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25616/tid=25630] ppid=25611 vsize=941204 CPUtime=2.9 cores=4-31
/proc/25616/task/25630/stat : 25630 (plingeling) R 25611 25611 25371 0 -1 4202560 3863 1240 0 0 289 0 0 1 22 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4236344 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=25616/tid=25631] ppid=25611 vsize=941204 CPUtime=3.05 cores=4-31
/proc/25616/task/25631/stat : 25631 (plingeling) R 25611 25611 25371 0 -1 4202560 5576 1240 0 0 303 1 0 1 20 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4303915 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=25616/tid=25632] ppid=25611 vsize=941204 CPUtime=2.93 cores=4-31
/proc/25616/task/25632/stat : 25632 (plingeling) R 25611 25611 25371 0 -1 4202560 4000 1240 0 0 290 2 0 1 22 0 29 0 641361419 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4303196 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=25616/tid=25633] ppid=25611 vsize=941204 CPUtime=2.94 cores=4-31
/proc/25616/task/25633/stat : 25633 (plingeling) R 25611 25611 25371 0 -1 4202560 5426 1240 0 0 288 5 0 1 25 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4216153 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=25616/tid=25634] ppid=25611 vsize=941204 CPUtime=2.88 cores=4-31
/proc/25616/task/25634/stat : 25634 (plingeling) R 25611 25611 25371 0 -1 4202560 4273 1240 0 0 285 2 0 1 25 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4211045 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=25616/tid=25635] ppid=25611 vsize=941204 CPUtime=2.79 cores=4-31
/proc/25616/task/25635/stat : 25635 (plingeling) R 25611 25611 25371 0 -1 4202560 3580 1240 0 0 274 4 0 1 25 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4304852 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25616/tid=25636] ppid=25611 vsize=941204 CPUtime=2.98 cores=4-31
/proc/25616/task/25636/stat : 25636 (plingeling) R 25611 25611 25371 0 -1 4202560 5351 1240 0 0 293 4 0 1 20 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4217720 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=25616/tid=25637] ppid=25611 vsize=941204 CPUtime=2.97 cores=4-31
/proc/25616/task/25637/stat : 25637 (plingeling) R 25611 25611 25371 0 -1 4202560 5801 1240 0 0 292 4 0 1 20 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4301721 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=25616/tid=25638] ppid=25611 vsize=941204 CPUtime=2.98 cores=4-31
/proc/25616/task/25638/stat : 25638 (plingeling) R 25611 25611 25371 0 -1 4202560 5150 1240 0 0 293 4 0 1 22 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4292802 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=25616/tid=25639] ppid=25611 vsize=941204 CPUtime=2.92 cores=4-31
/proc/25616/task/25639/stat : 25639 (plingeling) R 25611 25611 25371 0 -1 4202560 4519 1240 0 0 288 3 0 1 21 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4301736 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=25616/tid=25640] ppid=25611 vsize=941204 CPUtime=2.55 cores=4-31
/proc/25616/task/25640/stat : 25640 (plingeling) R 25611 25611 25371 0 -1 4202560 5212 1240 0 0 252 2 0 1 25 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4215047 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=25616/tid=25641] ppid=25611 vsize=941204 CPUtime=3.02 cores=4-31
/proc/25616/task/25641/stat : 25641 (plingeling) R 25611 25611 25371 0 -1 4202560 4021 1240 0 0 301 0 0 1 25 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4217733 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=25616/tid=25642] ppid=25611 vsize=941204 CPUtime=2.97 cores=4-31
/proc/25616/task/25642/stat : 25642 (plingeling) R 25611 25611 25371 0 -1 4202560 3578 1240 0 0 293 3 0 1 25 0 29 0 641361420 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4279656 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=25616/tid=25643] ppid=25611 vsize=941204 CPUtime=2.77 cores=4-31
/proc/25616/task/25643/stat : 25643 (plingeling) R 25611 25611 25371 0 -1 4202560 3620 1240 0 0 275 1 0 1 25 0 29 0 641361421 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4231984 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=25616/tid=25644] ppid=25611 vsize=941204 CPUtime=2.97 cores=4-31
/proc/25616/task/25644/stat : 25644 (plingeling) R 25611 25611 25371 0 -1 4202560 5303 1240 0 0 293 3 0 1 22 0 29 0 641361421 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4232593 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=25616/tid=25645] ppid=25611 vsize=941204 CPUtime=3.01 cores=4-31
/proc/25616/task/25645/stat : 25645 (plingeling) R 25611 25611 25371 0 -1 4202560 4812 1240 0 0 297 3 0 1 20 0 29 0 641361421 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4218160 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=25616/tid=25646] ppid=25611 vsize=941204 CPUtime=3.02 cores=4-31
/proc/25616/task/25646/stat : 25646 (plingeling) R 25611 25611 25371 0 -1 4202560 5182 1240 0 0 298 3 0 1 24 0 29 0 641361421 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4217730 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=25616/tid=25647] ppid=25611 vsize=941204 CPUtime=2.82 cores=4-31
/proc/25616/task/25647/stat : 25647 (plingeling) R 25611 25611 25371 0 -1 4202560 4278 1240 0 0 278 3 0 1 20 0 29 0 641361421 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4232951 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=25616/tid=25648] ppid=25611 vsize=941204 CPUtime=2.98 cores=4-31
/proc/25616/task/25648/stat : 25648 (plingeling) R 25611 25611 25371 0 -1 4202560 4653 1240 0 0 294 3 0 1 25 0 29 0 641361421 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4217742 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=25616/tid=25649] ppid=25611 vsize=941204 CPUtime=2.96 cores=4-31
/proc/25616/task/25649/stat : 25649 (plingeling) R 25611 25611 25371 0 -1 4202560 3938 1240 0 0 292 3 0 1 23 0 29 0 641361421 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4293958 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=25616/tid=25650] ppid=25611 vsize=941204 CPUtime=3.02 cores=4-31
/proc/25616/task/25650/stat : 25650 (plingeling) R 25611 25611 25371 0 -1 4202560 4524 1240 0 0 299 2 0 1 25 0 29 0 641361422 963792896 113876 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331086 0 0 4096 17442 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 104.16
Current children cumulated vsize (KiB) 1284888

[startup+12.7022 s]
/proc/loadavg: 27.04 30.16 30.72 32/446 25650
/proc/meminfo: memFree=254847848/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.05 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 0 2 3 15 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197976245633290 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
[pid=25612] ppid=25611 vsize=120328 CPUtime=12.68 cores=0
/proc/25612/stat : 25612 (cryptominisat) R 25611 25611 25371 0 -1 4202496 20003 0 0 0 1262 6 0 0 25 0 1 0 641361097 123215872 16807 268435456000 4194304 4587354 140736800319312 18446744073709551615 4238101 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25612/statm: 30082 16807 334 96 0 26644 0
[pid=25613] ppid=25611 vsize=57888 CPUtime=12.68 cores=1
/proc/25613/stat : 25613 (clasp) R 25611 25611 25371 0 -1 4202496 14096 0 0 0 1263 5 0 0 25 0 1 0 641361097 59277312 11558 268435456000 4194304 4785740 140736951626560 18446744073709551615 4287050 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25613/statm: 14472 11558 338 145 0 11517 0
[pid=25615] ppid=25611 vsize=157160 CPUtime=12.68 cores=3
/proc/25615/stat : 25615 (march_hi) R 25611 25611 25371 0 -1 4202496 82184 0 0 0 1249 19 0 0 25 0 1 0 641361097 160931840 20941 268435456000 4194304 4331988 140734492818064 18446744073709551615 4265452 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25615/statm: 39290 20941 174 34 0 37732 0
[pid=25616] ppid=25611 vsize=1052348 CPUtime=264.23 cores=4-31
/proc/25616/stat : 25616 (plingeling) S 25611 25611 25371 0 -1 4202496 246437 1240 0 0 26302 120 0 1 18 0 29 0 641361097 1077604352 141407 268435456000 4194304 4356188 140736902408352 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0

################
# More data... #
################

/proc/meminfo: memFree=252911212/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.06 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 1 2 3 15 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197976245633290 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
[pid=25612] ppid=25611 vsize=1453144 CPUtime=5099.04 cores=0
/proc/25612/stat : 25612 (cryptominisat) R 25611 25611 25371 0 -1 4202496 307207 0 0 0 509683 221 0 0 25 0 1 0 641361097 1488019456 39709 268435456000 4194304 4587354 140736800319312 18446744073709551615 4238578 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25612/statm: 363286 39709 334 96 0 359848 0
[pid=25613] ppid=25611 vsize=318312 CPUtime=5099.05 cores=1
/proc/25613/stat : 25613 (clasp) R 25611 25611 25371 0 -1 4202496 77081 0 0 0 509847 58 0 0 25 0 1 0 641361097 325951488 74543 268435456000 4194304 4785740 140736951626560 18446744073709551615 4550406 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25613/statm: 79578 74543 338 145 0 76623 0
[pid=25615] ppid=25611 vsize=173176 CPUtime=5099.04 cores=3
/proc/25615/stat : 25615 (march_hi) R 25611 25611 25371 0 -1 4202496 85708 0 0 0 509881 23 0 0 25 0 1 0 641361097 177332224 24465 268435456000 4194304 4331988 140734492818064 18446744073709551615 4263331 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25615/statm: 43294 24465 174 34 0 41736 0
[pid=25616] ppid=25611 vsize=2635300 CPUtime=140932 cores=4-31
/proc/25616/stat : 25616 (plingeling) S 25611 25611 25371 0 -1 4202496 4732968 1240 0 0 14090323 2897 0 1 18 0 29 0 641361097 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/25616/statm: 658825 538007 159 40 0 657369 0
[pid=25616/tid=25623] ppid=25611 vsize=2635300 CPUtime=5012.34 cores=4-31
/proc/25616/task/25623/stat : 25623 (plingeling) R 25611 25611 25371 0 -1 4202560 136943 1240 0 0 501165 68 0 1 25 0 29 0 641361418 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25616/tid=25624] ppid=25611 vsize=2635300 CPUtime=5023.24 cores=4-31
/proc/25616/task/25624/stat : 25624 (plingeling) R 25611 25611 25371 0 -1 4202560 164104 1240 0 0 502273 50 0 1 25 0 29 0 641361418 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=25616/tid=25625] ppid=25611 vsize=2635300 CPUtime=5095.53 cores=4-31
/proc/25616/task/25625/stat : 25625 (plingeling) R 25611 25611 25371 0 -1 4202560 183311 1240 0 0 509474 78 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25616/tid=25626] ppid=25611 vsize=2635300 CPUtime=5026.35 cores=4-31
/proc/25616/task/25626/stat : 25626 (plingeling) R 25611 25611 25371 0 -1 4202560 57855 1240 0 0 502589 45 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=25616/tid=25627] ppid=25611 vsize=2635300 CPUtime=5068.3 cores=4-31
/proc/25616/task/25627/stat : 25627 (plingeling) R 25611 25611 25371 0 -1 4202560 136237 1240 0 0 506759 70 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331401 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=25616/tid=25628] ppid=25611 vsize=2635300 CPUtime=5093.18 cores=4-31
/proc/25616/task/25628/stat : 25628 (plingeling) R 25611 25611 25371 0 -1 4202560 230431 1240 0 0 509155 162 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=25616/tid=25629] ppid=25611 vsize=2635300 CPUtime=5095.73 cores=4-31
/proc/25616/task/25629/stat : 25629 (plingeling) R 25611 25611 25371 0 -1 4202560 153983 1240 0 0 509495 77 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25616/tid=25630] ppid=25611 vsize=2635300 CPUtime=5095.67 cores=4-31
/proc/25616/task/25630/stat : 25630 (plingeling) R 25611 25611 25371 0 -1 4202560 263817 1240 0 0 509398 168 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=25616/tid=25631] ppid=25611 vsize=2635300 CPUtime=5061.92 cores=4-31
/proc/25616/task/25631/stat : 25631 (plingeling) R 25611 25611 25371 0 -1 4202560 164147 1240 0 0 506099 92 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=25616/tid=25632] ppid=25611 vsize=2635300 CPUtime=5065.66 cores=4-31
/proc/25616/task/25632/stat : 25632 (plingeling) R 25611 25611 25371 0 -1 4202560 188510 1240 0 0 506472 93 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25616/tid=25633] ppid=25611 vsize=2635300 CPUtime=5007.86 cores=4-31
/proc/25616/task/25633/stat : 25633 (plingeling) R 25611 25611 25371 0 -1 4202560 153508 1240 0 0 500698 87 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=25616/tid=25634] ppid=25611 vsize=2635300 CPUtime=5074.96 cores=4-31
/proc/25616/task/25634/stat : 25634 (plingeling) R 25611 25611 25371 0 -1 4202560 44316 1240 0 0 507464 31 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=25616/tid=25635] ppid=25611 vsize=2635300 CPUtime=4909.83 cores=4-31
/proc/25616/task/25635/stat : 25635 (plingeling) R 25611 25611 25371 0 -1 4202560 120199 1240 0 0 490895 87 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4281568 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=25616/tid=25636] ppid=25611 vsize=2635300 CPUtime=5089.19 cores=4-31
/proc/25616/task/25636/stat : 25636 (plingeling) R 25611 25611 25371 0 -1 4202560 150213 1240 0 0 508845 73 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4332109 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=25616/tid=25637] ppid=25611 vsize=2635300 CPUtime=4969.53 cores=4-31
/proc/25616/task/25637/stat : 25637 (plingeling) R 25611 25611 25371 0 -1 4202560 167004 1240 0 0 496809 143 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331345 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=25616/tid=25638] ppid=25611 vsize=2635300 CPUtime=5004.49 cores=4-31
/proc/25616/task/25638/stat : 25638 (plingeling) R 25611 25611 25371 0 -1 4202560 257222 1240 0 0 500229 219 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=25616/tid=25639] ppid=25611 vsize=2635300 CPUtime=4894.96 cores=4-31
/proc/25616/task/25639/stat : 25639 (plingeling) R 25611 25611 25371 0 -1 4202560 144819 1240 0 0 489422 73 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=25616/tid=25640] ppid=25611 vsize=2635300 CPUtime=4944.09 cores=4-31
/proc/25616/task/25640/stat : 25640 (plingeling) R 25611 25611 25371 0 -1 4202560 192490 1240 0 0 494274 134 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331456 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=25616/tid=25641] ppid=25611 vsize=2635300 CPUtime=4947.89 cores=4-31
/proc/25616/task/25641/stat : 25641 (plingeling) R 25611 25611 25371 0 -1 4202560 234734 1240 0 0 494666 122 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=25616/tid=25642] ppid=25611 vsize=2635300 CPUtime=5043.25 cores=4-31
/proc/25616/task/25642/stat : 25642 (plingeling) R 25611 25611 25371 0 -1 4202560 24324 1240 0 0 504299 25 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=25616/tid=25643] ppid=25611 vsize=2635300 CPUtime=4913.59 cores=4-31
/proc/25616/task/25643/stat : 25643 (plingeling) R 25611 25611 25371 0 -1 4202560 145402 1240 0 0 491288 70 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=25616/tid=25644] ppid=25611 vsize=2635300 CPUtime=5011.41 cores=4-31
/proc/25616/task/25644/stat : 25644 (plingeling) R 25611 25611 25371 0 -1 4202560 181681 1240 0 0 501052 88 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=25616/tid=25645] ppid=25611 vsize=2635300 CPUtime=5074.35 cores=4-31
/proc/25616/task/25645/stat : 25645 (plingeling) R 25611 25611 25371 0 -1 4202560 194820 1240 0 0 507301 133 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331007 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=25616/tid=25646] ppid=25611 vsize=2635300 CPUtime=5095.6 cores=4-31
/proc/25616/task/25646/stat : 25646 (plingeling) R 25611 25611 25371 0 -1 4202560 273560 1240 0 0 509298 261 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=25616/tid=25647] ppid=25611 vsize=2635300 CPUtime=5024.27 cores=4-31
/proc/25616/task/25647/stat : 25647 (plingeling) R 25611 25611 25371 0 -1 4202560 213068 1240 0 0 502272 154 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330995 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=25616/tid=25648] ppid=25611 vsize=2635300 CPUtime=5095.71 cores=4-31
/proc/25616/task/25648/stat : 25648 (plingeling) R 25611 25611 25371 0 -1 4202560 204800 1240 0 0 509495 75 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4279586 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=25616/tid=25649] ppid=25611 vsize=2635300 CPUtime=5095.72 cores=4-31
/proc/25616/task/25649/stat : 25649 (plingeling) R 25611 25611 25371 0 -1 4202560 229177 1240 0 0 509413 158 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=25616/tid=25650] ppid=25611 vsize=2635300 CPUtime=5094.39 cores=4-31
/proc/25616/task/25650/stat : 25650 (plingeling) R 25611 25611 25371 0 -1 4202560 37209 1240 0 0 509413 25 0 1 25 0 29 0 641361422 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 156229
Current children cumulated vsize (KiB) 4598256

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5100 s]
/proc/loadavg: 31.06 31.08 31.05 32/446 25809
/proc/meminfo: memFree=252911212/264257624 swapFree=67111528/67111528
[pid=25611] ppid=25609 vsize=18324 CPUtime=0.06 cores=0-31
/proc/25611/stat : 25611 (ppfolio) S 25609 25611 25371 0 -1 4202496 512 1195 0 0 0 1 2 3 15 0 1 0 641361096 18763776 386 268435456000 4194304 4287625 140735906147248 18446744073709551615 258650975999 0 0 4096 65536 9224197976245633290 0 0 17 4 0 0 0
/proc/25611/statm: 4581 386 338 23 0 91 0
[pid=25612] ppid=25611 vsize=1453144 CPUtime=5099.04 cores=0
/proc/25612/stat : 25612 (cryptominisat) R 25611 25611 25371 0 -1 4202496 307207 0 0 0 509683 221 0 0 25 0 1 0 641361097 1488019456 39709 268435456000 4194304 4587354 140736800319312 18446744073709551615 4238578 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/25612/statm: 363286 39709 334 96 0 359848 0
[pid=25613] ppid=25611 vsize=318312 CPUtime=5099.05 cores=1
/proc/25613/stat : 25613 (clasp) R 25611 25611 25371 0 -1 4202496 77081 0 0 0 509847 58 0 0 25 0 1 0 641361097 325951488 74543 268435456000 4194304 4785740 140736951626560 18446744073709551615 4550406 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/25613/statm: 79578 74543 338 145 0 76623 0
[pid=25615] ppid=25611 vsize=173176 CPUtime=5099.04 cores=3
/proc/25615/stat : 25615 (march_hi) R 25611 25611 25371 0 -1 4202496 85708 0 0 0 509881 23 0 0 25 0 1 0 641361097 177332224 24465 268435456000 4194304 4331988 140734492818064 18446744073709551615 4263331 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/25615/statm: 43294 24465 174 34 0 41736 0
[pid=25616] ppid=25611 vsize=2635300 CPUtime=140932 cores=4-31
/proc/25616/stat : 25616 (plingeling) S 25611 25611 25371 0 -1 4202496 4732968 1240 0 0 14090323 2897 0 1 18 0 29 0 641361097 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 12 0 0 0
/proc/25616/statm: 658825 538007 159 40 0 657369 0
[pid=25616/tid=25623] ppid=25611 vsize=2635300 CPUtime=5012.34 cores=4-31
/proc/25616/task/25623/stat : 25623 (plingeling) R 25611 25611 25371 0 -1 4202560 136943 1240 0 0 501165 68 0 1 25 0 29 0 641361418 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=25616/tid=25624] ppid=25611 vsize=2635300 CPUtime=5023.24 cores=4-31
/proc/25616/task/25624/stat : 25624 (plingeling) R 25611 25611 25371 0 -1 4202560 164104 1240 0 0 502273 50 0 1 25 0 29 0 641361418 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=25616/tid=25625] ppid=25611 vsize=2635300 CPUtime=5095.53 cores=4-31
/proc/25616/task/25625/stat : 25625 (plingeling) R 25611 25611 25371 0 -1 4202560 183311 1240 0 0 509474 78 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=25616/tid=25626] ppid=25611 vsize=2635300 CPUtime=5026.35 cores=4-31
/proc/25616/task/25626/stat : 25626 (plingeling) R 25611 25611 25371 0 -1 4202560 57855 1240 0 0 502589 45 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=25616/tid=25627] ppid=25611 vsize=2635300 CPUtime=5068.3 cores=4-31
/proc/25616/task/25627/stat : 25627 (plingeling) R 25611 25611 25371 0 -1 4202560 136237 1240 0 0 506759 70 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331401 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=25616/tid=25628] ppid=25611 vsize=2635300 CPUtime=5093.18 cores=4-31
/proc/25616/task/25628/stat : 25628 (plingeling) R 25611 25611 25371 0 -1 4202560 230431 1240 0 0 509155 162 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=25616/tid=25629] ppid=25611 vsize=2635300 CPUtime=5095.73 cores=4-31
/proc/25616/task/25629/stat : 25629 (plingeling) R 25611 25611 25371 0 -1 4202560 153983 1240 0 0 509495 77 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=25616/tid=25630] ppid=25611 vsize=2635300 CPUtime=5095.67 cores=4-31
/proc/25616/task/25630/stat : 25630 (plingeling) R 25611 25611 25371 0 -1 4202560 263817 1240 0 0 509398 168 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=25616/tid=25631] ppid=25611 vsize=2635300 CPUtime=5061.92 cores=4-31
/proc/25616/task/25631/stat : 25631 (plingeling) R 25611 25611 25371 0 -1 4202560 164147 1240 0 0 506099 92 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=25616/tid=25632] ppid=25611 vsize=2635300 CPUtime=5065.66 cores=4-31
/proc/25616/task/25632/stat : 25632 (plingeling) R 25611 25611 25371 0 -1 4202560 188510 1240 0 0 506472 93 0 1 25 0 29 0 641361419 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=25616/tid=25633] ppid=25611 vsize=2635300 CPUtime=5007.86 cores=4-31
/proc/25616/task/25633/stat : 25633 (plingeling) R 25611 25611 25371 0 -1 4202560 153508 1240 0 0 500698 87 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=25616/tid=25634] ppid=25611 vsize=2635300 CPUtime=5074.96 cores=4-31
/proc/25616/task/25634/stat : 25634 (plingeling) R 25611 25611 25371 0 -1 4202560 44316 1240 0 0 507464 31 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=25616/tid=25635] ppid=25611 vsize=2635300 CPUtime=4909.83 cores=4-31
/proc/25616/task/25635/stat : 25635 (plingeling) R 25611 25611 25371 0 -1 4202560 120199 1240 0 0 490895 87 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4281568 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=25616/tid=25636] ppid=25611 vsize=2635300 CPUtime=5089.19 cores=4-31
/proc/25616/task/25636/stat : 25636 (plingeling) R 25611 25611 25371 0 -1 4202560 150213 1240 0 0 508845 73 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4332109 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=25616/tid=25637] ppid=25611 vsize=2635300 CPUtime=4969.53 cores=4-31
/proc/25616/task/25637/stat : 25637 (plingeling) R 25611 25611 25371 0 -1 4202560 167004 1240 0 0 496809 143 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331345 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=25616/tid=25638] ppid=25611 vsize=2635300 CPUtime=5004.49 cores=4-31
/proc/25616/task/25638/stat : 25638 (plingeling) R 25611 25611 25371 0 -1 4202560 257222 1240 0 0 500229 219 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=25616/tid=25639] ppid=25611 vsize=2635300 CPUtime=4894.96 cores=4-31
/proc/25616/task/25639/stat : 25639 (plingeling) R 25611 25611 25371 0 -1 4202560 144819 1240 0 0 489422 73 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=25616/tid=25640] ppid=25611 vsize=2635300 CPUtime=4944.09 cores=4-31
/proc/25616/task/25640/stat : 25640 (plingeling) R 25611 25611 25371 0 -1 4202560 192490 1240 0 0 494274 134 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331456 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=25616/tid=25641] ppid=25611 vsize=2635300 CPUtime=4947.89 cores=4-31
/proc/25616/task/25641/stat : 25641 (plingeling) R 25611 25611 25371 0 -1 4202560 234734 1240 0 0 494666 122 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=25616/tid=25642] ppid=25611 vsize=2635300 CPUtime=5043.25 cores=4-31
/proc/25616/task/25642/stat : 25642 (plingeling) R 25611 25611 25371 0 -1 4202560 24324 1240 0 0 504299 25 0 1 25 0 29 0 641361420 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=25616/tid=25643] ppid=25611 vsize=2635300 CPUtime=4913.59 cores=4-31
/proc/25616/task/25643/stat : 25643 (plingeling) R 25611 25611 25371 0 -1 4202560 145402 1240 0 0 491288 70 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=25616/tid=25644] ppid=25611 vsize=2635300 CPUtime=5011.41 cores=4-31
/proc/25616/task/25644/stat : 25644 (plingeling) R 25611 25611 25371 0 -1 4202560 181681 1240 0 0 501052 88 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=25616/tid=25645] ppid=25611 vsize=2635300 CPUtime=5074.35 cores=4-31
/proc/25616/task/25645/stat : 25645 (plingeling) R 25611 25611 25371 0 -1 4202560 194820 1240 0 0 507301 133 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331007 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=25616/tid=25646] ppid=25611 vsize=2635300 CPUtime=5095.6 cores=4-31
/proc/25616/task/25646/stat : 25646 (plingeling) R 25611 25611 25371 0 -1 4202560 273560 1240 0 0 509298 261 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=25616/tid=25647] ppid=25611 vsize=2635300 CPUtime=5024.27 cores=4-31
/proc/25616/task/25647/stat : 25647 (plingeling) R 25611 25611 25371 0 -1 4202560 213068 1240 0 0 502272 154 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330995 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=25616/tid=25648] ppid=25611 vsize=2635300 CPUtime=5095.71 cores=4-31
/proc/25616/task/25648/stat : 25648 (plingeling) R 25611 25611 25371 0 -1 4202560 204800 1240 0 0 509495 75 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4279586 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=25616/tid=25649] ppid=25611 vsize=2635300 CPUtime=5095.72 cores=4-31
/proc/25616/task/25649/stat : 25649 (plingeling) R 25611 25611 25371 0 -1 4202560 229177 1240 0 0 509413 158 0 1 25 0 29 0 641361421 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=25616/tid=25650] ppid=25611 vsize=2635300 CPUtime=5094.39 cores=4-31
/proc/25616/task/25650/stat : 25650 (plingeling) R 25611 25611 25371 0 -1 4202560 37209 1240 0 0 509413 25 0 1 25 0 29 0 641361422 2698547200 538007 268435456000 4194304 4356188 140736902408352 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 156229
Current children cumulated vsize (KiB) 4598256

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 25611 and gives
#  childrusage.ru_utime.tv_sec=0
#  childrusage.ru_utime.tv_usec=24996
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=48992
# CPU time returned by wait4() is 0.073988
# while last known CPU time is 156229
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5100.04
CPU time (s): 156229
CPU user time (s): 156197
CPU system time (s): 32.04
CPU usage (%): 3063.3
Max. virtual memory (cumulated for all children) (KiB): 5916632

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 0.024996
system time used= 0.048992
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1707
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 487
involuntary context switches= 6

runsolver used 26.9139 second user time and 117.044 second system time

The end

Launcher Data

Begin job on node200 at 2011-05-13 16:59:07
IDJOB=3347537
IDBENCH=83116
IDSOLVER=1808
FILE ID=node200/3347537-1305298746
RUNJOBID= node200-1305293643-25385
PBS_JOBID= 13282108
Free space on /tmp= 62048 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-top24.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3347537-1305298746/watcher-3347537-1305298746 -o /tmp/evaluation-result-3347537-1305298746/solver-3347537-1305298746 -C 160000 -W 5100 -M 266338304  ppfolio HOME/instance-3347537-1305298746.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 1f485d737caaaca49e2c467c83cbdcec
RANDOM SEED=1723240900

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255623984 kB
Buffers:        433072 kB
Cached:        7800564 kB
SwapCached:          0 kB
Active:        7378600 kB
Inactive:       923592 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255623984 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:           13804 kB
Writeback:           0 kB
AnonPages:       69140 kB
Mapped:          14540 kB
Slab:           249808 kB
PageTables:       4532 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   187844 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 62032 MiB
End job on node200 at 2011-05-13 18:24:09