Trace number 3219183

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39971.8 5000.02

General information on the benchmark

Namerandom/large/
unif-k3-r4.2-v30000-c126000-S1854039067-041-UNKNOWN.cnf
MD5SUM30037b76a2fd7877580e038a49317f67
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark205.402
Satisfiable
(Un)Satisfiability was proved
Number of variables30000
Number of clauses126000
Sum of the clauses size378000
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3126000
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3219183-1304149029/watcher-3219183-1304149029 -o /tmp/evaluation-result-3219183-1304149029/solver-3219183-1304149029 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3219183-1304149029.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.03 8.05 7.96 2/170 3389
/proc/meminfo: memFree=30597660/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0

[startup+0.107517 s]
/proc/loadavg: 8.03 8.05 7.96 2/170 3389
/proc/meminfo: memFree=30597660/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.2005 s]
/proc/loadavg: 8.03 8.05 7.96 2/170 3389
/proc/meminfo: memFree=30597660/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 25 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300479 s]
/proc/loadavg: 8.03 8.05 7.96 2/170 3389
/proc/meminfo: memFree=30597660/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 23 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700343 s]
/proc/loadavg: 8.03 8.05 7.96 2/170 3389
/proc/meminfo: memFree=30597660/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 20 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50118 s]
/proc/loadavg: 8.03 8.05 7.96 9/180 3406
/proc/meminfo: memFree=30465892/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 20 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=51220 CPUtime=1.49 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 10889 0 0 0 148 1 0 0 20 0 1 0 526499593 52449280 8478 33554432000 4194304 4587354 140734160791056 18446744073709551615 4241312 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 12805 8478 326 96 0 9367 0
[pid=3392] ppid=3389 vsize=28752 CPUtime=1.48 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 5893 0 0 0 148 0 0 0 20 0 1 0 526499593 29442048 4460 33554432000 4194304 4785740 140734768519408 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 7188 4460 340 145 0 4233 0
[pid=3393] ppid=3389 vsize=464024 CPUtime=1.49 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5102 0 0 0 148 1 0 0 20 0 1 0 526499593 475160576 4999 33554432000 134512640 134534564 4288581152 18446744073709551615 134530728 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116006 4999 95 6 0 114672 0
[pid=3394] ppid=3389 vsize=67668 CPUtime=1.49 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 35588 0 0 0 144 5 0 0 20 0 1 0 526499593 69292032 6961 33554432000 4194304 4331988 140734282283664 18446744073709551615 4251963 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 16917 6961 151 34 0 15359 0
[pid=3395] ppid=3389 vsize=155880 CPUtime=4.73 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 20271 1250 0 0 471 2 0 0 25 0 5 0 526499593 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 38970 11557 163 40 0 37514 0
[pid=3395/tid=3402] ppid=3389 vsize=155880 CPUtime=1.1 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 2123 1250 0 0 110 0 0 0 19 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4274159 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=155880 CPUtime=1.1 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 2598 1250 0 0 110 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4286011 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=155880 CPUtime=1.08 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 2265 1250 0 0 108 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=155880 CPUtime=1.05 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 2704 1250 0 0 105 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4211028 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 10.68
Current children cumulated vsize (KiB) 785868

[startup+3.10086 s]
/proc/loadavg: 8.03 8.05 7.96 9/180 3406
/proc/meminfo: memFree=30424352/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=88724 CPUtime=3.08 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 11899 0 0 0 307 1 0 0 25 0 1 0 526499593 90853376 9484 33554432000 4194304 4587354 140734160791056 18446744073709551615 4238297 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 22181 9484 326 96 0 18743 0
[pid=3392] ppid=3389 vsize=34592 CPUtime=3.09 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 7364 0 0 0 308 1 0 0 25 0 1 0 526499593 35422208 5931 33554432000 4194304 4785740 140734768519408 18446744073709551615 4553003 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 8648 5931 340 145 0 5693 0
[pid=3393] ppid=3389 vsize=464024 CPUtime=3.09 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5102 0 0 0 308 1 0 0 25 0 1 0 526499593 475160576 4999 33554432000 134512640 134534564 4288581152 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116006 4999 95 6 0 114672 0
[pid=3394] ppid=3389 vsize=67296 CPUtime=3.09 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 45044 0 0 0 303 6 0 0 25 0 1 0 526499593 68911104 13827 33554432000 4194304 4331988 140734282283664 18446744073709551615 4212976 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 16824 13827 173 34 0 15266 0
[pid=3395] ppid=3389 vsize=155880 CPUtime=11.12 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 20271 1250 0 0 1110 2 0 0 25 0 5 0 526499593 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 38970 11557 163 40 0 37514 0
[pid=3395/tid=3402] ppid=3389 vsize=155880 CPUtime=2.7 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 2123 1250 0 0 270 0 0 0 22 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=155880 CPUtime=2.7 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 2598 1250 0 0 270 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=155880 CPUtime=2.68 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 2265 1250 0 0 268 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4331435 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=155880 CPUtime=2.64 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 2704 1250 0 0 264 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 23.47
Current children cumulated vsize (KiB) 828840

[startup+6.30123 s]
/proc/loadavg: 8.03 8.04 7.96 9/180 3406
/proc/meminfo: memFree=30400428/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=88724 CPUtime=6.28 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 13813 0 0 0 626 2 0 0 25 0 1 0 526499593 90853376 11391 33554432000 4194304 4587354 140734160791056 18446744073709551615 4238739 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 22181 11391 326 96 0 18743 0
[pid=3392] ppid=3389 vsize=47528 CPUtime=6.29 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 10608 0 0 0 627 2 0 0 25 0 1 0 526499593 48668672 9175 33554432000 4194304 4785740 140734768519408 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 11882 9175 340 145 0 8927 0
[pid=3393] ppid=3389 vsize=464024 CPUtime=6.29 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5102 0 0 0 628 1 0 0 25 0 1 0 526499593 475160576 4999 33554432000 134512640 134534564 4288581152 18446744073709551615 134530186 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116006 4999 95 6 0 114672 0
[pid=3394] ppid=3389 vsize=72100 CPUtime=6.29 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 46161 0 0 0 623 6 0 0 25 0 1 0 526499593 73830400 14944 33554432000 4194304 4331988 140734282283664 18446744073709551615 4237171 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 18025 14944 173 34 0 16467 0
[pid=3395] ppid=3389 vsize=155880 CPUtime=23.9 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 20271 1250 0 0 2388 2 0 0 25 0 5 0 526499593 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 38970 11557 163 40 0 37514 0
[pid=3395/tid=3402] ppid=3389 vsize=155880 CPUtime=5.9 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 2123 1250 0 0 590 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=155880 CPUtime=5.9 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 2598 1250 0 0 590 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4279717 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=155880 CPUtime=5.88 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 2265 1250 0 0 588 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4332020 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=155880 CPUtime=5.82 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 2704 1250 0 0 582 0 0 0 25 0 5 0 526499632 159621120 11557 33554432000 4194304 4356188 140735975657104 18446744073709551615 4283944 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.05
Current children cumulated vsize (KiB) 846580

[startup+12.701 s]
/proc/loadavg: 8.02 8.04 7.96 9/180 3407
/proc/meminfo: memFree=30357692/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=997760 CPUtime=12.68 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 19582 0 0 0 1264 4 0 0 25 0 1 0 526499593 1021706240 11128 33554432000 4194304 4587354 140734160791056 18446744073709551615 4238736 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 249440 11128 331 96 0 246002 0
[pid=3392] ppid=3389 vsize=75536 CPUtime=12.69 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 17563 0 0 0 1265 4 0 0 25 0 1 0 526499593 77348864 16130 33554432000 4194304 4785740 140734768519408 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 18884 16130 340 145 0 15929 0
[pid=3393] ppid=3389 vsize=464024 CPUtime=12.69 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5102 0 0 0 1268 1 0 0 25 0 1 0 526499593 475160576 4999 33554432000 134512640 134534564 4288581152 18446744073709551615 134529042 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116006 4999 95 6 0 114672 0
[pid=3394] ppid=3389 vsize=75008 CPUtime=12.69 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 46899 0 0 0 1262 7 0 0 25 0 1 0 526499593 76808192 15682 33554432000 4194304 4331988 140734282283664 18446744073709551615 4263152 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 18752 15682 173 34 0 17194 0
[pid=3395] ppid=3389 vsize=168660 CPUtime=49.47 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 23466 1250 0 0 4944 3 0 0 25 0 5 0 526499593 172707840 14254 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 42165 14254 163 40 0 40709 0
[pid=3395/tid=3402] ppid=3389 vsize=168660 CPUtime=12.29 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 3409 1250 0 0 1229 0 0 0 25 0 5 0 526499632 172707840 14254 33554432000 4194304 4356188 140735975657104 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=168660 CPUtime=12.29 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 4256 1250 0 0 1229 0 0 0 25 0 5 0 526499632 172707840 14254 33554432000 4194304 4356188 140735975657104 18446744073709551615 4210587 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=168660 CPUtime=12.28 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 2265 1250 0 0 1228 0 0 0 25 0 5 0 526499632 172707840 14254 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=168660 CPUtime=12.19 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 2955 1250 0 0 1219 0 0 0 25 0 5 0 526499632 172707840 14254 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.22
Current children cumulated vsize (KiB) 1799312

[startup+25.5014 s]
/proc/loadavg: 8.02 8.04 7.96 9/180 3407
/proc/meminfo: memFree=30280844/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=112780 CPUtime=25.48 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 60891 0 0 0 2537 11 0 0 25 0 1 0 526499593 115486720 22667 33554432000 4194304 4587354 140734160791056 18446744073709551615 4388336 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=3393] ppid=3389 vsize=467856 CPUtime=4841.86 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5113 0 0 0 484184 2 0 0 25 0 1 0 526499593 479084544 5010 33554432000 134512640 134534564 4288581152 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116964 5010 103 6 0 115630 0
[pid=3394] ppid=3389 vsize=84908 CPUtime=4842.52 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 49382 0 0 0 484243 9 0 0 25 0 1 0 526499593 86945792 18165 33554432000 4194304 4331988 140734282283664 18446744073709551615 4228983 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 21227 18165 173 34 0 19669 0
[pid=3395] ppid=3389 vsize=253520 CPUtime=19341.7 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 133310 1250 0 0 1934143 22 0 0 25 0 5 0 526499593 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 63380 35469 163 40 0 61924 0
[pid=3395/tid=3402] ppid=3389 vsize=253520 CPUtime=4842.12 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 23196 1250 0 0 484210 2 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4211032 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=253520 CPUtime=4841.98 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 38810 1250 0 0 484191 7 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=253520 CPUtime=4841.62 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 31608 1250 0 0 484156 6 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=253520 CPUtime=4815.5 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 29115 1250 0 0 481546 4 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4211054 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 38711
Current children cumulated vsize (KiB) 2586328

[startup+4902.3 s]
/proc/loadavg: 8.02 8.05 8.00 9/180 3561
/proc/meminfo: memFree=29552608/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=168724 CPUtime=4902.51 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 2369107 0 0 0 489233 1018 0 0 25 0 1 0 526499593 172773376 27977 33554432000 4194304 4587354 140734160791056 18446744073709551615 4237679 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 42181 27977 337 96 0 38743 0
[pid=3392] ppid=3389 vsize=719028 CPUtime=4902.51 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 176506 0 0 0 490190 61 0 0 25 0 1 0 526499593 736284672 175073 33554432000 4194304 4785740 140734768519408 18446744073709551615 4287008 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 179757 175073 340 145 0 176802 0
[pid=3393] ppid=3389 vsize=467856 CPUtime=4901.86 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5113 0 0 0 490184 2 0 0 25 0 1 0 526499593 479084544 5010 33554432000 134512640 134534564 4288581152 18446744073709551615 134528330 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116964 5010 103 6 0 115630 0
[pid=3394] ppid=3389 vsize=84908 CPUtime=4902.52 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 49390 0 0 0 490243 9 0 0 25 0 1 0 526499593 86945792 18173 33554432000 4194304 4331988 140734282283664 18446744073709551615 4228836 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 21227 18173 173 34 0 19669 0
[pid=3395] ppid=3389 vsize=253520 CPUtime=19581.3 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 133310 1250 0 0 1958109 22 0 0 25 0 5 0 526499593 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 63380 35469 163 40 0 61924 0
[pid=3395/tid=3402] ppid=3389 vsize=253520 CPUtime=4902.12 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 23196 1250 0 0 490210 2 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4331383 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=253520 CPUtime=4901.98 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 38810 1250 0 0 490191 7 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4332327 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=253520 CPUtime=4901.61 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 31608 1250 0 0 490155 6 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=253520 CPUtime=4875.18 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 29115 1250 0 0 487514 4 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39190.7
Current children cumulated vsize (KiB) 1712360

[startup+4962.3 s]
/proc/loadavg: 8.01 8.04 8.00 9/180 3563
/proc/meminfo: memFree=29592624/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=1061928 CPUtime=4962.51 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 2390187 0 0 0 495221 1030 0 0 25 0 1 0 526499593 1087414272 16600 33554432000 4194304 4587354 140734160791056 18446744073709551615 4237467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 265482 16600 337 96 0 262044 0
[pid=3392] ppid=3389 vsize=720024 CPUtime=4962.51 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 176730 0 0 0 496190 61 0 0 25 0 1 0 526499593 737304576 175297 33554432000 4194304 4785740 140734768519408 18446744073709551615 4286973 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 180006 175297 340 145 0 177051 0
[pid=3393] ppid=3389 vsize=467856 CPUtime=4961.86 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5113 0 0 0 496184 2 0 0 25 0 1 0 526499593 479084544 5010 33554432000 134512640 134534564 4288581152 18446744073709551615 134530186 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116964 5010 103 6 0 115630 0
[pid=3394] ppid=3389 vsize=84908 CPUtime=4962.52 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 49398 0 0 0 496243 9 0 0 25 0 1 0 526499593 86945792 18181 33554432000 4194304 4331988 140734282283664 18446744073709551615 4263246 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 21227 18181 173 34 0 19669 0
[pid=3395] ppid=3389 vsize=257616 CPUtime=19821 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 136382 1250 0 0 1982076 23 0 0 25 0 5 0 526499593 263798784 36493 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 64404 36493 163 40 0 62948 0
[pid=3395/tid=3402] ppid=3389 vsize=257616 CPUtime=4962.12 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 25244 1250 0 0 496210 2 0 0 25 0 5 0 526499632 263798784 36493 33554432000 4194304 4356188 140735975657104 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=257616 CPUtime=4961.98 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 38810 1250 0 0 496191 7 0 0 25 0 5 0 526499632 263798784 36493 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=257616 CPUtime=4961.61 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 31608 1250 0 0 496155 6 0 0 25 0 5 0 526499632 263798784 36493 33554432000 4194304 4356188 140735975657104 18446744073709551615 4281495 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=257616 CPUtime=4934.85 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 30139 1250 0 0 493481 4 0 0 25 0 5 0 526499632 263798784 36493 33554432000 4194304 4356188 140735975657104 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39670.4
Current children cumulated vsize (KiB) 2610656



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 3564
/proc/meminfo: memFree=29541392/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=1061928 CPUtime=5000.22 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 2403873 0 0 0 498985 1037 0 0 25 0 1 0 526499593 1087414272 30238 33554432000 4194304 4587354 140734160791056 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 265482 30238 337 96 0 262044 0
[pid=3392] ppid=3389 vsize=720024 CPUtime=5000.22 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 176738 0 0 0 499961 61 0 0 25 0 1 0 526499593 737304576 175305 33554432000 4194304 4785740 140734768519408 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 180006 175305 340 145 0 177051 0
[pid=3393] ppid=3389 vsize=467856 CPUtime=4999.57 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5113 0 0 0 499955 2 0 0 25 0 1 0 526499593 479084544 5010 33554432000 134512640 134534564 4288581152 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116964 5010 103 6 0 115630 0
[pid=3394] ppid=3389 vsize=84908 CPUtime=5000.23 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 49402 0 0 0 500014 9 0 0 25 0 1 0 526499593 86945792 18185 33554432000 4194304 4331988 140734282283664 18446744073709551615 4228875 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 21227 18185 173 34 0 19669 0
[pid=3395] ppid=3389 vsize=253520 CPUtime=19971.6 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 136382 1250 0 0 1997137 23 0 0 25 0 5 0 526499593 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 63380 35469 163 40 0 61924 0
[pid=3395/tid=3402] ppid=3389 vsize=253520 CPUtime=4999.83 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 25244 1250 0 0 499981 2 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=253520 CPUtime=4999.69 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 38810 1250 0 0 499962 7 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=253520 CPUtime=4999.31 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 31608 1250 0 0 499925 6 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=253520 CPUtime=4972.34 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 30139 1250 0 0 497230 4 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39971.8
Current children cumulated vsize (KiB) 2606560

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5000 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 3564
/proc/meminfo: memFree=29541392/32950928 swapFree=67111464/67111528
[pid=3389] ppid=3387 vsize=18324 CPUtime=0 cores=0-7
/proc/3389/stat : 3389 (ppfolio) S 3387 3389 3131 0 -1 4202496 521 0 0 0 0 0 0 0 15 0 1 0 526499593 18763776 389 33554432000 4194304 4287625 140735213683504 18446744073709551615 254500711951 0 0 4096 65536 2772318243296116735 0 0 17 4 0 0 0
/proc/3389/statm: 4581 389 341 23 0 91 0
[pid=3391] ppid=3389 vsize=1061928 CPUtime=5000.22 cores=0
/proc/3391/stat : 3391 (cryptominisat) R 3389 3389 3131 0 -1 4202496 2403873 0 0 0 498985 1037 0 0 25 0 1 0 526499593 1087414272 30238 33554432000 4194304 4587354 140734160791056 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/3391/statm: 265482 30238 337 96 0 262044 0
[pid=3392] ppid=3389 vsize=720024 CPUtime=5000.22 cores=1
/proc/3392/stat : 3392 (clasp) R 3389 3389 3131 0 -1 4202496 176738 0 0 0 499961 61 0 0 25 0 1 0 526499593 737304576 175305 33554432000 4194304 4785740 140734768519408 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/3392/statm: 180006 175305 340 145 0 177051 0
[pid=3393] ppid=3389 vsize=467856 CPUtime=4999.57 cores=2
/proc/3393/stat : 3393 (TNM) R 3389 3389 3131 0 -1 4202496 5113 0 0 0 499955 2 0 0 25 0 1 0 526499593 479084544 5010 33554432000 134512640 134534564 4288581152 18446744073709551615 134528275 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/3393/statm: 116964 5010 103 6 0 115630 0
[pid=3394] ppid=3389 vsize=84908 CPUtime=5000.23 cores=3
/proc/3394/stat : 3394 (march_hi) R 3389 3389 3131 0 -1 4202496 49402 0 0 0 500014 9 0 0 25 0 1 0 526499593 86945792 18185 33554432000 4194304 4331988 140734282283664 18446744073709551615 4228875 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/3394/statm: 21227 18185 173 34 0 19669 0
[pid=3395] ppid=3389 vsize=253520 CPUtime=19971.6 cores=4-7
/proc/3395/stat : 3395 (plingeling) S 3389 3389 3131 0 -1 4202496 136382 1250 0 0 1997137 23 0 0 25 0 5 0 526499593 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 254512494565 0 0 4096 17442 18446744073709551615 0 0 17 6 0 0 0
/proc/3395/statm: 63380 35469 163 40 0 61924 0
[pid=3395/tid=3402] ppid=3389 vsize=253520 CPUtime=4999.83 cores=4-7
/proc/3395/task/3402/stat : 3402 (plingeling) R 3389 3389 3131 0 -1 4202560 25244 1250 0 0 499981 2 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330862 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=3395/tid=3404] ppid=3389 vsize=253520 CPUtime=4999.69 cores=4-7
/proc/3395/task/3404/stat : 3404 (plingeling) R 3389 3389 3131 0 -1 4202560 38810 1250 0 0 499962 7 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=3395/tid=3405] ppid=3389 vsize=253520 CPUtime=4999.31 cores=4-7
/proc/3395/task/3405/stat : 3405 (plingeling) R 3389 3389 3131 0 -1 4202560 31608 1250 0 0 499925 6 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=3395/tid=3406] ppid=3389 vsize=253520 CPUtime=4972.34 cores=4-7
/proc/3395/task/3406/stat : 3406 (plingeling) R 3389 3389 3131 0 -1 4202560 30139 1250 0 0 497230 4 0 0 25 0 5 0 526499632 259604480 35469 33554432000 4194304 4356188 140735975657104 18446744073709551615 4210834 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 39971.8
Current children cumulated vsize (KiB) 2606560

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3389 and gives
#  childrusage.ru_utime.tv_sec=14989
#  childrusage.ru_utime.tv_usec=558240
#  childrusage.ru_stime.tv_sec=10
#  childrusage.ru_stime.tv_usec=504403
# CPU time returned by wait4() is 15000.1
# while last known CPU time is 39971.8
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39971.8
CPU user time (s): 39960.5
CPU system time (s): 11.32
CPU usage (%): 799.434
Max. virtual memory (cumulated for all children) (KiB): 3667276

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14989.6
system time used= 10.5044
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2458910
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 471
involuntary context switches= 81062

runsolver used 6.99594 second user time and 20.2859 second system time

The end

Launcher Data 

Begin job on node149 at 2011-04-30 09:37:09
IDJOB=3219183
IDBENCH=82547
IDSOLVER=1638
FILE ID=node149/3219183-1304149029
RUNJOBID= node149-1304144389-3145
PBS_JOBID= 13170856
Free space on /tmp= 73324 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/large/unif-k3-r4.2-v30000-c126000-S1854039067-041-UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3219183-1304149029/watcher-3219183-1304149029 -o /tmp/evaluation-result-3219183-1304149029/solver-3219183-1304149029 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3219183-1304149029.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 30037b76a2fd7877580e038a49317f67
RANDOM SEED=454727133

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30598188 kB
Buffers:        496280 kB
Cached:        1414600 kB
SwapCached:         60 kB
Active:         187768 kB
Inactive:      1787228 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30598188 kB
SwapTotal:    67111528 kB
SwapFree:     67111464 kB
Dirty:           12436 kB
Writeback:           0 kB
AnonPages:       64124 kB
Mapped:          14804 kB
Slab:           312256 kB
PageTables:       4088 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   185952 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73312 MiB
End job on node149 at 2011-04-30 11:00:31