Trace number 3221169

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39972.4 5000.02

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v650-c2769-S1235135694-059.UNKNOWN.cnf
MD5SUM4fe02968acd00bdecbd1c96009763619
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables650
Number of clauses2769
Sum of the clauses size8307
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32769
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3221169-1304161024/watcher-3221169-1304161024 -o /tmp/evaluation-result-3221169-1304161024/solver-3221169-1304161024 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3221169-1304161024.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.41 7.90 7.96 2/170 28952
/proc/meminfo: memFree=30489736/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 2826088513535 0 0 17 6 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0

[startup+0.154561 s]
/proc/loadavg: 7.41 7.90 7.96 2/170 28952
/proc/meminfo: memFree=30489736/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) R 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.200553 s]
/proc/loadavg: 7.41 7.90 7.96 2/170 28952
/proc/meminfo: memFree=30489736/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) R 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300534 s]
/proc/loadavg: 7.41 7.90 7.96 2/170 28952
/proc/meminfo: memFree=30489736/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) R 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 24 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700461 s]
/proc/loadavg: 7.41 7.90 7.96 2/170 28952
/proc/meminfo: memFree=30489736/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) R 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 21 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 0 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50131 s]
/proc/loadavg: 7.41 7.90 7.96 9/180 28967
/proc/meminfo: memFree=30472420/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1581988 CPUtime=1.49 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 3799 0 0 0 149 0 0 0 20 0 1 0 606332934 1619955712 2453 33554432000 4194304 4587354 140733417603024 18446744073709551615 4357600 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 395497 2453 337 96 0 392059 0
[pid=28954] ppid=28952 vsize=14444 CPUtime=1.49 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 1082 0 0 0 149 0 0 0 20 0 1 0 606332934 14790656 969 33554432000 4194304 4785740 140734821700496 18446744073709551615 4550406 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 3611 969 340 145 0 656 0
[pid=28955] ppid=28952 vsize=455960 CPUtime=1.49 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 365 0 0 0 149 0 0 0 25 0 1 0 606332934 466903040 262 33554432000 134512640 134534564 4289782480 18446744073709551615 134530832 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113990 262 95 6 0 112656 0
[pid=28956] ppid=28952 vsize=8348 CPUtime=1.49 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 690 0 0 0 149 0 0 0 20 0 1 0 606332934 8548352 583 33554432000 4194304 4331988 140737100049376 18446744073709551615 4229469 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 2087 583 173 34 0 529 0
[pid=28957] ppid=28952 vsize=123436 CPUtime=5.91 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 3751 1250 0 0 591 0 0 0 18 0 5 0 606332934 126398464 2981 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 30859 2981 164 40 0 29403 0
[pid=28957/tid=28964] ppid=28952 vsize=123436 CPUtime=1.48 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 870 1250 0 0 148 0 0 0 19 0 5 0 606332935 126398464 2981 33554432000 4194304 4356188 140735053151072 18446744073709551615 4279626 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=123436 CPUtime=1.44 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 837 1250 0 0 144 0 0 0 23 0 5 0 606332935 126398464 2981 33554432000 4194304 4356188 140735053151072 18446744073709551615 4284157 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=123436 CPUtime=1.48 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 824 1250 0 0 148 0 0 0 20 0 5 0 606332935 126398464 2981 33554432000 4194304 4356188 140735053151072 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=123436 CPUtime=1.48 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 691 1250 0 0 148 0 0 0 23 0 5 0 606332935 126398464 2981 33554432000 4194304 4356188 140735053151072 18446744073709551615 4332041 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.87
Current children cumulated vsize (KiB) 2202500

[startup+3.10102 s]
/proc/loadavg: 7.41 7.90 7.96 9/180 28967
/proc/meminfo: memFree=30464520/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1581800 CPUtime=3.09 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 4808 0 0 0 309 0 0 0 25 0 1 0 606332934 1619763200 2521 33554432000 4194304 4587354 140733417603024 18446744073709551615 4238752 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 395450 2521 337 96 0 392012 0
[pid=28954] ppid=28952 vsize=15244 CPUtime=3.09 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 1283 0 0 0 309 0 0 0 25 0 1 0 606332934 15609856 1170 33554432000 4194304 4785740 140734821700496 18446744073709551615 4551132 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 3811 1170 340 145 0 856 0
[pid=28955] ppid=28952 vsize=455960 CPUtime=3.09 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 365 0 0 0 309 0 0 0 25 0 1 0 606332934 466903040 262 33554432000 134512640 134534564 4289782480 18446744073709551615 5749150 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113990 262 95 6 0 112656 0
[pid=28956] ppid=28952 vsize=8904 CPUtime=3.09 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 766 0 0 0 309 0 0 0 25 0 1 0 606332934 9117696 659 33554432000 4194304 4331988 140737100049376 18446744073709551615 4229616 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 2226 659 173 34 0 668 0
[pid=28957] ppid=28952 vsize=123308 CPUtime=12.3 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 4071 1250 0 0 1230 0 0 0 18 0 5 0 606332934 126267392 2949 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 30827 2949 164 40 0 29371 0
[pid=28957/tid=28964] ppid=28952 vsize=123308 CPUtime=3.08 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 902 1250 0 0 308 0 0 0 25 0 5 0 606332935 126267392 2949 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331392 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=123308 CPUtime=3.03 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 997 1250 0 0 303 0 0 0 25 0 5 0 606332935 126267392 2949 33554432000 4194304 4356188 140735053151072 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=123308 CPUtime=3.08 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 888 1250 0 0 308 0 0 0 25 0 5 0 606332935 126267392 2949 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=123308 CPUtime=3.08 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 755 1250 0 0 308 0 0 0 25 0 5 0 606332935 126267392 2949 33554432000 4194304 4356188 140735053151072 18446744073709551615 4330973 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.66
Current children cumulated vsize (KiB) 2203540

[startup+6.30043 s]
/proc/loadavg: 7.54 7.92 7.96 9/180 28967
/proc/meminfo: memFree=30457948/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1584948 CPUtime=6.29 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 7289 0 0 0 628 1 0 0 25 0 1 0 606332934 1622986752 2613 33554432000 4194304 4587354 140733417603024 18446744073709551615 260020092564 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 396237 2613 337 96 0 392799 0
[pid=28954] ppid=28952 vsize=16716 CPUtime=6.29 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 1619 0 0 0 629 0 0 0 25 0 1 0 606332934 17117184 1506 33554432000 4194304 4785740 140734821700496 18446744073709551615 4553199 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 4179 1506 341 145 0 1224 0
[pid=28955] ppid=28952 vsize=455960 CPUtime=6.29 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 365 0 0 0 629 0 0 0 25 0 1 0 606332934 466903040 262 33554432000 134512640 134534564 4289782480 18446744073709551615 134529640 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113990 262 95 6 0 112656 0
[pid=28956] ppid=28952 vsize=9244 CPUtime=6.29 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 899 0 0 0 629 0 0 0 25 0 1 0 606332934 9465856 792 33554432000 4194304 4331988 140737100049376 18446744073709551615 4226333 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 2311 792 173 34 0 753 0
[pid=28957] ppid=28952 vsize=125640 CPUtime=25.09 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 6830 1250 0 0 2508 1 0 0 18 0 5 0 606332934 128655360 3500 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 31410 3500 164 40 0 29954 0
[pid=28957/tid=28964] ppid=28952 vsize=125640 CPUtime=6.28 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 1511 1250 0 0 628 0 0 0 25 0 5 0 606332935 128655360 3500 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331960 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=125640 CPUtime=6.21 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 1477 1250 0 0 621 0 0 0 25 0 5 0 606332935 128655360 3500 33554432000 4194304 4356188 140735053151072 18446744073709551615 4280065 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=125640 CPUtime=6.28 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 1562 1250 0 0 628 0 0 0 25 0 5 0 606332935 128655360 3500 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=125640 CPUtime=6.28 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 1751 1250 0 0 628 0 0 0 25 0 5 0 606332935 128655360 3500 33554432000 4194304 4356188 140735053151072 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.25
Current children cumulated vsize (KiB) 2210832

[startup+12.7013 s]
/proc/loadavg: 7.58 7.92 7.96 9/180 28967
/proc/meminfo: memFree=30442396/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1590844 CPUtime=12.69 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 12603 0 0 0 1267 2 0 0 25 0 1 0 606332934 1629024256 4414 33554432000 4194304 4587354 140733417603024 18446744073709551615 4238463 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 397711 4414 337 96 0 394273 0
[pid=28954] ppid=28952 vsize=18228 CPUtime=12.69 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 1999 0 0 0 1269 0 0 0 25 0 1 0 606332934 18665472 1886 33554432000 4194304 4785740 140734821700496 18446744073709551615 4287058 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 4557 1886 341 145 0 1602 0
[pid=28955] ppid=28952 vsize=455960 CPUtime=12.69 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 365 0 0 0 1269 0 0 0 25 0 1 0 606332934 466903040 262 33554432000 134512640 134534564 4289782480 18446744073709551615 134528659 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113990 262 95 6 0 112656 0
[pid=28956] ppid=28952 vsize=10816 CPUtime=12.69 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 1188 0 0 0 1269 0 0 0 25 0 1 0 606332934 11075584 1081 33554432000 4194304 4331988 140737100049376 18446744073709551615 4211568 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 2704 1081 173 34 0 1146 0
[pid=28957] ppid=28952 vsize=130664 CPUtime=50.64 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 12070 1250 0 0 5063 1 0 0 18 0 5 0 606332934 133799936 4772 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 32666 4772 164 40 0 31210 0
[pid=28957/tid=28964] ppid=28952 vsize=130664 CPUtime=12.68 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 2793 1250 0 0 1268 0 0 0 25 0 5 0 606332935 133799936 4772 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=130664 CPUtime=12.58 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 2919 1250 0 0 1258 0 0 0 25 0 5 0 606332935 133799936 4772 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=130664 CPUtime=12.68 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 2972 1250 0 0 1268 0 0 0 25 0 5 0 606332935 133799936 4772 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331199 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=130664 CPUtime=12.68 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 2857 1250 0 0 1268 0 0 0 25 0 5 0 606332935 133799936 4772 33554432000 4194304 4356188 140735053151072 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.4
Current children cumulated vsize (KiB) 2224836

[startup+25.5009 s]
/proc/loadavg: 7.67 7.92 7.96 9/180 28968
/proc/meminfo: memFree=30428396/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 7 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1601880 CPUtime=25.49 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 18393 0 0 0 2546 3 0 0 25 0 1 0 606332934 1640325120 5784 33554432000 4194304 4587354 140733417603024 18446744073709551615 4238736 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=28955] ppid=28952 vsize=455964 CPUtime=4842.43 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 374 0 0 0 484241 2 0 0 25 0 1 0 606332934 466907136 271 33554432000 134512640 134534564 4289782480 18446744073709551615 134530549 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113991 271 103 6 0 112657 0
[pid=28956] ppid=28952 vsize=830772 CPUtime=4842.44 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 152256 0 0 0 484132 112 0 0 25 0 1 0 606332934 850710528 152149 33554432000 4194304 4331988 140737100049376 18446744073709551615 4211503 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 207693 152149 173 34 0 206135 0
[pid=28957] ppid=28952 vsize=285728 CPUtime=19342.1 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 895876 1250 0 0 1934079 129 0 0 18 0 5 0 606332934 292585472 43554 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 71432 43554 164 40 0 69976 0
[pid=28957/tid=28964] ppid=28952 vsize=285728 CPUtime=4842.44 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 224493 1250 0 0 484214 30 0 0 25 0 5 0 606332935 292585472 43554 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=285728 CPUtime=4815.98 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 230347 1250 0 0 481567 31 0 0 25 0 5 0 606332935 292585472 43554 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=285728 CPUtime=4842.43 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 225198 1250 0 0 484211 32 0 0 25 0 5 0 606332935 292585472 43554 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=285728 CPUtime=4841.18 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 215309 1250 0 0 484084 34 0 0 25 0 5 0 606332935 292585472 43554 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331015 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38711.7
Current children cumulated vsize (KiB) 3353040

[startup+4902.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 29119
/proc/meminfo: memFree=29515200/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 3 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1696560 CPUtime=4902.43 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 221449 0 0 0 490139 104 0 0 25 0 1 0 606332934 1737277440 34774 33554432000 4194304 4587354 140733417603024 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 424140 34774 337 96 0 420702 0
[pid=28954] ppid=28952 vsize=65692 CPUtime=4902.28 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 12903 0 0 0 490226 2 0 0 25 0 1 0 606332934 67268608 12790 33554432000 4194304 4785740 140734821700496 18446744073709551615 4553740 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 16423 12790 341 145 0 13468 0
[pid=28955] ppid=28952 vsize=455964 CPUtime=4902.43 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 374 0 0 0 490241 2 0 0 25 0 1 0 606332934 466907136 271 33554432000 134512640 134534564 4289782480 18446744073709551615 134530522 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113991 271 103 6 0 112657 0
[pid=28956] ppid=28952 vsize=830772 CPUtime=4902.43 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 154084 0 0 0 490130 113 0 0 25 0 1 0 606332934 850710528 153977 33554432000 4194304 4331988 140737100049376 18446744073709551615 4240214 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 207693 153977 173 34 0 206135 0
[pid=28957] ppid=28952 vsize=282144 CPUtime=19581.7 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 901540 1250 0 0 1958043 130 0 0 18 0 5 0 606332934 288915456 42658 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 70536 42658 164 40 0 69080 0
[pid=28957/tid=28964] ppid=28952 vsize=282144 CPUtime=4902.43 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 225773 1250 0 0 490213 30 0 0 25 0 5 0 606332935 288915456 42658 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=282144 CPUtime=4875.66 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 231371 1250 0 0 487535 31 0 0 25 0 5 0 606332935 288915456 42658 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=282144 CPUtime=4902.43 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 226862 1250 0 0 490211 32 0 0 25 0 5 0 606332935 288915456 42658 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=282144 CPUtime=4901.17 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 217005 1250 0 0 490082 35 0 0 25 0 5 0 606332935 288915456 42658 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331305 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39191.3
Current children cumulated vsize (KiB) 3349456

[startup+4962.3 s]
/proc/loadavg: 8.00 8.02 8.00 9/180 29120
/proc/meminfo: memFree=29547472/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 3 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1741860 CPUtime=4962.43 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 222194 0 0 0 496139 104 0 0 25 0 1 0 606332934 1783664640 33554 33554432000 4194304 4587354 140733417603024 18446744073709551615 4238690 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 435465 33554 337 96 0 432027 0
[pid=28954] ppid=28952 vsize=65832 CPUtime=4962.28 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 12936 0 0 0 496226 2 0 0 25 0 1 0 606332934 67411968 12823 33554432000 4194304 4785740 140734821700496 18446744073709551615 4552974 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 16458 12823 341 145 0 13503 0
[pid=28955] ppid=28952 vsize=455964 CPUtime=4962.43 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 374 0 0 0 496241 2 0 0 25 0 1 0 606332934 466907136 271 33554432000 134512640 134534564 4289782480 18446744073709551615 134530339 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113991 271 103 6 0 112657 0
[pid=28956] ppid=28952 vsize=830772 CPUtime=4962.43 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 155829 0 0 0 496128 115 0 0 25 0 1 0 606332934 850710528 155722 33554432000 4194304 4331988 140737100049376 18446744073709551615 4211480 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 207693 155722 173 34 0 206135 0
[pid=28957] ppid=28952 vsize=246304 CPUtime=19821.4 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 905956 1250 0 0 1982010 130 0 0 18 0 5 0 606332934 252215296 33698 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 61576 33698 164 40 0 60120 0
[pid=28957/tid=28964] ppid=28952 vsize=246304 CPUtime=4962.43 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 227469 1250 0 0 496213 30 0 0 25 0 5 0 606332935 252215296 33698 33554432000 4194304 4356188 140735053151072 18446744073709551615 4281624 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=246304 CPUtime=4935.33 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 232523 1250 0 0 493501 32 0 0 25 0 5 0 606332935 252215296 33698 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=246304 CPUtime=4962.43 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 227662 1250 0 0 496211 32 0 0 25 0 5 0 606332935 252215296 33698 33554432000 4194304 4356188 140735053151072 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=246304 CPUtime=4961.17 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 217773 1250 0 0 496082 35 0 0 25 0 5 0 606332935 252215296 33698 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39671
Current children cumulated vsize (KiB) 3359056



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.08 8.03 8.01 9/180 29121
/proc/meminfo: memFree=29517416/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 3 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1739112 CPUtime=5000.15 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 223733 0 0 0 499910 105 0 0 25 0 1 0 606332934 1780850688 33036 33554432000 4194304 4587354 140733417603024 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 434778 33036 337 96 0 431340 0
[pid=28954] ppid=28952 vsize=66000 CPUtime=4999.99 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 12971 0 0 0 499997 2 0 0 25 0 1 0 606332934 67584000 12858 33554432000 4194304 4785740 140734821700496 18446744073709551615 4287168 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 16500 12858 341 145 0 13545 0
[pid=28955] ppid=28952 vsize=455964 CPUtime=5000.14 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 374 0 0 0 500012 2 0 0 25 0 1 0 606332934 466907136 271 33554432000 134512640 134534564 4289782480 18446744073709551615 134528918 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113991 271 103 6 0 112657 0
[pid=28956] ppid=28952 vsize=830772 CPUtime=5000.15 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 156904 0 0 0 499899 116 0 0 25 0 1 0 606332934 850710528 156797 33554432000 4194304 4331988 140737100049376 18446744073709551615 4229702 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 207693 156797 173 34 0 206135 0
[pid=28957] ppid=28952 vsize=274336 CPUtime=19972 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 915428 1250 0 0 1997070 132 0 0 18 0 5 0 606332934 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 68584 40706 164 40 0 67128 0
[pid=28957/tid=28964] ppid=28952 vsize=274336 CPUtime=5000.15 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 232333 1250 0 0 499984 31 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=274336 CPUtime=4972.83 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 234955 1250 0 0 497251 32 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=274336 CPUtime=5000.13 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 228174 1250 0 0 499981 32 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331807 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=274336 CPUtime=4998.87 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 219437 1250 0 0 499852 35 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39972.4
Current children cumulated vsize (KiB) 3384508

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5000 s]
/proc/loadavg: 8.08 8.03 8.01 9/180 29121
/proc/meminfo: memFree=29517416/32951124 swapFree=67111464/67111528
[pid=28952] ppid=28950 vsize=18324 CPUtime=0 cores=0-7
/proc/28952/stat : 28952 (ppfolio) S 28950 28952 28844 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 606332934 18763776 390 33554432000 4194304 4287625 140733316962576 18446744073709551615 260020416015 0 0 4096 65536 4235925312589791231 0 0 17 3 0 0 0
/proc/28952/statm: 4581 390 341 23 0 91 0
[pid=28953] ppid=28952 vsize=1739112 CPUtime=5000.15 cores=0
/proc/28953/stat : 28953 (cryptominisat) R 28952 28952 28844 0 -1 4202496 223733 0 0 0 499910 105 0 0 25 0 1 0 606332934 1780850688 33036 33554432000 4194304 4587354 140733417603024 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/28953/statm: 434778 33036 337 96 0 431340 0
[pid=28954] ppid=28952 vsize=66000 CPUtime=4999.99 cores=1
/proc/28954/stat : 28954 (clasp) R 28952 28952 28844 0 -1 4202496 12971 0 0 0 499997 2 0 0 25 0 1 0 606332934 67584000 12858 33554432000 4194304 4785740 140734821700496 18446744073709551615 4287168 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/28954/statm: 16500 12858 341 145 0 13545 0
[pid=28955] ppid=28952 vsize=455964 CPUtime=5000.14 cores=2
/proc/28955/stat : 28955 (TNM) R 28952 28952 28844 0 -1 4202496 374 0 0 0 500012 2 0 0 25 0 1 0 606332934 466907136 271 33554432000 134512640 134534564 4289782480 18446744073709551615 134528918 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/28955/statm: 113991 271 103 6 0 112657 0
[pid=28956] ppid=28952 vsize=830772 CPUtime=5000.15 cores=3
/proc/28956/stat : 28956 (march_hi) R 28952 28952 28844 0 -1 4202496 156904 0 0 0 499899 116 0 0 25 0 1 0 606332934 850710528 156797 33554432000 4194304 4331988 140737100049376 18446744073709551615 4229702 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/28956/statm: 207693 156797 173 34 0 206135 0
[pid=28957] ppid=28952 vsize=274336 CPUtime=19972 cores=4-7
/proc/28957/stat : 28957 (plingeling) S 28952 28952 28844 0 -1 4202496 915428 1250 0 0 1997070 132 0 0 18 0 5 0 606332934 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 260032198629 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/28957/statm: 68584 40706 164 40 0 67128 0
[pid=28957/tid=28964] ppid=28952 vsize=274336 CPUtime=5000.15 cores=4-7
/proc/28957/task/28964/stat : 28964 (plingeling) R 28952 28952 28844 0 -1 4202560 232333 1250 0 0 499984 31 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=28957/tid=28965] ppid=28952 vsize=274336 CPUtime=4972.83 cores=4-7
/proc/28957/task/28965/stat : 28965 (plingeling) R 28952 28952 28844 0 -1 4202560 234955 1250 0 0 497251 32 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331047 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=28957/tid=28966] ppid=28952 vsize=274336 CPUtime=5000.13 cores=4-7
/proc/28957/task/28966/stat : 28966 (plingeling) R 28952 28952 28844 0 -1 4202560 228174 1250 0 0 499981 32 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331807 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=28957/tid=28967] ppid=28952 vsize=274336 CPUtime=4998.87 cores=4-7
/proc/28957/task/28967/stat : 28967 (plingeling) R 28952 28952 28844 0 -1 4202560 219437 1250 0 0 499852 35 0 0 25 0 5 0 606332935 280920064 40706 33554432000 4194304 4356188 140735053151072 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39972.4
Current children cumulated vsize (KiB) 3384508

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 28952 and gives
#  childrusage.ru_utime.tv_sec=14999
#  childrusage.ru_utime.tv_usec=201774
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=120829
# CPU time returned by wait4() is 15000.3
# while last known CPU time is 39972.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39972.4
CPU user time (s): 39968.9
CPU system time (s): 3.57
CPU usage (%): 799.446
Max. virtual memory (cumulated for all children) (KiB): 10638680

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14999.2
system time used= 1.12083
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 237618
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 580
involuntary context switches= 74629

runsolver used 8.29574 second user time and 18.9851 second system time

The end

Launcher Data 

Begin job on node110 at 2011-04-30 12:57:04
IDJOB=3221169
IDBENCH=82878
IDSOLVER=1638
FILE ID=node110/3221169-1304161024
RUNJOBID= node110-1304161023-28858
PBS_JOBID= 13170927
Free space on /tmp= 73476 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v650-c2769-S1235135694-059.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3221169-1304161024/watcher-3221169-1304161024 -o /tmp/evaluation-result-3221169-1304161024/solver-3221169-1304161024 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3221169-1304161024.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 4fe02968acd00bdecbd1c96009763619
RANDOM SEED=969682179

node110.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30490016 kB
Buffers:        990252 kB
Cached:         990300 kB
SwapCached:         64 kB
Active:         290328 kB
Inactive:      1754236 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30490016 kB
SwapTotal:    67111528 kB
SwapFree:     67111464 kB
Dirty:           10160 kB
Writeback:          84 kB
AnonPages:       64024 kB
Mapped:          14628 kB
Slab:           349348 kB
PageTables:       4440 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   189368 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73464 MiB
End job on node110 at 2011-04-30 14:20:26