Trace number 3222225

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 37931.8 5000.02

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v90-c8010-S1897189325-062.UNKNOWN.cnf
MD5SUM28731ba8f9c5274ff0cd070fd7aeff45
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables90
Number of clauses8010
Sum of the clauses size56070
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 58010

Solver Data

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3222225-1304165008/watcher-3222225-1304165008 -o /tmp/evaluation-result-3222225-1304165008/solver-3222225-1304165008 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3222225-1304165008.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.42 7.93 7.96 2/170 12926
/proc/meminfo: memFree=31117552/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0

[startup+0.075637 s]
/proc/loadavg: 7.42 7.93 7.96 2/170 12926
/proc/meminfo: memFree=31117552/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100632 s]
/proc/loadavg: 7.42 7.93 7.96 2/170 12926
/proc/meminfo: memFree=31117552/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 25 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.300599 s]
/proc/loadavg: 7.42 7.93 7.96 2/170 12926
/proc/meminfo: memFree=31117552/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 24 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700533 s]
/proc/loadavg: 7.42 7.93 7.96 2/170 12926
/proc/meminfo: memFree=31117552/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) R 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 21 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.5004 s]
/proc/loadavg: 7.42 7.93 7.96 9/180 12942
/proc/meminfo: memFree=31097632/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
[pid=12927] ppid=12926 vsize=260908 CPUtime=1.49 cores=0
/proc/12927/stat : 12927 (cryptominisat) R 12926 12926 12900 0 -1 4202496 3299 0 0 0 149 0 0 0 20 0 1 0 528099286 267169792 1266 33554432000 4194304 4587354 140736423621728 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12927/statm: 65227 1266 330 96 0 61789 0
[pid=12928] ppid=12926 vsize=15436 CPUtime=1.49 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 1320 0 0 0 149 0 0 0 20 0 1 0 528099286 15806464 1212 33554432000 4194304 4785740 140736038427024 18446744073709551615 4553344 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 3859 1212 341 145 0 904 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=1.49 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 149 0 0 0 20 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134530630 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=8804 CPUtime=1.49 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 1854 0 0 0 149 0 0 0 25 0 1 0 528099286 9015296 631 33554432000 4194304 4331988 140736659254656 18446744073709551615 4237421 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 2201 631 174 34 0 643 0
[pid=12931] ppid=12926 vsize=62172 CPUtime=5.89 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 4637 1245 0 0 589 0 0 0 18 0 5 0 528099286 63664128 3998 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 15543 3998 161 40 0 14087 0
[pid=12931/tid=12938] ppid=12926 vsize=62172 CPUtime=1.47 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 872 1245 0 0 147 0 0 0 25 0 5 0 528099288 63664128 3998 33554432000 4194304 4356188 140735509979088 18446744073709551615 4281487 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=62172 CPUtime=1.46 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 819 1245 0 0 146 0 0 0 25 0 5 0 528099288 63664128 3998 33554432000 4194304 4356188 140735509979088 18446744073709551615 4330995 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=62172 CPUtime=1.46 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 922 1245 0 0 146 0 0 0 23 0 5 0 528099288 63664128 3998 33554432000 4194304 4356188 140735509979088 18446744073709551615 4234306 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=62172 CPUtime=1.47 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 839 1245 0 0 147 0 0 0 23 0 5 0 528099288 63664128 3998 33554432000 4194304 4356188 140735509979088 18446744073709551615 4273730 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.85
Current children cumulated vsize (KiB) 822412

[startup+3.10113 s]
/proc/loadavg: 7.42 7.93 7.96 9/180 12942
/proc/meminfo: memFree=31082160/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
[pid=12927] ppid=12926 vsize=269532 CPUtime=3.08 cores=0
/proc/12927/stat : 12927 (cryptominisat) R 12926 12926 12900 0 -1 4202496 6035 0 0 0 308 0 0 0 25 0 1 0 528099286 276000768 3461 33554432000 4194304 4587354 140736423621728 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12927/statm: 67383 3461 336 96 0 63945 0
[pid=12928] ppid=12926 vsize=16156 CPUtime=3.09 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 1478 0 0 0 309 0 0 0 25 0 1 0 528099286 16543744 1370 33554432000 4194304 4785740 140736038427024 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 4039 1370 341 145 0 1084 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=3.09 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 309 0 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134530551 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=9108 CPUtime=3.09 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 1952 0 0 0 309 0 0 0 25 0 1 0 528099286 9326592 729 33554432000 4194304 4331988 140736659254656 18446744073709551615 4237409 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 2277 729 174 34 0 719 0
[pid=12931] ppid=12926 vsize=61804 CPUtime=12.28 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 5217 1245 0 0 1228 0 0 0 18 0 5 0 528099286 63287296 3906 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 15451 3906 161 40 0 13995 0
[pid=12931/tid=12938] ppid=12926 vsize=61804 CPUtime=3.07 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 1001 1245 0 0 307 0 0 0 25 0 5 0 528099288 63287296 3906 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=61804 CPUtime=3.06 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 1045 1245 0 0 306 0 0 0 25 0 5 0 528099288 63287296 3906 33554432000 4194304 4356188 140735509979088 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=61804 CPUtime=3.05 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 1018 1245 0 0 305 0 0 0 25 0 5 0 528099288 63287296 3906 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331188 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=61804 CPUtime=3.07 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 968 1245 0 0 307 0 0 0 25 0 5 0 528099288 63287296 3906 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331340 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.63
Current children cumulated vsize (KiB) 831692

[startup+6.3006 s]
/proc/loadavg: 7.47 7.93 7.96 9/180 12942
/proc/meminfo: memFree=31079672/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
[pid=12927] ppid=12926 vsize=36504 CPUtime=6.29 cores=0
/proc/12927/stat : 12927 (cryptominisat) R 12926 12926 12900 0 -1 4202496 7905 0 0 0 628 1 0 0 25 0 1 0 528099286 37380096 3521 33554432000 4194304 4587354 140736423621728 18446744073709551615 4238576 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12927/statm: 9126 3521 336 96 0 5688 0
[pid=12928] ppid=12926 vsize=17560 CPUtime=6.29 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 1791 0 0 0 629 0 0 0 25 0 1 0 528099286 17981440 1683 33554432000 4194304 4785740 140736038427024 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 4390 1683 341 145 0 1435 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=6.29 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 629 0 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=9828 CPUtime=6.29 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 2050 0 0 0 629 0 0 0 25 0 1 0 528099286 10063872 827 33554432000 4194304 4331988 140736659254656 18446744073709551615 4315836 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 2457 827 175 34 0 899 0
[pid=12931] ppid=12926 vsize=63456 CPUtime=25.07 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 6543 1245 0 0 2506 1 0 0 18 0 5 0 528099286 64978944 4319 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 15864 4319 161 40 0 14408 0
[pid=12931/tid=12938] ppid=12926 vsize=63456 CPUtime=6.27 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 1265 1245 0 0 627 0 0 0 25 0 5 0 528099288 64978944 4319 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331028 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=63456 CPUtime=6.26 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 1399 1245 0 0 626 0 0 0 25 0 5 0 528099288 64978944 4319 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=63456 CPUtime=6.24 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 1340 1245 0 0 624 0 0 0 25 0 5 0 528099288 64978944 4319 33554432000 4194304 4356188 140735509979088 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=63456 CPUtime=6.27 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 1354 1245 0 0 627 0 0 0 25 0 5 0 528099288 64978944 4319 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.23
Current children cumulated vsize (KiB) 602440

[startup+12.7005 s]
/proc/loadavg: 7.51 7.93 7.96 9/180 12942
/proc/meminfo: memFree=31062616/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
[pid=12927] ppid=12926 vsize=44956 CPUtime=12.68 cores=0
/proc/12927/stat : 12927 (cryptominisat) R 12926 12926 12900 0 -1 4202496 12904 0 0 0 1267 1 0 0 25 0 1 0 528099286 46034944 5390 33554432000 4194304 4587354 140736423621728 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/12927/statm: 11239 5390 336 96 0 7801 0
[pid=12928] ppid=12926 vsize=19156 CPUtime=12.69 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 2153 0 0 0 1269 0 0 0 25 0 1 0 528099286 19615744 2045 33554432000 4194304 4785740 140736038427024 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 4789 2045 341 145 0 1834 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=12.69 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 1269 0 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134530151 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=10340 CPUtime=12.69 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 2246 0 0 0 1269 0 0 0 25 0 1 0 528099286 10588160 1023 33554432000 4194304 4331988 140736659254656 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 2585 1023 175 34 0 1027 0
[pid=12931] ppid=12926 vsize=70372 CPUtime=50.63 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 10544 1245 0 0 5062 1 0 0 18 0 5 0 528099286 72060928 6048 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 17593 6048 161 40 0 16137 0
[pid=12931/tid=12938] ppid=12926 vsize=70372 CPUtime=12.67 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 2482 1245 0 0 1267 0 0 0 25 0 5 0 528099288 72060928 6048 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=70372 CPUtime=12.66 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 2487 1245 0 0 1266 0 0 0 25 0 5 0 528099288 72060928 6048 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331212 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=70372 CPUtime=12.6 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 1852 1245 0 0 1260 0 0 0 25 0 5 0 528099288 72060928 6048 33554432000 4194304 4356188 140735509979088 18446744073709551615 4332023 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=70372 CPUtime=12.67 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 2538 1245 0 0 1267 0 0 0 25 0 5 0 528099288 72060928 6048 33554432000 4194304 4356188 140735509979088 18446744073709551615 4237486 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.38
Current children cumulated vsize (KiB) 619916

[startup+25.5004 s]
/proc/loadavg: 7.62 7.93 7.96 9/180 12943
/proc/meminfo: memFree=31049232/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=0 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 522 0 0 0 0 0 0 0 15 0 1 0 528099285 18763776 390 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 3456626930072158207 0 0 17 6 0 0 0
/proc/12926/statm: 4581 390 341 23 0 91 0
[pid=12927] ppid=12926 vsize=50028 CPUtime=25.49 cores=0
/proc/12927/stat : 12927 (cryptominisat) R 12926 12926 12900 0 -1 4202496 19402 0 0 0 2546 3 0 0 25 0 1 0 528099286 51228672 6977 33554432000 4194304 4587354 140736423621728 18446744073709551615 4238658 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 35989.3
Current children cumulated vsize (KiB) 1476004

[startup+4782.3 s]
/proc/loadavg: 7.02 7.04 7.16 8/179 13084
/proc/meminfo: memFree=30366576/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=2959.08 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 523 295555 0 0 0 0 295791 117 15 0 1 0 528099285 18763776 391 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12926/statm: 4581 391 342 23 0 91 0
[pid=12928] ppid=12926 vsize=64220 CPUtime=4782.36 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 14236 0 0 0 478233 3 0 0 25 0 1 0 528099286 65761280 11982 33554432000 4194304 4785740 140736038427024 18446744073709551615 4287017 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 16055 11982 341 145 0 13100 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=4781.58 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 478156 2 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134530632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=689196 CPUtime=4782.36 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 125132 0 0 0 478143 93 0 0 25 0 1 0 528099286 705736704 123909 33554432000 4194304 4331988 140736659254656 18446744073709551615 4287517 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 172299 123909 175 34 0 170741 0
[pid=12931] ppid=12926 vsize=253128 CPUtime=19103.6 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 479360 1245 0 0 1910279 79 0 0 18 0 5 0 528099286 259203072 51737 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 63282 51737 161 40 0 61826 0
[pid=12931/tid=12938] ppid=12926 vsize=253128 CPUtime=4782.33 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 119286 1245 0 0 478216 17 0 0 25 0 5 0 528099288 259203072 51737 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331792 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=253128 CPUtime=4781.85 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 120027 1245 0 0 478168 17 0 0 25 0 5 0 528099288 259203072 51737 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331576 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=253128 CPUtime=4757.03 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 120544 1245 0 0 475681 22 0 0 25 0 5 0 528099288 259203072 51737 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=253128 CPUtime=4782.32 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 118318 1245 0 0 478212 20 0 0 25 0 5 0 528099288 259203072 51737 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331792 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 36409
Current children cumulated vsize (KiB) 1481636

[startup+4842.3 s]
/proc/loadavg: 7.04 7.04 7.15 8/179 13086
/proc/meminfo: memFree=30363576/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=2959.08 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 523 295555 0 0 0 0 295791 117 15 0 1 0 528099285 18763776 391 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12926/statm: 4581 391 342 23 0 91 0
[pid=12928] ppid=12926 vsize=64220 CPUtime=4842.36 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 14240 0 0 0 484233 3 0 0 25 0 1 0 528099286 65761280 11986 33554432000 4194304 4785740 140736038427024 18446744073709551615 4286998 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 16055 11986 341 145 0 13100 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=4841.58 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 484156 2 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134529080 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=689196 CPUtime=4842.36 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 126429 0 0 0 484142 94 0 0 25 0 1 0 528099286 705736704 125206 33554432000 4194304 4331988 140736659254656 18446744073709551615 4226642 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 172299 125206 175 34 0 170741 0
[pid=12931] ppid=12926 vsize=249544 CPUtime=19343.3 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 484736 1245 0 0 1934247 80 0 0 18 0 5 0 528099286 255533056 50841 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 62386 50841 161 40 0 60930 0
[pid=12931/tid=12938] ppid=12926 vsize=249544 CPUtime=4842.33 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 121078 1245 0 0 484215 18 0 0 25 0 5 0 528099288 255533056 50841 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=249544 CPUtime=4841.85 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 121051 1245 0 0 484167 18 0 0 25 0 5 0 528099288 255533056 50841 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=249544 CPUtime=4816.74 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 121312 1245 0 0 481651 23 0 0 25 0 5 0 528099288 255533056 50841 33554432000 4194304 4356188 140735509979088 18446744073709551615 4210562 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=249544 CPUtime=4842.33 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 120110 1245 0 0 484212 21 0 0 25 0 5 0 528099288 255533056 50841 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 36828.7
Current children cumulated vsize (KiB) 1478052

[startup+4902.3 s]
/proc/loadavg: 7.05 7.05 7.14 8/179 13088
/proc/meminfo: memFree=30353500/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=2959.08 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 523 295555 0 0 0 0 295791 117 15 0 1 0 528099285 18763776 391 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12926/statm: 4581 391 342 23 0 91 0
[pid=12928] ppid=12926 vsize=64028 CPUtime=4902.36 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 14247 0 0 0 490233 3 0 0 25 0 1 0 528099286 65564672 11945 33554432000 4194304 4785740 140736038427024 18446744073709551615 4553440 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 16007 11945 341 145 0 13052 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=4901.58 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 490156 2 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134528944 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=689196 CPUtime=4902.36 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 128031 0 0 0 490140 96 0 0 25 0 1 0 528099286 705736704 126808 33554432000 4194304 4331988 140736659254656 18446744073709551615 4239528 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 172299 126808 175 34 0 170741 0
[pid=12931] ppid=12926 vsize=255944 CPUtime=19583 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 488128 1245 0 0 1958217 81 0 0 18 0 5 0 528099286 262086656 52441 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 63986 52441 161 40 0 62530 0
[pid=12931/tid=12938] ppid=12926 vsize=255944 CPUtime=4902.33 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 121526 1245 0 0 490215 18 0 0 25 0 5 0 528099288 262086656 52441 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331992 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=255944 CPUtime=4901.84 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 122267 1245 0 0 490166 18 0 0 25 0 5 0 528099288 262086656 52441 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=255944 CPUtime=4876.44 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 122272 1245 0 0 487621 23 0 0 25 0 5 0 528099288 262086656 52441 33554432000 4194304 4356188 140735509979088 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=255944 CPUtime=4902.33 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 120878 1245 0 0 490212 21 0 0 25 0 5 0 528099288 262086656 52441 33554432000 4194304 4356188 140735509979088 18446744073709551615 4332026 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37248.4
Current children cumulated vsize (KiB) 1484260

[startup+4962.3 s]
/proc/loadavg: 7.10 7.05 7.14 8/179 13089
/proc/meminfo: memFree=30349976/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=2959.08 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 523 295555 0 0 0 0 295791 117 15 0 1 0 528099285 18763776 391 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12926/statm: 4581 391 342 23 0 91 0
[pid=12928] ppid=12926 vsize=64028 CPUtime=4962.36 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 14252 0 0 0 496233 3 0 0 25 0 1 0 528099286 65564672 11950 33554432000 4194304 4785740 140736038427024 18446744073709551615 4549596 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 16007 11950 341 145 0 13052 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=4961.58 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 496156 2 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134530598 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=698292 CPUtime=4962.36 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 129473 0 0 0 496139 97 0 0 25 0 1 0 528099286 715051008 128250 33554432000 4194304 4331988 140736659254656 18446744073709551615 210204343801 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 174573 128250 175 34 0 173015 0
[pid=12931] ppid=12926 vsize=251080 CPUtime=19822.7 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 493824 1245 0 0 1982186 82 0 0 18 0 5 0 528099286 257105920 51225 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 62770 51225 161 40 0 61314 0
[pid=12931/tid=12938] ppid=12926 vsize=251080 CPUtime=4962.33 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 123382 1245 0 0 496215 18 0 0 25 0 5 0 528099288 257105920 51225 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=251080 CPUtime=4961.83 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 123291 1245 0 0 496165 18 0 0 25 0 5 0 528099288 257105920 51225 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=251080 CPUtime=4936.14 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 124064 1245 0 0 493591 23 0 0 25 0 5 0 528099288 257105920 51225 33554432000 4194304 4356188 140735509979088 18446744073709551615 4332260 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=251080 CPUtime=4962.33 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 121902 1245 0 0 496212 21 0 0 25 0 5 0 528099288 257105920 51225 33554432000 4194304 4356188 140735509979088 18446744073709551615 4279435 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37668.1
Current children cumulated vsize (KiB) 1488492



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 7.05 7.04 7.13 8/179 13090
/proc/meminfo: memFree=30346848/32951124 swapFree=67107188/67111528
[pid=12926] ppid=12924 vsize=18324 CPUtime=2959.08 cores=0-7
/proc/12926/stat : 12926 (ppfolio) S 12924 12926 12900 0 -1 4202496 523 295555 0 0 0 0 295791 117 15 0 1 0 528099285 18763776 391 33554432000 4194304 4287625 140737360601328 18446744073709551615 210204667407 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/12926/statm: 4581 391 342 23 0 91 0
[pid=12928] ppid=12926 vsize=64028 CPUtime=5000.07 cores=1
/proc/12928/stat : 12928 (clasp) R 12926 12926 12900 0 -1 4202496 14257 0 0 0 500004 3 0 0 25 0 1 0 528099286 65564672 11955 33554432000 4194304 4785740 140736038427024 18446744073709551615 4553363 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/12928/statm: 16007 11955 341 145 0 13052 0
[pid=12929] ppid=12926 vsize=456768 CPUtime=4999.28 cores=2
/proc/12929/stat : 12929 (TNM) R 12926 12926 12900 0 -1 4202496 596 0 0 0 499926 2 0 0 25 0 1 0 528099286 467730432 493 33554432000 134512640 134534564 4293206944 18446744073709551615 134528687 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/12929/statm: 114192 493 95 6 0 112858 0
[pid=12930] ppid=12926 vsize=698292 CPUtime=5000.07 cores=3
/proc/12930/stat : 12930 (march_hi) R 12926 12926 12900 0 -1 4202496 130463 0 0 0 499909 98 0 0 25 0 1 0 528099286 715051008 129240 33554432000 4194304 4331988 140736659254656 18446744073709551615 4299299 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/12930/statm: 174573 129240 175 34 0 173015 0
[pid=12931] ppid=12926 vsize=250056 CPUtime=19973.3 cores=4-7
/proc/12931/stat : 12931 (plingeling) S 12926 12926 12900 0 -1 4202496 495872 1245 0 0 1997247 83 0 0 18 0 5 0 528099286 256057344 50969 33554432000 4194304 4356188 140735509979088 18446744073709551615 210216450021 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/12931/statm: 62514 50969 161 40 0 61058 0
[pid=12931/tid=12938] ppid=12926 vsize=250056 CPUtime=5000.03 cores=4-7
/proc/12931/task/12938/stat : 12938 (plingeling) R 12926 12926 12900 0 -1 4202560 123382 1245 0 0 499985 18 0 0 25 0 5 0 528099288 256057344 50969 33554432000 4194304 4356188 140735509979088 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=12931/tid=12939] ppid=12926 vsize=250056 CPUtime=4999.54 cores=4-7
/proc/12931/task/12939/stat : 12939 (plingeling) R 12926 12926 12900 0 -1 4202560 124315 1245 0 0 499936 18 0 0 25 0 5 0 528099288 256057344 50969 33554432000 4194304 4356188 140735509979088 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=12931/tid=12940] ppid=12926 vsize=250056 CPUtime=4973.63 cores=4-7
/proc/12931/task/12940/stat : 12940 (plingeling) R 12926 12926 12900 0 -1 4202560 124064 1245 0 0 497340 23 0 0 25 0 5 0 528099288 256057344 50969 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=12931/tid=12941] ppid=12926 vsize=250056 CPUtime=5000.03 cores=4-7
/proc/12931/task/12941/stat : 12941 (plingeling) R 12926 12926 12900 0 -1 4202560 122926 1245 0 0 499982 21 0 0 25 0 5 0 528099288 256057344 50969 33554432000 4194304 4356188 140735509979088 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37931.8
Current children cumulated vsize (KiB) 1487468

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 12926 and gives
#  childrusage.ru_utime.tv_sec=12957
#  childrusage.ru_utime.tv_usec=227202
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=246810
# CPU time returned by wait4() is 12958.5
# while last known CPU time is 37931.8
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 37931.8
CPU user time (s): 37928.8
CPU system time (s): 3.03
CPU usage (%): 758.633
Max. virtual memory (cumulated for all children) (KiB): 17338140

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 12957.2
system time used= 1.24681
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 310945
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 765
involuntary context switches= 27206

runsolver used 8.00778 second user time and 18.2322 second system time

The end

Launcher Data

Begin job on node130 at 2011-04-30 14:03:28
IDJOB=3222225
IDBENCH=83054
IDSOLVER=1638
FILE ID=node130/3222225-1304165008
RUNJOBID= node130-1304165008-12914
PBS_JOBID= 13170945
Free space on /tmp= 72700 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k7-r89-v90-c8010-S1897189325-062.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3222225-1304165008/watcher-3222225-1304165008 -o /tmp/evaluation-result-3222225-1304165008/solver-3222225-1304165008 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3222225-1304165008.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 28731ba8f9c5274ff0cd070fd7aeff45
RANDOM SEED=1878092943

node130.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.838
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      31118080 kB
Buffers:        197208 kB
Cached:        1296256 kB
SwapCached:       1788 kB
Active:         360096 kB
Inactive:      1194828 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      31118080 kB
SwapTotal:    67111528 kB
SwapFree:     67107188 kB
Dirty:           10108 kB
Writeback:           0 kB
AnonPages:       60728 kB
Mapped:          14412 kB
Slab:           213140 kB
PageTables:       4616 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   183976 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72692 MiB
End job on node130 at 2011-04-30 15:26:50