Trace number 3220347

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 39971.4 5000.02

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v150-c3195-S873866006-055.UNKNOWN.cnf
MD5SUM9ec6e0c9f7f3592c1fc9e4fe986eba4f
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables150
Number of clauses3195
Sum of the clauses size15975
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 53195
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3220347-1304157429/watcher-3220347-1304157429 -o /tmp/evaluation-result-3220347-1304157429/solver-3220347-1304157429 -C 40000 -W 5000 -M 31000 ppfolio HOME/instance-3220347-1304157429.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.84 7.80 7.92 2/170 13797
/proc/meminfo: memFree=20827020/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 22 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0

[startup+0.818191 s]
/proc/loadavg: 6.84 7.80 7.92 2/170 13797
/proc/meminfo: memFree=20827020/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 17 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.900175 s]
/proc/loadavg: 6.84 7.80 7.92 2/170 13797
/proc/meminfo: memFree=20827020/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.00016 s]
/proc/loadavg: 6.84 7.80 7.92 2/170 13797
/proc/meminfo: memFree=20827020/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.10014 s]
/proc/loadavg: 6.84 7.80 7.92 2/170 13797
/proc/meminfo: memFree=20827020/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50007 s]
/proc/loadavg: 6.84 7.80 7.92 2/170 13797
/proc/meminfo: memFree=20827020/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+3.10078 s]
/proc/loadavg: 6.84 7.80 7.92 9/180 13812
/proc/meminfo: memFree=20794356/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=627136 CPUtime=3.08 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 6326 0 0 0 308 0 0 0 25 0 1 0 527342633 642187264 3768 33554432000 4194304 4587354 140736074696064 18446744073709551615 4364688 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 156784 3768 337 96 0 153346 0
[pid=13799] ppid=13797 vsize=15036 CPUtime=3.09 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 1220 0 0 0 309 0 0 0 25 0 1 0 527342633 15396864 1111 33554432000 4194304 4785740 140733468018432 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 3759 1111 340 145 0 804 0
[pid=13800] ppid=13797 vsize=456088 CPUtime=3.09 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 387 0 0 0 309 0 0 0 25 0 1 0 527342633 467034112 284 33554432000 134512640 134534564 4293006704 18446744073709551615 134528726 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114022 284 95 6 0 112688 0
[pid=13801] ppid=13797 vsize=8436 CPUtime=3.09 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 691 0 0 0 309 0 0 0 25 0 1 0 527342633 8638464 576 33554432000 4194304 4331988 140734382647200 18446744073709551615 4316635 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 2109 576 175 34 0 551 0
[pid=13802] ppid=13797 vsize=126940 CPUtime=12.33 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 5417 1246 0 0 1232 1 0 0 24 0 5 0 527342633 129986560 3808 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 31735 3808 162 40 0 30279 0
[pid=13802/tid=13809] ppid=13797 vsize=126940 CPUtime=3.06 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 1141 1246 0 0 306 0 0 0 25 0 5 0 527342634 129986560 3808 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331984 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=126940 CPUtime=3.08 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 1276 1246 0 0 308 0 0 0 25 0 5 0 527342634 129986560 3808 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=126940 CPUtime=3.08 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 1106 1246 0 0 308 0 0 0 25 0 5 0 527342634 129986560 3808 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331328 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=126940 CPUtime=3.08 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 1284 1246 0 0 308 0 0 0 25 0 5 0 527342634 129986560 3808 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331050 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.68
Current children cumulated vsize (KiB) 1251960

[startup+6.30621 s]
/proc/loadavg: 6.93 7.80 7.92 9/180 13812
/proc/meminfo: memFree=20788048/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=627392 CPUtime=6.29 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 8610 0 0 0 628 1 0 0 25 0 1 0 527342633 642449408 4097 33554432000 4194304 4587354 140736074696064 18446744073709551615 4238611 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 156848 4097 337 96 0 153410 0
[pid=13799] ppid=13797 vsize=16128 CPUtime=6.3 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 1479 0 0 0 630 0 0 0 25 0 1 0 527342633 16515072 1370 33554432000 4194304 4785740 140733468018432 18446744073709551615 4552910 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 4032 1370 340 145 0 1077 0
[pid=13800] ppid=13797 vsize=456088 CPUtime=6.29 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 387 0 0 0 629 0 0 0 25 0 1 0 527342633 467034112 284 33554432000 134512640 134534564 4293006704 18446744073709551615 134528396 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114022 284 95 6 0 112688 0
[pid=13801] ppid=13797 vsize=8948 CPUtime=6.3 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 880 0 0 0 630 0 0 0 25 0 1 0 527342633 9162752 765 33554432000 4194304 4331988 140734382647200 18446744073709551615 4239528 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 2237 765 175 34 0 679 0
[pid=13802] ppid=13797 vsize=128004 CPUtime=25.13 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 7027 1246 0 0 2512 1 0 0 24 0 5 0 527342633 131076096 4074 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 32001 4074 162 40 0 30545 0
[pid=13802/tid=13809] ppid=13797 vsize=128004 CPUtime=6.25 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 1509 1246 0 0 625 0 0 0 25 0 5 0 527342634 131076096 4074 33554432000 4194304 4356188 140733775029936 18446744073709551615 4332041 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=128004 CPUtime=6.28 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 1668 1246 0 0 628 0 0 0 25 0 5 0 527342634 131076096 4074 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=128004 CPUtime=6.29 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 1539 1246 0 0 629 0 0 0 25 0 5 0 527342634 131076096 4074 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=128004 CPUtime=6.28 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 1701 1246 0 0 628 0 0 0 25 0 5 0 527342634 131076096 4074 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.31
Current children cumulated vsize (KiB) 1254884

[startup+12.7011 s]
/proc/loadavg: 7.01 7.81 7.92 9/180 13812
/proc/meminfo: memFree=20783364/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=630840 CPUtime=12.69 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 13608 0 0 0 1267 2 0 0 25 0 1 0 527342633 645980160 4854 33554432000 4194304 4587354 140736074696064 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 157710 4854 337 96 0 154272 0
[pid=13799] ppid=13797 vsize=17260 CPUtime=12.69 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 1737 0 0 0 1269 0 0 0 25 0 1 0 527342633 17674240 1628 33554432000 4194304 4785740 140733468018432 18446744073709551615 4553375 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 4315 1628 340 145 0 1360 0
[pid=13800] ppid=13797 vsize=456088 CPUtime=12.69 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 387 0 0 0 1269 0 0 0 25 0 1 0 527342633 467034112 284 33554432000 134512640 134534564 4293006704 18446744073709551615 134528991 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114022 284 95 6 0 112688 0
[pid=13801] ppid=13797 vsize=10864 CPUtime=12.69 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 1212 0 0 0 1269 0 0 0 25 0 1 0 527342633 11124736 1097 33554432000 4194304 4331988 140734382647200 18446744073709551615 4299360 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 2716 1097 175 34 0 1158 0
[pid=13802] ppid=13797 vsize=132368 CPUtime=50.68 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 10326 1246 0 0 5066 2 0 0 24 0 5 0 527342633 135544832 5165 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 33092 5165 162 40 0 31636 0
[pid=13802/tid=13809] ppid=13797 vsize=132368 CPUtime=12.62 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 2117 1246 0 0 1262 0 0 0 25 0 5 0 527342634 135544832 5165 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=132368 CPUtime=12.68 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 2341 1246 0 0 1268 0 0 0 25 0 5 0 527342634 135544832 5165 33554432000 4194304 4356188 140733775029936 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=132368 CPUtime=12.68 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 2820 1246 0 0 1268 0 0 0 25 0 5 0 527342634 135544832 5165 33554432000 4194304 4356188 140733775029936 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=132368 CPUtime=12.68 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 2438 1246 0 0 1268 0 0 0 25 0 5 0 527342634 135544832 5165 33554432000 4194304 4356188 140733775029936 18446744073709551615 4238402 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.44
Current children cumulated vsize (KiB) 1265744

[startup+25.5008 s]
/proc/loadavg: 7.23 7.81 7.92 9/180 13812
/proc/meminfo: memFree=20767944/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=636724 CPUtime=25.49 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 21493 0 0 0 2546 3 0 0 25 0 1 0 527342633 652005376 6054 33554432000 4194304 4587354 140736074696064 18446744073709551615 4238392 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 159181 6054 337 96 0 155743 0
[pid=13799] ppid=13797 vsize=19052 CPUtime=25.49 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 2160 0 0 0 2549 0 0 0 25 0 1 0 527342633 19509248 2051 33554432000 4194304 4785740 140733468018432 18446744073709551615 4287124 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 4763 2051 340 145 0 1808 0
[pid=13800] ppid=13797 vsize=456088 CPUtime=25.49 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 387 0 0 0 2549 0 0 0 25 0 1 0 527342633 467034112 284 33554432000 134512640 134534564 4293006704 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114022 284 95 6 0 112688 0
[pid=13801] ppid=13797 vsize=15416 CPUtime=25.48 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 1953 0 0 0 2548 0 0 0 25 0 1 0 527342633 15785984 1838 33554432000 4194304 4331988 140734382647200 18446744073709551615 4239602 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 3854 1838 175 34 0 2296 0
[pid=13802] ppid=13797 vsize=136740 CPUtime=101.81 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 17019 1246 0 0 10179 2 0 0 24 0 5 0 527342633 140021760 6258 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 34185 6258 162 40 0 32729 0
[pid=13802/tid=13809] ppid=13797 vsize=136740 CPUtime=25.34 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 3911 1246 0 0 2534 0 0 0 25 0 5 0 527342634 140021760 6258 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331268 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=136740 CPUtime=25.48 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 4327 1246 0 0 2547 1 0 0 25 0 5 0 527342634 140021760 6258 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=136740 CPUtime=25.48 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 4037 1246 0 0 2548 0 0 0 25 0 5 0 527342634 140021760 6258 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=136740 CPUtime=25.47 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 4134 1246 0 0 2547 0 0 0 25 0 5 0 527342634 140021760 6258 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331992 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.76
Current children cumulated vsize (KiB) 1282344

################
# More data... #
################

[pid=13800] ppid=13797 vsize=456092 CPUtime=4781.94 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 396 0 0 0 478192 2 0 0 25 0 1 0 527342633 467038208 293 33554432000 134512640 134534564 4293006704 18446744073709551615 134529632 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114023 293 103 6 0 112689 0
[pid=13801] ppid=13797 vsize=1075680 CPUtime=4780.78 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 199553 0 0 0 477931 147 0 0 25 0 1 0 527342633 1101496320 199438 33554432000 4194304 4331988 140734382647200 18446744073709551615 4239578 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 268920 199438 175 34 0 267362 0
[pid=13802] ppid=13797 vsize=250892 CPUtime=19103.6 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 736773 1246 0 0 1910247 108 0 0 24 0 5 0 527342633 256913408 34796 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 62723 34796 162 40 0 61267 0
[pid=13802/tid=13809] ppid=13797 vsize=250892 CPUtime=4756.32 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 191114 1246 0 0 475604 28 0 0 25 0 5 0 527342634 256913408 34796 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=250892 CPUtime=4782.4 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 184969 1246 0 0 478213 27 0 0 25 0 5 0 527342634 256913408 34796 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=250892 CPUtime=4782.4 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 184135 1246 0 0 478216 24 0 0 25 0 5 0 527342634 256913408 34796 33554432000 4194304 4356188 140733775029936 18446744073709551615 4332003 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=250892 CPUtime=4782.4 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 175945 1246 0 0 478212 28 0 0 25 0 5 0 527342634 256913408 34796 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331305 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38230.9
Current children cumulated vsize (KiB) 6144796

[startup+4842.3 s]
/proc/loadavg: 8.07 8.07 8.02 9/180 13965
/proc/meminfo: memFree=19681700/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=4297332 CPUtime=4842.4 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 368306 0 0 0 484089 151 0 0 25 0 1 0 527342633 4400467968 38360 33554432000 4194304 4587354 140736074696064 18446744073709551615 4241080 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 1074333 38360 337 96 0 1070895 0
[pid=13799] ppid=13797 vsize=46476 CPUtime=4842.25 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 8280 0 0 0 484224 1 0 0 25 0 1 0 527342633 47591424 8171 33554432000 4194304 4785740 140733468018432 18446744073709551615 4287194 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 11619 8171 341 145 0 8664 0
[pid=13800] ppid=13797 vsize=456092 CPUtime=4841.93 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 396 0 0 0 484191 2 0 0 25 0 1 0 527342633 467038208 293 33554432000 134512640 134534564 4293006704 18446744073709551615 134529775 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114023 293 103 6 0 112689 0
[pid=13801] ppid=13797 vsize=1075680 CPUtime=4840.79 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 201971 0 0 0 483930 149 0 0 25 0 1 0 527342633 1101496320 201856 33554432000 4194304 4331988 140734382647200 18446744073709551615 4299282 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 268920 201856 175 34 0 267362 0
[pid=13802] ppid=13797 vsize=268812 CPUtime=19343.2 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 744965 1246 0 0 1934213 109 0 0 24 0 5 0 527342633 275263488 39276 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 67203 39276 162 40 0 65747 0
[pid=13802/tid=13809] ppid=13797 vsize=268812 CPUtime=4816 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 191498 1246 0 0 481572 28 0 0 25 0 5 0 527342634 275263488 39276 33554432000 4194304 4356188 140733775029936 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=268812 CPUtime=4842.4 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 186185 1246 0 0 484213 27 0 0 25 0 5 0 527342634 275263488 39276 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331183 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=268812 CPUtime=4842.4 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 190087 1246 0 0 484215 25 0 0 25 0 5 0 527342634 275263488 39276 33554432000 4194304 4356188 140733775029936 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=268812 CPUtime=4842.4 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 176585 1246 0 0 484212 28 0 0 25 0 5 0 527342634 275263488 39276 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38710.6
Current children cumulated vsize (KiB) 6162716

[startup+4902.3 s]
/proc/loadavg: 8.06 8.06 8.02 9/180 13967
/proc/meminfo: memFree=19675864/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=4297332 CPUtime=4902.41 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 369637 0 0 0 490089 152 0 0 25 0 1 0 527342633 4400467968 37319 33554432000 4194304 4587354 140736074696064 18446744073709551615 4238406 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 1074333 37319 337 96 0 1070895 0
[pid=13799] ppid=13797 vsize=46476 CPUtime=4902.26 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 8280 0 0 0 490225 1 0 0 25 0 1 0 527342633 47591424 8171 33554432000 4194304 4785740 140733468018432 18446744073709551615 4550848 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 11619 8171 341 145 0 8664 0
[pid=13800] ppid=13797 vsize=456092 CPUtime=4901.93 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 396 0 0 0 490191 2 0 0 25 0 1 0 527342633 467038208 293 33554432000 134512640 134534564 4293006704 18446744073709551615 134528295 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114023 293 103 6 0 112689 0
[pid=13801] ppid=13797 vsize=1075680 CPUtime=4900.79 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 204608 0 0 0 489928 151 0 0 25 0 1 0 527342633 1101496320 204493 33554432000 4194304 4331988 140734382647200 18446744073709551615 4316920 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 268920 204493 175 34 0 267362 0
[pid=13802] ppid=13797 vsize=265996 CPUtime=19582.9 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 756165 1246 0 0 1958181 111 0 0 24 0 5 0 527342633 272379904 38572 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 66499 38572 162 40 0 65043 0
[pid=13802/tid=13809] ppid=13797 vsize=265996 CPUtime=4875.68 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 197066 1246 0 0 487539 29 0 0 25 0 5 0 527342634 272379904 38572 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=265996 CPUtime=4902.41 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 186825 1246 0 0 490214 27 0 0 25 0 5 0 527342634 272379904 38572 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=265996 CPUtime=4902.41 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 190087 1246 0 0 490216 25 0 0 25 0 5 0 527342634 272379904 38572 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=265996 CPUtime=4902.4 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 181577 1246 0 0 490211 29 0 0 25 0 5 0 527342634 272379904 38572 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39190.3
Current children cumulated vsize (KiB) 6159900

[startup+4962.3 s]
/proc/loadavg: 8.06 8.06 8.02 9/180 13968
/proc/meminfo: memFree=19686508/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=752856 CPUtime=4962.4 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 371720 0 0 0 496087 153 0 0 25 0 1 0 527342633 770924544 37008 33554432000 4194304 4587354 140736074696064 18446744073709551615 4238955 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 188214 37008 337 96 0 184776 0
[pid=13799] ppid=13797 vsize=46476 CPUtime=4962.25 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 8281 0 0 0 496224 1 0 0 25 0 1 0 527342633 47591424 8172 33554432000 4194304 4785740 140733468018432 18446744073709551615 4287204 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 11619 8172 341 145 0 8664 0
[pid=13800] ppid=13797 vsize=456092 CPUtime=4961.92 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 396 0 0 0 496190 2 0 0 25 0 1 0 527342633 467038208 293 33554432000 134512640 134534564 4293006704 18446744073709551615 134530568 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114023 293 103 6 0 112689 0
[pid=13801] ppid=13797 vsize=1075680 CPUtime=4960.79 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 206982 0 0 0 495925 154 0 0 25 0 1 0 527342633 1101496320 206867 33554432000 4194304 4331988 140734382647200 18446744073709551615 4287504 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 268920 206867 175 34 0 267362 0
[pid=13802] ppid=13797 vsize=247052 CPUtime=19822.6 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 759237 1246 0 0 1982147 112 0 0 24 0 5 0 527342633 252981248 33836 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 61763 33836 162 40 0 60307 0
[pid=13802/tid=13809] ppid=13797 vsize=247052 CPUtime=4935.35 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 197066 1246 0 0 493506 29 0 0 25 0 5 0 527342634 252981248 33836 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=247052 CPUtime=4962.4 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 187721 1246 0 0 496213 27 0 0 25 0 5 0 527342634 252981248 33836 33554432000 4194304 4356188 140733775029936 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=247052 CPUtime=4962.4 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 191303 1246 0 0 496215 25 0 0 25 0 5 0 527342634 252981248 33836 33554432000 4194304 4356188 140733775029936 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=247052 CPUtime=4962.4 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 182537 1246 0 0 496211 29 0 0 25 0 5 0 527342634 252981248 33836 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39669.9
Current children cumulated vsize (KiB) 2596480



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5000 s]
/proc/loadavg: 8.03 8.05 8.01 9/180 13969
/proc/meminfo: memFree=19652148/32950904 swapFree=67082284/67111528
[pid=13797] ppid=13795 vsize=18324 CPUtime=0 cores=0-7
/proc/13797/stat : 13797 (ppfolio) S 13795 13797 13771 0 -1 4202496 519 0 0 0 0 0 0 0 15 0 1 0 527342633 18763776 389 33554432000 4194304 4287625 140736897084288 18446744073709551615 237278899727 0 0 4096 65536 4422201411471671295 0 0 17 6 0 0 0
/proc/13797/statm: 4581 389 341 23 0 91 0
[pid=13798] ppid=13797 vsize=752856 CPUtime=5000.11 cores=0
/proc/13798/stat : 13798 (cryptominisat) R 13797 13797 13771 0 -1 4202496 372418 0 0 0 499857 154 0 0 25 0 1 0 527342633 770924544 37689 33554432000 4194304 4587354 140736074696064 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/13798/statm: 188214 37689 337 96 0 184776 0
[pid=13799] ppid=13797 vsize=46728 CPUtime=4999.95 cores=1
/proc/13799/stat : 13799 (clasp) R 13797 13797 13771 0 -1 4202496 8304 0 0 0 499994 1 0 0 25 0 1 0 527342633 47849472 8195 33554432000 4194304 4785740 140733468018432 18446744073709551615 4287241 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/13799/statm: 11682 8195 341 145 0 8727 0
[pid=13800] ppid=13797 vsize=456092 CPUtime=4999.62 cores=2
/proc/13800/stat : 13800 (TNM) R 13797 13797 13771 0 -1 4202496 396 0 0 0 499960 2 0 0 25 0 1 0 527342633 467038208 293 33554432000 134512640 134534564 4293006704 18446744073709551615 134530693 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/13800/statm: 114023 293 103 6 0 112689 0
[pid=13801] ppid=13797 vsize=1075680 CPUtime=4998.5 cores=3
/proc/13801/stat : 13801 (march_hi) R 13797 13797 13771 0 -1 4202496 208581 0 0 0 499695 155 0 0 25 0 1 0 527342633 1101496320 208466 33554432000 4194304 4331988 140734382647200 18446744073709551615 4210430 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/13801/statm: 268920 208466 175 34 0 267362 0
[pid=13802] ppid=13797 vsize=273420 CPUtime=19973.2 cores=4-7
/proc/13802/stat : 13802 (plingeling) S 13797 13797 13771 0 -1 4202496 765829 1246 0 0 1997207 113 0 0 24 0 5 0 527342633 279982080 40428 33554432000 4194304 4356188 140733775029936 18446744073709551615 237290682341 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/13802/statm: 68355 40428 162 40 0 66899 0
[pid=13802/tid=13809] ppid=13797 vsize=273420 CPUtime=4972.84 cores=4-7
/proc/13802/task/13809/stat : 13809 (plingeling) R 13797 13797 13771 0 -1 4202560 197962 1246 0 0 497255 29 0 0 25 0 5 0 527342634 279982080 40428 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=13802/tid=13810] ppid=13797 vsize=273420 CPUtime=5000.11 cores=4-7
/proc/13802/task/13810/stat : 13810 (plingeling) R 13797 13797 13771 0 -1 4202560 192777 1246 0 0 499983 28 0 0 25 0 5 0 527342634 279982080 40428 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=13802/tid=13811] ppid=13797 vsize=273420 CPUtime=5000.12 cores=4-7
/proc/13802/task/13811/stat : 13811 (plingeling) R 13797 13797 13771 0 -1 4202560 191943 1246 0 0 499986 26 0 0 25 0 5 0 527342634 279982080 40428 33554432000 4194304 4356188 140733775029936 18446744073709551615 4332003 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=13802/tid=13812] ppid=13797 vsize=273420 CPUtime=5000.1 cores=4-7
/proc/13802/task/13812/stat : 13812 (plingeling) R 13797 13797 13771 0 -1 4202560 182537 1246 0 0 499981 29 0 0 25 0 5 0 527342634 279982080 40428 33554432000 4194304 4356188 140733775029936 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39971.4
Current children cumulated vsize (KiB) 2623100

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 13797 and gives
#  childrusage.ru_utime.tv_sec=14998
#  childrusage.ru_utime.tv_usec=136936
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=602756
# CPU time returned by wait4() is 14999.7
# while last known CPU time is 39971.4
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5000.02
CPU time (s): 39971.4
CPU user time (s): 39967.1
CPU system time (s): 4.25
CPU usage (%): 799.425
Max. virtual memory (cumulated for all children) (KiB): 10283208

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 14998.1
system time used= 1.60276
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 381652
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 809
involuntary context switches= 41363

runsolver used 8.36273 second user time and 18.9111 second system time

The end

Launcher Data 

Begin job on node121 at 2011-04-30 11:57:09
IDJOB=3220347
IDBENCH=82741
IDSOLVER=1638
FILE ID=node121/3220347-1304157429
RUNJOBID= node121-1304157429-13785
PBS_JOBID= 13170911
Free space on /tmp= 73564 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v150-c3195-S873866006-055.UNKNOWN.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3220347-1304157429/watcher-3220347-1304157429 -o /tmp/evaluation-result-3220347-1304157429/solver-3220347-1304157429 -C 40000 -W 5000 -M 31000  ppfolio HOME/instance-3220347-1304157429.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9ec6e0c9f7f3592c1fc9e4fe986eba4f
RANDOM SEED=461905912

node121.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.817
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950904 kB
MemFree:      20827300 kB
Buffers:       2187864 kB
Cached:        9383008 kB
SwapCached:       2776 kB
Active:        8606368 kB
Inactive:      3002700 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950904 kB
LowFree:      20827300 kB
SwapTotal:    67111528 kB
SwapFree:     67082284 kB
Dirty:           10228 kB
Writeback:           0 kB
AnonPages:       36240 kB
Mapped:          13140 kB
Slab:           446196 kB
PageTables:       4924 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586980 kB
Committed_AS:   185944 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73552 MiB
End job on node121 at 2011-04-30 13:20:31