Trace number 3310324

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 35672.6 5100.02

General information on the benchmark

Nameapplication/fuhs/slp-synthesis-AES/
slp-synthesis-aes-bottom21.cnf
MD5SUMe6407843d1fbd00b68d4ad600a25dfb2
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables48639
Number of clauses162901
Sum of the clauses size415741
Maximum clause length1542
Minimum clause length1
Number of clauses of size 11
Number of clauses of size 283161
Number of clauses of size 378900
Number of clauses of size 4230
Number of clauses of size 519
Number of clauses of size over 5590

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3310324-1304011500/watcher-3310324-1304011500 -o /tmp/evaluation-result-3310324-1304011500/solver-3310324-1304011500 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3310324-1304011500.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.12 8.07 8.01 1/170 16335
/proc/meminfo: memFree=12809852/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0 cores=0-7
/proc/16335/stat : 16335 (ppfolio) R 16333 16335 16061 0 -1 4202496 518 0 0 0 0 0 0 0 25 0 1 0 591380730 18763776 389 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 0 0 0 17 6 0 0 0
/proc/16335/statm: 4581 389 341 23 0 91 0

[startup+0.0439861 s]
/proc/loadavg: 8.12 8.07 8.01 1/170 16335
/proc/meminfo: memFree=12809852/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 25 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195386380386304 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.10098 s]
/proc/loadavg: 8.12 8.07 8.01 1/170 16335
/proc/meminfo: memFree=12809852/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 25 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.300886 s]
/proc/loadavg: 8.12 8.07 8.01 1/170 16335
/proc/meminfo: memFree=12809852/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 25 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+0.700811 s]
/proc/loadavg: 8.12 8.07 8.01 1/170 16335
/proc/meminfo: memFree=12809852/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 25 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 18324

[startup+1.50066 s]
/proc/loadavg: 8.11 8.07 8.01 8/179 16351
/proc/meminfo: memFree=12674124/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195738567704576 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=65360 CPUtime=1.49 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 15230 0 0 0 147 2 0 0 20 0 1 0 591380731 66928640 11315 33554432000 4194304 4587354 140733770771664 18446744073709551615 256415056815 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 16340 11315 317 96 0 12902 0
[pid=16337] ppid=16335 vsize=40516 CPUtime=1.49 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 9484 0 0 0 148 1 0 0 20 0 1 0 591380731 41488384 7315 33554432000 4194304 4785740 140734643542480 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 10129 7315 340 145 0 7174 0
[pid=16339] ppid=16335 vsize=107404 CPUtime=1.48 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 57892 0 0 0 140 8 0 0 20 0 1 0 591380731 109981696 12771 33554432000 4194304 4331988 140735271879216 18446744073709551615 4226496 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 26851 12771 201 34 0 25293 0
[pid=16340] ppid=16335 vsize=171684 CPUtime=5.34 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 22514 1251 0 0 531 3 0 0 18 0 5 0 591380731 175804416 14991 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 42921 14991 164 40 0 41465 0
[pid=16340/tid=16348] ppid=16335 vsize=171684 CPUtime=1.28 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 3814 1251 0 0 128 0 0 0 19 0 5 0 591380752 175804416 14991 33554432000 4194304 4356188 140735132377712 18446744073709551615 4276888 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=171684 CPUtime=1.28 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 4381 1251 0 0 128 0 0 0 25 0 5 0 591380752 175804416 14991 33554432000 4194304 4356188 140735132377712 18446744073709551615 4232153 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=171684 CPUtime=1.28 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 2920 1251 0 0 128 0 0 0 19 0 5 0 591380752 175804416 14991 33554432000 4194304 4356188 140735132377712 18446744073709551615 4305961 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=171684 CPUtime=1.27 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 2775 1251 0 0 127 0 0 0 22 0 5 0 591380752 175804416 14991 33554432000 4194304 4356188 140735132377712 18446744073709551615 4236874 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 9.81
Current children cumulated vsize (KiB) 403288

[startup+3.10135 s]
/proc/loadavg: 8.11 8.07 8.01 8/179 16351
/proc/meminfo: memFree=12610016/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=69704 CPUtime=3.09 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 16488 0 0 0 307 2 0 0 25 0 1 0 591380731 71376896 12553 33554432000 4194304 4587354 140733770771664 18446744073709551615 4239583 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 17426 12553 330 96 0 13988 0
[pid=16337] ppid=16335 vsize=41048 CPUtime=3.09 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 9671 0 0 0 308 1 0 0 25 0 1 0 591380731 42033152 7502 33554432000 4194304 4785740 140734643542480 18446744073709551615 4552993 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 10262 7502 340 145 0 7307 0
[pid=16339] ppid=16335 vsize=115416 CPUtime=3.09 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 60237 0 0 0 300 9 0 0 25 0 1 0 591380731 118185984 15090 33554432000 4194304 4331988 140735271879216 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 28854 15090 175 34 0 27296 0
[pid=16340] ppid=16335 vsize=174292 CPUtime=11.73 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 23167 1251 0 0 1170 3 0 0 18 0 5 0 591380731 178475008 15643 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 43573 15643 164 40 0 42117 0
[pid=16340/tid=16348] ppid=16335 vsize=174292 CPUtime=2.88 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 3814 1251 0 0 288 0 0 0 24 0 5 0 591380752 178475008 15643 33554432000 4194304 4356188 140735132377712 18446744073709551615 4211827 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=174292 CPUtime=2.88 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 4381 1251 0 0 288 0 0 0 25 0 5 0 591380752 178475008 15643 33554432000 4194304 4356188 140735132377712 18446744073709551615 4211749 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=174292 CPUtime=2.88 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 3247 1251 0 0 288 0 0 0 24 0 5 0 591380752 178475008 15643 33554432000 4194304 4356188 140735132377712 18446744073709551615 4302272 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=174292 CPUtime=2.86 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 3101 1251 0 0 286 0 0 0 25 0 5 0 591380752 178475008 15643 33554432000 4194304 4356188 140735132377712 18446744073709551615 4210544 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 21.01
Current children cumulated vsize (KiB) 418784

[startup+6.30075 s]
/proc/loadavg: 8.02 8.05 8.00 8/179 16351
/proc/meminfo: memFree=12611084/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=782160 CPUtime=6.29 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 18963 0 0 0 627 2 0 0 25 0 1 0 591380731 800931840 12282 33554432000 4194304 4587354 140733770771664 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 195540 12282 333 96 0 192102 0
[pid=16337] ppid=16335 vsize=48132 CPUtime=6.29 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 11417 0 0 0 627 2 0 0 25 0 1 0 591380731 49287168 9248 33554432000 4194304 4785740 140734643542480 18446744073709551615 256415052485 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 12033 9248 340 145 0 9078 0
[pid=16339] ppid=16335 vsize=119904 CPUtime=6.29 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 60951 0 0 0 620 9 0 0 25 0 1 0 591380731 122781696 15804 33554432000 4194304 4331988 140735271879216 18446744073709551615 4299068 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 29976 15804 175 34 0 28418 0
[pid=16340] ppid=16335 vsize=161776 CPUtime=24.51 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 23167 1251 0 0 2448 3 0 0 18 0 5 0 591380731 165658624 12514 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 40444 12514 164 40 0 38988 0
[pid=16340/tid=16348] ppid=16335 vsize=161776 CPUtime=6.08 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 3814 1251 0 0 608 0 0 0 25 0 5 0 591380752 165658624 12514 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330845 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=161776 CPUtime=6.08 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 4381 1251 0 0 608 0 0 0 25 0 5 0 591380752 165658624 12514 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=161776 CPUtime=6.08 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 3247 1251 0 0 608 0 0 0 25 0 5 0 591380752 165658624 12514 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330827 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=161776 CPUtime=6.04 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 3101 1251 0 0 604 0 0 0 25 0 5 0 591380752 165658624 12514 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331264 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 43.39
Current children cumulated vsize (KiB) 1130296

[startup+12.7005 s]
/proc/loadavg: 7.94 8.03 8.00 8/179 16351
/proc/meminfo: memFree=12580720/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=798536 CPUtime=12.68 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 25220 0 0 0 1265 3 0 0 25 0 1 0 591380731 817700864 15726 33554432000 4194304 4587354 140733770771664 18446744073709551615 256415085544 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 199634 15726 336 96 0 196196 0
[pid=16337] ppid=16335 vsize=55012 CPUtime=12.69 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 13119 0 0 0 1266 3 0 0 25 0 1 0 591380731 56332288 10950 33554432000 4194304 4785740 140734643542480 18446744073709551615 4552628 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 13753 10950 340 145 0 10798 0
[pid=16339] ppid=16335 vsize=120880 CPUtime=12.68 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 61218 0 0 0 1259 9 0 0 25 0 1 0 591380731 123781120 16071 33554432000 4194304 4331988 140735271879216 18446744073709551615 4299264 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 30220 16071 175 34 0 28662 0
[pid=16340] ppid=16335 vsize=168420 CPUtime=50.07 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 24836 1251 0 0 5004 3 0 0 18 0 5 0 591380731 172462080 14175 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 42105 14175 164 40 0 40649 0
[pid=16340/tid=16348] ppid=16335 vsize=168420 CPUtime=12.47 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 4839 1251 0 0 1247 0 0 0 25 0 5 0 591380752 172462080 14175 33554432000 4194304 4356188 140735132377712 18446744073709551615 4234169 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=168420 CPUtime=12.47 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 4381 1251 0 0 1247 0 0 0 25 0 5 0 591380752 172462080 14175 33554432000 4194304 4356188 140735132377712 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=168420 CPUtime=12.48 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 3247 1251 0 0 1248 0 0 0 25 0 5 0 591380752 172462080 14175 33554432000 4194304 4356188 140735132377712 18446744073709551615 4332003 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=168420 CPUtime=12.41 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 3745 1251 0 0 1241 0 0 0 25 0 5 0 591380752 172462080 14175 33554432000 4194304 4356188 140735132377712 18446744073709551615 4211010 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 88.13
Current children cumulated vsize (KiB) 1161172

[startup+25.5011 s]
/proc/loadavg: 7.80 8.00 7.98 8/179 16351
/proc/meminfo: memFree=12555328/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=816512 CPUtime=25.49 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 30996 0 0 0 2544 5 0 0 25 0 1 0 591380731 836108288 19029 33554432000 4194304 4587354 140733770771664 18446744073709551615 4238616 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 204128 19029 336 96 0 200690 0
[pid=16337] ppid=16335 vsize=64432 CPUtime=25.49 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 15460 0 0 0 2545 4 0 0 25 0 1 0 591380731 65978368 13291 33554432000 4194304 4785740 140734643542480 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 16108 13291 340 145 0 13153 0
[pid=16339] ppid=16335 vsize=122096 CPUtime=25.49 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 61539 0 0 0 2539 10 0 0 25 0 1 0 591380731 125026304 16392 33554432000 4194304 4331988 140735271879216 18446744073709551615 4299872 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 30524 16392 175 34 0 28966 0
[pid=16340] ppid=16335 vsize=171696 CPUtime=101.2 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 25658 1251 0 0 10117 3 0 0 18 0 5 0 591380731 175816704 14994 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 42924 14994 164 40 0 41468 0
[pid=16340/tid=16348] ppid=16335 vsize=171696 CPUtime=25.28 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 4839 1251 0 0 2528 0 0 0 25 0 5 0 591380752 175816704 14994 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330982 0 0 4096 17442 0 0 0 -1 4 0 0 0

################
# More data... #
################

Current children cumulated CPU time (s) 33870
Current children cumulated vsize (KiB) 1919332

[startup+4902.3 s]
/proc/loadavg: 7.00 7.01 7.00 8/179 16496
/proc/meminfo: memFree=11889928/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639361 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=893412 CPUtime=4902.47 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 816889 0 0 0 489943 304 0 0 25 0 1 0 591380731 914853888 44310 33554432000 4194304 4587354 140733770771664 18446744073709551615 4238618 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 223353 44310 336 96 0 219915 0
[pid=16337] ppid=16335 vsize=365452 CPUtime=4902.45 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 88007 0 0 0 490207 38 0 0 25 0 1 0 591380731 374222848 85838 33554432000 4194304 4785740 140734643542480 18446744073709551615 4277627 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 91363 85838 340 145 0 88408 0
[pid=16339] ppid=16335 vsize=207924 CPUtime=4902.32 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 80190 0 0 0 490212 20 0 0 25 0 1 0 591380731 212914176 34976 33554432000 4194304 4331988 140735271879216 18446744073709551615 4298764 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 51981 34976 175 34 0 50423 0
[pid=16340] ppid=16335 vsize=370228 CPUtime=19582.5 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 615216 1251 0 0 1958162 87 0 0 18 0 5 0 591380731 379113472 64627 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 92557 64627 164 40 0 91101 0
[pid=16340/tid=16348] ppid=16335 vsize=370228 CPUtime=4902.25 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 167685 1251 0 0 490201 24 0 0 25 0 5 0 591380752 379113472 64627 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331675 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=370228 CPUtime=4902.24 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 172157 1251 0 0 490203 21 0 0 25 0 5 0 591380752 379113472 64627 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331328 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=370228 CPUtime=4902.24 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 210767 1251 0 0 490194 30 0 0 25 0 5 0 591380752 379113472 64627 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330969 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=370228 CPUtime=4875.51 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 55983 1251 0 0 487543 8 0 0 25 0 5 0 591380752 379113472 64627 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34289.7
Current children cumulated vsize (KiB) 1855340

[startup+4962.3 s]
/proc/loadavg: 7.00 7.01 7.00 8/179 16498
/proc/meminfo: memFree=11896340/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=918248 CPUtime=4962.46 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 827329 0 0 0 495939 307 0 0 25 0 1 0 591380731 940285952 45356 33554432000 4194304 4587354 140733770771664 18446744073709551615 4239240 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 229562 45356 336 96 0 226124 0
[pid=16337] ppid=16335 vsize=365452 CPUtime=4962.45 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 88029 0 0 0 496207 38 0 0 25 0 1 0 591380731 374222848 85860 33554432000 4194304 4785740 140734643542480 18446744073709551615 4553409 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 91363 85860 340 145 0 88408 0
[pid=16339] ppid=16335 vsize=221284 CPUtime=4962.33 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 80529 0 0 0 496212 21 0 0 25 0 1 0 591380731 226594816 35315 33554432000 4194304 4331988 140735271879216 18446744073709551615 4243048 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 55321 35315 175 34 0 53763 0
[pid=16340] ppid=16335 vsize=358968 CPUtime=19822.2 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 621617 1251 0 0 1982129 88 0 0 18 0 5 0 591380731 367583232 61812 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 89742 61812 164 40 0 88286 0
[pid=16340/tid=16348] ppid=16335 vsize=358968 CPUtime=4962.25 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 169733 1251 0 0 496201 24 0 0 25 0 5 0 591380752 367583232 61812 33554432000 4194304 4356188 140735132377712 18446744073709551615 4332125 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=358968 CPUtime=4962.24 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 173693 1251 0 0 496203 21 0 0 25 0 5 0 591380752 367583232 61812 33554432000 4194304 4356188 140735132377712 18446744073709551615 4234124 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=358968 CPUtime=4962.24 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 211791 1251 0 0 496194 30 0 0 25 0 5 0 591380752 367583232 61812 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=358968 CPUtime=4935.18 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 57776 1251 0 0 493510 8 0 0 25 0 5 0 591380752 367583232 61812 33554432000 4194304 4356188 140735132377712 18446744073709551615 4332041 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 34709.4
Current children cumulated vsize (KiB) 1882276

[startup+5022.3 s]
/proc/loadavg: 7.00 7.00 7.00 8/179 16499
/proc/meminfo: memFree=11869016/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=918248 CPUtime=5022.47 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 835185 0 0 0 501936 311 0 0 25 0 1 0 591380731 940285952 53116 33554432000 4194304 4587354 140733770771664 18446744073709551615 4241101 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 229562 53116 336 96 0 226124 0
[pid=16337] ppid=16335 vsize=368228 CPUtime=5022.46 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 88599 0 0 0 502207 39 0 0 25 0 1 0 591380731 377065472 86430 33554432000 4194304 4785740 140734643542480 18446744073709551615 4552550 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 92057 86430 340 145 0 89102 0
[pid=16339] ppid=16335 vsize=221548 CPUtime=5022.33 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 80749 0 0 0 502212 21 0 0 25 0 1 0 591380731 226865152 35535 33554432000 4194304 4331988 140735271879216 18446744073709551615 4226450 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 55387 35535 175 34 0 53829 0
[pid=16340] ppid=16335 vsize=351800 CPUtime=20061.8 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 628273 1251 0 0 2006097 88 0 0 18 0 5 0 591380731 360243200 60020 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 87950 60020 164 40 0 86494 0
[pid=16340/tid=16348] ppid=16335 vsize=351800 CPUtime=5022.25 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 169733 1251 0 0 502201 24 0 0 25 0 5 0 591380752 360243200 60020 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=351800 CPUtime=5022.24 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 173693 1251 0 0 502203 21 0 0 25 0 5 0 591380752 360243200 60020 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331065 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=351800 CPUtime=5022.24 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 217935 1251 0 0 502193 31 0 0 25 0 5 0 591380752 360243200 60020 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330817 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=351800 CPUtime=4994.86 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 58288 1251 0 0 499478 8 0 0 25 0 5 0 591380752 360243200 60020 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 35129.1
Current children cumulated vsize (KiB) 1878148

[startup+5082.3 s]
/proc/loadavg: 7.06 7.02 7.00 8/179 16501
/proc/meminfo: memFree=11899732/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=907560 CPUtime=5082.47 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 838757 0 0 0 507933 314 0 0 25 0 1 0 591380731 929341440 42856 33554432000 4194304 4587354 140733770771664 18446744073709551615 4238467 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 226890 42856 336 96 0 223452 0
[pid=16337] ppid=16335 vsize=368936 CPUtime=5082.47 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 88733 0 0 0 508208 39 0 0 25 0 1 0 591380731 377790464 86564 33554432000 4194304 4785740 140734643542480 18446744073709551615 4553402 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 92234 86564 340 145 0 89279 0
[pid=16339] ppid=16335 vsize=221956 CPUtime=5082.33 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 81050 0 0 0 508212 21 0 0 25 0 1 0 591380731 227282944 35836 33554432000 4194304 4331988 140735271879216 18446744073709551615 4237419 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 55489 35836 175 34 0 53931 0
[pid=16340] ppid=16335 vsize=361016 CPUtime=20301.5 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 633649 1251 0 0 2030064 89 0 0 18 0 5 0 591380731 369680384 62324 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 90254 62324 164 40 0 88798 0
[pid=16340/tid=16348] ppid=16335 vsize=361016 CPUtime=5082.26 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 171269 1251 0 0 508202 24 0 0 25 0 5 0 591380752 369680384 62324 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331504 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=361016 CPUtime=5082.25 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 173693 1251 0 0 508204 21 0 0 25 0 5 0 591380752 369680384 62324 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331371 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=361016 CPUtime=5082.24 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 221007 1251 0 0 508193 31 0 0 25 0 5 0 591380752 369680384 62324 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=361016 CPUtime=5054.53 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 59056 1251 0 0 505445 8 0 0 25 0 5 0 591380752 369680384 62324 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 35548.8
Current children cumulated vsize (KiB) 1877792



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 7.04 7.01 7.00 8/179 16501
/proc/meminfo: memFree=11891044/32950928 swapFree=67111524/67111528
[pid=16335] ppid=16333 vsize=18324 CPUtime=0.01 cores=0-7
/proc/16335/stat : 16335 (ppfolio) S 16333 16335 16061 0 -1 4202496 519 1195 0 0 0 0 0 1 15 0 1 0 591380730 18763776 390 33554432000 4194304 4287625 140734377372608 18446744073709551615 256415411727 0 0 4096 65536 9224195742862639362 0 0 17 6 0 0 0
/proc/16335/statm: 4581 390 342 23 0 91 0
[pid=16336] ppid=16335 vsize=907560 CPUtime=5100.18 cores=0
/proc/16336/stat : 16336 (cryptominisat) R 16335 16335 16061 0 -1 4202496 840750 0 0 0 509703 315 0 0 25 0 1 0 591380731 929341440 44826 33554432000 4194304 4587354 140733770771664 18446744073709551615 256415088273 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/16336/statm: 226890 44826 336 96 0 223452 0
[pid=16337] ppid=16335 vsize=371608 CPUtime=5100.17 cores=1
/proc/16337/stat : 16337 (clasp) R 16335 16335 16061 0 -1 4202496 89376 0 0 0 509978 39 0 0 25 0 1 0 591380731 380526592 87207 33554432000 4194304 4785740 140734643542480 18446744073709551615 4287042 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/16337/statm: 92902 87207 340 145 0 89947 0
[pid=16339] ppid=16335 vsize=222096 CPUtime=5100.03 cores=3
/proc/16339/stat : 16339 (march_hi) R 16335 16335 16061 0 -1 4202496 81148 0 0 0 509982 21 0 0 25 0 1 0 591380731 227426304 35934 33554432000 4194304 4331988 140735271879216 18446744073709551615 4211123 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/16339/statm: 55524 35934 175 34 0 53966 0
[pid=16340] ppid=16335 vsize=357944 CPUtime=20372.2 cores=4-7
/proc/16340/stat : 16340 (plingeling) S 16335 16335 16061 0 -1 4202496 635697 1251 0 0 2037136 89 0 0 18 0 5 0 591380731 366534656 61556 33554432000 4194304 4356188 140735132377712 18446744073709551615 256427194341 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/16340/statm: 89486 61556 164 40 0 88030 0
[pid=16340/tid=16348] ppid=16335 vsize=357944 CPUtime=5099.96 cores=4-7
/proc/16340/task/16348/stat : 16348 (plingeling) R 16335 16335 16061 0 -1 4202560 171269 1251 0 0 509972 24 0 0 25 0 5 0 591380752 366534656 61556 33554432000 4194304 4356188 140735132377712 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=16340/tid=16349] ppid=16335 vsize=357944 CPUtime=5099.96 cores=4-7
/proc/16340/task/16349/stat : 16349 (plingeling) R 16335 16335 16061 0 -1 4202560 175741 1251 0 0 509974 22 0 0 25 0 5 0 591380752 366534656 61556 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330964 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=16340/tid=16350] ppid=16335 vsize=357944 CPUtime=5099.95 cores=4-7
/proc/16340/task/16350/stat : 16350 (plingeling) R 16335 16335 16061 0 -1 4202560 221007 1251 0 0 509964 31 0 0 25 0 5 0 591380752 366534656 61556 33554432000 4194304 4356188 140735132377712 18446744073709551615 4331296 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=16340/tid=16351] ppid=16335 vsize=357944 CPUtime=5072.14 cores=4-7
/proc/16340/task/16351/stat : 16351 (plingeling) R 16335 16335 16061 0 -1 4202560 59056 1251 0 0 507206 8 0 0 25 0 5 0 591380752 366534656 61556 33554432000 4194304 4356188 140735132377712 18446744073709551615 4330990 0 0 4096 17442 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 35672.6
Current children cumulated vsize (KiB) 1877532

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 16335 and gives
#  childrusage.ru_utime.tv_sec=10196
#  childrusage.ru_utime.tv_usec=868840
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=405482
# CPU time returned by wait4() is 10200.3
# while last known CPU time is 35672.6
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5100.02
CPU time (s): 35672.6
CPU user time (s): 35668
CPU system time (s): 4.65
CPU usage (%): 699.461
Max. virtual memory (cumulated for all children) (KiB): 2607844

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 10196.9
system time used= 3.40548
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 923612
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 610
involuntary context switches= 16989

runsolver used 8.49371 second user time and 19.3331 second system time

The end

Launcher Data 

Begin job on node115 at 2011-04-28 19:25:01
IDJOB=3310324
IDBENCH=83135
IDSOLVER=1638
FILE ID=node115/3310324-1304011500
RUNJOBID= node115-1304006040-16075
PBS_JOBID= 13168822
Free space on /tmp= 72728 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT11/application/fuhs/slp-synthesis-AES/slp-synthesis-aes-bottom21.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3310324-1304011500/watcher-3310324-1304011500 -o /tmp/evaluation-result-3310324-1304011500/solver-3310324-1304011500 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3310324-1304011500.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= e6407843d1fbd00b68d4ad600a25dfb2
RANDOM SEED=365136005

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12810132 kB
Buffers:       2313648 kB
Cached:       17202552 kB
SwapCached:          4 kB
Active:        7528576 kB
Inactive:     12054440 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12810132 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           12816 kB
Writeback:          52 kB
AnonPages:       66620 kB
Mapped:          14552 kB
Slab:           492368 kB
PageTables:       3800 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   187220 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72716 MiB
End job on node115 at 2011-04-28 20:50:03