Trace number 3275961

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
ppfolio par? (TO) 40000.6 5003.42

General information on the benchmark

NameSAT03/handmade/stanion/hwb/
hwb-n30-02-S77299857.shuffled-as.sat03-1633.cnf
MD5SUMbdf2c49c74bfdbd3bd4663dca402d24f
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark3326.88
Satisfiable
(Un)Satisfiability was proved
Number of variables194
Number of clauses954
Sum of the clauses size3252
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 210
Number of clauses of size 3544
Number of clauses of size 4400
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.00/0.00	This is pico-portfolio, a naive parallel portfolio
0.00/0.00	Copyright (C) 2011 Olivier Roussel
0.00/0.00	Using 8 processing units
0.00/0.00	This portfolio uses the following solvers:
0.00/0.00	[1] cryptominisat 2.7.1 (Mate Soos)
0.00/0.00	[2] clasp 1.3.6 (Martin Gebser, Benjamin Kaufmann, and Torsten Schaub)
0.00/0.00	[3] TNM 2009 (Wanxia Wei and Chu Min Li)
0.00/0.00	[4] march_hi 2009 (Marijn Heule and Hans Van Maaren)
0.00/0.00	[5] plingeling 276-6264d55-100731 (Armin Biere)
0.00/0.00	Starting cryptominisat 2.7.1 on core(s) 0
0.00/0.00	Starting clasp 1.3.6 on core(s) 1
0.00/0.00	Starting TNM 2009 on core(s) 2
0.00/0.00	Starting march_hi 2009 on core(s) 3
0.00/0.00	Starting plingeling 276-6264d55-100731 on core(s) 4 5 6 7

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3275961-1304172362/watcher-3275961-1304172362 -o /tmp/evaluation-result-3275961-1304172362/solver-3275961-1304172362 -C 40000 -W 5100 -M 31000 ppfolio HOME/instance-3275961-1304172362.cnf 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.02 8.02 8.00 2/170 9725
/proc/meminfo: memFree=29003564/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 25 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0

[startup+0.0234509 s]
/proc/loadavg: 8.02 8.02 8.00 2/170 9725
/proc/meminfo: memFree=29003564/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 18 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.100437 s]
/proc/loadavg: 8.02 8.02 8.00 2/170 9725
/proc/meminfo: memFree=29003564/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 17 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.3004 s]
/proc/loadavg: 8.02 8.02 8.00 2/170 9725
/proc/meminfo: memFree=29003564/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+0.700326 s]
/proc/loadavg: 8.02 8.02 8.00 2/170 9725
/proc/meminfo: memFree=29003564/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 18324

[startup+1.50118 s]
/proc/loadavg: 8.02 8.02 8.00 9/180 9740
/proc/meminfo: memFree=28987628/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1000428 CPUtime=1.49 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 3342 0 0 0 149 0 0 0 20 0 1 0 607466880 1024438272 2294 33554432000 4194304 4587354 140733418851648 18446744073709551615 4238763 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 250107 2294 337 96 0 246669 0
[pid=9727] ppid=9725 vsize=12956 CPUtime=1.49 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 693 0 0 0 149 0 0 0 19 0 1 0 607466880 13266944 584 33554432000 4194304 4785740 140735248883440 18446744073709551615 4552528 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 3239 584 340 145 0 284 0
[pid=9728] ppid=9725 vsize=455692 CPUtime=1.49 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 289 0 0 0 149 0 0 0 20 0 1 0 607466880 466628608 186 33554432000 134512640 134534564 4294556896 18446744073709551615 134528302 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113923 186 95 6 0 112589 0
[pid=9729] ppid=9725 vsize=7488 CPUtime=1.49 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 500 0 0 0 149 0 0 0 20 0 1 0 607466880 7667712 392 33554432000 4194304 4331988 140737141778224 18446744073709551615 4211717 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 1872 392 174 34 0 314 0
[pid=9730] ppid=9725 vsize=189304 CPUtime=5.94 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 4689 1257 0 0 594 0 0 0 18 0 5 0 607466880 193847296 3120 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 47326 3120 164 40 0 45870 0
[pid=9730/tid=9737] ppid=9725 vsize=189304 CPUtime=1.48 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 1222 1257 0 0 148 0 0 0 20 0 5 0 607466881 193847296 3120 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=189304 CPUtime=1.49 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 1001 1257 0 0 149 0 0 0 20 0 5 0 607466881 193847296 3120 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=189304 CPUtime=1.48 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 1018 1257 0 0 148 0 0 0 23 0 5 0 607466881 193847296 3120 33554432000 4194304 4356188 140736050712704 18446744073709551615 4332239 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=189304 CPUtime=1.48 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 1089 1257 0 0 148 0 0 0 20 0 5 0 607466881 193847296 3120 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.9
Current children cumulated vsize (KiB) 1684192

[startup+3.10088 s]
/proc/loadavg: 8.02 8.02 8.00 9/180 9740
/proc/meminfo: memFree=28976344/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1003808 CPUtime=3.09 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 4899 0 0 0 309 0 0 0 25 0 1 0 607466880 1027899392 3050 33554432000 4194304 4587354 140733418851648 18446744073709551615 4232219 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 250952 3050 337 96 0 247514 0
[pid=9727] ppid=9725 vsize=13228 CPUtime=3.09 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 758 0 0 0 309 0 0 0 25 0 1 0 607466880 13545472 649 33554432000 4194304 4785740 140735248883440 18446744073709551615 4287028 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 3307 649 340 145 0 352 0
[pid=9728] ppid=9725 vsize=455692 CPUtime=3.09 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 289 0 0 0 309 0 0 0 25 0 1 0 607466880 466628608 186 33554432000 134512640 134534564 4294556896 18446744073709551615 134530353 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113923 186 95 6 0 112589 0
[pid=9729] ppid=9725 vsize=7716 CPUtime=3.09 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 588 0 0 0 309 0 0 0 25 0 1 0 607466880 7901184 480 33554432000 4194304 4331988 140737141778224 18446744073709551615 4211996 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 1929 480 174 34 0 371 0
[pid=9730] ppid=9725 vsize=194948 CPUtime=12.34 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 7619 1257 0 0 1233 1 0 0 18 0 5 0 607466880 199626752 4546 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 48737 4546 164 40 0 47281 0
[pid=9730/tid=9737] ppid=9725 vsize=194948 CPUtime=3.08 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 2136 1257 0 0 308 0 0 0 25 0 5 0 607466881 199626752 4546 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331401 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=194948 CPUtime=3.08 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 1737 1257 0 0 308 0 0 0 25 0 5 0 607466881 199626752 4546 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331168 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=194948 CPUtime=3.07 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 1722 1257 0 0 307 0 0 0 25 0 5 0 607466881 199626752 4546 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=194948 CPUtime=3.08 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 1665 1257 0 0 308 0 0 0 25 0 5 0 607466881 199626752 4546 33554432000 4194304 4356188 140736050712704 18446744073709551615 4281491 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.7
Current children cumulated vsize (KiB) 1693716

[startup+6.30129 s]
/proc/loadavg: 8.02 8.02 8.00 9/180 9740
/proc/meminfo: memFree=28968888/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1005080 CPUtime=6.28 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 7462 0 0 0 628 0 0 0 25 0 1 0 607466880 1029201920 2950 33554432000 4194304 4587354 140733418851648 18446744073709551615 4238763 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 251270 2950 337 96 0 247832 0
[pid=9727] ppid=9725 vsize=13496 CPUtime=6.29 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 829 0 0 0 629 0 0 0 25 0 1 0 607466880 13819904 720 33554432000 4194304 4785740 140735248883440 18446744073709551615 4553344 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 3374 720 340 145 0 419 0
[pid=9728] ppid=9725 vsize=455692 CPUtime=6.29 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 289 0 0 0 629 0 0 0 25 0 1 0 607466880 466628608 186 33554432000 134512640 134534564 4294556896 18446744073709551615 134530158 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113923 186 95 6 0 112589 0
[pid=9729] ppid=9725 vsize=9080 CPUtime=6.29 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 811 0 0 0 629 0 0 0 25 0 1 0 607466880 9297920 703 33554432000 4194304 4331988 140737141778224 18446744073709551615 4211806 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 2270 703 174 34 0 712 0
[pid=9730] ppid=9725 vsize=197016 CPUtime=25.12 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 10920 1257 0 0 2511 1 0 0 18 0 5 0 607466880 201744384 5063 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 49254 5063 164 40 0 47798 0
[pid=9730/tid=9737] ppid=9725 vsize=197016 CPUtime=6.28 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 2969 1257 0 0 628 0 0 0 25 0 5 0 607466881 201744384 5063 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=197016 CPUtime=6.28 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 2570 1257 0 0 628 0 0 0 25 0 5 0 607466881 201744384 5063 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331069 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=197016 CPUtime=6.25 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 2620 1257 0 0 625 0 0 0 25 0 5 0 607466881 201744384 5063 33554432000 4194304 4356188 140736050712704 18446744073709551615 4279694 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=197016 CPUtime=6.28 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 2402 1257 0 0 628 0 0 0 25 0 5 0 607466881 201744384 5063 33554432000 4194304 4356188 140736050712704 18446744073709551615 4333399 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 50.27
Current children cumulated vsize (KiB) 1698688

[startup+12.7011 s]
/proc/loadavg: 8.01 8.02 8.00 9/180 9740
/proc/meminfo: memFree=28953972/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1006456 CPUtime=12.68 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 11720 0 0 0 1267 1 0 0 25 0 1 0 607466880 1030610944 3632 33554432000 4194304 4587354 140733418851648 18446744073709551615 4237665 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 251614 3632 337 96 0 248176 0
[pid=9727] ppid=9725 vsize=13896 CPUtime=12.69 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 932 0 0 0 1269 0 0 0 25 0 1 0 607466880 14229504 823 33554432000 4194304 4785740 140735248883440 18446744073709551615 4277565 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 3474 823 340 145 0 519 0
[pid=9728] ppid=9725 vsize=455692 CPUtime=12.69 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 289 0 0 0 1269 0 0 0 25 0 1 0 607466880 466628608 186 33554432000 134512640 134534564 4294556896 18446744073709551615 134528643 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113923 186 95 6 0 112589 0
[pid=9729] ppid=9725 vsize=11508 CPUtime=12.69 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 1178 0 0 0 1269 0 0 0 25 0 1 0 607466880 11784192 1070 33554432000 4194304 4331988 140737141778224 18446744073709551615 4211952 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 2877 1070 174 34 0 1319 0
[pid=9730] ppid=9725 vsize=206628 CPUtime=50.69 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 18155 1257 0 0 5067 2 0 0 18 0 5 0 607466880 211587072 7466 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 51657 7466 164 40 0 50201 0
[pid=9730/tid=9737] ppid=9725 vsize=206628 CPUtime=12.68 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 4729 1257 0 0 1268 0 0 0 25 0 5 0 607466881 211587072 7466 33554432000 4194304 4356188 140736050712704 18446744073709551615 4332162 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=206628 CPUtime=12.68 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 4363 1257 0 0 1268 0 0 0 25 0 5 0 607466881 211587072 7466 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=206628 CPUtime=12.62 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 4477 1257 0 0 1262 0 0 0 25 0 5 0 607466881 211587072 7466 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=206628 CPUtime=12.68 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 4227 1257 0 0 1268 0 0 0 25 0 5 0 607466881 211587072 7466 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331960 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 101.44
Current children cumulated vsize (KiB) 1712504

[startup+25.5007 s]
/proc/loadavg: 8.01 8.02 8.00 9/180 9740
/proc/meminfo: memFree=28935476/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1021988 CPUtime=25.49 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 20323 0 0 0 2546 3 0 0 25 0 1 0 607466880 1046515712 6675 33554432000 4194304 4587354 140733418851648 18446744073709551615 4357656 0 0 4096 2 0 0 0 17 0 0 0 0

################
# More data... #
################

[pid=9728] ppid=9725 vsize=455696 CPUtime=4781.64 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 299 0 0 0 478163 1 0 0 25 0 1 0 607466880 466632704 196 33554432000 134512640 134534564 4294556896 18446744073709551615 134528708 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113924 196 103 6 0 112590 0
[pid=9729] ppid=9725 vsize=1276196 CPUtime=4782.44 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 255702 0 0 0 478101 143 0 0 25 0 1 0 607466880 1306824704 255594 33554432000 4194304 4331988 140737141778224 18446744073709551615 4226479 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 319049 255594 174 34 0 317491 0
[pid=9730] ppid=9725 vsize=392684 CPUtime=19103.9 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 1477365 1257 0 0 1910185 208 0 0 18 0 5 0 607466880 402108416 53942 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 98171 53942 164 40 0 96715 0
[pid=9730/tid=9737] ppid=9725 vsize=392684 CPUtime=4782.32 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 363291 1257 0 0 478180 52 0 0 25 0 5 0 607466881 402108416 53942 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=392684 CPUtime=4782.44 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 365870 1257 0 0 478195 49 0 0 25 0 5 0 607466881 402108416 53942 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=392684 CPUtime=4757.18 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 374239 1257 0 0 475665 53 0 0 25 0 5 0 607466881 402108416 53942 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330960 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=392684 CPUtime=4781.96 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 373606 1257 0 0 478144 52 0 0 25 0 5 0 607466881 402108416 53942 33554432000 4194304 4356188 140736050712704 18446744073709551615 4332165 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38232.9
Current children cumulated vsize (KiB) 3298276

[startup+4842.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/180 9881
/proc/meminfo: memFree=27568656/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1136584 CPUtime=4842.44 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 550706 0 0 0 484057 187 0 0 25 0 1 0 607466880 1163862016 39124 33554432000 4194304 4587354 140733418851648 18446744073709551615 4231923 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 284146 39124 337 96 0 280708 0
[pid=9727] ppid=9725 vsize=18804 CPUtime=4842.43 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 2121 0 0 0 484243 0 0 0 25 0 1 0 607466880 19255296 2012 33554432000 4194304 4785740 140735248883440 18446744073709551615 4286953 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 4701 2012 340 145 0 1746 0
[pid=9728] ppid=9725 vsize=455696 CPUtime=4841.64 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 299 0 0 0 484163 1 0 0 25 0 1 0 607466880 466632704 196 33554432000 134512640 134534564 4294556896 18446744073709551615 134528864 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113924 196 103 6 0 112590 0
[pid=9729] ppid=9725 vsize=1276196 CPUtime=4842.44 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 258851 0 0 0 484099 145 0 0 25 0 1 0 607466880 1306824704 258743 33554432000 4194304 4331988 140737141778224 18446744073709551615 4315863 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 319049 258743 174 34 0 317491 0
[pid=9730] ppid=9725 vsize=411884 CPUtime=19343.6 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 1495541 1257 0 0 1934150 211 0 0 18 0 5 0 607466880 421769216 58742 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 102971 58742 164 40 0 101515 0
[pid=9730/tid=9737] ppid=9725 vsize=411884 CPUtime=4842.32 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 368795 1257 0 0 484179 53 0 0 25 0 5 0 607466881 421769216 58742 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=411884 CPUtime=4842.44 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 373486 1257 0 0 484194 50 0 0 25 0 5 0 607466881 421769216 58742 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331350 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=411884 CPUtime=4816.85 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 377503 1257 0 0 481632 53 0 0 25 0 5 0 607466881 421769216 58742 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330995 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=411884 CPUtime=4841.95 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 375398 1257 0 0 484143 52 0 0 25 0 5 0 607466881 421769216 58742 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331035 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 38712.6
Current children cumulated vsize (KiB) 3317488

[startup+4902.3 s]
/proc/loadavg: 8.03 8.05 8.00 9/180 9882
/proc/meminfo: memFree=27542464/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1136576 CPUtime=4902.45 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 551758 0 0 0 490057 188 0 0 25 0 1 0 607466880 1163853824 39017 33554432000 4194304 4587354 140733418851648 18446744073709551615 4231923 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 284144 39017 337 96 0 280706 0
[pid=9727] ppid=9725 vsize=18804 CPUtime=4902.43 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 2121 0 0 0 490243 0 0 0 25 0 1 0 607466880 19255296 2012 33554432000 4194304 4785740 140735248883440 18446744073709551615 4286912 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 4701 2012 340 145 0 1746 0
[pid=9728] ppid=9725 vsize=455696 CPUtime=4901.64 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 299 0 0 0 490163 1 0 0 25 0 1 0 607466880 466632704 196 33554432000 134512640 134534564 4294556896 18446744073709551615 134530701 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113924 196 103 6 0 112590 0
[pid=9729] ppid=9725 vsize=1276196 CPUtime=4902.45 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 262063 0 0 0 490098 147 0 0 25 0 1 0 607466880 1306824704 261955 33554432000 4194304 4331988 140737141778224 18446744073709551615 4211648 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 319049 261955 174 34 0 317491 0
[pid=9730] ppid=9725 vsize=425196 CPUtime=19583.3 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 1507221 1257 0 0 1958118 212 0 0 18 0 5 0 607466880 435400704 62070 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 106299 62070 164 40 0 104843 0
[pid=9730/tid=9737] ppid=9725 vsize=425196 CPUtime=4902.32 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 370747 1257 0 0 490178 54 0 0 25 0 5 0 607466881 435400704 62070 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=425196 CPUtime=4902.44 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 375726 1257 0 0 490194 50 0 0 25 0 5 0 607466881 435400704 62070 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=425196 CPUtime=4876.56 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 383007 1257 0 0 487602 54 0 0 25 0 5 0 607466881 435400704 62070 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330992 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=425196 CPUtime=4901.95 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 377382 1257 0 0 490142 53 0 0 25 0 5 0 607466881 435400704 62070 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331060 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39192.3
Current children cumulated vsize (KiB) 3330792

[startup+4962.3 s]
/proc/loadavg: 8.01 8.03 8.00 9/180 9884
/proc/meminfo: memFree=27523456/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1138276 CPUtime=4962.45 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 552932 0 0 0 496056 189 0 0 25 0 1 0 607466880 1165594624 39136 33554432000 4194304 4587354 140733418851648 18446744073709551615 4231923 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 284569 39136 337 96 0 281131 0
[pid=9727] ppid=9725 vsize=18804 CPUtime=4962.44 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 2126 0 0 0 496244 0 0 0 25 0 1 0 607466880 19255296 2017 33554432000 4194304 4785740 140735248883440 18446744073709551615 4553396 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 4701 2017 340 145 0 1746 0
[pid=9728] ppid=9725 vsize=455696 CPUtime=4961.64 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 299 0 0 0 496163 1 0 0 25 0 1 0 607466880 466632704 196 33554432000 134512640 134534564 4294556896 18446744073709551615 134528473 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113924 196 103 6 0 112590 0
[pid=9729] ppid=9725 vsize=1538416 CPUtime=4962.45 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 265046 0 0 0 496096 149 0 0 25 0 1 0 607466880 1575337984 264938 33554432000 4194304 4331988 140737141778224 18446744073709551615 4211818 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 384604 264938 174 34 0 383046 0
[pid=9730] ppid=9725 vsize=431596 CPUtime=19823 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 1518101 1257 0 0 1982085 214 0 0 18 0 5 0 607466880 441954304 63670 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 107899 63670 164 40 0 106443 0
[pid=9730/tid=9737] ppid=9725 vsize=431596 CPUtime=4962.32 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 372603 1257 0 0 496178 54 0 0 25 0 5 0 607466881 441954304 63670 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331019 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=431596 CPUtime=4962.44 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 377838 1257 0 0 496193 51 0 0 25 0 5 0 607466881 441954304 63670 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331355 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=431596 CPUtime=4936.24 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 386527 1257 0 0 493570 54 0 0 25 0 5 0 607466881 441954304 63670 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331154 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=431596 CPUtime=4961.94 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 380774 1257 0 0 496141 53 0 0 25 0 5 0 607466881 441954304 63670 33554432000 4194304 4356188 140736050712704 18446744073709551615 4280608 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 39672
Current children cumulated vsize (KiB) 3601112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5003.4 s]
/proc/loadavg: 8.00 8.03 8.00 9/180 9885
/proc/meminfo: memFree=27537320/32950928 swapFree=67111352/67111528
[pid=9725] ppid=9723 vsize=18324 CPUtime=0 cores=0-7
/proc/9725/stat : 9725 (ppfolio) S 9723 9725 9020 0 -1 4202496 523 0 0 0 0 0 0 0 15 0 1 0 607466880 18763776 390 33554432000 4194304 4287625 140735573248176 18446744073709551615 251367566863 0 0 4096 65536 4154526817518092287 0 0 17 4 0 0 0
/proc/9725/statm: 4581 390 341 23 0 91 0
[pid=9726] ppid=9725 vsize=1136572 CPUtime=5003.55 cores=0
/proc/9726/stat : 9726 (cryptominisat) R 9725 9725 9020 0 -1 4202496 553015 0 0 0 500166 189 0 0 25 0 1 0 607466880 1163849728 38090 33554432000 4194304 4587354 140733418851648 18446744073709551615 4237681 0 0 4096 2 0 0 0 17 0 0 0 0
/proc/9726/statm: 284143 38090 337 96 0 280705 0
[pid=9727] ppid=9725 vsize=18804 CPUtime=5003.54 cores=1
/proc/9727/stat : 9727 (clasp) R 9725 9725 9020 0 -1 4202496 2126 0 0 0 500354 0 0 0 25 0 1 0 607466880 19255296 2017 33554432000 4194304 4785740 140735248883440 18446744073709551615 4550365 0 0 4096 16386 0 0 0 17 1 0 0 0
/proc/9727/statm: 4701 2017 340 145 0 1746 0
[pid=9728] ppid=9725 vsize=455696 CPUtime=5002.75 cores=2
/proc/9728/stat : 9728 (TNM) R 9725 9725 9020 0 -1 4202496 299 0 0 0 500274 1 0 0 25 0 1 0 607466880 466632704 196 33554432000 134512640 134534564 4294556896 18446744073709551615 134528918 0 0 4096 0 0 0 0 17 2 0 0 0
/proc/9728/statm: 113924 196 103 6 0 112590 0
[pid=9729] ppid=9725 vsize=1538416 CPUtime=5003.55 cores=3
/proc/9729/stat : 9729 (march_hi) R 9725 9725 9020 0 -1 4202496 267351 0 0 0 500204 151 0 0 25 0 1 0 607466880 1575337984 267243 33554432000 4194304 4331988 140737141778224 18446744073709551615 4297334 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/9729/statm: 384604 267243 174 34 0 383046 0
[pid=9730] ppid=9725 vsize=412140 CPUtime=19987.2 cores=4-7
/proc/9730/stat : 9730 (plingeling) S 9725 9725 9020 0 -1 4202496 1523605 1257 0 0 1998505 215 0 0 18 0 5 0 607466880 422031360 58806 33554432000 4194304 4356188 140736050712704 18446744073709551615 251379349477 0 0 4096 17442 18446744073709551615 0 0 17 5 0 0 0
/proc/9730/statm: 103035 58806 164 40 0 101579 0
[pid=9730/tid=9737] ppid=9725 vsize=412140 CPUtime=5003.43 cores=4-7
/proc/9730/task/9737/stat : 9737 (plingeling) R 9725 9725 9020 0 -1 4202560 374331 1257 0 0 500289 54 0 0 25 0 5 0 607466881 422031360 58806 33554432000 4194304 4356188 140736050712704 18446744073709551615 4330986 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=9730/tid=9738] ppid=9725 vsize=412140 CPUtime=5003.55 cores=4-7
/proc/9730/task/9738/stat : 9738 (plingeling) R 9725 9725 9020 0 -1 4202560 379886 1257 0 0 500304 51 0 0 25 0 5 0 607466881 422031360 58806 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331055 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=9730/tid=9739] ppid=9725 vsize=412140 CPUtime=4977.13 cores=4-7
/proc/9730/task/9739/stat : 9739 (plingeling) R 9725 9725 9020 0 -1 4202560 387615 1257 0 0 497659 54 0 0 25 0 5 0 607466881 422031360 58806 33554432000 4194304 4356188 140736050712704 18446744073709551615 4331062 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=9730/tid=9740] ppid=9725 vsize=412140 CPUtime=5003.05 cores=4-7
/proc/9730/task/9740/stat : 9740 (plingeling) R 9725 9725 9020 0 -1 4202560 381414 1257 0 0 500251 54 0 0 25 0 5 0 607466881 422031360 58806 33554432000 4194304 4356188 140736050712704 18446744073709551615 4332046 0 0 4096 17442 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 3579952

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9725 and gives
#  childrusage.ru_utime.tv_sec=15007
#  childrusage.ru_utime.tv_usec=964442
#  childrusage.ru_stime.tv_sec=1
#  childrusage.ru_stime.tv_usec=935705
# CPU time returned by wait4() is 15009.9
# while last known CPU time is 40000.6
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5003.42
CPU time (s): 40000.6
CPU user time (s): 39995
CPU system time (s): 5.56
CPU usage (%): 799.465
Max. virtual memory (cumulated for all children) (KiB): 9927764

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 15008
system time used= 1.93571
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 555980
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 974
involuntary context switches= 80652

runsolver used 8.47471 second user time and 17.9413 second system time

The end

Launcher Data 

Begin job on node113 at 2011-04-30 16:06:02
IDJOB=3275961
IDBENCH=84028
IDSOLVER=1638
FILE ID=node113/3275961-1304172362
RUNJOBID= node113-1304162219-9034
PBS_JOBID= 13170938
Free space on /tmp= 72396 MiB

SOLVER NAME= ppfolio par
BENCH NAME= SAT03/handmade/stanion/hwb/hwb-n30-02-S77299857.shuffled-as.sat03-1633.cnf
COMMAND LINE= ppfolio BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3275961-1304172362/watcher-3275961-1304172362 -o /tmp/evaluation-result-3275961-1304172362/solver-3275961-1304172362 -C 40000 -W 5100 -M 31000  ppfolio HOME/instance-3275961-1304172362.cnf

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= bdf2c49c74bfdbd3bd4663dca402d24f
RANDOM SEED=188390995

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      29004216 kB
Buffers:        769620 kB
Cached:        2608840 kB
SwapCached:         12 kB
Active:         216936 kB
Inactive:      3224912 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      29004216 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:           10096 kB
Writeback:           0 kB
AnonPages:       63340 kB
Mapped:          14700 kB
Slab:           438564 kB
PageTables:       4636 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:   186088 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72384 MiB
End job on node113 at 2011-04-30 17:29:28