Trace number 3498488

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-industrial (fixed)? (TO) 5000.04 4993.46

General information on the benchmark

NameSAT03/industrial/maris/CNF/
gripper13u.shuffled-as.sat03-395.cnf
MD5SUM3889b7ed90cc1fbe8bc8d140a1763832
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark1519.95
Satisfiable
(Un)Satisfiability was proved
Number of variables4268
Number of clauses38965
Sum of the clauses size86077
Maximum clause length28
Minimum clause length1
Number of clauses of size 169
Number of clauses of size 235984
Number of clauses of size 3104
Number of clauses of size 42704
Number of clauses of size 50
Number of clauses of size over 5104

Solver Data 

0.35/0.69	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.35/0.69	c INFO: solving HOME/instance-3498488-1307955512.cnf
0.35/0.69	c INFO: computing task features
0.53/0.80	c INFO: computed variable-clause graph statistics
0.53/0.80	c INFO: computed clause balance statistics
1.03/1.34	c INFO: computed variable balance statistics
1.43/1.77	c INFO: computed small-clause counts
1.43/1.77	c INFO: computed Horn-clause counts
1.53/1.82	c INFO: computed variable graph statistics
1.53/1.82	c INFO: collected features for HOME/instance-3498488-1307955512.cnf in 1.14 s
1.53/1.85	c INFO: running lingeling-276@0 for 97 with 4998 remaining (b = 0.27)
100.71/100.98	c INFO: running lingeling-276@99 for 485 with 4899 remaining (b = 0.40)
586.80/586.48	c INFO: running SApperloT@0 for 97 with 4414 remaining (b = 0.15)
687.09/686.63	c INFO: running cryptominisat-2.9.0@0 for 485 with 4314 remaining (b = 0.39)
1176.85/1175.79	c INFO: running precosat-570@0 for 388 with 3825 remaining (b = 0.28)
1565.87/1564.26	c INFO: running lingeling-276@0 for 582 with 3436 remaining (b = 0.35)
2150.34/2147.98	c INFO: running lingeling-276@0 for 97 with 2853 remaining (b = 0.08)
2250.66/2248.13	c INFO: running cryptominisat-2.9.0@0 for 582 with 2753 remaining (b = 0.22)
2834.24/2830.78	c INFO: running lingeling-276@100 for 97 with 2170 remaining (b = 0.05)
2931.58/2927.91	c INFO: running precosat-570@0 for 194 with 2073 remaining (b = 0.07)
3128.01/3124.20	c INFO: running lingeling-276@584 for 485 with 1877 remaining (b = 0.17)
3616.28/3611.77	c INFO: running cryptominisat-2.9.0@582 for 1389 with 1389 remaining (b = 0.38)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /var/tmp/evaluation-result-3498488-1307955512/watcher-3498488-1307955512 -o /var/tmp/evaluation-result-3498488-1307955512/solver-3498488-1307955512 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3498488-1307955512.cnf -seed 356937537 -budget 5000 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.00 1.82 2/182 29062
/proc/meminfo: memFree=7032112/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=15184 CPUtime=0 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 729 0 0 0 0 0 0 0 17 0 1 0 907151663 15548416 617 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/29062/statm: 3796 617 322 364 0 283 0

[startup+0.0592511 s]
/proc/loadavg: 2.00 2.00 1.82 2/182 29062
/proc/meminfo: memFree=7032112/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=29788 CPUtime=0.04 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 1415 0 0 0 3 1 0 0 17 0 1 0 907151663 30502912 1257 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/29062/statm: 7447 1257 443 364 0 817 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 29788

[startup+0.100943 s]
/proc/loadavg: 2.00 2.00 1.82 2/182 29062
/proc/meminfo: memFree=7032112/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=85620 CPUtime=0.06 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 2632 0 0 0 4 2 0 0 17 0 1 0 907151663 87674880 2348 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/29062/statm: 21405 2348 713 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85620

[startup+0.300971 s]
/proc/loadavg: 2.00 2.00 1.82 2/182 29062
/proc/meminfo: memFree=7032112/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=135748 CPUtime=0.19 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 5972 934 1 0 13 6 0 0 17 0 1 0 907151663 139005952 4149 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446943232 0 2147483391 16781312 2 18446744071699199166 0 0 17 6 0 0 0
/proc/29062/statm: 33937 4149 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135748

[startup+0.705484 s]
/proc/loadavg: 2.00 2.00 1.82 2/182 29062
/proc/meminfo: memFree=7032112/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=279668 CPUtime=0.44 cores=0,2,4,6
/proc/29062/stat : 29062 (python) R 29060 29062 28636 0 -1 4202496 9563 934 1 0 31 13 0 0 16 0 1 0 907151663 286380032 7234 33554432000 4194304 5683644 140733664339824 18446744073709551615 4602336 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/29062/statm: 69917 7234 1761 364 0 5841 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 279668

[startup+1.50031 s]
/proc/loadavg: 2.00 2.00 1.82 3/183 29067
/proc/meminfo: memFree=7004492/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=284556 CPUtime=1.23 cores=0,2,4,6
/proc/29062/stat : 29062 (python) R 29060 29062 28636 0 -1 4202496 12023 934 1 0 110 13 0 0 17 0 1 0 907151663 291385344 8525 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446636464 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/29062/statm: 71139 8525 1827 364 0 7063 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 284556

[startup+3.10099 s]
/proc/loadavg: 2.08 2.02 1.82 3/186 29070
/proc/meminfo: memFree=6999136/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.58 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 12482 934 1 0 145 13 0 0 19 0 2 0 907151663 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 73697 8568 1875 364 0 9615 0
[pid=29068] ppid=29062 vsize=284544 CPUtime=0 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 809 0 0 0 0 0 0 0 18 0 1 0 907151848 291373056 7047 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965263 0 0 16781312 514 17331131476829274111 0 0 17 6 0 0 0
/proc/29068/statm: 71136 7047 358 364 0 7054 0
[pid=29070] ppid=29068 vsize=9872 CPUtime=1.24 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) R 29068 29070 29070 0 -1 4202496 2141 0 0 0 124 0 0 0 25 0 1 0 907151849 10108928 1637 33554432000 4194304 4352236 140734520664416 18446744073709551615 4328568 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/29070/statm: 2468 1637 129 39 0 1548 0
[pid=29062/tid=29069] ppid=29060 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 2 0 907151848 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.82
Current children cumulated vsize (KiB) 589204

[startup+6.30045 s]
/proc/loadavg: 2.08 2.02 1.82 3/186 29070
/proc/meminfo: memFree=6995544/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.58 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 12482 934 1 0 145 13 0 0 19 0 2 0 907151663 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 73697 8568 1875 364 0 9615 0
[pid=29068] ppid=29062 vsize=284544 CPUtime=0 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 958 0 0 0 0 0 0 0 18 0 1 0 907151848 291373056 7069 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965263 0 0 16781312 514 17331131476829241601 0 0 17 6 0 0 0
/proc/29068/statm: 71136 7069 380 364 0 7054 0
[pid=29070] ppid=29068 vsize=10592 CPUtime=4.44 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) R 29068 29070 29070 0 -1 4202496 2557 0 0 0 444 0 0 0 25 0 1 0 907151849 10846208 1837 33554432000 4194304 4352236 140734520664416 18446744073709551615 4280978 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/29070/statm: 2648 1837 129 39 0 1728 0
[pid=29062/tid=29069] ppid=29060 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 2 0 907151848 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 6.02
Current children cumulated vsize (KiB) 589924

[startup+12.7013 s]
/proc/loadavg: 2.07 2.02 1.83 3/186 29070
/proc/meminfo: memFree=6999940/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.58 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 12482 934 1 0 145 13 0 0 19 0 2 0 907151663 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 73697 8568 1875 364 0 9615 0
[pid=29068] ppid=29062 vsize=284544 CPUtime=0.01 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 962 0 0 0 0 1 0 0 18 0 1 0 907151848 291373056 7070 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965263 0 0 16781312 514 17331131476829241601 0 0 17 6 0 0 0
/proc/29068/statm: 71136 7070 381 364 0 7054 0
[pid=29070] ppid=29068 vsize=14880 CPUtime=10.84 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) R 29068 29070 29070 0 -1 4202496 3613 0 0 0 1084 0 0 0 25 0 1 0 907151849 15237120 2889 33554432000 4194304 4352236 140734520664416 18446744073709551615 4328022 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/29070/statm: 3720 2889 129 39 0 2800 0
[pid=29062/tid=29069] ppid=29060 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 2 0 907151848 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.43
Current children cumulated vsize (KiB) 594212

[startup+25.5009 s]
/proc/loadavg: 2.06 2.01 1.83 3/186 29071
/proc/meminfo: memFree=6998760/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.58 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 12482 934 1 0 145 13 0 0 19 0 2 0 907151663 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 73697 8568 1875 364 0 9615 0
[pid=29068] ppid=29062 vsize=284544 CPUtime=0.02 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 965 0 0 0 1 1 0 0 18 0 1 0 907151848 291373056 7070 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965263 0 0 16781312 514 4970003663780020481 0 0 17 6 0 0 0
/proc/29068/statm: 71136 7070 381 364 0 7054 0
[pid=29070] ppid=29068 vsize=16144 CPUtime=23.63 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) R 29068 29070 29070 0 -1 4202496 4062 0 0 0 2363 0 0 0 25 0 1 0 907151849 16531456 3205 33554432000 4194304 4352236 140734520664416 18446744073709551615 4328100 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/29070/statm: 4036 3205 129 39 0 3116 0
[pid=29062/tid=29069] ppid=29060 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 2 0 907151848 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.23
Current children cumulated vsize (KiB) 595476

[startup+51.1011 s]
/proc/loadavg: 2.04 2.01 1.83 3/186 29071
/proc/meminfo: memFree=6994868/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.58 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 12482 934 1 0 145 13 0 0 19 0 2 0 907151663 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 73697 8568 1875 364 0 9615 0
[pid=29068] ppid=29062 vsize=284544 CPUtime=0.04 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 974 0 0 0 2 2 0 0 18 0 1 0 907151848 291373056 7070 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965263 0 0 16781312 514 4970003663780020481 0 0 17 2 0 0 0
/proc/29068/statm: 71136 7070 381 364 0 7054 0
[pid=29070] ppid=29068 vsize=17668 CPUtime=49.23 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) R 29068 29070 29070 0 -1 4202496 6644 0 0 0 4923 0 0 0 25 0 1 0 907151849 18092032 3586 33554432000 4194304 4352236 140734520664416 18446744073709551615 4328105 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/29070/statm: 4417 3586 129 39 0 3497 0
[pid=29062/tid=29069] ppid=29060 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 2 0 907151848 301862912 8568 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.85
Current children cumulated vsize (KiB) 597000

[startup+102.307 s]
/proc/loadavg: 2.06 2.02 1.84 3/187 29074
/proc/meminfo: memFree=6986620/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.6 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 12963 934 1 0 147 13 0 0 18 0 2 0 907151663 301862912 8569 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 73697 8569 1876 364 0 9615 0
[pid=29068] ppid=29062 vsize=294788 CPUtime=0.11 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 1153 0 0 0 6 5 0 0 18 0 2 0 907151848 301862912 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/29068/statm: 73697 7109 415 364 0 9615 0
[pid=29070] ppid=29068 vsize=21644 CPUtime=100.41 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) R 29068 29070 29070 0 -1 4202496 12631 0 0 0 10040 1 0 0 25 0 1 0 907151849 22163456 4612 33554432000 4194304 4352236 140734520664416 18446744073709551615 4328832 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/29070/statm: 5411 4612 129 39 0 4491 0
[pid=29068/tid=29074] ppid=29062 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29068/task/29074/stat : 29074 (python) S 29062 29062 28636 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 907161759 301862912 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604444063009856 0 0 -1 2 0 0 0
[pid=29062/tid=29069] ppid=29060 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 2 0 907151848 301862912 8569 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 102.12
Current children cumulated vsize (KiB) 611220

[startup+162.301 s]
/proc/loadavg: 2.02 2.02 1.85 3/187 29084
/proc/meminfo: memFree=6977836/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.6 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 12963 934 1 0 147 13 0 0 18 0 2 0 907151663 301862912 8569 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 73697 8569 1876 364 0 9615 0
[pid=29068] ppid=29062 vsize=294788 CPUtime=0.18 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 1173 0 0 0 11 7 0 0 18 0 2 0 907151848 301862912 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/29068/statm: 73697 7109 415 364 0 9615 0
[pid=29070] ppid=29068 vsize=24336 CPUtime=160.42 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) R 29068 29070 29070 0 -1 4202496 19051 0 0 0 16039 3 0 0 25 0 1 0 907151849 24920064 5285 33554432000 4194304 4352236 140734520664416 18446744073709551615 4328039 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/29070/statm: 6084 5285 129 39 0 5164 0
[pid=29068/tid=29074] ppid=29062 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29068/task/29074/stat : 29074 (python) S 29062 29062 28636 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 907161759 301862912 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604444063009856 0 0 -1 2 0 0 0
[pid=29062/tid=29069] ppid=29060 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 2 0 907151848 301862912 8569 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 162.2
Current children cumulated vsize (KiB) 613912

[startup+222.3 s]
/proc/loadavg: 2.01 2.01 1.85 3/187 29086
/proc/meminfo: memFree=6968220/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=294788 CPUtime=1.6 cores=0,2,4,6

################
# More data... #
################

[pid=29134] ppid=29062 vsize=315312 CPUtime=0.51 cores=0,2,4,6
/proc/29134/stat : 29134 (python) S 29062 29062 28636 0 -1 4202560 1152 0 0 0 28 23 0 0 19 0 2 0 907269242 322879488 7101 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/29134/statm: 78828 7101 399 364 0 14737 0
[pid=29136] ppid=29134 vsize=50084 CPUtime=388.45 cores=0,2,4,6
/proc/29136/stat : 29136 (precosat) T 29134 29136 29136 0 -1 4202496 12807 0 0 0 38843 2 0 0 25 0 1 0 907269243 51286016 12324 33554432000 4194304 4890832 140734780226336 18446744073709551615 4273122 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/29136/statm: 12521 12324 86 171 0 12348 0
[pid=29134/tid=29153] ppid=29062 vsize=315312 CPUtime=0 cores=0,2,4,6
/proc/29134/task/29153/stat : 29153 (python) S 29062 29062 28636 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 907308088 322879488 7101 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453855405184 0 0 -1 4 0 0 0
[pid=29154] ppid=29062 vsize=325568 CPUtime=0.78 cores=0,2,4,6
/proc/29154/stat : 29154 (python) S 29062 29062 28636 0 -1 4202560 1147 0 0 0 44 34 0 0 19 0 2 0 907308090 333381632 7105 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/29154/statm: 81392 7105 399 364 0 17298 0
[pid=29156] ppid=29154 vsize=40580 CPUtime=583.71 cores=0,2,4,6
/proc/29156/stat : 29156 (lingeling) T 29154 29156 29156 0 -1 4202496 80646 0 0 0 58360 11 0 0 25 0 1 0 907308090 41553920 9315 33554432000 4194304 4352236 140734770660992 18446744073709551615 4329024 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/29156/statm: 10145 9315 129 39 0 9225 0
[pid=29154/tid=29176] ppid=29062 vsize=325568 CPUtime=0 cores=0,2,4,6
/proc/29154/task/29176/stat : 29176 (python) S 29062 29062 28636 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 907366460 333381632 7105 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453985323712 0 0 -1 6 0 0 0
[pid=29177] ppid=29062 vsize=335824 CPUtime=0.26 cores=0,2,4,6
/proc/29177/stat : 29177 (python) S 29062 29062 28636 0 -1 4202560 1133 0 0 0 16 10 0 0 18 0 2 0 907366462 343883776 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/29177/statm: 83956 7109 399 364 0 19859 0
[pid=29179] ppid=29177 vsize=25712 CPUtime=197.23 cores=0,2,4,6
/proc/29179/stat : 29179 (lingeling) T 29177 29179 29179 0 -1 4202496 28255 0 0 0 19719 4 0 0 25 0 1 0 907366462 26329088 5631 33554432000 4194304 4352236 140737104826080 18446744073709551615 4328308 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/29179/statm: 6428 5631 129 39 0 5508 0
[pid=29177/tid=29183] ppid=29062 vsize=335824 CPUtime=0 cores=0,2,4,6
/proc/29177/task/29183/stat : 29183 (python) S 29062 29062 28636 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 907376474 343883776 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453856444800 0 0 -1 4 0 0 0
[pid=29184] ppid=29062 vsize=346080 CPUtime=2.93 cores=0,2,4,6
/proc/29184/stat : 29184 (python) S 29062 29062 28636 0 -1 4202560 1198 0 0 0 169 124 0 0 15 0 2 0 907376476 354385920 7114 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/29184/statm: 86520 7114 400 364 0 22420 0
[pid=29186] ppid=29184 vsize=188312 CPUtime=1933.22 cores=0,2,4,6
/proc/29186/stat : 29186 (cryptominisat-2) R 29184 29186 29186 0 -1 4202496 317007 0 0 0 193256 66 0 0 25 0 1 0 907376476 192831488 38782 33554432000 4194304 4677096 140736600194736 18446744073709551615 4320231 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/29186/statm: 47078 38782 390 118 0 42037 0
[pid=29184/tid=29206] ppid=29062 vsize=346080 CPUtime=0 cores=0,2,4,6
/proc/29184/task/29206/stat : 29206 (python) S 29062 29062 28636 0 -1 4202560 52 0 0 0 0 0 0 0 16 0 2 0 907434740 354385920 7114 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453877033984 0 0 -1 0 0 0 0
[pid=29210] ppid=29062 vsize=356336 CPUtime=0.29 cores=0,2,4,6
/proc/29210/stat : 29210 (python) S 29062 29062 28636 0 -1 4202560 1185 0 0 0 17 12 0 0 19 0 2 0 907444455 364888064 7123 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/29210/statm: 89084 7123 405 364 0 24981 0
[pid=29212] ppid=29210 vsize=37852 CPUtime=196.21 cores=0,2,4,6
/proc/29212/stat : 29212 (precosat) T 29210 29212 29212 0 -1 4202496 10105 0 0 0 19619 2 0 0 25 0 1 0 907444455 38760448 9289 33554432000 4194304 4890832 140735271608272 18446744073709551615 4293310 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/29212/statm: 9463 9289 86 171 0 9290 0
[pid=29210/tid=29218] ppid=29062 vsize=356336 CPUtime=0 cores=0,2,4,6
/proc/29210/task/29218/stat : 29218 (python) S 29062 29062 28636 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 907464082 364888064 7123 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604452249059712 0 0 -1 4 0 0 0
[pid=29062/tid=29069] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 35 934 0 0 0 0 0 0 18 0 9 0 907151848 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
[pid=29062/tid=29103] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29103/stat : 29103 (python) S 29060 29062 28636 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 9 0 907210312 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 6 0 0 0
[pid=29062/tid=29111] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29111/stat : 29111 (python) S 29060 29062 28636 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 9 0 907220327 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29135] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29135/stat : 29135 (python) S 29060 29062 28636 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 9 0 907269243 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29155] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29155/stat : 29155 (python) S 29060 29062 28636 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 9 0 907308090 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29178] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29178/stat : 29178 (python) S 29060 29062 28636 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 9 0 907366462 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29185] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29185/stat : 29185 (python) S 29060 29062 28636 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 9 0 907376476 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29211] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29211/stat : 29211 (python) S 29060 29062 28636 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 9 0 907444455 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4968.9
Current children cumulated vsize (KiB) 3518004



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+4993.4 s]
/proc/loadavg: 1.05 1.04 1.26 2/201 29305
/proc/meminfo: memFree=6900528/32951124 swapFree=67064380/67111528
[pid=29062] ppid=29060 vsize=366580 CPUtime=1.79 cores=0,2,4,6
/proc/29062/stat : 29062 (python) S 29060 29062 28636 0 -1 4202496 18812 934 1 0 164 15 0 0 18 0 9 0 907151663 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29062/statm: 91645 8611 1890 364 0 27542 0
[pid=29068] ppid=29062 vsize=294788 CPUtime=1.33 cores=0,2,4,6
/proc/29068/stat : 29068 (python) S 29062 29062 28636 0 -1 4202560 1208 0 0 0 75 58 0 0 18 0 2 0 907151848 301862912 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/29068/statm: 73697 7109 415 364 0 9615 0
[pid=29070] ppid=29068 vsize=55836 CPUtime=1072.18 cores=0,2,4,6
/proc/29070/stat : 29070 (lingeling) T 29068 29070 29070 0 -1 4202496 132193 0 0 0 107199 19 0 0 25 0 1 0 907151849 57176064 13160 33554432000 4194304 4352236 140734520664416 18446744073709551615 4277074 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/29070/statm: 13959 13160 129 39 0 13039 0
[pid=29068/tid=29074] ppid=29062 vsize=294788 CPUtime=0 cores=0,2,4,6
/proc/29068/task/29074/stat : 29074 (python) S 29062 29062 28636 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 907161759 301862912 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604444063009856 0 0 -1 2 0 0 0
[pid=29102] ppid=29062 vsize=294800 CPUtime=0.12 cores=0,2,4,6
/proc/29102/stat : 29102 (python) S 29062 29062 28636 0 -1 4202560 1093 0 0 0 7 5 0 0 19 0 2 0 907210312 301875200 7093 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/29102/statm: 73700 7093 399 364 0 9615 0
[pid=29104] ppid=29102 vsize=55352 CPUtime=100.08 cores=0,2,4,6
/proc/29104/stat : 29104 (SApperloT-base) T 29102 29104 29104 0 -1 4202496 13719 0 0 0 10005 3 0 0 25 0 1 0 907210312 56680448 13425 33554432000 134512640 135674940 4288817200 18446744073709551615 134564219 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/29104/statm: 13838 13425 100 284 0 13552 0
[pid=29102/tid=29109] ppid=29062 vsize=294800 CPUtime=0 cores=0,2,4,6
/proc/29102/task/29109/stat : 29109 (python) S 29062 29062 28636 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 907220324 301875200 7093 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453857728320 0 0 -1 0 0 0 0
[pid=29110] ppid=29062 vsize=305056 CPUtime=0.67 cores=0,2,4,6
/proc/29110/stat : 29110 (python) S 29062 29062 28636 0 -1 4202560 1187 0 0 0 37 30 0 0 16 0 2 0 907220326 312377344 7098 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/29110/statm: 76264 7098 400 364 0 12176 0
[pid=29112] ppid=29110 vsize=123932 CPUtime=489.14 cores=0,2,4,6
/proc/29112/stat : 29112 (cryptominisat-2) T 29110 29112 29112 0 -1 4202496 120154 0 0 0 48888 26 0 0 25 0 1 0 907220327 126906368 24025 33554432000 4194304 4677096 140734694806832 18446744073709551615 4320006 0 0 16781312 2 18446744071562691360 0 0 17 0 0 0 0
/proc/29112/statm: 30983 24025 390 118 0 25942 0
[pid=29110/tid=29133] ppid=29062 vsize=305056 CPUtime=0 cores=0,2,4,6
/proc/29110/task/29133/stat : 29133 (python) S 29062 29062 28636 0 -1 4202560 44 0 0 0 0 0 0 0 16 0 2 0 907269240 312377344 7098 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604444063008960 0 0 -1 2 0 0 0
[pid=29134] ppid=29062 vsize=315312 CPUtime=0.51 cores=0,2,4,6
/proc/29134/stat : 29134 (python) S 29062 29062 28636 0 -1 4202560 1152 0 0 0 28 23 0 0 19 0 2 0 907269242 322879488 7101 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/29134/statm: 78828 7101 399 364 0 14737 0
[pid=29136] ppid=29134 vsize=50084 CPUtime=388.45 cores=0,2,4,6
/proc/29136/stat : 29136 (precosat) T 29134 29136 29136 0 -1 4202496 12807 0 0 0 38843 2 0 0 25 0 1 0 907269243 51286016 12324 33554432000 4194304 4890832 140734780226336 18446744073709551615 4273122 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/29136/statm: 12521 12324 86 171 0 12348 0
[pid=29134/tid=29153] ppid=29062 vsize=315312 CPUtime=0 cores=0,2,4,6
/proc/29134/task/29153/stat : 29153 (python) S 29062 29062 28636 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 907308088 322879488 7101 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453855405184 0 0 -1 4 0 0 0
[pid=29154] ppid=29062 vsize=325568 CPUtime=0.78 cores=0,2,4,6
/proc/29154/stat : 29154 (python) S 29062 29062 28636 0 -1 4202560 1147 0 0 0 44 34 0 0 19 0 2 0 907308090 333381632 7105 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/29154/statm: 81392 7105 399 364 0 17298 0
[pid=29156] ppid=29154 vsize=40580 CPUtime=583.71 cores=0,2,4,6
/proc/29156/stat : 29156 (lingeling) T 29154 29156 29156 0 -1 4202496 80646 0 0 0 58360 11 0 0 25 0 1 0 907308090 41553920 9315 33554432000 4194304 4352236 140734770660992 18446744073709551615 4329024 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/29156/statm: 10145 9315 129 39 0 9225 0
[pid=29154/tid=29176] ppid=29062 vsize=325568 CPUtime=0 cores=0,2,4,6
/proc/29154/task/29176/stat : 29176 (python) S 29062 29062 28636 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 907366460 333381632 7105 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453985323712 0 0 -1 6 0 0 0
[pid=29177] ppid=29062 vsize=335824 CPUtime=0.26 cores=0,2,4,6
/proc/29177/stat : 29177 (python) S 29062 29062 28636 0 -1 4202560 1133 0 0 0 16 10 0 0 18 0 2 0 907366462 343883776 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/29177/statm: 83956 7109 399 364 0 19859 0
[pid=29179] ppid=29177 vsize=25712 CPUtime=197.23 cores=0,2,4,6
/proc/29179/stat : 29179 (lingeling) T 29177 29179 29179 0 -1 4202496 28255 0 0 0 19719 4 0 0 25 0 1 0 907366462 26329088 5631 33554432000 4194304 4352236 140737104826080 18446744073709551615 4328308 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/29179/statm: 6428 5631 129 39 0 5508 0
[pid=29177/tid=29183] ppid=29062 vsize=335824 CPUtime=0 cores=0,2,4,6
/proc/29177/task/29183/stat : 29183 (python) S 29062 29062 28636 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 907376474 343883776 7109 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453856444800 0 0 -1 4 0 0 0
[pid=29184] ppid=29062 vsize=346080 CPUtime=2.97 cores=0,2,4,6
/proc/29184/stat : 29184 (python) S 29062 29062 28636 0 -1 4202560 1198 0 0 0 171 126 0 0 15 0 2 0 907376476 354385920 7114 33554432000 4194304 5683644 140733664339824 18446744073709551615 244446965318 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/29184/statm: 86520 7114 400 364 0 22420 0
[pid=29186] ppid=29184 vsize=188312 CPUtime=1964.32 cores=0,2,4,6
/proc/29186/stat : 29186 (cryptominisat-2) R 29184 29186 29186 0 -1 4202496 318661 0 0 0 196366 66 0 0 25 0 1 0 907376476 192831488 40436 33554432000 4194304 4677096 140736600194736 18446744073709551615 4320006 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/29186/statm: 47078 40436 390 118 0 42037 0
[pid=29184/tid=29206] ppid=29062 vsize=346080 CPUtime=0 cores=0,2,4,6
/proc/29184/task/29206/stat : 29206 (python) S 29062 29062 28636 0 -1 4202560 52 0 0 0 0 0 0 0 16 0 2 0 907434740 354385920 7114 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604453877033984 0 0 -1 0 0 0 0
[pid=29210] ppid=29062 vsize=356336 CPUtime=0.29 cores=0,2,4,6
/proc/29210/stat : 29210 (python) S 29062 29062 28636 0 -1 4202560 1185 0 0 0 17 12 0 0 19 0 2 0 907444455 364888064 7123 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458771915 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/29210/statm: 89084 7123 405 364 0 24981 0
[pid=29212] ppid=29210 vsize=37852 CPUtime=196.21 cores=0,2,4,6
/proc/29212/stat : 29212 (precosat) T 29210 29212 29212 0 -1 4202496 10105 0 0 0 19619 2 0 0 25 0 1 0 907444455 38760448 9289 33554432000 4194304 4890832 140735271608272 18446744073709551615 4293310 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/29212/statm: 9463 9289 86 171 0 9290 0
[pid=29210/tid=29218] ppid=29062 vsize=356336 CPUtime=0 cores=0,2,4,6
/proc/29210/task/29218/stat : 29218 (python) S 29062 29062 28636 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 907464082 364888064 7123 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 514 18446604452249059712 0 0 -1 4 0 0 0
[pid=29062/tid=29069] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29069/stat : 29069 (python) S 29060 29062 28636 0 -1 4202560 35 934 0 0 0 0 0 0 18 0 9 0 907151848 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
[pid=29062/tid=29103] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29103/stat : 29103 (python) S 29060 29062 28636 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 9 0 907210312 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 6 0 0 0
[pid=29062/tid=29111] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29111/stat : 29111 (python) S 29060 29062 28636 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 9 0 907220327 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29135] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29135/stat : 29135 (python) S 29060 29062 28636 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 9 0 907269243 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29155] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29155/stat : 29155 (python) S 29060 29062 28636 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 9 0 907308090 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29178] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29178/stat : 29178 (python) S 29060 29062 28636 0 -1 4202560 31 934 0 0 0 0 0 0 18 0 9 0 907366462 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29185] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29185/stat : 29185 (python) S 29060 29062 28636 0 -1 4202560 30 934 0 0 0 0 0 0 18 0 9 0 907376476 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 2 0 0 0
[pid=29062/tid=29211] ppid=29060 vsize=366580 CPUtime=0 cores=0,2,4,6
/proc/29062/task/29211/stat : 29211 (python) S 29060 29062 28636 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 9 0 907444455 375377920 8611 33554432000 4194304 5683644 140733664339824 18446744073709551615 244458768817 0 0 16781312 2 18446604453888666496 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5000.04
Current children cumulated vsize (KiB) 3518004

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29062 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=647749
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=160975
# CPU time returned by wait4() is 1.80872
# while last known CPU time is 5000.04
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 4993.46
CPU time (s): 5000.04
CPU user time (s): 4995.58
CPU system time (s): 4.46
CPU usage (%): 100.132
Max. virtual memory (cumulated for all children) (KiB): 3606408

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.64775
system time used= 0.160975
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 19750
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2891
involuntary context switches= 36

runsolver used 17.7863 second user time and 34.9287 second system time

The end

Launcher Data 

Begin job on node105 at 2011-06-13 10:58:32
IDJOB=3498488
IDBENCH=84180
IDSOLVER=1991
FILE ID=node105/3498488-1307955512
RUNJOBID= node105-1307953128-28654
PBS_JOBID= 13559723
Free space on /var/tmp= 2856 MiB

SOLVER NAME= borg-sat 11.03.02-seq-industrial (fixed)
BENCH NAME= SAT03/industrial/maris/CNF/gripper13u.shuffled-as.sat03-395.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /var/tmp/evaluation-result-3498488-1307955512/watcher-3498488-1307955512 -o /var/tmp/evaluation-result-3498488-1307955512/solver-3498488-1307955512 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3498488-1307955512.cnf -seed 356937537 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3889b7ed90cc1fbe8bc8d140a1763832
RANDOM SEED=356937537

node105.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5410.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.825
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       7032672 kB
Buffers:        100024 kB
Cached:       24564484 kB
SwapCached:      19904 kB
Active:        9640248 kB
Inactive:     15458856 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       7032672 kB
SwapTotal:    67111528 kB
SwapFree:     67064380 kB
Dirty:           32248 kB
Writeback:           0 kB
AnonPages:      432344 kB
Mapped:          16408 kB
Slab:           750540 kB
PageTables:       6876 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   804804 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /var/tmp at the end= 2860 MiB
End job on node105 at 2011-06-13 12:21:48