Trace number 3498268

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-industrial (fixed)? (TO) 5000.1 5022.56

General information on the benchmark

Nameindustrial/IBM_FV_2002_13_rule_1/
SAT_dat.k20.cnf
MD5SUM2bee695ce4278b72ee9295631ad94252
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark162.983
Satisfiable
(Un)Satisfiability was proved
Number of variables31357
Number of clauses133420
Sum of the clauses size370492
Maximum clause length37
Minimum clause length1
Number of clauses of size 1278
Number of clauses of size 258046
Number of clauses of size 369900
Number of clauses of size 4166
Number of clauses of size 5305
Number of clauses of size over 54725

Solver Data 

0.56/28.83	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.56/28.83	c INFO: solving HOME/instance-3498268-1307953114.cnf
0.56/28.83	c INFO: computing task features
0.92/29.29	c INFO: computed variable-clause graph statistics
1.02/29.31	c INFO: computed clause balance statistics
3.12/31.49	c INFO: computed variable balance statistics
4.82/33.12	c INFO: computed small-clause counts
4.82/33.14	c INFO: computed Horn-clause counts
6.02/34.39	c INFO: computed variable graph statistics
6.02/34.39	c INFO: collected features for HOME/instance-3498268-1307953114.cnf in 5.54 s
6.12/34.42	c INFO: running lingeling-276@0 for 97 with 4994 remaining (b = 0.30)
105.30/133.56	c INFO: running cryptominisat-2.9.0@0 for 485 with 4895 remaining (b = 0.31)
595.18/622.83	c INFO: running lingeling-276@99 for 97 with 4406 remaining (b = 0.12)
692.39/719.96	c INFO: running lingeling-276@0 for 97 with 4309 remaining (b = 0.16)
792.58/820.09	c INFO: running lingeling-276@196 for 97 with 4209 remaining (b = 0.10)
889.89/917.22	c INFO: running lingeling-276@293 for 291 with 4112 remaining (b = 0.17)
1182.52/1209.54	c INFO: running lingeling-276@100 for 97 with 3820 remaining (b = 0.09)
1279.70/1306.67	c INFO: running precosat-570@0 for 97 with 3722 remaining (b = 0.14)
1379.93/1406.82	c INFO: running cryptominisat-2.9.0@488 for 97 with 3622 remaining (b = 0.07)
1476.36/1503.17	c INFO: running lingeling-276@197 for 97 with 3526 remaining (b = 0.07)
1573.67/1600.30	c INFO: running precosat-570@99 for 194 with 3429 remaining (b = 0.12)
1766.96/1793.55	c INFO: running SApperloT@0 for 97 with 3236 remaining (b = 0.09)
1867.26/1893.70	c INFO: running lingeling-276@294 for 97 with 3136 remaining (b = 0.06)
1964.48/1990.83	c INFO: running lingeling-276@391 for 97 with 3039 remaining (b = 0.06)
2059.71/2085.96	c INFO: running precosat-570@293 for 97 with 2944 remaining (b = 0.06)
2157.08/2183.11	c INFO: running lingeling-276@0 for 97 with 2847 remaining (b = 0.09)
2257.26/2283.26	c INFO: running cryptominisat-2.9.0@0 for 485 with 2746 remaining (b = 0.18)
2746.34/2771.74	c INFO: running lingeling-276@100 for 97 with 2259 remaining (b = 0.04)
2843.57/2868.87	c INFO: running precosat-570@0 for 97 with 2162 remaining (b = 0.04)
2943.87/2969.02	c INFO: running lingeling-276@585 for 679 with 2061 remaining (b = 0.20)
3620.56/3644.84	c INFO: running cryptominisat-2.9.0@584 for 1386 with 1386 remaining (b = 0.38)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.1 (svn: 989)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /var/tmp/evaluation-result-3498268-1307953114/watcher-3498268-1307953114 -o /var/tmp/evaluation-result-3498268-1307953114/solver-3498268-1307953114 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3498268-1307953114.cnf -seed 462826349 -budget 5000 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 1/171 3800
/proc/meminfo: memFree=3922764/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=11804 CPUtime=0 cores=0,2,4,6
/proc/3800/stat : 3800 (runsolver) S 3798 3800 3763 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 1 0 887727359 12087296 50 33554432000 4194304 5436572 140737177222208 18446744073709551615 4883175 0 2147483391 4096 24578 18446744071699968823 0 0 17 6 0 0 0
/proc/3800/statm: 2951 50 31 304 0 2643 0

[startup+0.00816804 s]
/proc/loadavg: 0.00 0.00 0.00 1/171 3800
/proc/meminfo: memFree=3922764/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=11804 CPUtime=0 cores=0,2,4,6
/proc/3800/stat : 3800 (runsolver) S 3798 3800 3763 0 -1 4202560 42 0 0 0 0 0 0 0 20 0 1 0 887727359 12087296 50 33554432000 4194304 5436572 140737177222208 18446744073709551615 4883175 0 2147483391 4096 24578 18446744071699968823 0 0 17 6 0 0 0
/proc/3800/statm: 2951 50 31 304 0 2643 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 11804

[startup+0.101172 s]
/proc/loadavg: 0.00 0.00 0.00 1/171 3800
/proc/meminfo: memFree=3922764/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=11804 CPUtime=0 cores=0,2,4,6
/proc/3800/stat : 3800 (runsolver) S 3798 3800 3763 0 -1 4202560 47 0 0 0 0 0 0 0 21 0 1 0 887727359 12087296 55 33554432000 4194304 5436572 140737177222208 18446744073709551615 5057063 0 2147483391 4096 24578 18446744071699207358 0 0 17 4 0 0 0
/proc/3800/statm: 2951 55 36 304 0 2643 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 11804

[startup+0.301123 s]
/proc/loadavg: 0.00 0.00 0.00 1/171 3800
/proc/meminfo: memFree=3922764/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=11804 CPUtime=0 cores=0,2,4,6
/proc/3800/stat : 3800 (runsolver) S 3798 3800 3763 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 1 0 887727359 12087296 55 33554432000 4194304 5436572 140737177222208 18446744073709551615 5057063 0 2147483391 4096 24578 18446744071699207358 0 0 17 4 0 0 0
/proc/3800/statm: 2951 55 36 304 0 2643 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 11804

[startup+0.70108 s]
/proc/loadavg: 0.00 0.00 0.00 1/171 3800
/proc/meminfo: memFree=3922764/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=1840 CPUtime=0 cores=0,2,4,6
/proc/3800/stat : 3800 (python) D 3798 3800 3763 0 -1 4194304 48 0 0 0 0 0 0 0 17 0 1 0 887727359 1884160 1 33554432000 0 0 140736988911235 18446744073709551615 5057063 0 0 4096 0 18446744071562234166 0 0 17 6 0 0 10
/proc/3800/statm: 460 1 0 0 0 36 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 1840

[startup+1.50089 s]
/proc/loadavg: 0.00 0.00 0.00 2/172 3801
/proc/meminfo: memFree=3892360/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=14820 CPUtime=0 cores=0,2,4,6
/proc/3800/stat : 3800 (python) S 3798 3800 3763 0 -1 4202496 506 0 7 0 0 0 0 0 15 0 1 0 887727359 15175680 419 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726400869 0 2147483391 4096 0 18446744071699207358 0 0 17 4 0 0 58
/proc/3800/statm: 3705 419 259 364 0 192 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 14820

[startup+3.10059 s]
/proc/loadavg: 0.00 0.00 0.00 2/172 3801
/proc/meminfo: memFree=3848468/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/3800/stat : 3800 (python) S 3798 3800 3763 0 -1 4202496 805 0 7 0 0 0 0 0 15 0 1 0 887727359 15749120 696 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726403072 0 2147483391 16781312 2 18446744071699207358 0 0 17 4 0 0 66
/proc/3800/statm: 3845 696 334 364 0 332 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15380
heavy processes:

[startup+6.30097 s]
/proc/loadavg: 0.08 0.02 0.01 1/175 3811
/proc/meminfo: memFree=3798072/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=25316 CPUtime=0.04 cores=0,2,4,6
/proc/3800/stat : 3800 (python) D 3798 3800 3763 0 -1 4202496 1339 0 11 0 3 1 0 0 16 0 1 0 887727359 25923584 1194 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726403072 0 0 16781312 2 0 0 0 17 4 0 0 85
/proc/3800/statm: 6329 1194 438 364 0 730 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 25316
heavy processes:

[startup+12.7008 s]
/proc/loadavg: 0.15 0.03 0.01 1/175 3811
/proc/meminfo: memFree=3792484/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=50012 CPUtime=0.07 cores=0,2,4,6
/proc/3800/stat : 3800 (python) D 3798 3800 3763 0 -1 4202496 1937 0 18 0 4 3 0 0 16 0 1 0 887727359 51212288 1717 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726403072 0 0 16781312 2 0 0 0 17 4 0 0 108
/proc/3800/statm: 12503 1717 476 364 0 1203 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 50012
heavy processes:

[startup+25.5003 s]
/proc/loadavg: 0.19 0.05 0.01 1/175 3819
/proc/meminfo: memFree=3744296/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=221924 CPUtime=0.4 cores=0,2,4,6
/proc/3800/stat : 3800 (python) D 3798 3800 3763 0 -1 4202496 7408 925 89 8 26 14 0 0 16 0 1 0 887727359 227250176 5480 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726404864 0 0 16781312 2 18446744071562234166 0 0 17 2 0 0 307
/proc/3800/statm: 55481 5480 1544 364 0 4134 0
Current children cumulated CPU time (s) 0.4
Current children cumulated vsize (KiB) 221924
heavy processes:

[startup+51.1005 s]
/proc/loadavg: 0.69 0.17 0.05 3/178 3823
/proc/meminfo: memFree=3067872/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=316044 CPUtime=6.14 cores=0,2,4,6
/proc/3800/stat : 3800 (python) S 3798 3800 3763 0 -1 4202496 20700 925 116 8 593 21 0 0 19 0 2 0 887727359 323629056 13882 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 370
/proc/3800/statm: 79011 13882 1875 364 0 14929 0
[pid=3821] ppid=3800 vsize=305800 CPUtime=0.01 cores=0,2,4,6
/proc/3821/stat : 3821 (python) S 3800 3800 3763 0 -1 4202560 985 0 0 0 0 1 0 0 18 0 1 0 887730801 313139200 12384 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726425103 0 0 16781312 514 11368334504692252932 0 0 17 6 0 0 0
/proc/3821/statm: 76450 12384 381 364 0 12368 0
[pid=3823] ppid=3821 vsize=16104 CPUtime=16.65 cores=0,2,4,6
/proc/3823/stat : 3823 (lingeling) R 3821 3823 3823 0 -1 4202496 6102 0 0 0 1665 0 0 0 25 0 1 0 887730802 16490496 3090 33554432000 4194304 4352236 140737427087312 18446744073709551615 4328109 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/3823/statm: 4026 3090 129 39 0 3106 0
[pid=3800/tid=3822] ppid=3798 vsize=316044 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3822/stat : 3822 (python) S 3798 3800 3763 0 -1 4202560 30 925 0 8 0 0 0 0 19 0 2 0 887730802 323629056 13882 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 22.8
Current children cumulated vsize (KiB) 637948
heavy processes:

[startup+102.307 s]
/proc/loadavg: 1.43 0.45 0.15 3/178 3824
/proc/meminfo: memFree=3057740/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=316044 CPUtime=6.14 cores=0,2,4,6
/proc/3800/stat : 3800 (python) S 3798 3800 3763 0 -1 4202496 20700 925 116 8 593 21 0 0 19 0 2 0 887727359 323629056 13882 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 370
/proc/3800/statm: 79011 13882 1875 364 0 14929 0
[pid=3821] ppid=3800 vsize=305800 CPUtime=0.07 cores=0,2,4,6
/proc/3821/stat : 3821 (python) S 3800 3800 3763 0 -1 4202560 1005 0 0 0 3 4 0 0 18 0 1 0 887730801 313139200 12384 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726425103 0 0 16781312 514 11368334504692252932 0 0 17 6 0 0 0
/proc/3821/statm: 76450 12384 381 364 0 12368 0
[pid=3823] ppid=3821 vsize=26784 CPUtime=67.86 cores=0,2,4,6
/proc/3823/stat : 3823 (lingeling) R 3821 3823 3823 0 -1 4202496 9200 0 0 0 6785 1 0 0 25 0 1 0 887730802 27426816 5866 33554432000 4194304 4352236 140737427087312 18446744073709551615 4328159 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/3823/statm: 6696 5866 129 39 0 5776 0
[pid=3800/tid=3822] ppid=3798 vsize=316044 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3822/stat : 3822 (python) S 3798 3800 3763 0 -1 4202560 30 925 0 8 0 0 0 0 19 0 2 0 887730802 323629056 13882 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 74.07
Current children cumulated vsize (KiB) 648628
heavy processes:

[startup+162.301 s]
/proc/loadavg: 1.84 0.75 0.27 3/185 3833
/proc/meminfo: memFree=2994344/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=326300 CPUtime=6.16 cores=0,2,4,6
/proc/3800/stat : 3800 (python) S 3798 3800 3763 0 -1 4202496 21495 925 116 8 595 21 0 0 20 0 3 0 887727359 334131200 13889 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 370
/proc/3800/statm: 81575 13889 1878 364 0 17490 0
[pid=3821] ppid=3800 vsize=316044 CPUtime=0.11 cores=0,2,4,6
/proc/3821/stat : 3821 (python) S 3800 3800 3763 0 -1 4202560 1173 0 0 0 5 6 0 0 19 0 2 0 887730801 323629056 12422 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/3821/statm: 79011 12422 414 364 0 14929 0
[pid=3823] ppid=3821 vsize=26784 CPUtime=99.09 cores=0,2,4,6
/proc/3823/stat : 3823 (lingeling) T 3821 3823 3823 0 -1 4202496 9200 0 0 0 9908 1 0 0 25 0 1 0 887730802 27426816 5866 33554432000 4194304 4352236 140737427087312 18446744073709551615 4329060 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/3823/statm: 6696 5866 129 39 0 5776 0
[pid=3821/tid=3826] ppid=3800 vsize=316044 CPUtime=0 cores=0,2,4,6
/proc/3821/task/3826/stat : 3826 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 887740712 323629056 12422 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453947385920 0 0 -1 0 0 0 0
[pid=3827] ppid=3800 vsize=316056 CPUtime=0.04 cores=0,2,4,6
/proc/3827/stat : 3827 (python) S 3800 3800 3763 0 -1 4202560 929 0 0 0 2 2 0 0 15 0 1 0 887740715 323641344 12377 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726425158 0 0 16781312 514 4609588154536526084 0 0 17 6 0 0 0
/proc/3827/statm: 79014 12377 370 364 0 14929 0
[pid=3829] ppid=3827 vsize=197216 CPUtime=28.72 cores=0,2,4,6
/proc/3829/stat : 3829 (cryptominisat-2) R 3827 3829 3829 0 -1 4202496 54864 0 1 0 2864 8 0 0 25 0 1 0 887740715 201949184 39044 33554432000 4194304 4677096 140735262188624 18446744073709551615 4319160 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3829/statm: 49304 39044 393 118 0 44263 0
[pid=3800/tid=3822] ppid=3798 vsize=326300 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3822/stat : 3822 (python) S 3798 3800 3763 0 -1 4202560 30 925 0 8 0 0 0 0 19 0 3 0 887730802 334131200 13889 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
[pid=3800/tid=3828] ppid=3798 vsize=326300 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3828/stat : 3828 (python) S 3798 3800 3763 0 -1 4202560 28 925 0 8 0 0 0 0 20 0 3 0 887740715 334131200 13889 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 134.12
Current children cumulated vsize (KiB) 1182400

[startup+222.301 s]
/proc/loadavg: 1.94 0.97 0.38 3/185 3835
/proc/meminfo: memFree=2932216/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=326300 CPUtime=6.16 cores=0,2,4,6
/proc/3800/stat : 3800 (python) S 3798 3800 3763 0 -1 4202496 21495 925 116 8 595 21 0 0 20 0 3 0 887727359 334131200 13889 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 370
/proc/3800/statm: 81575 13889 1878 364 0 17490 0
[pid=3821] ppid=3800 vsize=316044 CPUtime=0.11 cores=0,2,4,6
/proc/3821/stat : 3821 (python) S 3800 3800 3763 0 -1 4202560 1173 0 0 0 5 6 0 0 19 0 2 0 887730801 323629056 12422 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/3821/statm: 79011 12422 414 364 0 14929 0
[pid=3823] ppid=3821 vsize=26784 CPUtime=99.09 cores=0,2,4,6
/proc/3823/stat : 3823 (lingeling) T 3821 3823 3823 0 -1 4202496 9200 0 0 0 9908 1 0 0 25 0 1 0 887730802 27426816 5866 33554432000 4194304 4352236 140737427087312 18446744073709551615 4329060 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/3823/statm: 6696 5866 129 39 0 5776 0
[pid=3821/tid=3826] ppid=3800 vsize=316044 CPUtime=0 cores=0,2,4,6
/proc/3821/task/3826/stat : 3826 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 887740712 323629056 12422 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453947385920 0 0 -1 0 0 0 0
[pid=3827] ppid=3800 vsize=316056 CPUtime=0.12 cores=0,2,4,6
/proc/3827/stat : 3827 (python) S 3800 3800 3763 0 -1 4202560 959 0 0 0 7 5 0 0 15 0 1 0 887740715 323641344 12377 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726425158 0 0 16781312 514 11368334504692252932 0 0 17 4 0 0 0
/proc/3827/statm: 79014 12377 370 364 0 14929 0
[pid=3829] ppid=3827 vsize=236740 CPUtime=88.71 cores=0,2,4,6
/proc/3829/stat : 3829 (cryptominisat-2) R 3827 3829 3829 0 -1 4202496 100981 0 1 0 8856 15 0 0 25 0 1 0 887740715 242421760 47607 33554432000 4194304 4677096 140735262188624 18446744073709551615 4465929 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/3829/statm: 59185 47607 393 118 0 54144 0
[pid=3800/tid=3822] ppid=3798 vsize=326300 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3822/stat : 3822 (python) S 3798 3800 3763 0 -1 4202560 30 925 0 8 0 0 0 0 19 0 3 0 887730802 334131200 13889 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
[pid=3800/tid=3828] ppid=3798 vsize=326300 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3828/stat : 3828 (python) S 3798 3800 3763 0 -1 4202560 28 925 0 8 0 0 0 0 20 0 3 0 887740715 334131200 13889 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0

################
# More data... #
################

/proc/3860/stat : 3860 (lingeling) T 3858 3860 3860 0 -1 4202496 12622 0 0 0 48645 2 0 0 25 0 1 0 887799355 29208576 6238 33554432000 4194304 4352236 140737260682336 18446744073709551615 4328359 0 0 16781312 17506 18446744071562691360 0 0 17 4 0 0 0
/proc/3860/statm: 7131 6238 129 39 0 6211 0
[pid=3858/tid=3863] ppid=3800 vsize=326312 CPUtime=0 cores=0,2,4,6
/proc/3858/task/3863/stat : 3863 (python) S 3800 3800 3763 0 -1 4202560 50 0 0 0 0 0 0 0 18 0 2 0 887809366 334143488 12411 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453943019456 0 0 -1 2 0 0 0
[pid=3884] ppid=3800 vsize=336568 CPUtime=0.55 cores=0,2,4,6
/proc/3884/stat : 3884 (python) S 3800 3800 3763 0 -1 4202560 1174 0 0 0 30 25 0 0 18 0 2 0 887858026 344645632 12415 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/3884/statm: 84142 12415 399 364 0 20051 0
[pid=3886] ppid=3884 vsize=66328 CPUtime=390.19 cores=0,2,4,6
/proc/3886/stat : 3886 (precosat) T 3884 3886 3886 0 -1 4202496 17795 0 0 0 39016 3 0 0 25 0 1 0 887858026 67919872 16276 33554432000 4194304 4890832 140735791669680 18446744073709551615 4238594 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/3886/statm: 16582 16276 86 171 0 16409 0
[pid=3884/tid=3890] ppid=3800 vsize=336568 CPUtime=0 cores=0,2,4,6
/proc/3884/task/3890/stat : 3890 (python) S 3800 3800 3763 0 -1 4202560 51 0 0 0 0 0 0 0 18 0 2 0 887868038 344645632 12415 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453956606656 0 0 -1 6 0 0 0
[pid=3902] ppid=3800 vsize=346824 CPUtime=0.13 cores=0,2,4,6
/proc/3902/stat : 3902 (python) S 3800 3800 3763 0 -1 4202560 1134 0 0 0 9 4 0 0 19 0 2 0 887906714 355147776 12419 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3902/statm: 86706 12419 399 364 0 22612 0
[pid=3904] ppid=3902 vsize=80656 CPUtime=100.1 cores=0,2,4,6
/proc/3904/stat : 3904 (SApperloT-base) T 3902 3904 3904 0 -1 4202496 20046 0 0 0 10007 3 0 0 25 0 1 0 887906715 82591744 19681 33554432000 134512640 135674940 4294785760 18446744073709551615 134552120 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/3904/statm: 20164 19681 100 284 0 19878 0
[pid=3902/tid=3908] ppid=3800 vsize=346824 CPUtime=0 cores=0,2,4,6
/proc/3902/task/3908/stat : 3908 (python) S 3800 3800 3763 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 887916727 355147776 12419 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453961883904 0 0 -1 4 0 0 0
[pid=3925] ppid=3800 vsize=357080 CPUtime=0.27 cores=0,2,4,6
/proc/3925/stat : 3925 (python) S 3800 3800 3763 0 -1 4202560 1157 0 0 0 16 11 0 0 18 0 2 0 887945670 365649920 12423 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3925/statm: 89270 12423 399 364 0 25173 0
[pid=3927] ppid=3925 vsize=27140 CPUtime=197.18 cores=0,2,4,6
/proc/3927/stat : 3927 (lingeling) T 3925 3927 3927 0 -1 4202496 9600 0 0 0 19717 1 0 0 25 0 1 0 887945671 27791360 5955 33554432000 4194304 4352236 140733553993856 18446744073709551615 4241364 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/3927/statm: 6785 5955 129 39 0 5865 0
[pid=3925/tid=3933] ppid=3800 vsize=357080 CPUtime=0 cores=0,2,4,6
/proc/3925/task/3933/stat : 3933 (python) S 3800 3800 3763 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 887955683 365649920 12423 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453946852544 0 0 -1 6 0 0 0
[pid=3934] ppid=3800 vsize=367336 CPUtime=0.72 cores=0,2,4,6
/proc/3934/stat : 3934 (python) S 3800 3800 3763 0 -1 4202560 1176 0 0 0 41 31 0 0 16 0 2 0 887955685 376152064 12427 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3934/statm: 91834 12427 399 364 0 27734 0
[pid=3936] ppid=3934 vsize=401000 CPUtime=488.33 cores=0,2,4,6
/proc/3936/stat : 3936 (cryptominisat-2) T 3934 3936 3936 0 -1 4202496 287289 0 0 0 48778 55 0 0 25 0 1 0 887955685 410624000 88627 33554432000 4194304 4677096 140735635675568 18446744073709551615 4465886 0 0 16781312 2 18446744071562691360 0 0 17 0 0 0 0
/proc/3936/statm: 100250 88627 393 118 0 95209 0
[pid=3934/tid=3963] ppid=3800 vsize=367336 CPUtime=0 cores=0,2,4,6
/proc/3934/task/3963/stat : 3963 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 888004531 376152064 12427 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453921675264 0 0 -1 4 0 0 0
[pid=3968] ppid=3800 vsize=377592 CPUtime=0.14 cores=0,2,4,6
/proc/3968/stat : 3968 (python) S 3800 3800 3763 0 -1 4202560 1179 0 0 0 8 6 0 0 19 0 2 0 888014246 386654208 12437 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3968/statm: 94398 12437 405 364 0 30295 0
[pid=3970] ppid=3968 vsize=45556 CPUtime=100.09 cores=0,2,4,6
/proc/3970/stat : 3970 (precosat) T 3968 3970 3970 0 -1 4202496 11847 0 0 0 10008 1 0 0 25 0 1 0 888014246 46649344 11085 33554432000 4194304 4890832 140737046556368 18446744073709551615 4299152 0 0 16781312 17442 18446744071562691360 0 0 17 0 0 0 0
/proc/3970/statm: 11389 11085 86 171 0 11216 0
[pid=3968/tid=3974] ppid=3800 vsize=377592 CPUtime=0 cores=0,2,4,6
/proc/3968/task/3974/stat : 3974 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 888024259 386654208 12437 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604440279314816 0 0 -1 0 0 0 0
[pid=3800/tid=3822] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3822/stat : 3822 (python) S 3798 3800 3763 0 -1 4202560 43 925 0 8 0 0 0 0 18 0 9 0 887730802 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
[pid=3800/tid=3828] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3828/stat : 3828 (python) S 3798 3800 3763 0 -1 4202560 37 925 0 8 0 0 0 0 18 0 9 0 887740715 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
[pid=3800/tid=3859] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3859/stat : 3859 (python) S 3798 3800 3763 0 -1 4202560 36 925 0 8 0 0 0 0 18 0 9 0 887799355 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 0 0 0 0
[pid=3800/tid=3885] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3885/stat : 3885 (python) S 3798 3800 3763 0 -1 4202560 26 925 0 8 0 0 0 0 18 0 9 0 887858026 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 2 0 0 0
[pid=3800/tid=3903] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3903/stat : 3903 (python) S 3798 3800 3763 0 -1 4202560 28 925 0 8 0 0 0 0 20 0 9 0 887906714 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 0 0 0 0
[pid=3800/tid=3926] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3926/stat : 3926 (python) S 3798 3800 3763 0 -1 4202560 33 925 0 8 0 0 0 0 18 0 9 0 887945671 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 4 0 0 0
[pid=3800/tid=3935] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3935/stat : 3935 (python) S 3798 3800 3763 0 -1 4202560 31 925 0 8 0 0 0 0 20 0 9 0 887955685 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 4 0 0 0
[pid=3800/tid=3969] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3969/stat : 3969 (python) S 3798 3800 3763 0 -1 4202560 32 925 0 8 0 0 0 0 20 0 9 0 888014246 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5000.1
Current children cumulated vsize (KiB) 4628000

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5022.5 s]
/proc/loadavg: 2.07 2.03 1.98 3/218 4177
/proc/meminfo: memFree=5712900/32951124 swapFree=67064576/67111528
[pid=3800] ppid=3798 vsize=387836 CPUtime=6.54 cores=0,2,4,6
/proc/3800/stat : 3800 (python) S 3798 3800 3763 0 -1 4202496 27681 925 116 8 631 23 0 0 18 0 9 0 887727359 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 370
/proc/3800/statm: 96959 13925 1890 364 0 32856 0
[pid=3821] ppid=3800 vsize=316044 CPUtime=1.61 cores=0,2,4,6
/proc/3821/stat : 3821 (python) S 3800 3800 3763 0 -1 4202560 1217 0 0 0 90 71 0 0 18 0 2 0 887730801 323629056 12423 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/3821/statm: 79011 12423 415 364 0 14929 0
[pid=3823] ppid=3821 vsize=39992 CPUtime=1261.38 cores=0,2,4,6
/proc/3823/stat : 3823 (lingeling) T 3821 3823 3823 0 -1 4202496 38593 0 0 0 126130 8 0 0 25 0 1 0 887730802 40951808 9168 33554432000 4194304 4352236 140737427087312 18446744073709551615 4328423 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/3823/statm: 9998 9168 129 39 0 9078 0
[pid=3821/tid=3826] ppid=3800 vsize=316044 CPUtime=0 cores=0,2,4,6
/proc/3821/task/3826/stat : 3826 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 887740712 323629056 12423 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453947385920 0 0 -1 0 0 0 0
[pid=3827] ppid=3800 vsize=316056 CPUtime=2.62 cores=0,2,4,6
/proc/3827/stat : 3827 (python) S 3800 3800 3763 0 -1 4202560 1179 0 0 0 145 117 0 0 15 0 2 0 887740715 323641344 12408 33554432000 4194304 5683644 140736988907552 18446744073709551615 264726425158 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3827/statm: 79014 12408 400 364 0 14929 0
[pid=3829] ppid=3827 vsize=807156 CPUtime=1963.15 cores=0,2,4,6
/proc/3829/stat : 3829 (cryptominisat-2) R 3827 3829 3829 0 -1 4202496 682570 0 1 0 196188 127 0 0 25 0 1 0 887740715 826527744 155113 33554432000 4194304 4677096 140735262188624 18446744073709551615 4454496 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/3829/statm: 201789 155113 393 118 0 196748 0
[pid=3827/tid=3854] ppid=3800 vsize=316056 CPUtime=0 cores=0,2,4,6
/proc/3827/task/3854/stat : 3854 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 15 0 2 0 887789640 323641344 12408 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453921673472 0 0 -1 4 0 0 0
[pid=3858] ppid=3800 vsize=326312 CPUtime=0.63 cores=0,2,4,6
/proc/3858/stat : 3858 (python) S 3800 3800 3763 0 -1 4202560 1156 0 0 0 36 27 0 0 18 0 2 0 887799355 334143488 12411 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/3858/statm: 81578 12411 399 364 0 17490 0
[pid=3860] ppid=3858 vsize=28524 CPUtime=486.47 cores=0,2,4,6
/proc/3860/stat : 3860 (lingeling) T 3858 3860 3860 0 -1 4202496 12622 0 0 0 48645 2 0 0 25 0 1 0 887799355 29208576 6238 33554432000 4194304 4352236 140737260682336 18446744073709551615 4328359 0 0 16781312 17506 18446744071562691360 0 0 17 4 0 0 0
/proc/3860/statm: 7131 6238 129 39 0 6211 0
[pid=3858/tid=3863] ppid=3800 vsize=326312 CPUtime=0 cores=0,2,4,6
/proc/3858/task/3863/stat : 3863 (python) S 3800 3800 3763 0 -1 4202560 50 0 0 0 0 0 0 0 18 0 2 0 887809366 334143488 12411 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453943019456 0 0 -1 2 0 0 0
[pid=3884] ppid=3800 vsize=336568 CPUtime=0.55 cores=0,2,4,6
/proc/3884/stat : 3884 (python) S 3800 3800 3763 0 -1 4202560 1174 0 0 0 30 25 0 0 18 0 2 0 887858026 344645632 12415 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/3884/statm: 84142 12415 399 364 0 20051 0
[pid=3886] ppid=3884 vsize=66328 CPUtime=390.19 cores=0,2,4,6
/proc/3886/stat : 3886 (precosat) T 3884 3886 3886 0 -1 4202496 17795 0 0 0 39016 3 0 0 25 0 1 0 887858026 67919872 16276 33554432000 4194304 4890832 140735791669680 18446744073709551615 4238594 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/3886/statm: 16582 16276 86 171 0 16409 0
[pid=3884/tid=3890] ppid=3800 vsize=336568 CPUtime=0 cores=0,2,4,6
/proc/3884/task/3890/stat : 3890 (python) S 3800 3800 3763 0 -1 4202560 51 0 0 0 0 0 0 0 18 0 2 0 887868038 344645632 12415 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453956606656 0 0 -1 6 0 0 0
[pid=3902] ppid=3800 vsize=346824 CPUtime=0.13 cores=0,2,4,6
/proc/3902/stat : 3902 (python) S 3800 3800 3763 0 -1 4202560 1134 0 0 0 9 4 0 0 19 0 2 0 887906714 355147776 12419 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3902/statm: 86706 12419 399 364 0 22612 0
[pid=3904] ppid=3902 vsize=80656 CPUtime=100.1 cores=0,2,4,6
/proc/3904/stat : 3904 (SApperloT-base) T 3902 3904 3904 0 -1 4202496 20046 0 0 0 10007 3 0 0 25 0 1 0 887906715 82591744 19681 33554432000 134512640 135674940 4294785760 18446744073709551615 134552120 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/3904/statm: 20164 19681 100 284 0 19878 0
[pid=3902/tid=3908] ppid=3800 vsize=346824 CPUtime=0 cores=0,2,4,6
/proc/3902/task/3908/stat : 3908 (python) S 3800 3800 3763 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 887916727 355147776 12419 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453961883904 0 0 -1 4 0 0 0
[pid=3925] ppid=3800 vsize=357080 CPUtime=0.27 cores=0,2,4,6
/proc/3925/stat : 3925 (python) S 3800 3800 3763 0 -1 4202560 1157 0 0 0 16 11 0 0 18 0 2 0 887945670 365649920 12423 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3925/statm: 89270 12423 399 364 0 25173 0
[pid=3927] ppid=3925 vsize=27140 CPUtime=197.18 cores=0,2,4,6
/proc/3927/stat : 3927 (lingeling) T 3925 3927 3927 0 -1 4202496 9600 0 0 0 19717 1 0 0 25 0 1 0 887945671 27791360 5955 33554432000 4194304 4352236 140733553993856 18446744073709551615 4241364 0 0 16781312 17506 18446744071562691360 0 0 17 0 0 0 0
/proc/3927/statm: 6785 5955 129 39 0 5865 0
[pid=3925/tid=3933] ppid=3800 vsize=357080 CPUtime=0 cores=0,2,4,6
/proc/3925/task/3933/stat : 3933 (python) S 3800 3800 3763 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 887955683 365649920 12423 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453946852544 0 0 -1 6 0 0 0
[pid=3934] ppid=3800 vsize=367336 CPUtime=0.72 cores=0,2,4,6
/proc/3934/stat : 3934 (python) S 3800 3800 3763 0 -1 4202560 1176 0 0 0 41 31 0 0 16 0 2 0 887955685 376152064 12427 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3934/statm: 91834 12427 399 364 0 27734 0
[pid=3936] ppid=3934 vsize=401000 CPUtime=488.33 cores=0,2,4,6
/proc/3936/stat : 3936 (cryptominisat-2) T 3934 3936 3936 0 -1 4202496 287289 0 0 0 48778 55 0 0 25 0 1 0 887955685 410624000 88627 33554432000 4194304 4677096 140735635675568 18446744073709551615 4465886 0 0 16781312 2 18446744071562691360 0 0 17 0 0 0 0
/proc/3936/statm: 100250 88627 393 118 0 95209 0
[pid=3934/tid=3963] ppid=3800 vsize=367336 CPUtime=0 cores=0,2,4,6
/proc/3934/task/3963/stat : 3963 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 888004531 376152064 12427 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604453921675264 0 0 -1 4 0 0 0
[pid=3968] ppid=3800 vsize=377592 CPUtime=0.14 cores=0,2,4,6
/proc/3968/stat : 3968 (python) S 3800 3800 3763 0 -1 4202560 1179 0 0 0 8 6 0 0 19 0 2 0 888014246 386654208 12437 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738231755 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/3968/statm: 94398 12437 405 364 0 30295 0
[pid=3970] ppid=3968 vsize=45556 CPUtime=100.09 cores=0,2,4,6
/proc/3970/stat : 3970 (precosat) T 3968 3970 3970 0 -1 4202496 11847 0 0 0 10008 1 0 0 25 0 1 0 888014246 46649344 11085 33554432000 4194304 4890832 140737046556368 18446744073709551615 4299152 0 0 16781312 17442 18446744071562691360 0 0 17 0 0 0 0
/proc/3970/statm: 11389 11085 86 171 0 11216 0
[pid=3968/tid=3974] ppid=3800 vsize=377592 CPUtime=0 cores=0,2,4,6
/proc/3968/task/3974/stat : 3974 (python) S 3800 3800 3763 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 888024259 386654208 12437 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 514 18446604440279314816 0 0 -1 0 0 0 0
[pid=3800/tid=3822] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3822/stat : 3822 (python) S 3798 3800 3763 0 -1 4202560 43 925 0 8 0 0 0 0 18 0 9 0 887730802 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
[pid=3800/tid=3828] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3828/stat : 3828 (python) S 3798 3800 3763 0 -1 4202560 37 925 0 8 0 0 0 0 18 0 9 0 887740715 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 6 0 0 0
[pid=3800/tid=3859] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3859/stat : 3859 (python) S 3798 3800 3763 0 -1 4202560 36 925 0 8 0 0 0 0 18 0 9 0 887799355 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 0 0 0 0
[pid=3800/tid=3885] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3885/stat : 3885 (python) S 3798 3800 3763 0 -1 4202560 26 925 0 8 0 0 0 0 18 0 9 0 887858026 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 2 0 0 0
[pid=3800/tid=3903] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3903/stat : 3903 (python) S 3798 3800 3763 0 -1 4202560 28 925 0 8 0 0 0 0 20 0 9 0 887906714 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 0 0 0 0
[pid=3800/tid=3926] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3926/stat : 3926 (python) S 3798 3800 3763 0 -1 4202560 33 925 0 8 0 0 0 0 18 0 9 0 887945671 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 4 0 0 0
[pid=3800/tid=3935] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3935/stat : 3935 (python) S 3798 3800 3763 0 -1 4202560 31 925 0 8 0 0 0 0 20 0 9 0 887955685 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 4 0 0 0
[pid=3800/tid=3969] ppid=3798 vsize=387836 CPUtime=0 cores=0,2,4,6
/proc/3800/task/3969/stat : 3969 (python) S 3798 3800 3763 0 -1 4202560 32 925 0 8 0 0 0 0 20 0 9 0 888014246 397144064 13925 33554432000 4194304 5683644 140736988907552 18446744073709551615 264738228657 0 0 16781312 2 18446604453943018560 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5000.1
Current children cumulated vsize (KiB) 4628000

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 3800 and gives
#  childrusage.ru_utime.tv_sec=6
#  childrusage.ru_utime.tv_usec=312040
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=245962
# CPU time returned by wait4() is 6.558
# while last known CPU time is 5000.1
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 5022.56
CPU time (s): 5000.1
CPU user time (s): 4994.95
CPU system time (s): 5.15
CPU usage (%): 99.5527
Max. virtual memory (cumulated for all children) (KiB): 4639808

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 6.31204
system time used= 0.245962
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 28610
page faults= 124
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 18313
involuntary context switches= 55

runsolver used 17.9133 second user time and 34.2798 second system time

The end

Launcher Data 

Begin job on node141 at 2011-06-13 10:18:35
IDJOB=3498268
IDBENCH=20529
IDSOLVER=1991
FILE ID=node141/3498268-1307953114
RUNJOBID= node141-1307953114-3781
PBS_JOBID= 13559687
Free space on /var/tmp= 2912 MiB

SOLVER NAME= borg-sat 11.03.02-seq-industrial (fixed)
BENCH NAME= SAT07/industrial/IBM_FV_2002_13_rule_1/SAT_dat.k20.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /var/tmp/evaluation-result-3498268-1307953114/watcher-3498268-1307953114 -o /var/tmp/evaluation-result-3498268-1307953114/solver-3498268-1307953114 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3498268-1307953114.cnf -seed 462826349 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 2bee695ce4278b72ee9295631ad94252
RANDOM SEED=462826349

node141.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5362.96
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:       3923548 kB
Buffers:        107552 kB
Cached:       27995712 kB
SwapCached:       9036 kB
Active:       12580144 kB
Inactive:     15540720 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:       3923548 kB
SwapTotal:    67111528 kB
SwapFree:     67064576 kB
Dirty:           35116 kB
Writeback:           0 kB
AnonPages:       15344 kB
Mapped:           9540 kB
Slab:           844008 kB
PageTables:       4112 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   157748 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /var/tmp at the end= 2848 MiB
End job on node141 at 2011-06-13 11:42:22