Trace number 3317920

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 4214.25 1300.06

General information on the benchmark

Nameapplication/kullmann/AES/Benchmarks/
128/aes_128_10_keyfind_1.cnf
MD5SUMf35f3aee72237c1bf3d673feb4883756
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables8080
Number of clauses96704
Sum of the clauses size539496
Maximum clause length8
Minimum clause length1
Number of clauses of size 1336
Number of clauses of size 21152
Number of clauses of size 315040
Number of clauses of size 42944
Number of clauses of size 518432
Number of clauses of size over 558800

Solver Data 

0.34/0.63	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.34/0.63	c INFO: solving HOME/instance-3317920-1304270208.cnf
0.34/0.63	c INFO: computing task features
0.93/1.20	c INFO: computed variable-clause graph statistics
0.93/1.21	c INFO: computed clause balance statistics
3.83/4.10	c INFO: computed variable balance statistics
5.24/5.52	c INFO: computed small-clause counts
5.24/5.53	c INFO: computed Horn-clause counts
5.34/5.69	c INFO: computed variable graph statistics
5.34/5.69	c INFO: collected features for HOME/instance-3317920-1304270208.cnf in 5.04 s
5.43/5.72	c INFO: running lingeling-276@0 for 97 with 4794 remaining (b = 0.26)
5.43/5.74	c INFO: running cryptominisat-2.9.0@0 for 485 with 4794 remaining (b = 0.31)
5.43/5.77	c INFO: running lingeling-276@0 for 582 with 4794 remaining (b = 0.36)
5.43/5.79	c INFO: running lingeling-276@0 for 1261 with 4794 remaining (b = 0.42)
649.28/189.98	c INFO: running lingeling-276@0 for 1164 with 4610 remaining (b = 0.35)
1708.44/492.88	c INFO: running SApperloT@0 for 97 with 4307 remaining (b = 0.11)
2007.65/592.62	c INFO: running precosat-570@0 for 388 with 4208 remaining (b = 0.24)
2296.81/689.18	c INFO: running cryptominisat-2.9.0@0 for 388 with 4111 remaining (b = 0.22)
3723.47/1150.29	c INFO: running cryptominisat-2.9.0@391 for 97 with 3650 remaining (b = 0.10)
4021.52/1235.51	c INFO: running cryptominisat-2.9.0@0 for 1649 with 3565 remaining (b = 0.48)
4064.03/1249.70	c INFO: running cryptominisat-2.9.0@0 for 485 with 3550 remaining (b = 0.21)
4214.25/1300.03	c INFO: running precosat-570@97 for 97 with 3500 remaining (b = 0.07)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317920-1304270208/watcher-3317920-1304270208 -o /tmp/evaluation-result-3317920-1304270208/solver-3317920-1304270208 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317920-1304270208.cnf -seed 809406488 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 9.09 7.46 6.08 6/212 24121
/proc/meminfo: memFree=20727724/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 792 0 0 0 0 0 0 0 25 0 1 0 538620446 15749120 679 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473965568 0 2147483391 16781312 2 18446744071699215550 0 0 17 4 0 0 0
/proc/24121/statm: 3845 679 322 364 0 332 0

[startup+0.0839231 s]
/proc/loadavg: 9.09 7.46 6.08 6/212 24121
/proc/meminfo: memFree=20727724/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=54540 CPUtime=0.05 cores=0,2,4,6
/proc/24121/stat : 24121 (python) R 24119 24121 23557 0 -1 4202496 2068 0 0 0 4 1 0 0 25 0 1 0 538620446 55848960 1825 33554432000 4194304 5683644 140736554223520 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/24121/statm: 13635 1829 492 364 0 1285 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 54540

[startup+0.100918 s]
/proc/loadavg: 9.09 7.46 6.08 6/212 24121
/proc/meminfo: memFree=20727724/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=93960 CPUtime=0.06 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 2695 0 0 0 4 2 0 0 25 0 1 0 538620446 96215040 2407 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473965568 0 2147483391 16781312 2 18446744071699215550 0 0 17 4 0 0 0
/proc/24121/statm: 23490 2407 764 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 93960

[startup+0.300883 s]
/proc/loadavg: 9.09 7.46 6.08 6/212 24121
/proc/meminfo: memFree=20727724/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=135748 CPUtime=0.17 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 5961 935 1 0 13 4 0 0 25 0 1 0 538620446 139005952 4150 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473963525 0 2147483391 16781312 2 18446744071699215550 0 0 17 4 0 0 0
/proc/24121/statm: 33937 4150 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.700755 s]
/proc/loadavg: 9.09 7.46 6.08 6/212 24121
/proc/meminfo: memFree=20727724/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=284588 CPUtime=0.43 cores=0,2,4,6
/proc/24121/stat : 24121 (python) R 24119 24121 23557 0 -1 4202496 10777 935 1 0 34 9 0 0 25 0 1 0 538620446 291418112 8459 33554432000 4194304 5683644 140736554223520 18446744073709551615 4602441 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/24121/statm: 71147 8461 1761 364 0 7071 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 284588

[startup+1.50067 s]
/proc/loadavg: 9.09 7.46 6.08 7/213 24126
/proc/meminfo: memFree=20680348/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=313960 CPUtime=1.23 cores=0,2,4,6
/proc/24121/stat : 24121 (python) R 24119 24121 23557 0 -1 4202496 22630 935 1 0 112 11 0 0 25 0 1 0 538620446 321495040 15875 33554432000 4194304 5683644 140736554223520 18446744073709551615 47503299735773 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/24121/statm: 78490 15875 1826 364 0 14414 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 313960

[startup+3.10033 s]
/proc/loadavg: 9.09 7.46 6.08 7/213 24126
/proc/meminfo: memFree=20667704/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=313960 CPUtime=2.83 cores=0,2,4,6
/proc/24121/stat : 24121 (python) R 24119 24121 23557 0 -1 4202496 22630 935 1 0 272 11 0 0 25 0 1 0 538620446 321495040 15875 33554432000 4194304 5683644 140736554223520 18446744073709551615 47503299665536 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/24121/statm: 78490 15875 1826 364 0 14414 0
Current children cumulated CPU time (s) 2.83
Current children cumulated vsize (KiB) 313960

[startup+6.30078 s]
/proc/loadavg: 8.84 7.44 6.08 7/213 24126
/proc/meminfo: memFree=20664100/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=352360 CPUtime=5.53 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 25386 935 1 0 541 12 0 0 25 0 5 0 538620446 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/24121/statm: 88090 15287 1881 364 0 23999 0
Current children cumulated CPU time (s) 5.53
Current children cumulated vsize (KiB) 352360

[startup+12.7007 s]
/proc/loadavg: 8.86 7.46 6.10 10/225 24138
/proc/meminfo: memFree=20564572/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=352360 CPUtime=5.53 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 25386 935 1 0 541 12 0 0 25 0 5 0 538620446 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/24121/statm: 88090 15287 1881 364 0 23999 0
[pid=24127] ppid=24121 vsize=311348 CPUtime=0 cores=0,2,4,6
/proc/24127/stat : 24127 (python) S 24121 24121 23557 0 -1 4202560 984 0 0 0 0 0 0 0 18 0 1 0 538621018 318820352 13771 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987599 0 0 16781312 514 3984912442965590276 0 0 17 2 0 0 0
/proc/24127/statm: 77837 13771 381 364 0 13755 0
[pid=24129] ppid=24127 vsize=18048 CPUtime=6.91 cores=0,2,4,6
/proc/24129/stat : 24129 (lingeling) R 24127 24129 24129 0 -1 4202496 4810 0 0 0 691 0 0 0 25 0 1 0 538621019 18481152 3681 33554432000 4194304 4352236 140734877286224 18446744073709551615 4278317 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/24129/statm: 4512 3681 129 39 0 3592 0
[pid=24130] ppid=24121 vsize=321604 CPUtime=0 cores=0,2,4,6
/proc/24130/stat : 24130 (python) S 24121 24121 23557 0 -1 4202560 916 0 0 0 0 0 0 0 15 0 1 0 538621021 329322496 13764 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 4160129791334252804 0 0 17 2 0 0 0
/proc/24130/statm: 80401 13764 370 364 0 16316 0
[pid=24132] ppid=24130 vsize=68380 CPUtime=6.91 cores=0,2,4,6
/proc/24132/stat : 24132 (cryptominisat-2) R 24130 24132 24132 0 -1 4202496 15889 0 0 0 689 2 0 0 25 0 1 0 538621021 70021120 10318 33554432000 4194304 4677096 140735037539136 18446744073709551615 4346878 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/24132/statm: 17095 10318 392 118 0 12054 0
[pid=24133] ppid=24121 vsize=331860 CPUtime=0 cores=0,2,4,6
/proc/24133/stat : 24133 (python) S 24121 24121 23557 0 -1 4202560 905 0 0 0 0 0 0 0 18 0 1 0 538621023 339824640 13767 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 4100500252102983940 0 0 17 4 0 0 0
/proc/24133/statm: 82965 13767 369 364 0 18877 0
[pid=24137] ppid=24133 vsize=17784 CPUtime=6.86 cores=0,2,4,6
/proc/24137/stat : 24137 (lingeling) R 24133 24137 24137 0 -1 4202496 4741 0 0 0 685 1 0 0 25 0 1 0 538621026 18210816 3614 33554432000 4194304 4352236 140737168287360 18446744073709551615 4329086 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24137/statm: 4446 3614 129 39 0 3526 0
[pid=24135] ppid=24121 vsize=342116 CPUtime=0 cores=0,2,4,6
/proc/24135/stat : 24135 (python) S 24121 24121 23557 0 -1 4202560 941 0 0 0 0 0 0 0 18 0 1 0 538621026 350326784 13772 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 4100504100393681156 0 0 17 4 0 0 0
/proc/24135/statm: 85529 13772 370 364 0 21438 0
[pid=24138] ppid=24135 vsize=17552 CPUtime=3.41 cores=0,2,4,6
/proc/24138/stat : 24138 (lingeling) R 24135 24138 24138 0 -1 4202496 4685 0 0 0 341 0 0 0 25 0 1 0 538621027 17973248 3557 33554432000 4194304 4352236 140734059045184 18446744073709551615 4238992 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/24138/statm: 4388 3557 129 39 0 3468 0
[pid=24121/tid=24128] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24128/stat : 24128 (python) S 24119 24121 23557 0 -1 4202560 24 935 0 0 0 0 0 0 25 0 5 0 538621019 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 6 0 0 0
[pid=24121/tid=24131] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24131/stat : 24131 (python) S 24119 24121 23557 0 -1 4202560 22 935 0 0 0 0 0 0 25 0 5 0 538621021 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
[pid=24121/tid=24134] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24134/stat : 24134 (python) S 24119 24121 23557 0 -1 4202560 31 935 0 0 0 0 0 0 25 0 5 0 538621024 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24136] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24136/stat : 24136 (python) S 24119 24121 23557 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 538621026 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 29.62
Current children cumulated vsize (KiB) 1781052

[startup+25.5014 s]
/proc/loadavg: 8.89 7.54 6.14 10/225 24138
/proc/meminfo: memFree=20540644/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=352360 CPUtime=5.53 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 25386 935 1 0 541 12 0 0 25 0 5 0 538620446 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/24121/statm: 88090 15287 1881 364 0 23999 0
[pid=24127] ppid=24121 vsize=311348 CPUtime=0.02 cores=0,2,4,6
/proc/24127/stat : 24127 (python) S 24121 24121 23557 0 -1 4202560 991 0 0 0 1 1 0 0 18 0 1 0 538621018 318820352 13771 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987599 0 0 16781312 514 3909880940557467908 0 0 17 2 0 0 0
/proc/24127/statm: 77837 13771 381 364 0 13755 0
[pid=24129] ppid=24127 vsize=20184 CPUtime=17.91 cores=0,2,4,6
/proc/24129/stat : 24129 (lingeling) R 24127 24129 24129 0 -1 4202496 5372 0 0 0 1790 1 0 0 25 0 1 0 538621019 20668416 4243 33554432000 4194304 4352236 140734877286224 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/24129/statm: 5046 4243 129 39 0 4126 0
[pid=24130] ppid=24121 vsize=321604 CPUtime=0.04 cores=0,2,4,6
/proc/24130/stat : 24130 (python) S 24121 24121 23557 0 -1 4202560 934 0 0 0 2 2 0 0 15 0 1 0 538621021 329322496 13764 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 4160129791334252804 0 0 17 2 0 0 0
/proc/24130/statm: 80401 13764 370 364 0 16316 0
[pid=24132] ppid=24130 vsize=74000 CPUtime=19.66 cores=0,2,4,6
/proc/24132/stat : 24132 (cryptominisat-2) R 24130 24132 24132 0 -1 4202496 26310 0 0 0 1961 5 0 0 25 0 1 0 538621021 75776000 11708 33554432000 4194304 4677096 140735037539136 18446744073709551615 4320039 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/24132/statm: 18500 11708 392 118 0 13459 0
[pid=24133] ppid=24121 vsize=331860 CPUtime=0.01 cores=0,2,4,6
/proc/24133/stat : 24133 (python) S 24121 24121 23557 0 -1 4202560 911 0 0 0 1 0 0 0 18 0 1 0 538621023 339824640 13768 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 3909880940557467908 0 0 17 4 0 0 0
/proc/24133/statm: 82965 13768 370 364 0 18877 0
[pid=24137] ppid=24133 vsize=22124 CPUtime=19.62 cores=0,2,4,6
/proc/24137/stat : 24137 (lingeling) R 24133 24137 24137 0 -1 4202496 7874 0 0 0 1961 1 0 0 25 0 1 0 538621026 22654976 4699 33554432000 4194304 4352236 140737168287360 18446744073709551615 4235598 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24137/statm: 5531 4699 129 39 0 4611 0
[pid=24135] ppid=24121 vsize=342116 CPUtime=0.02 cores=0,2,4,6
/proc/24135/stat : 24135 (python) S 24121 24121 23557 0 -1 4202560 947 0 0 0 1 1 0 0 18 0 1 0 538621026 350326784 13772 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 3909880940557467908 0 0 17 4 0 0 0
/proc/24135/statm: 85529 13772 370 364 0 21438 0
[pid=24138] ppid=24135 vsize=18976 CPUtime=11.56 cores=0,2,4,6
/proc/24138/stat : 24138 (lingeling) R 24135 24138 24138 0 -1 4202496 5065 0 0 0 1156 0 0 0 25 0 1 0 538621027 19431424 3937 33554432000 4194304 4352236 140734059045184 18446744073709551615 4328126 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/24138/statm: 4744 3937 129 39 0 3824 0
[pid=24121/tid=24128] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24128/stat : 24128 (python) S 24119 24121 23557 0 -1 4202560 24 935 0 0 0 0 0 0 25 0 5 0 538621019 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 6 0 0 0
[pid=24121/tid=24131] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24131/stat : 24131 (python) S 24119 24121 23557 0 -1 4202560 22 935 0 0 0 0 0 0 25 0 5 0 538621021 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
[pid=24121/tid=24134] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24134/stat : 24134 (python) S 24119 24121 23557 0 -1 4202560 31 935 0 0 0 0 0 0 25 0 5 0 538621024 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24136] ppid=24119 vsize=352360 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24136/stat : 24136 (python) S 24119 24121 23557 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 5 0 538621026 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 74.37
Current children cumulated vsize (KiB) 1794572

[startup+51.1009 s]
/proc/loadavg: 8.92 7.66 6.22 10/225 24139
/proc/meminfo: memFree=20515340/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=352360 CPUtime=5.53 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 25386 935 1 0 541 12 0 0 25 0 5 0 538620446 360816640 15287 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/24121/statm: 88090 15287 1881 364 0 23999 0
[pid=24127] ppid=24121 vsize=311348 CPUtime=0.05 cores=0,2,4,6
/proc/24127/stat : 24127 (python) S 24121 24121 23557 0 -1 4202560 1004 0 0 0 3 2 0 0 18 0 1 0 538621018 318820352 13771 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987599 0 0 16781312 514 3909880940557467908 0 0 17 2 0 0 0
/proc/24127/statm: 77837 13771 381 364 0 13755 0
[pid=24129] ppid=24127 vsize=31048 CPUtime=30.6 cores=0,2,4,6
/proc/24129/stat : 24129 (lingeling) R 24127 24129 24129 0 -1 4202496 10108 0 0 0 3059 1 0 0 25 0 1 0 538621019 31793152 6931 33554432000 4194304 4352236 140734877286224 18446744073709551615 4328100 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/24129/statm: 7762 6931 129 39 0 6842 0
[pid=24130] ppid=24121 vsize=321604 CPUtime=0.08 cores=0,2,4,6
/proc/24130/stat : 24130 (python) S 24121 24121 23557 0 -1 4202560 944 0 0 0 5 3 0 0 15 0 1 0 538621021 329322496 13764 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 4160129791334252804 0 0 17 2 0 0 0
/proc/24130/statm: 80401 13764 370 364 0 16316 0
[pid=24132] ppid=24130 vsize=77920 CPUtime=45.19 cores=0,2,4,6
/proc/24132/stat : 24132 (cryptominisat-2) R 24130 24132 24132 0 -1 4202496 43561 0 0 0 4509 10 0 0 25 0 1 0 538621021 79790080 12576 33554432000 4194304 4677096 140735037539136 18446744073709551615 4320122 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/24132/statm: 19480 12576 392 118 0 14439 0
[pid=24133] ppid=24121 vsize=331860 CPUtime=0.05 cores=0,2,4,6
/proc/24133/stat : 24133 (python) S 24121 24121 23557 0 -1 4202560 921 0 0 0 3 2 0 0 18 0 1 0 538621023 339824640 13768 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 3909880940557467908 0 0 17 4 0 0 0
/proc/24133/statm: 82965 13768 370 364 0 18877 0
[pid=24137] ppid=24133 vsize=23816 CPUtime=45.16 cores=0,2,4,6
/proc/24137/stat : 24137 (lingeling) R 24133 24137 24137 0 -1 4202496 14441 0 0 0 4514 2 0 0 25 0 1 0 538621026 24387584 5122 33554432000 4194304 4352236 140737168287360 18446744073709551615 4329000 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24137/statm: 5954 5122 129 39 0 5034 0
[pid=24135] ppid=24121 vsize=342116 CPUtime=0.05 cores=0,2,4,6
/proc/24135/stat : 24135 (python) S 24121 24121 23557 0 -1 4202560 956 0 0 0 2 3 0 0 18 0 1 0 538621026 350326784 13772 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 3909880940557467908 0 0 17 4 0 0 0
/proc/24135/statm: 85529 13772 370 364 0 21438 0

################
# More data... #
################

/proc/24129/stat : 24129 (lingeling) R 24127 24129 24129 0 -1 4202496 82628 0 0 0 85685 12 0 0 25 0 1 0 538621019 41422848 9307 33554432000 4194304 4352236 140734877286224 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/24129/statm: 10113 9307 129 39 0 9193 0
[pid=24127/tid=24144] ppid=24121 vsize=321592 CPUtime=0 cores=0,2,4,6
/proc/24127/task/24144/stat : 24144 (python) S 24121 24121 23557 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 538639442 329310208 13810 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453840502976 0 0 -1 2 0 0 0
[pid=24130] ppid=24121 vsize=321604 CPUtime=1.71 cores=0,2,4,6
/proc/24130/stat : 24130 (python) S 24121 24121 23557 0 -1 4202560 1211 0 0 0 88 83 0 0 15 0 2 0 538621021 329322496 13795 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/24130/statm: 80401 13795 400 364 0 16316 0
[pid=24132] ppid=24130 vsize=203760 CPUtime=968.95 cores=0,2,4,6
/proc/24132/stat : 24132 (cryptominisat-2) R 24130 24132 24132 0 -1 4202496 332477 0 0 0 96811 84 0 0 25 0 1 0 538621021 208650240 42043 33554432000 4194304 4677096 140735037539136 18446744073709551615 4320231 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/24132/statm: 50940 42043 392 118 0 45899 0
[pid=24130/tid=24158] ppid=24121 vsize=321604 CPUtime=0 cores=0,2,4,6
/proc/24130/task/24158/stat : 24158 (python) S 24121 24121 23557 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 538669733 329322496 13795 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453558881280 0 0 -1 2 0 0 0
[pid=24133] ppid=24121 vsize=331860 CPUtime=0.87 cores=0,2,4,6
/proc/24133/stat : 24133 (python) S 24121 24121 23557 0 -1 4202560 1137 0 0 0 46 41 0 0 19 0 2 0 538621023 339824640 13798 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/24133/statm: 82965 13798 399 364 0 18877 0
[pid=24137] ppid=24133 vsize=39460 CPUtime=585.14 cores=0,2,4,6
/proc/24137/stat : 24137 (lingeling) T 24133 24137 24137 0 -1 4202496 58512 0 0 0 58506 8 0 0 25 0 1 0 538621026 40407040 9033 33554432000 4194304 4352236 140737168287360 18446744073709551615 4328385 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/24137/statm: 9865 9033 129 39 0 8945 0
[pid=24133/tid=24165] ppid=24121 vsize=331860 CPUtime=0 cores=0,2,4,6
/proc/24133/task/24165/stat : 24165 (python) S 24121 24121 23557 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538679706 339824640 13798 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453867415360 0 0 -1 2 0 0 0
[pid=24135] ppid=24121 vsize=342116 CPUtime=1.78 cores=0,2,4,6
/proc/24135/stat : 24135 (python) S 24121 24121 23557 0 -1 4202560 1004 0 0 0 91 87 0 0 18 0 1 0 538621026 350326784 13772 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 3909880940557467908 0 0 17 4 0 0 0
/proc/24135/statm: 85529 13772 370 364 0 21438 0
[pid=24138] ppid=24135 vsize=46512 CPUtime=1127.91 cores=0,2,4,6
/proc/24138/stat : 24138 (lingeling) R 24135 24138 24138 0 -1 4202496 140604 0 0 0 112772 19 0 0 25 0 1 0 538621027 47628288 10829 33554432000 4194304 4352236 140734059045184 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/24138/statm: 11628 10829 129 39 0 10708 0
[pid=24159] ppid=24121 vsize=352372 CPUtime=0.29 cores=0,2,4,6
/proc/24159/stat : 24159 (python) S 24121 24121 23557 0 -1 4202560 1167 0 0 0 13 16 0 0 19 0 2 0 538669735 360828928 13806 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/24159/statm: 88093 13806 399 364 0 23999 0
[pid=24161] ppid=24159 vsize=68236 CPUtime=97.5 cores=0,2,4,6
/proc/24161/stat : 24161 (SApperloT-base) T 24159 24161 24161 0 -1 4202496 16641 0 0 0 9745 5 0 0 25 0 1 0 538669736 69873664 16277 33554432000 134512640 135674940 4287012224 18446744073709551615 134564660 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/24161/statm: 17059 16277 100 284 0 16773 0
[pid=24159/tid=24171] ppid=24121 vsize=352372 CPUtime=0 cores=0,2,4,6
/proc/24159/task/24171/stat : 24171 (python) S 24121 24121 23557 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538689362 360828928 13806 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453864233664 0 0 -1 4 0 0 0
[pid=24166] ppid=24121 vsize=362628 CPUtime=0.94 cores=0,2,4,6
/proc/24166/stat : 24166 (python) S 24121 24121 23557 0 -1 4202560 1177 0 0 0 49 45 0 0 19 0 2 0 538679708 371331072 13810 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/24166/statm: 90657 13810 399 364 0 26560 0
[pid=24168] ppid=24166 vsize=54268 CPUtime=388.97 cores=0,2,4,6
/proc/24168/stat : 24168 (precosat) T 24166 24168 24168 0 -1 4202496 15454 0 0 0 38894 3 0 0 25 0 1 0 538679709 55570432 13414 33554432000 4194304 4890832 140736121753968 18446744073709551615 4238599 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/24168/statm: 13567 13414 86 171 0 13394 0
[pid=24166/tid=24252] ppid=24121 vsize=362628 CPUtime=0 cores=0,2,4,6
/proc/24166/task/24252/stat : 24252 (python) S 24121 24121 23557 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538743996 371331072 13810 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453892854976 0 0 -1 6 0 0 0
[pid=24253] ppid=24121 vsize=372884 CPUtime=0 cores=0,2,4,6
/proc/24253/stat : 24253 (python) S 24121 24121 23557 0 -1 4202560 958 0 0 0 0 0 0 0 15 0 1 0 538743998 381833216 13784 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 12111300904154530049 0 0 17 4 0 0 0
/proc/24253/statm: 93221 13784 370 364 0 29121 0
[pid=24255] ppid=24253 vsize=64520 CPUtime=3.38 cores=0,2,4,6
/proc/24255/stat : 24255 (cryptominisat-2) R 24253 24255 24255 0 -1 4202496 13133 0 0 0 336 2 0 0 25 0 1 0 538743998 66068480 8938 33554432000 4194304 4677096 140736366728032 18446744073709551615 4320184 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/24255/statm: 16130 8938 392 118 0 11089 0
[pid=24121/tid=24128] ppid=24119 vsize=383128 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24128/stat : 24128 (python) S 24119 24121 23557 0 -1 4202560 29 935 0 0 0 0 0 0 18 0 8 0 538621019 392323072 15306 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 6 0 0 0
[pid=24121/tid=24131] ppid=24119 vsize=383128 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24131/stat : 24131 (python) S 24119 24121 23557 0 -1 4202560 26 935 0 0 0 0 0 0 18 0 8 0 538621021 392323072 15306 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
[pid=24121/tid=24134] ppid=24119 vsize=383128 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24134/stat : 24134 (python) S 24119 24121 23557 0 -1 4202560 31 935 0 0 0 0 0 0 25 0 8 0 538621024 392323072 15306 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24136] ppid=24119 vsize=383128 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24136/stat : 24136 (python) S 24119 24121 23557 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 8 0 538621026 392323072 15306 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24160] ppid=24119 vsize=383128 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24160/stat : 24160 (python) S 24119 24121 23557 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 8 0 538669735 392323072 15306 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24167] ppid=24119 vsize=383128 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24167/stat : 24167 (python) S 24119 24121 23557 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 8 0 538679709 392323072 15306 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
[pid=24121/tid=24254] ppid=24119 vsize=383128 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24254/stat : 24254 (python) S 24119 24121 23557 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 8 0 538743998 392323072 15306 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4041.85
Current children cumulated vsize (KiB) 3305392



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 6.26 7.63 7.77 7/220 24259
/proc/meminfo: memFree=20163228/32950928 swapFree=67111524/67111528
[pid=24121] ppid=24119 vsize=393384 CPUtime=5.69 cores=0,2,4,6
/proc/24121/stat : 24121 (python) S 24119 24121 23557 0 -1 4202496 28802 935 1 0 556 13 0 0 20 0 9 0 538620446 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/24121/statm: 98346 15312 1890 364 0 34243 0
[pid=24127] ppid=24121 vsize=321592 CPUtime=1.86 cores=0,2,4,6
/proc/24127/stat : 24127 (python) S 24121 24121 23557 0 -1 4202560 1223 0 0 0 107 79 0 0 18 0 2 0 538621018 329310208 13810 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/24127/statm: 80398 13810 415 364 0 16316 0
[pid=24129] ppid=24127 vsize=47108 CPUtime=885.44 cores=0,2,4,6
/proc/24129/stat : 24129 (lingeling) R 24127 24129 24129 0 -1 4202496 84292 0 0 0 88532 12 0 0 25 0 1 0 538621019 48238592 10971 33554432000 4194304 4352236 140734877286224 18446744073709551615 4328000 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/24129/statm: 11777 10971 129 39 0 10857 0
[pid=24127/tid=24144] ppid=24121 vsize=321592 CPUtime=0 cores=0,2,4,6
/proc/24127/task/24144/stat : 24144 (python) S 24121 24121 23557 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 538639442 329310208 13810 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453840502976 0 0 -1 2 0 0 0
[pid=24130] ppid=24121 vsize=321604 CPUtime=1.73 cores=0,2,4,6
/proc/24130/stat : 24130 (python) S 24121 24121 23557 0 -1 4202560 1211 0 0 0 89 84 0 0 15 0 2 0 538621021 329322496 13795 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/24130/statm: 80401 13795 400 364 0 16316 0
[pid=24132] ppid=24130 vsize=210280 CPUtime=976.31 cores=0,2,4,6
/proc/24132/stat : 24132 (cryptominisat-2) T 24130 24132 24132 0 -1 4202496 334032 0 0 0 97547 84 0 0 25 0 1 0 538621021 215326720 43598 33554432000 4194304 4677096 140735037539136 18446744073709551615 4320006 0 0 16781312 2 18446744071562691360 0 0 17 2 0 0 0
/proc/24132/statm: 52570 43598 413 118 0 47529 0
[pid=24130/tid=24158] ppid=24121 vsize=321604 CPUtime=0 cores=0,2,4,6
/proc/24130/task/24158/stat : 24158 (python) S 24121 24121 23557 0 -1 4202560 49 0 0 0 0 0 0 0 15 0 2 0 538669733 329322496 13795 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453558881280 0 0 -1 2 0 0 0
[pid=24133] ppid=24121 vsize=331860 CPUtime=0.87 cores=0,2,4,6
/proc/24133/stat : 24133 (python) S 24121 24121 23557 0 -1 4202560 1137 0 0 0 46 41 0 0 19 0 2 0 538621023 339824640 13798 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/24133/statm: 82965 13798 399 364 0 18877 0
[pid=24137] ppid=24133 vsize=39460 CPUtime=585.14 cores=0,2,4,6
/proc/24137/stat : 24137 (lingeling) T 24133 24137 24137 0 -1 4202496 58512 0 0 0 58506 8 0 0 25 0 1 0 538621026 40407040 9033 33554432000 4194304 4352236 140737168287360 18446744073709551615 4328385 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/24137/statm: 9865 9033 129 39 0 8945 0
[pid=24133/tid=24165] ppid=24121 vsize=331860 CPUtime=0 cores=0,2,4,6
/proc/24133/task/24165/stat : 24165 (python) S 24121 24121 23557 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538679706 339824640 13798 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453867415360 0 0 -1 2 0 0 0
[pid=24135] ppid=24121 vsize=342116 CPUtime=1.85 cores=0,2,4,6
/proc/24135/stat : 24135 (python) S 24121 24121 23557 0 -1 4202560 1004 0 0 0 95 90 0 0 18 0 1 0 538621026 350326784 13772 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 3909880940557467908 0 0 17 4 0 0 0
/proc/24135/statm: 85529 13772 370 364 0 21438 0
[pid=24138] ppid=24135 vsize=59184 CPUtime=1185.53 cores=0,2,4,6
/proc/24138/stat : 24138 (lingeling) R 24135 24138 24138 0 -1 4202496 147068 0 0 0 118533 20 0 0 25 0 1 0 538621027 60604416 13997 33554432000 4194304 4352236 140734059045184 18446744073709551615 4328194 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/24138/statm: 14796 13997 129 39 0 13876 0
[pid=24159] ppid=24121 vsize=352372 CPUtime=0.29 cores=0,2,4,6
/proc/24159/stat : 24159 (python) S 24121 24121 23557 0 -1 4202560 1167 0 0 0 13 16 0 0 19 0 2 0 538669735 360828928 13806 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/24159/statm: 88093 13806 399 364 0 23999 0
[pid=24161] ppid=24159 vsize=68236 CPUtime=97.5 cores=0,2,4,6
/proc/24161/stat : 24161 (SApperloT-base) T 24159 24161 24161 0 -1 4202496 16641 0 0 0 9745 5 0 0 25 0 1 0 538669736 69873664 16277 33554432000 134512640 135674940 4287012224 18446744073709551615 134564660 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/24161/statm: 17059 16277 100 284 0 16773 0
[pid=24159/tid=24171] ppid=24121 vsize=352372 CPUtime=0 cores=0,2,4,6
/proc/24159/task/24171/stat : 24171 (python) S 24121 24121 23557 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 538689362 360828928 13806 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453864233664 0 0 -1 4 0 0 0
[pid=24166] ppid=24121 vsize=362628 CPUtime=0.94 cores=0,2,4,6
/proc/24166/stat : 24166 (python) S 24121 24121 23557 0 -1 4202560 1177 0 0 0 49 45 0 0 19 0 2 0 538679708 371331072 13810 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485794251 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/24166/statm: 90657 13810 399 364 0 26560 0
[pid=24168] ppid=24166 vsize=54268 CPUtime=388.97 cores=0,2,4,6
/proc/24168/stat : 24168 (precosat) T 24166 24168 24168 0 -1 4202496 15454 0 0 0 38894 3 0 0 25 0 1 0 538679709 55570432 13414 33554432000 4194304 4890832 140736121753968 18446744073709551615 4238599 0 0 16781312 17442 18446744071562691360 0 0 17 4 0 0 0
/proc/24168/statm: 13567 13414 86 171 0 13394 0
[pid=24166/tid=24252] ppid=24121 vsize=362628 CPUtime=0 cores=0,2,4,6
/proc/24166/task/24252/stat : 24252 (python) S 24121 24121 23557 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 538743996 371331072 13810 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 514 18446604453892854976 0 0 -1 6 0 0 0
[pid=24253] ppid=24121 vsize=372884 CPUtime=0.1 cores=0,2,4,6
/proc/24253/stat : 24253 (python) S 24121 24121 23557 0 -1 4202560 987 0 0 0 6 4 0 0 15 0 1 0 538743998 381833216 13784 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 12111300904154530049 0 0 17 4 0 0 0
/proc/24253/statm: 93221 13784 370 364 0 29121 0
[pid=24255] ppid=24253 vsize=77716 CPUtime=31.8 cores=0,2,4,6
/proc/24255/stat : 24255 (cryptominisat-2) R 24253 24255 24255 0 -1 4202496 35454 0 0 0 3172 8 0 0 25 0 1 0 538743998 79581184 12359 33554432000 4194304 4677096 140736366728032 18446744073709551615 4337904 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/24255/statm: 19429 12359 392 118 0 14388 0
[pid=24256] ppid=24121 vsize=383140 CPUtime=0.09 cores=0,2,4,6
/proc/24256/stat : 24256 (python) S 24121 24121 23557 0 -1 4202560 1031 0 0 0 5 4 0 0 15 0 1 0 538745417 392335360 13794 33554432000 4194304 5683644 140736554223520 18446744073709551615 274473987654 0 0 16781312 514 4121761233571381506 0 0 17 2 0 0 0
/proc/24256/statm: 95785 13794 376 364 0 31682 0
[pid=24258] ppid=24256 vsize=91452 CPUtime=50.14 cores=0,2,4,6
/proc/24258/stat : 24258 (cryptominisat-2) R 24256 24258 24258 0 -1 4202496 48753 0 0 0 5004 10 0 0 25 0 1 0 538745417 93646848 16044 33554432000 4194304 4677096 140737206370112 18446744073709551615 4319978 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/24258/statm: 22863 16044 392 118 0 17822 0
[pid=24121/tid=24128] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24128/stat : 24128 (python) S 24119 24121 23557 0 -1 4202560 29 935 0 0 0 0 0 0 18 0 9 0 538621019 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 6 0 0 0
[pid=24121/tid=24131] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24131/stat : 24131 (python) S 24119 24121 23557 0 -1 4202560 26 935 0 0 0 0 0 0 18 0 9 0 538621021 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
[pid=24121/tid=24134] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24134/stat : 24134 (python) S 24119 24121 23557 0 -1 4202560 31 935 0 0 0 0 0 0 25 0 9 0 538621024 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24136] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24136/stat : 24136 (python) S 24119 24121 23557 0 -1 4202560 25 935 0 0 0 0 0 0 25 0 9 0 538621026 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24160] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24160/stat : 24160 (python) S 24119 24121 23557 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 9 0 538669735 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 4 0 0 0
[pid=24121/tid=24167] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24167/stat : 24167 (python) S 24119 24121 23557 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 9 0 538679709 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
[pid=24121/tid=24254] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24254/stat : 24254 (python) S 24119 24121 23557 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 9 0 538743998 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
[pid=24121/tid=24257] ppid=24119 vsize=393384 CPUtime=0 cores=0,2,4,6
/proc/24121/task/24257/stat : 24257 (python) S 24119 24121 23557 0 -1 4202560 33 935 0 0 0 0 0 0 20 0 9 0 538745417 402825216 15312 33554432000 4194304 5683644 140736554223520 18446744073709551615 274485791153 0 0 16781312 2 18446604453881939904 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 4214.25
Current children cumulated vsize (KiB) 3829284

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 24121 and gives
#  childrusage.ru_utime.tv_sec=5
#  childrusage.ru_utime.tv_usec=582151
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=144977
# CPU time returned by wait4() is 5.72713
# while last known CPU time is 4214.25
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.06
CPU time (s): 4214.25
CPU user time (s): 4208.99
CPU system time (s): 5.26
CPU usage (%): 324.158
Max. virtual memory (cumulated for all children) (KiB): 3842180

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5.58215
system time used= 0.144977
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30284
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2975
involuntary context switches= 40

runsolver used 2.58461 second user time and 6.30104 second system time

The end

Launcher Data 

Begin job on node125 at 2011-05-01 19:16:48
IDJOB=3317920
IDBENCH=83413
IDSOLVER=1706
FILE ID=node125/3317920-1304270208
RUNJOBID= node125-1304267283-23575
PBS_JOBID= 13173954
Free space on /tmp= 73060 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT11/application/kullmann/AES/Benchmarks/128/aes_128_10_keyfind_1.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317920-1304270208/watcher-3317920-1304270208 -o /tmp/evaluation-result-3317920-1304270208/solver-3317920-1304270208 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317920-1304270208.cnf -seed 809406488 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f35f3aee72237c1bf3d673feb4883756
RANDOM SEED=809406488

node125.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5266.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.84
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      20728012 kB
Buffers:       2140920 kB
Cached:        7282436 kB
SwapCached:          4 kB
Active:        4915956 kB
Inactive:      6766128 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      20728012 kB
SwapTotal:    67111528 kB
SwapFree:     67111524 kB
Dirty:           34816 kB
Writeback:           0 kB
AnonPages:     2258016 kB
Mapped:          24512 kB
Slab:           465424 kB
PageTables:      14480 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS:  3160240 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73052 MiB
End job on node125 at 2011-05-01 19:38:30