Trace number 3317688

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-industrial? (TO) 4115.46 1300.14

General information on the benchmark

NameSAT05/industrial/grieu05/vmpc/
vmpc_35.renamed-as.sat05-1921.cnf
MD5SUM5fda0e0fb8a21967728dfe29bf2d4ea5
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark473.18
Satisfiable
(Un)Satisfiability was proved
Number of variables1225
Number of clauses211785
Sum of the clauses size554190
Maximum clause length35
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 283405
Number of clauses of size 3128310
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 570

Solver Data 

0.43/0.73	c INFO: loaded portfolio from HOME/etc/borg-mix+class.industrial.pickle
0.43/0.73	c INFO: solving HOME/instance-3317688-1304271104.cnf
0.43/0.73	c INFO: computing task features
1.02/1.39	c INFO: computed variable-clause graph statistics
1.12/1.42	c INFO: computed clause balance statistics
4.13/4.41	c INFO: computed variable balance statistics
6.72/7.04	c INFO: computed small-clause counts
6.72/7.05	c INFO: computed Horn-clause counts
6.82/7.10	c INFO: computed variable graph statistics
6.82/7.10	c INFO: collected features for HOME/instance-3317688-1304271104.cnf in 6.35 s
6.82/7.12	c INFO: running lingeling-276@0 for 97 with 4793 remaining (b = 0.19)
6.82/7.15	c INFO: running cryptominisat-2.9.0@0 for 485 with 4793 remaining (b = 0.24)
6.82/7.17	c INFO: running lingeling-276@0 for 582 with 4793 remaining (b = 0.32)
6.91/7.20	c INFO: running lingeling-276@0 for 1261 with 4793 remaining (b = 0.39)
590.96/202.50	c INFO: running lingeling-276@0 for 1164 with 4598 remaining (b = 0.33)
1676.55/550.83	c INFO: running SApperloT@0 for 97 with 4249 remaining (b = 0.07)
2264.98/747.19	c INFO: running precosat-570@0 for 291 with 4053 remaining (b = 0.17)
2383.01/786.56	c INFO: running cryptominisat-2.9.0@0 for 291 with 4014 remaining (b = 0.12)
3383.15/1075.76	c INFO: running precosat-570@584 for 97 with 3724 remaining (b = 0.06)
3714.74/1176.95	c INFO: running cryptominisat-2.9.0@0 for 1649 with 3623 remaining (b = 0.45)
4026.14/1274.08	c INFO: running precosat-570@679 for 97 with 3526 remaining (b = 0.06)
4115.46/1300.14	c INFO: running lingeling-276@0 for 97 with 3500 remaining (b = 0.06)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317688-1304271104/watcher-3317688-1304271104 -o /tmp/evaluation-result-3317688-1304271104/solver-3317688-1304271104 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317688-1304271104.cnf -seed 1130412693 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.54 8.99 9.20 4/242 24636
/proc/meminfo: memFree=28813108/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=15384 CPUtime=0 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 820 0 0 0 0 0 0 0 17 0 1 0 617341075 15753216 705 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905392128 0 2147483391 16781312 2 18446744071699953811 0 0 17 0 0 0 0
/proc/24635/statm: 3846 705 334 364 0 333 0

[startup+0.0122489 s]
/proc/loadavg: 6.54 8.99 9.20 4/242 24636
/proc/meminfo: memFree=28813108/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=15776 CPUtime=0 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 895 0 0 0 0 0 0 0 17 0 1 0 617341075 16154624 777 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905392128 0 2147483391 16781312 2 18446744071699953811 0 0 17 6 0 0 0
/proc/24635/statm: 3944 777 343 364 0 431 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15776

[startup+0.100552 s]
/proc/loadavg: 6.54 8.99 9.20 4/242 24636
/proc/meminfo: memFree=28813108/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=54536 CPUtime=0.06 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 2071 0 0 0 4 2 0 0 17 0 1 0 617341075 55844864 1828 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905392128 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/24635/statm: 13634 1828 492 364 0 1284 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 54536

[startup+0.300472 s]
/proc/loadavg: 6.54 8.99 9.20 4/242 24636
/proc/meminfo: memFree=28813108/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=135752 CPUtime=0.19 cores=0,2,4,6
/proc/24635/stat : 24635 (python) R 24631 24635 23228 0 -1 4202496 5965 934 1 0 12 7 0 0 18 0 1 0 617341075 139010048 4150 33554432000 4194304 5683644 140737037229584 18446744073709551615 5294181 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/24635/statm: 33938 4150 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.700397 s]
/proc/loadavg: 6.54 8.99 9.20 4/242 24636
/proc/meminfo: memFree=28813108/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=259460 CPUtime=0.43 cores=0,2,4,6
/proc/24635/stat : 24635 (python) R 24631 24635 23228 0 -1 4202496 8593 934 1 0 28 15 0 0 16 0 1 0 617341075 265687040 6432 33554432000 4194304 5683644 140737037229584 18446744073709551615 4871718 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/24635/statm: 64865 6432 1656 364 0 5223 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 259460

[startup+1.50035 s]
/proc/loadavg: 6.54 8.99 9.20 6/244 24646
/proc/meminfo: memFree=28731732/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=320592 CPUtime=1.22 cores=0,2,4,6
/proc/24635/stat : 24635 (python) R 24631 24635 23228 0 -1 4202496 24840 934 1 0 104 18 0 0 17 0 1 0 617341075 328286208 17533 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905052951 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/24635/statm: 80148 17533 1826 364 0 16072 0
Current children cumulated CPU time (s) 1.22
Current children cumulated vsize (KiB) 320592

[startup+3.10095 s]
/proc/loadavg: 6.54 8.99 9.20 6/244 24646
/proc/meminfo: memFree=28679280/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=320592 CPUtime=2.82 cores=0,2,4,6
/proc/24635/stat : 24635 (python) R 24631 24635 23228 0 -1 4202496 24840 934 1 0 264 18 0 0 19 0 1 0 617341075 328286208 17533 33554432000 4194304 5683644 140737037229584 18446744073709551615 47498150246192 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/24635/statm: 80148 17533 1826 364 0 16072 0
Current children cumulated CPU time (s) 2.82
Current children cumulated vsize (KiB) 320592

[startup+6.30047 s]
/proc/loadavg: 6.41 8.92 9.18 6/244 24647
/proc/meminfo: memFree=28677916/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=320592 CPUtime=6.02 cores=0,2,4,6
/proc/24635/stat : 24635 (python) R 24631 24635 23228 0 -1 4202496 24841 934 1 0 584 18 0 0 25 0 1 0 617341075 328286208 17534 33554432000 4194304 5683644 140737037229584 18446744073709551615 4602370 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/24635/statm: 80148 17534 1827 364 0 16072 0
Current children cumulated CPU time (s) 6.02
Current children cumulated vsize (KiB) 320592

[startup+12.7012 s]
/proc/loadavg: 6.86 8.97 9.19 12/268 24671
/proc/meminfo: memFree=28501216/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=358088 CPUtime=6.92 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 27578 934 1 0 673 19 0 0 25 0 5 0 617341075 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/24635/statm: 89522 16719 1881 364 0 25431 0
[pid=24648] ppid=24635 vsize=317076 CPUtime=0 cores=0,2,4,6
/proc/24648/stat : 24648 (python) S 24635 24635 23228 0 -1 4202560 980 0 0 0 0 0 0 0 18 0 1 0 617341787 324685824 15203 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414159 0 0 16781312 514 5517534345306276100 0 0 17 6 0 0 0
/proc/24648/statm: 79269 15203 381 364 0 15187 0
[pid=24650] ppid=24648 vsize=16328 CPUtime=2.83 cores=0,2,4,6
/proc/24650/stat : 24650 (lingeling) R 24648 24650 24650 0 -1 4202496 7193 0 0 0 283 0 0 0 25 0 1 0 617341788 16719872 3247 33554432000 4194304 4352236 140734574534224 18446744073709551615 4328026 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24650/statm: 4082 3247 129 39 0 3162 0
[pid=24651] ppid=24635 vsize=327332 CPUtime=0 cores=0,2,4,6
/proc/24651/stat : 24651 (python) S 24635 24635 23228 0 -1 4202560 901 0 0 0 0 0 0 0 15 0 1 0 617341790 335187968 15196 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 4061952748822692099 0 0 17 6 0 0 0
/proc/24651/statm: 81833 15196 370 364 0 17748 0
[pid=24653] ppid=24651 vsize=57040 CPUtime=5.47 cores=0,2,4,6
/proc/24653/stat : 24653 (cryptominisat-2) R 24651 24653 24653 0 -1 4202496 10370 0 0 0 546 1 0 0 25 0 1 0 617341790 58408960 7269 33554432000 4194304 4677096 140735699481936 18446744073709551615 4319754 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/24653/statm: 14260 7269 383 118 0 9219 0
[pid=24654] ppid=24635 vsize=337588 CPUtime=0 cores=0,2,4,6
/proc/24654/stat : 24654 (python) S 24635 24635 23228 0 -1 4202560 926 0 0 0 0 0 0 0 18 0 1 0 617341793 345690112 15200 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 10317635425048428804 0 0 17 0 0 0 0
/proc/24654/statm: 84397 15200 370 364 0 20309 0
[pid=24658] ppid=24654 vsize=16328 CPUtime=5.46 cores=0,2,4,6
/proc/24658/stat : 24658 (lingeling) R 24654 24658 24658 0 -1 4202496 7195 0 0 0 545 1 0 0 25 0 1 0 617341796 16719872 3246 33554432000 4194304 4352236 140733310141664 18446744073709551615 4328075 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/24658/statm: 4082 3246 129 39 0 3162 0
[pid=24656] ppid=24635 vsize=347844 CPUtime=0 cores=0,2,4,6
/proc/24656/stat : 24656 (python) S 24635 24635 23228 0 -1 4202560 939 0 0 0 0 0 0 0 18 0 1 0 617341795 356192256 15203 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 10317631576757731586 0 0 17 0 0 0 0
/proc/24656/statm: 86961 15203 369 364 0 22870 0
[pid=24659] ppid=24656 vsize=16328 CPUtime=2.71 cores=0,2,4,6
/proc/24659/stat : 24659 (lingeling) R 24656 24659 24659 0 -1 4202496 7197 0 0 0 271 0 0 0 25 0 1 0 617341796 16719872 3246 33554432000 4194304 4352236 140733603622736 18446744073709551615 4329300 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24659/statm: 4082 3246 129 39 0 3162 0
[pid=24635/tid=24649] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24649/stat : 24649 (python) S 24631 24635 23228 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 617341788 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24652] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24652/stat : 24652 (python) S 24631 24635 23228 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 5 0 617341790 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24655] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24655/stat : 24655 (python) S 24631 24635 23228 0 -1 4202560 41 934 0 0 0 0 0 0 25 0 5 0 617341793 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24657] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24657/stat : 24657 (python) S 24631 24635 23228 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 617341796 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 23.39
Current children cumulated vsize (KiB) 1793952

[startup+25.5009 s]
/proc/loadavg: 7.55 9.02 9.21 12/268 24671
/proc/meminfo: memFree=28468312/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=358088 CPUtime=6.92 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 27578 934 1 0 673 19 0 0 25 0 5 0 617341075 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/24635/statm: 89522 16719 1881 364 0 25431 0
[pid=24648] ppid=24635 vsize=317076 CPUtime=0.02 cores=0,2,4,6
/proc/24648/stat : 24648 (python) S 24635 24635 23228 0 -1 4202560 986 0 0 0 1 1 0 0 18 0 1 0 617341787 324685824 15203 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414159 0 0 16781312 514 240625920715489540 0 0 17 6 0 0 0
/proc/24648/statm: 79269 15203 381 364 0 15187 0
[pid=24650] ppid=24648 vsize=20316 CPUtime=9.23 cores=0,2,4,6
/proc/24650/stat : 24650 (lingeling) R 24648 24650 24650 0 -1 4202496 8190 0 0 0 923 0 0 0 25 0 1 0 617341788 20803584 4244 33554432000 4194304 4352236 140734574534224 18446744073709551615 4328009 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24650/statm: 5079 4244 129 39 0 4159 0
[pid=24651] ppid=24635 vsize=327332 CPUtime=0.03 cores=0,2,4,6
/proc/24651/stat : 24651 (python) S 24635 24635 23228 0 -1 4202560 918 0 0 0 2 1 0 0 15 0 1 0 617341790 335187968 15196 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 8847215919148007683 0 0 17 6 0 0 0
/proc/24651/statm: 81833 15196 370 364 0 17748 0
[pid=24653] ppid=24651 vsize=67908 CPUtime=18.12 cores=0,2,4,6
/proc/24653/stat : 24653 (cryptominisat-2) R 24651 24653 24653 0 -1 4202496 15091 0 0 0 1810 2 0 0 25 0 1 0 617341790 69537792 10458 33554432000 4194304 4677096 140735699481936 18446744073709551615 4320006 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/24653/statm: 16977 10458 388 118 0 11936 0
[pid=24654] ppid=24635 vsize=337588 CPUtime=0.02 cores=0,2,4,6
/proc/24654/stat : 24654 (python) S 24635 24635 23228 0 -1 4202560 933 0 0 0 1 1 0 0 18 0 1 0 617341793 345690112 15200 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 240625920715489540 0 0 17 0 0 0 0
/proc/24654/statm: 84397 15200 370 364 0 20309 0
[pid=24658] ppid=24654 vsize=20752 CPUtime=18.21 cores=0,2,4,6
/proc/24658/stat : 24658 (lingeling) R 24654 24658 24658 0 -1 4202496 8301 0 0 0 1820 1 0 0 25 0 1 0 617341796 21250048 4352 33554432000 4194304 4352236 140733310141664 18446744073709551615 4328109 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/24658/statm: 5188 4352 129 39 0 4268 0
[pid=24656] ppid=24635 vsize=347844 CPUtime=0.02 cores=0,2,4,6
/proc/24656/stat : 24656 (python) S 24635 24635 23228 0 -1 4202560 946 0 0 0 1 1 0 0 18 0 1 0 617341795 356192256 15204 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 240625920715489538 0 0 17 0 0 0 0
/proc/24656/statm: 86961 15204 370 364 0 22870 0
[pid=24659] ppid=24656 vsize=20420 CPUtime=9.11 cores=0,2,4,6
/proc/24659/stat : 24659 (lingeling) R 24656 24659 24659 0 -1 4202496 8220 0 0 0 911 0 0 0 25 0 1 0 617341796 20910080 4269 33554432000 4194304 4352236 140733603622736 18446744073709551615 4328109 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24659/statm: 5105 4269 129 39 0 4185 0
[pid=24635/tid=24649] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24649/stat : 24649 (python) S 24631 24635 23228 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 617341788 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24652] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24652/stat : 24652 (python) S 24631 24635 23228 0 -1 4202560 21 934 0 0 0 0 0 0 25 0 5 0 617341790 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24655] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24655/stat : 24655 (python) S 24631 24635 23228 0 -1 4202560 41 934 0 0 0 0 0 0 25 0 5 0 617341793 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24657] ppid=24631 vsize=358088 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24657/stat : 24657 (python) S 24631 24635 23228 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 5 0 617341796 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 61.68
Current children cumulated vsize (KiB) 1817324

[startup+51.1012 s]
/proc/loadavg: 8.45 9.11 9.23 12/268 24672
/proc/meminfo: memFree=28447368/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=358088 CPUtime=6.92 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 27578 934 1 0 673 19 0 0 25 0 5 0 617341075 366682112 16719 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/24635/statm: 89522 16719 1881 364 0 25431 0
[pid=24648] ppid=24635 vsize=317076 CPUtime=0.07 cores=0,2,4,6
/proc/24648/stat : 24648 (python) S 24635 24635 23228 0 -1 4202560 999 0 0 0 4 3 0 0 18 0 1 0 617341787 324685824 15203 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414159 0 0 16781312 514 240625920715489540 0 0 17 6 0 0 0
/proc/24648/statm: 79269 15203 381 364 0 15187 0
[pid=24650] ppid=24648 vsize=20920 CPUtime=22.02 cores=0,2,4,6
/proc/24650/stat : 24650 (lingeling) R 24648 24650 24650 0 -1 4202496 8341 0 0 0 2202 0 0 0 25 0 1 0 617341788 21422080 4395 33554432000 4194304 4352236 140734574534224 18446744073709551615 4328013 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24650/statm: 5230 4395 129 39 0 4310 0
[pid=24651] ppid=24635 vsize=327332 CPUtime=0.09 cores=0,2,4,6
/proc/24651/stat : 24651 (python) S 24635 24635 23228 0 -1 4202560 931 0 0 0 5 4 0 0 15 0 1 0 617341790 335187968 15196 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 8847215919148007683 0 0 17 6 0 0 0
/proc/24651/statm: 81833 15196 370 364 0 17748 0
[pid=24653] ppid=24651 vsize=74728 CPUtime=43.4 cores=0,2,4,6
/proc/24653/stat : 24653 (cryptominisat-2) R 24651 24653 24653 0 -1 4202496 23478 0 0 0 4337 3 0 0 25 0 1 0 617341790 76521472 11713 33554432000 4194304 4677096 140735699481936 18446744073709551615 4336232 0 0 16781312 2 0 0 0 17 6 0 0 0
/proc/24653/statm: 18682 11713 388 118 0 13641 0
[pid=24654] ppid=24635 vsize=337588 CPUtime=0.07 cores=0,2,4,6
/proc/24654/stat : 24654 (python) S 24635 24635 23228 0 -1 4202560 942 0 0 0 4 3 0 0 18 0 1 0 617341793 345690112 15200 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 240625920715489540 0 0 17 0 0 0 0
/proc/24654/statm: 84397 15200 370 364 0 20309 0
[pid=24658] ppid=24654 vsize=21496 CPUtime=43.73 cores=0,2,4,6
/proc/24658/stat : 24658 (lingeling) R 24654 24658 24658 0 -1 4202496 8487 0 0 0 4372 1 0 0 25 0 1 0 617341796 22011904 4538 33554432000 4194304 4352236 140733310141664 18446744073709551615 4241453 0 0 16781312 17506 0 0 0 17 0 0 0 0
/proc/24658/statm: 5374 4538 129 39 0 4454 0
[pid=24656] ppid=24635 vsize=347844 CPUtime=0.06 cores=0,2,4,6
/proc/24656/stat : 24656 (python) S 24635 24635 23228 0 -1 4202560 956 0 0 0 3 3 0 0 18 0 1 0 617341795 356192256 15204 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 240625920715489538 0 0 17 0 0 0 0
/proc/24656/statm: 86961 15204 370 364 0 22870 0

################
# More data... #
################

/proc/meminfo: memFree=27936524/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=388856 CPUtime=7.06 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 30079 934 1 0 685 21 0 0 20 0 8 0 617341075 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/24635/statm: 97214 16738 1888 364 0 33114 0
[pid=24648] ppid=24635 vsize=327320 CPUtime=2.27 cores=0,2,4,6
/proc/24648/stat : 24648 (python) S 24635 24635 23228 0 -1 4202560 1209 0 0 0 124 103 0 0 18 0 2 0 617341787 335175680 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/24648/statm: 81830 15242 415 364 0 17748 0
[pid=24650] ppid=24648 vsize=46952 CPUtime=937.47 cores=0,2,4,6
/proc/24650/stat : 24650 (lingeling) R 24648 24650 24650 0 -1 4202496 14849 0 0 0 93744 3 0 0 25 0 1 0 617341788 48078848 10903 33554432000 4194304 4352236 140734574534224 18446744073709551615 4328114 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/24650/statm: 11738 10903 129 39 0 10818 0
[pid=24648/tid=24678] ppid=24635 vsize=327320 CPUtime=0 cores=0,2,4,6
/proc/24648/task/24678/stat : 24678 (python) S 24635 24635 23228 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 617361323 335175680 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604445607409728 0 0 -1 6 0 0 0
[pid=24651] ppid=24635 vsize=327332 CPUtime=1.97 cores=0,2,4,6
/proc/24651/stat : 24651 (python) S 24635 24635 23228 0 -1 4202560 1169 0 0 0 107 90 0 0 15 0 2 0 617341790 335187968 15227 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/24651/statm: 81833 15227 400 364 0 17748 0
[pid=24653] ppid=24651 vsize=112204 CPUtime=749.98 cores=0,2,4,6
/proc/24653/stat : 24653 (cryptominisat-2) R 24651 24653 24653 0 -1 4202496 140683 0 0 0 74961 37 0 0 25 0 1 0 617341790 114896896 21099 33554432000 4194304 4677096 140735699481936 18446744073709551615 4319802 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/24653/statm: 28051 21099 388 118 0 23010 0
[pid=24651/tid=24691] ppid=24635 vsize=327332 CPUtime=0 cores=0,2,4,6
/proc/24651/task/24691/stat : 24691 (python) S 24635 24635 23228 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 617396156 335187968 15227 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604453858440320 0 0 -1 6 0 0 0
[pid=24654] ppid=24635 vsize=337588 CPUtime=1.43 cores=0,2,4,6
/proc/24654/stat : 24654 (python) S 24635 24635 23228 0 -1 4202560 1149 0 0 0 83 60 0 0 19 0 2 0 617341793 345690112 15230 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/24654/statm: 84397 15230 399 364 0 20309 0
[pid=24658] ppid=24654 vsize=44860 CPUtime=584.84 cores=0,2,4,6
/proc/24658/stat : 24658 (lingeling) T 24654 24658 24658 0 -1 4202496 14328 0 0 0 58480 4 0 0 25 0 1 0 617341796 45936640 10379 33554432000 4194304 4352236 140733310141664 18446744073709551615 4328116 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/24658/statm: 11215 10379 129 39 0 10295 0
[pid=24654/tid=24709] ppid=24635 vsize=337588 CPUtime=0 cores=0,2,4,6
/proc/24654/task/24709/stat : 24709 (python) S 24635 24635 23228 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 617419729 345690112 15230 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604446725020352 0 0 -1 2 0 0 0
[pid=24656] ppid=24635 vsize=347844 CPUtime=2.21 cores=0,2,4,6
/proc/24656/stat : 24656 (python) S 24635 24635 23228 0 -1 4202560 1005 0 0 0 120 101 0 0 18 0 1 0 617341795 356192256 15204 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 240625920715489538 0 0 17 0 0 0 0
/proc/24656/statm: 86961 15204 370 364 0 22870 0
[pid=24659] ppid=24656 vsize=50060 CPUtime=1106.78 cores=0,2,4,6
/proc/24659/stat : 24659 (lingeling) R 24656 24659 24659 0 -1 4202496 15630 0 0 0 110676 2 0 0 25 0 1 0 617341796 51261440 11679 33554432000 4194304 4352236 140733603622736 18446744073709551615 4328026 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24659/statm: 12515 11679 129 39 0 11595 0
[pid=24692] ppid=24635 vsize=358100 CPUtime=0.35 cores=0,2,4,6
/proc/24692/stat : 24692 (python) S 24635 24635 23228 0 -1 4202560 1167 0 0 0 18 17 0 0 19 0 2 0 617396158 366694400 15238 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/24692/statm: 89525 15238 399 364 0 25431 0
[pid=24694] ppid=24692 vsize=56056 CPUtime=98.19 cores=0,2,4,6
/proc/24694/stat : 24694 (SApperloT-base) T 24692 24694 24694 0 -1 4202496 13636 0 0 0 9817 2 0 0 25 0 1 0 617396159 57401344 13141 33554432000 134512640 135674940 4291985904 18446744073709551615 134564660 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/24694/statm: 14014 13141 99 284 0 13728 0
[pid=24692/tid=24704] ppid=24635 vsize=358100 CPUtime=0 cores=0,2,4,6
/proc/24692/task/24704/stat : 24704 (python) S 24635 24635 23228 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 617415792 366694400 15238 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604453860911872 0 0 -1 6 0 0 0
[pid=24705] ppid=24635 vsize=368356 CPUtime=0.78 cores=0,2,4,6
/proc/24705/stat : 24705 (python) S 24635 24635 23228 0 -1 4202560 1166 0 0 0 40 38 0 0 18 0 2 0 617415794 377196544 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/24705/statm: 92089 15242 399 364 0 27992 0
[pid=24707] ppid=24705 vsize=55124 CPUtime=389.54 cores=0,2,4,6
/proc/24707/stat : 24707 (precosat) T 24705 24707 24707 0 -1 4202496 29815 0 0 0 38944 10 0 0 25 0 1 0 617415794 56446976 13653 33554432000 4194304 4890832 140733481336752 18446744073709551615 4299152 0 0 16781312 17442 18446744071562691360 0 0 17 6 0 0 0
/proc/24707/statm: 13781 13653 85 171 0 13608 0
[pid=24705/tid=24723] ppid=24635 vsize=368356 CPUtime=0 cores=0,2,4,6
/proc/24705/task/24723/stat : 24723 (python) S 24635 24635 23228 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 617448649 377196544 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604453873586432 0 0 -1 4 0 0 0
[pid=24728] ppid=24635 vsize=378612 CPUtime=0.13 cores=0,2,4,6
/proc/24728/stat : 24728 (python) S 24635 24635 23228 0 -1 4202560 967 0 0 0 6 7 0 0 15 0 1 0 617458770 387698688 15216 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 240625920715489537 0 0 17 6 0 0 0
/proc/24728/statm: 94653 15216 370 364 0 30553 0
[pid=24730] ppid=24728 vsize=71488 CPUtime=32.6 cores=0,2,4,6
/proc/24730/stat : 24730 (cryptominisat-2) R 24728 24730 24730 0 -1 4202496 19452 0 0 0 3257 3 0 0 25 0 1 0 617458771 73203712 11179 33554432000 4194304 4677096 140733920990368 18446744073709551615 4462728 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/24730/statm: 17872 11179 388 118 0 12831 0
[pid=24635/tid=24649] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24649/stat : 24649 (python) S 24631 24635 23228 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 8 0 617341788 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24652] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24652/stat : 24652 (python) S 24631 24635 23228 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 8 0 617341790 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24655] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24655/stat : 24655 (python) S 24631 24635 23228 0 -1 4202560 41 934 0 0 0 0 0 0 25 0 8 0 617341793 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24657] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24657/stat : 24657 (python) S 24631 24635 23228 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 8 0 617341796 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24693] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24693/stat : 24693 (python) S 24631 24635 23228 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 8 0 617396158 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24706] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24706/stat : 24706 (python) S 24631 24635 23228 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 8 0 617415794 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24729] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24729/stat : 24729 (python) S 24631 24635 23228 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 8 0 617458770 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3915.6
Current children cumulated vsize (KiB) 3270752



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+1300 s]
/proc/loadavg: 10.15 10.13 9.89 13/296 24736
/proc/meminfo: memFree=27868792/32950928 swapFree=67054472/67111528
[pid=24635] ppid=24631 vsize=388856 CPUtime=7.08 cores=0,2,4,6
/proc/24635/stat : 24635 (python) S 24631 24635 23228 0 -1 4202496 30585 934 1 0 687 21 0 0 18 0 8 0 617341075 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/24635/statm: 97214 16738 1888 364 0 33114 0
[pid=24648] ppid=24635 vsize=327320 CPUtime=2.38 cores=0,2,4,6
/proc/24648/stat : 24648 (python) S 24635 24635 23228 0 -1 4202560 1209 0 0 0 129 109 0 0 18 0 2 0 617341787 335175680 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/24648/statm: 81830 15242 415 364 0 17748 0
[pid=24650] ppid=24648 vsize=46952 CPUtime=992.53 cores=0,2,4,6
/proc/24650/stat : 24650 (lingeling) R 24648 24650 24650 0 -1 4202496 14849 0 0 0 99250 3 0 0 25 0 1 0 617341788 48078848 10903 33554432000 4194304 4352236 140734574534224 18446744073709551615 4277685 0 0 16781312 17506 0 0 0 17 6 0 0 0
/proc/24650/statm: 11738 10903 129 39 0 10818 0
[pid=24648/tid=24678] ppid=24635 vsize=327320 CPUtime=0 cores=0,2,4,6
/proc/24648/task/24678/stat : 24678 (python) S 24635 24635 23228 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 617361323 335175680 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604445607409728 0 0 -1 6 0 0 0
[pid=24651] ppid=24635 vsize=327332 CPUtime=2.05 cores=0,2,4,6
/proc/24651/stat : 24651 (python) S 24635 24635 23228 0 -1 4202560 1169 0 0 0 111 94 0 0 16 0 2 0 617341790 335187968 15227 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/24651/statm: 81833 15227 400 364 0 17748 0
[pid=24653] ppid=24651 vsize=112244 CPUtime=781.61 cores=0,2,4,6
/proc/24653/stat : 24653 (cryptominisat-2) T 24651 24653 24653 0 -1 4202496 145471 0 0 0 78123 38 0 0 25 0 1 0 617341790 114937856 22407 33554432000 4194304 4677096 140735699481936 18446744073709551615 4334573 0 0 16781312 2 18446744071562691360 0 0 17 0 0 0 0
/proc/24653/statm: 28061 22407 388 118 0 23020 0
[pid=24651/tid=24691] ppid=24635 vsize=327332 CPUtime=0 cores=0,2,4,6
/proc/24651/task/24691/stat : 24691 (python) S 24635 24635 23228 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 617396156 335187968 15227 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604453858440320 0 0 -1 6 0 0 0
[pid=24654] ppid=24635 vsize=337588 CPUtime=1.43 cores=0,2,4,6
/proc/24654/stat : 24654 (python) S 24635 24635 23228 0 -1 4202560 1149 0 0 0 83 60 0 0 19 0 2 0 617341793 345690112 15230 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/24654/statm: 84397 15230 399 364 0 20309 0
[pid=24658] ppid=24654 vsize=44860 CPUtime=584.84 cores=0,2,4,6
/proc/24658/stat : 24658 (lingeling) T 24654 24658 24658 0 -1 4202496 14328 0 0 0 58480 4 0 0 25 0 1 0 617341796 45936640 10379 33554432000 4194304 4352236 140733310141664 18446744073709551615 4328116 0 0 16781312 17506 18446744071562691360 0 0 17 2 0 0 0
/proc/24658/statm: 11215 10379 129 39 0 10295 0
[pid=24654/tid=24709] ppid=24635 vsize=337588 CPUtime=0 cores=0,2,4,6
/proc/24654/task/24709/stat : 24709 (python) S 24635 24635 23228 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 617419729 345690112 15230 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604446725020352 0 0 -1 2 0 0 0
[pid=24656] ppid=24635 vsize=347844 CPUtime=2.31 cores=0,2,4,6
/proc/24656/stat : 24656 (python) S 24635 24635 23228 0 -1 4202560 1005 0 0 0 125 106 0 0 18 0 1 0 617341795 356192256 15204 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 240625920715489538 0 0 17 0 0 0 0
/proc/24656/statm: 86961 15204 370 364 0 22870 0
[pid=24659] ppid=24656 vsize=50060 CPUtime=1164.43 cores=0,2,4,6
/proc/24659/stat : 24659 (lingeling) R 24656 24659 24659 0 -1 4202496 15630 0 0 0 116441 2 0 0 25 0 1 0 617341796 51261440 11679 33554432000 4194304 4352236 140733603622736 18446744073709551615 4328095 0 0 16781312 17506 0 0 0 17 4 0 0 0
/proc/24659/statm: 12515 11679 129 39 0 11595 0
[pid=24692] ppid=24635 vsize=358100 CPUtime=0.35 cores=0,2,4,6
/proc/24692/stat : 24692 (python) S 24635 24635 23228 0 -1 4202560 1167 0 0 0 18 17 0 0 19 0 2 0 617396158 366694400 15238 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917220811 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/24692/statm: 89525 15238 399 364 0 25431 0
[pid=24694] ppid=24692 vsize=56056 CPUtime=98.19 cores=0,2,4,6
/proc/24694/stat : 24694 (SApperloT-base) T 24692 24694 24694 0 -1 4202496 13636 0 0 0 9817 2 0 0 25 0 1 0 617396159 57401344 13141 33554432000 134512640 135674940 4291985904 18446744073709551615 134564660 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/24694/statm: 14014 13141 99 284 0 13728 0
[pid=24692/tid=24704] ppid=24635 vsize=358100 CPUtime=0 cores=0,2,4,6
/proc/24692/task/24704/stat : 24704 (python) S 24635 24635 23228 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 617415792 366694400 15238 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604453860911872 0 0 -1 6 0 0 0
[pid=24705] ppid=24635 vsize=368356 CPUtime=0.84 cores=0,2,4,6
/proc/24705/stat : 24705 (python) S 24635 24635 23228 0 -1 4202560 1166 0 0 0 42 42 0 0 18 0 2 0 617415794 377196544 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/24705/statm: 92089 15242 399 364 0 27992 0
[pid=24707] ppid=24705 vsize=58388 CPUtime=402.85 cores=0,2,4,6
/proc/24707/stat : 24707 (precosat) R 24705 24707 24707 0 -1 4202496 30476 0 0 0 40275 10 0 0 25 0 1 0 617415794 59789312 14314 33554432000 4194304 4890832 140733481336752 18446744073709551615 4238686 0 0 16781312 17442 0 0 0 17 2 0 0 0
/proc/24707/statm: 14597 14314 85 171 0 14424 0
[pid=24705/tid=24723] ppid=24635 vsize=368356 CPUtime=0 cores=0,2,4,6
/proc/24705/task/24723/stat : 24723 (python) S 24635 24635 23228 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 617448649 377196544 15242 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 514 18446604453873586432 0 0 -1 4 0 0 0
[pid=24728] ppid=24635 vsize=378612 CPUtime=0.25 cores=0,2,4,6
/proc/24728/stat : 24728 (python) S 24635 24635 23228 0 -1 4202560 986 0 0 0 13 12 0 0 15 0 1 0 617458770 387698688 15216 33554432000 4194304 5683644 140737037229584 18446744073709551615 260905414214 0 0 16781312 514 9453521815075782913 0 0 17 0 0 0 0
/proc/24728/statm: 94653 15216 370 364 0 30553 0
[pid=24730] ppid=24728 vsize=99984 CPUtime=74.32 cores=0,2,4,6
/proc/24730/stat : 24730 (cryptominisat-2) R 24728 24730 24730 0 -1 4202496 27314 0 0 0 7428 4 0 0 25 0 1 0 617458771 102383616 17095 33554432000 4194304 4677096 140733920990368 18446744073709551615 4319787 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/24730/statm: 24996 17095 388 118 0 19955 0
[pid=24635/tid=24649] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24649/stat : 24649 (python) S 24631 24635 23228 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 8 0 617341788 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24652] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24652/stat : 24652 (python) S 24631 24635 23228 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 8 0 617341790 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 6 0 0 0
[pid=24635/tid=24655] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24655/stat : 24655 (python) S 24631 24635 23228 0 -1 4202560 41 934 0 0 0 0 0 0 25 0 8 0 617341793 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24657] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24657/stat : 24657 (python) S 24631 24635 23228 0 -1 4202560 24 934 0 0 0 0 0 0 25 0 8 0 617341796 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24693] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24693/stat : 24693 (python) S 24631 24635 23228 0 -1 4202560 35 934 0 0 0 0 0 0 20 0 8 0 617396158 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24706] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24706/stat : 24706 (python) S 24631 24635 23228 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 8 0 617415794 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
[pid=24635/tid=24729] ppid=24631 vsize=388856 CPUtime=0 cores=0,2,4,6
/proc/24635/task/24729/stat : 24729 (python) S 24631 24635 23228 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 8 0 617458770 398188544 16738 33554432000 4194304 5683644 140737037229584 18446744073709551615 260917217713 0 0 16781312 2 18446604453984475840 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4115.46
Current children cumulated vsize (KiB) 3302552

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 24635 and gives
#  childrusage.ru_utime.tv_sec=1653
#  childrusage.ru_utime.tv_usec=956560
#  childrusage.ru_stime.tv_sec=4
#  childrusage.ru_stime.tv_usec=856261
# CPU time returned by wait4() is 1658.81
# while last known CPU time is 4115.46
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 1300.14
CPU time (s): 4115.46
CPU user time (s): 4110.22
CPU system time (s): 5.24
CPU usage (%): 316.539
Max. virtual memory (cumulated for all children) (KiB): 3334896

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1653.96
system time used= 4.85626
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 113813
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 17649
involuntary context switches= 20619

runsolver used 3.42348 second user time and 7.8678 second system time

The end

Launcher Data 

Begin job on node116 at 2011-05-01 19:31:44
IDJOB=3317688
IDBENCH=85987
IDSOLVER=1706
FILE ID=node116/3317688-1304271104
RUNJOBID= node116-1304271104-24617
PBS_JOBID= 13173963
Free space on /tmp= 73252 MiB

SOLVER NAME= borg-sat 11.03.02-industrial
BENCH NAME= SAT05/industrial/grieu05/vmpc/vmpc_35.renamed-as.sat05-1921.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.industrial.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317688-1304271104/watcher-3317688-1304271104 -o /tmp/evaluation-result-3317688-1304271104/solver-3317688-1304271104 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.industrial.pickle HOME/instance-3317688-1304271104.cnf -seed 1130412693 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5fda0e0fb8a21967728dfe29bf2d4ea5
RANDOM SEED=1130412693

node116.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.826
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      28813436 kB
Buffers:       1048308 kB
Cached:        1712952 kB
SwapCached:       9656 kB
Active:        1401476 kB
Inactive:      2376532 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      28813436 kB
SwapTotal:    67111528 kB
SwapFree:     67054472 kB
Dirty:           69852 kB
Writeback:           0 kB
AnonPages:     1014596 kB
Mapped:          19176 kB
Slab:           284904 kB
PageTables:      13644 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 35644004 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73224 MiB
End job on node116 at 2011-05-01 19:53:26