Trace number 3386548

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 3457.24 3445.24

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1500-c30000-S303265982-092-UNKNOWN.cnf
MD5SUM1b4a544dd042630233219a622c9437c7
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark186.925
Satisfiable
(Un)Satisfiability was proved
Number of variables1500
Number of clauses30000
Sum of the clauses size150000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 530000
Number of clauses of size over 50

Solver Data 

0.36/0.66	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.36/0.67	c INFO: solving HOME/instance-3386548-1305586492.cnf
0.36/0.67	c INFO: computing task features
0.53/0.82	c INFO: computed variable-clause graph statistics
0.53/0.83	c INFO: computed clause balance statistics
1.33/1.60	c INFO: computed variable balance statistics
1.73/2.02	c INFO: computed small-clause counts
1.73/2.03	c INFO: computed Horn-clause counts
1.73/2.06	c INFO: computed variable graph statistics
1.73/2.06	c INFO: collected features for HOME/instance-3386548-1305586492.cnf in 1.39 s
1.73/2.09	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.38)
102.57/102.50	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.14)
202.30/201.92	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.09)
301.92/301.31	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.07)
402.67/401.69	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.06)
503.44/502.09	c INFO: running gnovelty+2@0 for 97 with 4498 remaining (b = 0.05)
603.28/601.56	c INFO: running FH@0 for 97 with 4399 remaining (b = 0.04)
703.93/701.95	c INFO: running adaptg2wsat2009++@0 for 97 with 4299 remaining (b = 0.03)
802.68/800.37	c INFO: running adaptg2wsat2009++@0 for 97 with 4200 remaining (b = 0.03)
903.27/900.77	c INFO: running FH@0 for 97 with 4100 remaining (b = 0.02)
1003.92/1001.17	c INFO: running hybridGM3@0 for 194 with 4000 remaining (b = 0.04)
1200.44/1197.00	c INFO: running gnovelty+2@0 for 97 with 3804 remaining (b = 0.02)
1299.18/1295.47	c INFO: running hybridGM3@0 for 194 with 3706 remaining (b = 0.03)
1495.00/1490.42	c INFO: running hybridGM3@0 for 194 with 3511 remaining (b = 0.03)
1694.63/1689.31	c INFO: running TNM@0 for 1552 with 3312 remaining (b = 0.21)
3257.05/3245.73	c INFO: running adaptg2wsat2009++@0 for 97 with 1755 remaining (b = 0.02)
3357.83/3346.15	c INFO: running FH@0 for 97 with 1655 remaining (b = 0.01)
3456.59/3444.58	c INFO: running hybridGM3@0 for 194 with 1557 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386548-1305586492/watcher-3386548-1305586492 -o /tmp/evaluation-result-3386548-1305586492/solver-3386548-1305586492 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386548-1305586492.cnf -seed 679681468 -budget 5000 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.07 2.44 2.27 4/469 1724
/proc/meminfo: memFree=13117760/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=14644 CPUtime=0 cores=1,3,5,7
/proc/1724/stat : 1724 (python) R 1722 1724 1545 0 -1 4202496 450 0 0 0 0 0 0 0 25 0 1 0 748879927 14995456 359 33554432000 4194304 5683644 140735616883520 18446744073709551615 4540219 0 0 4096 0 0 0 0 17 3 0 0 0
/proc/1724/statm: 3661 366 229 364 0 148 0

[startup+0.0221649 s]
/proc/loadavg: 3.07 2.44 2.27 4/469 1724
/proc/meminfo: memFree=13117760/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=16240 CPUtime=0 cores=1,3,5,7
/proc/1724/stat : 1724 (python) R 1722 1724 1545 0 -1 4202496 1044 0 0 0 0 0 0 0 17 0 1 0 748879927 16629760 916 33554432000 4194304 5683644 140735616883520 18446744073709551615 5294195 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/1724/statm: 4060 916 381 364 0 547 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 16240

[startup+0.10114 s]
/proc/loadavg: 3.07 2.44 2.27 4/469 1724
/proc/meminfo: memFree=13117760/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=85624 CPUtime=0.06 cores=1,3,5,7
/proc/1724/stat : 1724 (python) R 1722 1724 1545 0 -1 4202496 2628 0 0 0 4 2 0 0 17 0 1 0 748879927 87678976 2348 33554432000 4194304 5683644 140735616883520 18446744073709551615 4619296 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/1724/statm: 21406 2348 713 364 0 1623 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85624

[startup+0.3011 s]
/proc/loadavg: 3.07 2.44 2.27 4/469 1724
/proc/meminfo: memFree=13117760/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=135752 CPUtime=0.19 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 5971 934 1 0 12 7 0 0 17 0 1 0 748879927 139010048 4149 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415387653 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/1724/statm: 33938 4149 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 135752

[startup+0.701029 s]
/proc/loadavg: 3.07 2.44 2.27 4/469 1724
/proc/meminfo: memFree=13117760/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=281072 CPUtime=0.43 cores=1,3,5,7
/proc/1724/stat : 1724 (python) R 1722 1724 1545 0 -1 4202496 9917 934 1 0 28 15 0 0 15 0 1 0 748879927 287817728 7591 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415089109 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/1724/statm: 70268 7591 1761 364 0 6192 0
Current children cumulated CPU time (s) 0.43
Current children cumulated vsize (KiB) 281072

[startup+1.50087 s]
/proc/loadavg: 3.07 2.44 2.27 4/470 1729
/proc/meminfo: memFree=13085824/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=287580 CPUtime=1.23 cores=1,3,5,7
/proc/1724/stat : 1724 (python) R 1722 1724 1545 0 -1 4202496 12688 934 1 0 108 15 0 0 16 0 1 0 748879927 294481920 9279 33554432000 4194304 5683644 140735616883520 18446744073709551615 4418304 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/1724/statm: 71895 9279 1826 364 0 7819 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 287580

[startup+3.10058 s]
/proc/loadavg: 3.07 2.44 2.27 4/473 1733
/proc/meminfo: memFree=13087908/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=292808 CPUtime=1.82 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 12978 934 1 0 167 15 0 0 19 0 2 0 748879927 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/1724/statm: 73202 8098 1869 364 0 9120 0
[pid=1731] ppid=1724 vsize=283508 CPUtime=0 cores=1,3,5,7
/proc/1731/stat : 1731 (python) S 1724 1724 1545 0 -1 4202560 734 0 0 0 0 0 0 0 19 0 1 0 748880137 290312192 6581 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411727 0 0 16781312 514 5904293882568376319 0 0 17 3 0 0 0
/proc/1731/statm: 70877 6581 355 364 0 6795 0
[pid=1733] ppid=1731 vsize=1018996 CPUtime=1 cores=1,3,5,7
/proc/1733/stat : 1733 (gnovelty+2) R 1731 1733 1733 0 -1 4202496 961 0 0 0 100 0 0 0 23 0 1 0 748880137 1043451904 698 33554432000 134512640 135663704 4289505264 18446744073709551615 134545566 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/1733/statm: 254749 698 113 282 0 254465 0
[pid=1724/tid=1732] ppid=1722 vsize=292808 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 748880137 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2.82
Current children cumulated vsize (KiB) 1595312

[startup+6.30096 s]
/proc/loadavg: 3.06 2.45 2.27 4/473 1733
/proc/meminfo: memFree=13085048/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=292808 CPUtime=1.82 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 12978 934 1 0 167 15 0 0 19 0 2 0 748879927 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/1724/statm: 73202 8098 1869 364 0 9120 0
[pid=1731] ppid=1724 vsize=283508 CPUtime=0.02 cores=1,3,5,7
/proc/1731/stat : 1731 (python) S 1724 1724 1545 0 -1 4202560 881 0 0 0 1 1 0 0 18 0 1 0 748880137 290312192 6604 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411727 0 0 16781312 514 5904293882568343815 0 0 17 3 0 0 0
/proc/1731/statm: 70877 6604 378 364 0 6795 0
[pid=1733] ppid=1731 vsize=1018996 CPUtime=4.2 cores=1,3,5,7
/proc/1733/stat : 1733 (gnovelty+2) R 1731 1733 1733 0 -1 4202496 961 0 0 0 420 0 0 0 25 0 1 0 748880137 1043451904 698 33554432000 134512640 135663704 4289505264 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/1733/statm: 254749 698 113 282 0 254465 0
[pid=1724/tid=1732] ppid=1722 vsize=292808 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 748880137 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 6.04
Current children cumulated vsize (KiB) 1595312

[startup+12.7008 s]
/proc/loadavg: 3.06 2.45 2.28 4/473 1733
/proc/meminfo: memFree=13084544/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=292808 CPUtime=1.82 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 12978 934 1 0 167 15 0 0 19 0 2 0 748879927 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/1724/statm: 73202 8098 1869 364 0 9120 0
[pid=1731] ppid=1724 vsize=283508 CPUtime=0.03 cores=1,3,5,7
/proc/1731/stat : 1731 (python) S 1724 1724 1545 0 -1 4202560 883 0 0 0 1 2 0 0 18 0 1 0 748880137 290312192 6604 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411727 0 0 16781312 514 14007971651912696071 0 0 17 3 0 0 0
/proc/1731/statm: 70877 6604 378 364 0 6795 0
[pid=1733] ppid=1731 vsize=1018996 CPUtime=10.6 cores=1,3,5,7
/proc/1733/stat : 1733 (gnovelty+2) R 1731 1733 1733 0 -1 4202496 961 0 0 0 1060 0 0 0 25 0 1 0 748880137 1043451904 698 33554432000 134512640 135663704 4289505264 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/1733/statm: 254749 698 113 282 0 254465 0
[pid=1724/tid=1732] ppid=1722 vsize=292808 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 748880137 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 12.45
Current children cumulated vsize (KiB) 1595312

[startup+25.5003 s]
/proc/loadavg: 3.05 2.48 2.29 4/473 1733
/proc/meminfo: memFree=13085392/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=292808 CPUtime=1.82 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 12978 934 1 0 167 15 0 0 19 0 2 0 748879927 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/1724/statm: 73202 8098 1869 364 0 9120 0
[pid=1731] ppid=1724 vsize=283508 CPUtime=0.07 cores=1,3,5,7
/proc/1731/stat : 1731 (python) S 1724 1724 1545 0 -1 4202560 887 0 0 0 3 4 0 0 18 0 1 0 748880137 290312192 6604 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411727 0 0 16781312 514 14007971651912696071 0 0 17 7 0 0 0
/proc/1731/statm: 70877 6604 378 364 0 6795 0
[pid=1733] ppid=1731 vsize=1018996 CPUtime=23.39 cores=1,3,5,7
/proc/1733/stat : 1733 (gnovelty+2) R 1731 1733 1733 0 -1 4202496 961 0 0 0 2339 0 0 0 25 0 1 0 748880137 1043451904 698 33554432000 134512640 135663704 4289505264 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/1733/statm: 254749 698 113 282 0 254465 0
[pid=1724/tid=1732] ppid=1722 vsize=292808 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 748880137 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 25.28
Current children cumulated vsize (KiB) 1595312

[startup+51.1005 s]
/proc/loadavg: 3.03 2.52 2.30 4/473 1734
/proc/meminfo: memFree=13085288/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=292808 CPUtime=1.82 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 12978 934 1 0 167 15 0 0 19 0 2 0 748879927 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/1724/statm: 73202 8098 1869 364 0 9120 0
[pid=1731] ppid=1724 vsize=283508 CPUtime=0.16 cores=1,3,5,7
/proc/1731/stat : 1731 (python) S 1724 1724 1545 0 -1 4202560 896 0 0 0 8 8 0 0 18 0 1 0 748880137 290312192 6604 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411727 0 0 16781312 514 14007971651912696071 0 0 17 3 0 0 0
/proc/1731/statm: 70877 6604 378 364 0 6795 0
[pid=1733] ppid=1731 vsize=1018996 CPUtime=48.99 cores=1,3,5,7
/proc/1733/stat : 1733 (gnovelty+2) R 1731 1733 1733 0 -1 4202496 961 0 0 0 4899 0 0 0 25 0 1 0 748880137 1043451904 698 33554432000 134512640 135663704 4289505264 18446744073709551615 134525382 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/1733/statm: 254749 698 113 282 0 254465 0
[pid=1724/tid=1732] ppid=1722 vsize=292808 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 748880137 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 50.97
Current children cumulated vsize (KiB) 1595312

[startup+102.318 s]
/proc/loadavg: 3.01 2.59 2.34 4/477 1739
/proc/meminfo: memFree=13073920/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=292808 CPUtime=1.82 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 12978 934 1 0 167 15 0 0 19 0 2 0 748879927 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/1724/statm: 73202 8098 1869 364 0 9120 0
[pid=1731] ppid=1724 vsize=283508 CPUtime=0.33 cores=1,3,5,7
/proc/1731/stat : 1731 (python) S 1724 1724 1545 0 -1 4202560 912 0 0 0 16 17 0 0 18 0 1 0 748880137 290312192 6604 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411727 0 0 16781312 514 14007971651912696071 0 0 17 3 0 0 0
/proc/1731/statm: 70877 6604 378 364 0 6795 0
[pid=1733] ppid=1731 vsize=1018996 CPUtime=100.21 cores=1,3,5,7
/proc/1733/stat : 1733 (gnovelty+2) R 1731 1733 1733 0 -1 4202496 961 0 0 0 10021 0 0 0 25 0 1 0 748880137 1043451904 698 33554432000 134512640 135663704 4289505264 18446744073709551615 134525046 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/1733/statm: 254749 698 113 282 0 254465 0
[pid=1724/tid=1732] ppid=1722 vsize=292808 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 748880137 299835392 8098 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 102.36
Current children cumulated vsize (KiB) 1595312

[startup+162.3 s]
/proc/loadavg: 3.00 2.66 2.38 4/485 1749
/proc/meminfo: memFree=13059056/32950928 swapFree=67111200/67111528
[pid=1724] ppid=1722 vsize=303524 CPUtime=1.85 cores=1,3,5,7
/proc/1724/stat : 1724 (python) S 1722 1724 1545 0 -1 4202496 13876 934 1 0 170 15 0 0 20 0 3 0 748879927 310808576 8188 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/1724/statm: 75881 8188 1872 364 0 11796 0
[pid=1731] ppid=1724 vsize=293752 CPUtime=0.35 cores=1,3,5,7
/proc/1731/stat : 1731 (python) S 1724 1724 1545 0 -1 4202560 1070 0 0 0 17 18 0 0 19 0 2 0 748880137 300802048 6641 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/1731/statm: 73438 6641 410 364 0 9356 0
[pid=1733] ppid=1731 vsize=1018996 CPUtime=100.37 cores=1,3,5,7
/proc/1733/stat : 1733 (gnovelty+2) T 1731 1733 1733 0 -1 4202496 961 0 0 0 10037 0 0 0 25 0 1 0 748880137 1043451904 698 33554432000 134512640 135663704 4289505264 18446744073709551615 134524897 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/1733/statm: 254749 698 113 282 0 254465 0
[pid=1731/tid=1740] ppid=1724 vsize=293752 CPUtime=0 cores=1,3,5,7
/proc/1731/task/1740/stat : 1740 (python) S 1724 1724 1545 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 748890175 300802048 6641 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604467172295680 0 0 -1 3 0 0 0
[pid=1741] ppid=1724 vsize=293696 CPUtime=0.2 cores=1,3,5,7
/proc/1741/stat : 1741 (python) S 1724 1724 1545 0 -1 4202560 910 0 0 0 11 9 0 0 18 0 1 0 748890178 300744704 6685 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411782 0 0 16781312 514 14007971651912696069 0 0 17 7 0 0 0
/proc/1741/statm: 73424 6685 372 364 0 9339 0
[pid=1743] ppid=1741 vsize=1018996 CPUtime=59.79 cores=1,3,5,7
/proc/1743/stat : 1743 (gnovelty+2) R 1741 1743 1743 0 -1 4202496 961 0 0 0 5979 0 0 0 25 0 1 0 748890178 1043451904 697 33554432000 134512640 135663704 4291141488 18446744073709551615 134524921 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/1743/statm: 254749 697 113 282 0 254465 0
[pid=1724/tid=1732] ppid=1722 vsize=303524 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 3 0 748880137 310808576 8188 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
[pid=1724/tid=1742] ppid=1722 vsize=303524 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1742/stat : 1742 (python) S 1722 1724 1545 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 3 0 748890178 310808576 8188 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 162.56

################
# More data... #
################

[pid=1762] ppid=1724 vsize=314208 CPUtime=0.36 cores=1,3,5,7
/proc/1762/stat : 1762 (python) S 1724 1724 1545 0 -1 4202560 1094 0 0 0 18 18 0 0 19 0 2 0 748910059 321748992 6722 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1762/statm: 78552 6722 400 364 0 14461 0
[pid=1764] ppid=1762 vsize=421428 CPUtime=100.34 cores=1,3,5,7
/proc/1764/stat : 1764 (adaptg2wsat2009) T 1762 1764 1764 0 -1 4202496 2051 0 0 0 10034 0 0 0 25 0 1 0 748910059 431542272 1781 33554432000 134512640 135107411 4291828992 18446744073709551615 134532275 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/1764/statm: 105357 1781 59 146 0 104233 0
[pid=1762/tid=1781] ppid=1724 vsize=314208 CPUtime=0 cores=1,3,5,7
/proc/1762/task/1781/stat : 1781 (python) S 1724 1724 1545 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 748920093 321748992 6722 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604464682436416 0 0 -1 7 0 0 0
[pid=1782] ppid=1724 vsize=324464 CPUtime=0.36 cores=1,3,5,7
/proc/1782/stat : 1782 (python) S 1724 1724 1545 0 -1 4202560 1120 0 0 0 21 15 0 0 19 0 2 0 748920097 332251136 6726 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1782/statm: 81116 6726 400 364 0 17022 0
[pid=1784] ppid=1782 vsize=421428 CPUtime=100.36 cores=1,3,5,7
/proc/1784/stat : 1784 (adaptg2wsat2009) T 1782 1784 1784 0 -1 4202496 2053 0 0 0 10036 0 0 0 25 0 1 0 748920097 431542272 1782 33554432000 134512640 135107411 4293963792 18446744073709551615 134531781 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/1784/statm: 105357 1782 59 146 0 104233 0
[pid=1782/tid=1792] ppid=1724 vsize=324464 CPUtime=0 cores=1,3,5,7
/proc/1782/task/1792/stat : 1792 (python) S 1724 1724 1545 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 748930133 332251136 6726 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604470719275392 0 0 -1 7 0 0 0
[pid=1793] ppid=1724 vsize=334824 CPUtime=0.36 cores=1,3,5,7
/proc/1793/stat : 1793 (python) S 1724 1724 1545 0 -1 4202560 1123 0 0 0 19 17 0 0 19 0 2 0 748930137 342859776 6730 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1793/statm: 83706 6730 400 364 0 19609 0
[pid=1795] ppid=1793 vsize=1018996 CPUtime=99.42 cores=1,3,5,7
/proc/1795/stat : 1795 (gnovelty+2) T 1793 1795 1795 0 -1 4202496 964 0 0 0 9942 0 0 0 25 0 1 0 748930137 1043451904 697 33554432000 134512640 135663704 4293888912 18446744073709551615 134525286 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/1795/statm: 254749 697 113 282 0 254465 0
[pid=1793/tid=1802] ppid=1724 vsize=334824 CPUtime=0 cores=1,3,5,7
/proc/1793/task/1802/stat : 1802 (python) S 1724 1724 1545 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 748940080 342859776 6730 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604470719277184 0 0 -1 7 0 0 0
[pid=1803] ppid=1724 vsize=344976 CPUtime=0.37 cores=1,3,5,7
/proc/1803/stat : 1803 (python) S 1724 1724 1545 0 -1 4202560 1178 0 0 0 19 18 0 0 19 0 2 0 748940083 353255424 6749 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/1803/statm: 86244 6749 406 364 0 22144 0
[pid=1805] ppid=1803 vsize=251296 CPUtime=100.23 cores=1,3,5,7
/proc/1805/stat : 1805 (FH) T 1803 1805 1805 0 -1 4202496 1947 0 0 0 10023 0 0 0 25 0 1 0 748940084 257327104 1682 33554432000 134512640 134936988 4288203344 18446744073709551615 134554594 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/1805/statm: 62824 1682 41 104 0 62328 0
[pid=1803/tid=1813] ppid=1724 vsize=344976 CPUtime=0 cores=1,3,5,7
/proc/1803/task/1813/stat : 1813 (python) S 1724 1724 1545 0 -1 4202560 97 0 0 0 0 0 0 0 19 0 2 0 748950119 353255424 6749 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604456350459008 0 0 -1 7 0 0 0
[pid=1814] ppid=1724 vsize=355840 CPUtime=0.37 cores=1,3,5,7
/proc/1814/stat : 1814 (python) S 1724 1724 1545 0 -1 4202560 1152 0 0 0 20 17 0 0 19 0 2 0 748950123 364380160 6753 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1814/statm: 88960 6753 406 364 0 24857 0
[pid=1816] ppid=1814 vsize=421428 CPUtime=98.37 cores=1,3,5,7
/proc/1816/stat : 1816 (adaptg2wsat2009) T 1814 1816 1816 0 -1 4202496 2054 0 0 0 9837 0 0 0 25 0 1 0 748950123 431542272 1781 33554432000 134512640 135107411 4292021136 18446744073709551615 134532376 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/1816/statm: 105357 1781 59 146 0 104233 0
[pid=1814/tid=1820] ppid=1724 vsize=355840 CPUtime=0 cores=1,3,5,7
/proc/1814/task/1820/stat : 1820 (python) S 1724 1724 1545 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 748959961 364380160 6753 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604456350458112 0 0 -1 7 0 0 0
[pid=1821] ppid=1724 vsize=365612 CPUtime=0.37 cores=1,3,5,7
/proc/1821/stat : 1821 (python) S 1724 1724 1545 0 -1 4202560 1136 0 0 0 19 18 0 0 19 0 2 0 748959965 374386688 6876 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/1821/statm: 91403 6876 399 364 0 27297 0
[pid=1823] ppid=1821 vsize=421428 CPUtime=100.18 cores=1,3,5,7
/proc/1823/stat : 1823 (adaptg2wsat2009) T 1821 1823 1823 0 -1 4202496 2052 0 0 0 10018 0 0 0 25 0 1 0 748959966 431542272 1781 33554432000 134512640 135107411 4288269808 18446744073709551615 134531584 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/1823/statm: 105357 1781 59 146 0 104233 0
[pid=1821/tid=1830] ppid=1724 vsize=365612 CPUtime=0 cores=1,3,5,7
/proc/1821/task/1830/stat : 1830 (python) S 1724 1724 1545 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 748970001 374386688 6876 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604464073532672 0 0 -1 7 0 0 0
[pid=1831] ppid=1724 vsize=376128 CPUtime=0.37 cores=1,3,5,7
/proc/1831/stat : 1831 (python) S 1724 1724 1545 0 -1 4202560 1138 0 0 0 19 18 0 0 19 0 2 0 748970005 385155072 6881 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/1831/statm: 94032 6881 400 364 0 29923 0
[pid=1833] ppid=1831 vsize=251296 CPUtime=100.24 cores=1,3,5,7
/proc/1833/stat : 1833 (FH) T 1831 1833 1833 0 -1 4202496 1953 0 0 0 10024 0 0 0 25 0 1 0 748970005 257327104 1682 33554432000 134512640 134936988 4290493392 18446744073709551615 134545357 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/1833/statm: 62824 1682 41 104 0 62328 0
[pid=1831/tid=1841] ppid=1724 vsize=376128 CPUtime=0 cores=1,3,5,7
/proc/1831/task/1841/stat : 1841 (python) S 1724 1724 1545 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 748980041 385155072 6881 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604464073533568 0 0 -1 7 0 0 0
[pid=1842] ppid=1724 vsize=386276 CPUtime=0.74 cores=1,3,5,7
/proc/1842/stat : 1842 (python) S 1724 1724 1545 0 -1 4202560 1144 0 0 0 36 38 0 0 19 0 2 0 748980045 395546624 6885 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1842/statm: 96569 6885 400 364 0 32457 0
[pid=1844] ppid=1842 vsize=268420 CPUtime=195.65 cores=1,3,5,7
/proc/1844/stat : 1844 (hybridGM3) T 1842 1844 1844 0 -1 4202496 5814 0 0 0 19565 0 0 0 25 0 1 0 748980046 274862080 5543 33554432000 134512640 135204385 4287750368 18446744073709551615 134538432 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/1844/statm: 67105 5543 63 169 0 66935 0
[pid=1842/tid=1854] ppid=1724 vsize=386276 CPUtime=0 cores=1,3,5,7
/proc/1842/task/1854/stat : 1854 (python) S 1724 1724 1545 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 748999624 395546624 6885 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604464003585344 0 0 -1 7 0 0 0
[pid=1855] ppid=1724 vsize=396832 CPUtime=0.39 cores=1,3,5,7
/proc/1855/stat : 1855 (python) S 1724 1724 1545 0 -1 4202560 1130 0 0 0 19 20 0 0 19 0 2 0 748999628 406355968 6889 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/1855/statm: 99208 6889 400 364 0 35093 0
[pid=1857] ppid=1855 vsize=1018996 CPUtime=98.41 cores=1,3,5,7
/proc/1857/stat : 1857 (gnovelty+2) T 1855 1857 1857 0 -1 4202496 969 0 0 0 9841 0 0 0 25 0 1 0 748999628 1043451904 697 33554432000 134512640 135663704 4294922752 18446744073709551615 134524916 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/1857/statm: 254749 697 113 282 0 254465 0
[pid=1855/tid=1865] ppid=1724 vsize=396832 CPUtime=0 cores=1,3,5,7
/proc/1855/task/1865/stat : 1865 (python) S 1724 1724 1545 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 749009471 406355968 6889 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604464003586240 0 0 -1 5 0 0 0
[pid=1866] ppid=1724 vsize=406828 CPUtime=0.78 cores=1,3,5,7
/proc/1866/stat : 1866 (python) S 1724 1724 1545 0 -1 4202560 1161 0 0 0 40 38 0 0 19 0 2 0 749009474 416591872 6893 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1866/statm: 101707 6893 400 364 0 37589 0
[pid=1868] ppid=1866 vsize=268428 CPUtime=194.91 cores=1,3,5,7
/proc/1868/stat : 1868 (hybridGM3) T 1866 1868 1868 0 -1 4202496 5814 0 0 0 19491 0 0 0 25 0 1 0 749009475 274870272 5543 33554432000 134512640 135204385 4294014992 18446744073709551615 134538264 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/1868/statm: 67107 5543 63 169 0 66937 0
[pid=1866/tid=1878] ppid=1724 vsize=406828 CPUtime=0 cores=1,3,5,7
/proc/1866/task/1878/stat : 1878 (python) S 1724 1724 1545 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 749028966 416591872 6893 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604455303877568 0 0 -1 7 0 0 0
[pid=1879] ppid=1724 vsize=417412 CPUtime=0.77 cores=1,3,5,7
/proc/1879/stat : 1879 (python) S 1724 1724 1545 0 -1 4202560 1181 0 0 0 40 37 0 0 19 0 2 0 749028970 427429888 6897 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1879/statm: 104353 6897 400 364 0 40232 0
[pid=1881] ppid=1879 vsize=268428 CPUtime=198.82 cores=1,3,5,7
/proc/1881/stat : 1881 (hybridGM3) T 1879 1881 1881 0 -1 4202496 5814 0 0 0 19882 0 0 0 25 0 1 0 749028970 274870272 5543 33554432000 134512640 135204385 4291532752 18446744073709551615 134538395 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/1881/statm: 67107 5543 63 169 0 66937 0
[pid=1879/tid=1963] ppid=1724 vsize=417412 CPUtime=0 cores=1,3,5,7
/proc/1879/task/1963/stat : 1963 (python) S 1724 1724 1545 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 749048855 427429888 6897 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604455303878464 0 0 -1 7 0 0 0
[pid=1964] ppid=1724 vsize=427368 CPUtime=6.06 cores=1,3,5,7
/proc/1964/stat : 1964 (python) S 1724 1724 1545 0 -1 4202560 1225 0 0 0 325 281 0 0 19 0 2 0 749048859 437624832 6917 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/1964/statm: 106842 6917 407 364 0 42718 0
[pid=1966] ppid=1964 vsize=459768 CPUtime=1556.37 cores=1,3,5,7
/proc/1966/stat : 1966 (TNM) T 1964 1966 1966 0 -1 4202496 1927 0 0 0 155637 0 0 0 25 0 1 0 749048859 470802432 1652 33554432000 134512640 135008748 4294368080 18446744073709551615 134527477 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/1966/statm: 114942 1652 46 122 0 113842 0
[pid=1964/tid=2086] ppid=1724 vsize=427368 CPUtime=0 cores=1,3,5,7
/proc/1964/task/2086/stat : 2086 (python) S 1724 1724 1545 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 749204498 437624832 6917 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604460735980544 0 0 -1 1 0 0 0
[pid=2087] ppid=1724 vsize=437984 CPUtime=0.39 cores=1,3,5,7
/proc/2087/stat : 2087 (python) S 1724 1724 1545 0 -1 4202560 1186 0 0 0 19 20 0 0 19 0 2 0 749204500 448495616 6922 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/2087/statm: 109496 6922 408 364 0 45369 0
[pid=2089] ppid=2087 vsize=421428 CPUtime=100.39 cores=1,3,5,7
/proc/2089/stat : 2089 (adaptg2wsat2009) T 2087 2089 2089 0 -1 4202496 2056 0 0 0 10039 0 0 0 25 0 1 0 749204501 431542272 1782 33554432000 134512640 135107411 4287591152 18446744073709551615 134531239 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/2089/statm: 105357 1782 59 146 0 104233 0
[pid=2087/tid=2093] ppid=1724 vsize=437984 CPUtime=0 cores=1,3,5,7
/proc/2087/task/2093/stat : 2093 (python) S 1724 1724 1545 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 749214541 448495616 6922 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604460735981440 0 0 -1 3 0 0 0
[pid=2094] ppid=1724 vsize=448168 CPUtime=0.39 cores=1,3,5,7
/proc/2094/stat : 2094 (python) S 1724 1724 1545 0 -1 4202560 1169 0 0 0 20 19 0 0 16 0 2 0 749214543 458924032 6934 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/2094/statm: 112042 6934 401 364 0 47912 0
[pid=2096] ppid=2094 vsize=251292 CPUtime=98.4 cores=1,3,5,7
/proc/2096/stat : 2096 (FH) T 2094 2096 2096 0 -1 4202496 1959 0 0 0 9840 0 0 0 25 0 1 0 749214544 257323008 1683 33554432000 134512640 134936988 4291365024 18446744073709551615 134552455 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/2096/statm: 62823 1683 41 104 0 62327 0
[pid=2094/tid=2100] ppid=1724 vsize=448168 CPUtime=0 cores=1,3,5,7
/proc/2094/task/2100/stat : 2100 (python) S 1724 1724 1545 0 -1 4202560 47 0 0 0 0 0 0 0 16 0 2 0 749224384 458924032 6934 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 514 18446604463500678208 0 0 -1 7 0 0 0
[pid=2101] ppid=1724 vsize=458604 CPUtime=0 cores=1,3,5,7
/proc/2101/stat : 2101 (python) S 1724 1724 1545 0 -1 4202560 853 0 0 0 0 0 0 0 17 0 1 0 749224386 469610496 6886 33554432000 4194304 5683644 140735616883520 18446744073709551615 256415411782 0 0 16781312 514 8581564838906429439 0 0 17 3 0 0 0
/proc/2101/statm: 114651 6886 350 364 0 50518 0
[pid=2103] ppid=2101 vsize=268424 CPUtime=0.53 cores=1,3,5,7
/proc/2103/stat : 2103 (hybridGM3) R 2101 2103 2103 0 -1 4202496 5816 0 0 0 52 1 0 0 19 0 1 0 749224387 274866176 5543 33554432000 134512640 135204385 4288968400 18446744073709551615 134537939 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/2103/statm: 67106 5543 63 169 0 66936 0
[pid=1724/tid=1732] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1732/stat : 1732 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 19 0 748880137 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
[pid=1724/tid=1742] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1742/stat : 1742 (python) S 1722 1724 1545 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 748890178 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
[pid=1724/tid=1753] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1753/stat : 1753 (python) S 1722 1724 1545 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 748900120 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 3 0 0 0
[pid=1724/tid=1763] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1763/stat : 1763 (python) S 1722 1724 1545 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 748910059 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 3 0 0 0
[pid=1724/tid=1783] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1783/stat : 1783 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 748920097 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 5 0 0 0
[pid=1724/tid=1794] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1794/stat : 1794 (python) S 1722 1724 1545 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 748930137 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
[pid=1724/tid=1804] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1804/stat : 1804 (python) S 1722 1724 1545 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 748940084 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 5 0 0 0
[pid=1724/tid=1815] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1815/stat : 1815 (python) S 1722 1724 1545 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 748950123 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 5 0 0 0
[pid=1724/tid=1822] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1822/stat : 1822 (python) S 1722 1724 1545 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 19 0 748959965 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 5 0 0 0
[pid=1724/tid=1832] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1832/stat : 1832 (python) S 1722 1724 1545 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 748970005 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 5 0 0 0
[pid=1724/tid=1843] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1843/stat : 1843 (python) S 1722 1724 1545 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 748980045 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 5 0 0 0
[pid=1724/tid=1856] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1856/stat : 1856 (python) S 1722 1724 1545 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 748999628 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 3 0 0 0
[pid=1724/tid=1867] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1867/stat : 1867 (python) S 1722 1724 1545 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 749009475 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
[pid=1724/tid=1880] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1880/stat : 1880 (python) S 1722 1724 1545 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 749028970 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 3 0 0 0
[pid=1724/tid=1965] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/1965/stat : 1965 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 749048859 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
[pid=1724/tid=2088] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/2088/stat : 2088 (python) S 1722 1724 1545 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 749204501 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
[pid=1724/tid=2095] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/2095/stat : 2095 (python) S 1722 1724 1545 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 749214543 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 3 0 0 0
[pid=1724/tid=2102] ppid=1722 vsize=468160 CPUtime=0 cores=1,3,5,7
/proc/1724/task/2102/stat : 2102 (python) S 1722 1724 1545 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 749224386 479395840 8451 33554432000 4194304 5683644 140735616883520 18446744073709551615 256427215281 0 0 16781312 2 18446604460723417280 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3457.24
Current children cumulated vsize (KiB) 16046988

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 1724 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=223661
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=194970
# CPU time returned by wait4() is 2.41863
# while last known CPU time is 3457.24
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3445.24
CPU time (s): 3457.24
CPU user time (s): 3450.79
CPU system time (s): 6.45
CPU usage (%): 100.348
Max. virtual memory (cumulated for all children) (KiB): 16046988

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.22366
system time used= 0.19497
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30504
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2919
involuntary context switches= 51

runsolver used 27.8158 second user time and 59.7409 second system time

The end

Launcher Data 

Begin job on node115 at 2011-05-17 00:54:52
IDJOB=3386548
IDBENCH=82613
IDSOLVER=1872
FILE ID=node115/3386548-1305586492
RUNJOBID= node115-1305586148-1564
PBS_JOBID= 13350363
Free space on /tmp= 73236 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v1500-c30000-S303265982-092-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386548-1305586492/watcher-3386548-1305586492 -o /tmp/evaluation-result-3386548-1305586492/solver-3386548-1305586492 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386548-1305586492.cnf -seed 679681468 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1b4a544dd042630233219a622c9437c7
RANDOM SEED=679681468

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      13117800 kB
Buffers:        286544 kB
Cached:        3327252 kB
SwapCached:          0 kB
Active:       16402324 kB
Inactive:      3096192 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      13117800 kB
SwapTotal:    67111528 kB
SwapFree:     67111200 kB
Dirty:           32608 kB
Writeback:           0 kB
AnonPages:    15884872 kB
Mapped:          42588 kB
Slab:           197772 kB
PageTables:      69192 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 125721228 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73072 MiB
End job on node115 at 2011-05-17 01:52:20