Trace number 3378172

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 2109.71 685.437

General information on the benchmark

Namerandom/large/
unif-k7-r85-v250-c21250-S317257086-003-UNKNOWN.cnf
MD5SUM6f9c7c042784ec486e9e8bc570309805
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables250
Number of clauses21250
Sum of the clauses size148750
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 521250

Solver Data 

0.35/0.63	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.63	c INFO: solving HOME/instance-3378172-1305463362.cnf
0.35/0.63	c INFO: computing task features
0.44/0.77	c INFO: computed variable-clause graph statistics
0.44/0.78	c INFO: computed clause balance statistics
1.23/1.50	c INFO: computed variable balance statistics
1.53/1.80	c INFO: computed small-clause counts
1.53/1.80	c INFO: computed Horn-clause counts
1.53/1.82	c INFO: computed variable graph statistics
1.53/1.82	c INFO: collected features for HOME/instance-3378172-1305463362.cnf in 1.19 s
1.53/1.83	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.36)
1.53/1.84	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.13)
1.53/1.85	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.08)
1.53/1.86	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.07)
301.75/102.25	c INFO: running adaptg2wsat2009++@0 for 97 with 1098 remaining (b = 0.05)
301.75/102.31	c INFO: running gnovelty+2@0 for 97 with 1098 remaining (b = 0.04)
445.94/150.42	c INFO: running FH@0 for 97 with 1050 remaining (b = 0.03)
530.05/174.53	c INFO: running adaptg2wsat2009++@0 for 97 with 1026 remaining (b = 0.03)
614.05/202.62	c INFO: running adaptg2wsat2009++@0 for 97 with 998 remaining (b = 0.02)
739.99/238.87	c INFO: running FH@0 for 97 with 961 remaining (b = 0.02)
811.33/262.82	c INFO: running hybridGM3@0 for 194 with 937 remaining (b = 0.04)
966.55/315.05	c INFO: running gnovelty+2@0 for 97 with 885 remaining (b = 0.02)
972.49/317.05	c INFO: running hybridGM3@0 for 194 with 883 remaining (b = 0.03)
1134.60/369.37	c INFO: running adaptg2wsat2009++@0 for 97 with 831 remaining (b = 0.02)
1462.65/475.64	c INFO: running hybridGM3@0 for 194 with 724 remaining (b = 0.03)
1506.16/489.71	c INFO: running hybridGM3@97 for 97 with 710 remaining (b = 0.01)
1625.74/524.00	c INFO: running FH@0 for 97 with 676 remaining (b = 0.01)
1826.74/591.06	c INFO: running hybridGM3@0 for 194 with 609 remaining (b = 0.02)
1866.00/604.15	c INFO: running hybridGM3@100 for 97 with 596 remaining (b = 0.01)
2071.22/672.52	c INFO: running adaptg2wsat2009++@99 for 97 with 528 remaining (b = 0.01)
2106.93/684.48	c INFO: running gnovelty+2@0 for 97 with 516 remaining (b = 0.01)
2109.71/685.41	c INFO: running hybridGM3@0 for 194 with 515 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378172-1305463362/watcher-3378172-1305463362 -o /tmp/evaluation-result-3378172-1305463362/solver-3378172-1305463362 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378172-1305463362.cnf -seed 916887850 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.89 8.18 6.04 9/470 17284
/proc/meminfo: memFree=24902280/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=16240 CPUtime=0 cores=0,2,4,6
/proc/17284/stat : 17284 (python) S 17282 17284 16982 0 -1 4202496 1000 0 0 0 0 0 0 0 25 0 1 0 657938864 16629760 873 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634513408 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/17284/statm: 4060 873 371 364 0 547 0

[startup+0.103134 s]
/proc/loadavg: 8.89 8.18 6.04 9/470 17284
/proc/meminfo: memFree=24902280/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=94180 CPUtime=0.05 cores=0,2,4,6
/proc/17284/stat : 17284 (python) R 17282 17284 16982 0 -1 4202496 2767 0 0 0 4 1 0 0 17 0 1 0 657938864 96440320 2475 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634515088 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/17284/statm: 23545 2475 764 364 0 1677 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 94180

[startup+0.201114 s]
/proc/loadavg: 8.89 8.18 6.04 9/470 17284
/proc/meminfo: memFree=24902280/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=126296 CPUtime=0.12 cores=0,2,4,6
/proc/17284/stat : 17284 (python) R 17282 17284 16982 0 -1 4202496 4707 934 1 0 9 3 0 0 18 0 1 0 657938864 129327104 3895 33554432000 4194304 5683644 140734495140144 18446744073709551615 5177504 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/17284/statm: 31574 3895 1001 364 0 2869 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 126296

[startup+0.301101 s]
/proc/loadavg: 8.89 8.18 6.04 9/470 17284
/proc/meminfo: memFree=24902280/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=135748 CPUtime=0.17 cores=0,2,4,6
/proc/17284/stat : 17284 (python) R 17282 17284 16982 0 -1 4202496 5964 934 1 0 13 4 0 0 18 0 1 0 657938864 139005952 4151 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634511365 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/17284/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135748

[startup+0.701048 s]
/proc/loadavg: 8.89 8.18 6.04 9/470 17284
/proc/meminfo: memFree=24902280/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=280656 CPUtime=0.44 cores=0,2,4,6
/proc/17284/stat : 17284 (python) R 17282 17284 16982 0 -1 4202496 9911 934 1 0 35 9 0 0 17 0 1 0 657938864 287391744 7476 33554432000 4194304 5683644 140734495140144 18446744073709551615 5007426 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/17284/statm: 70164 7476 1761 364 0 6088 0
Current children cumulated CPU time (s) 0.44
Current children cumulated vsize (KiB) 280656

[startup+1.50088 s]
/proc/loadavg: 8.89 8.18 6.04 8/471 17289
/proc/meminfo: memFree=24873756/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=283912 CPUtime=1.23 cores=0,2,4,6
/proc/17284/stat : 17284 (python) R 17282 17284 16982 0 -1 4202496 11843 934 1 0 114 9 0 0 18 0 1 0 657938864 290725888 8363 33554432000 4194304 5683644 140734495140144 18446744073709551615 4539648 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/17284/statm: 70978 8363 1826 364 0 6902 0
Current children cumulated CPU time (s) 1.23
Current children cumulated vsize (KiB) 283912

[startup+3.10057 s]
/proc/loadavg: 8.89 8.18 6.04 11/483 17301
/proc/meminfo: memFree=24851588/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=321092 CPUtime=1.6 cores=0,2,4,6
/proc/17284/stat : 17284 (python) S 17282 17284 16982 0 -1 4202496 14601 934 1 0 150 10 0 0 25 0 5 0 657938864 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/17284/statm: 80273 7472 1883 364 0 16182 0
[pid=17290] ppid=17284 vsize=279924 CPUtime=0 cores=0,2,4,6
/proc/17290/stat : 17290 (python) S 17284 17284 16982 0 -1 4202560 724 0 0 0 0 0 0 0 18 0 1 0 657939047 286642176 5769 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535439 0 0 16781312 514 1297938296512413955 0 0 17 4 0 0 0
/proc/17290/statm: 69981 5769 355 364 0 5899 0
[pid=17292] ppid=17290 vsize=1018820 CPUtime=0.71 cores=0,2,4,6
/proc/17292/stat : 17292 (gnovelty+2) R 17290 17292 17292 0 -1 4202496 861 0 0 0 71 0 0 0 25 0 1 0 657939048 1043271680 596 33554432000 134512640 135663704 4286591632 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/17292/statm: 254705 596 113 282 0 254421 0
[pid=17293] ppid=17284 vsize=290820 CPUtime=0 cores=0,2,4,6
/proc/17293/stat : 17293 (python) S 17284 17284 16982 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 657939048 297799680 5792 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 5127656318189928705 0 0 17 2 0 0 0
/proc/17293/statm: 72705 5792 349 364 0 8620 0
[pid=17295] ppid=17293 vsize=1018820 CPUtime=1.24 cores=0,2,4,6
/proc/17295/stat : 17295 (gnovelty+2) R 17293 17295 17295 0 -1 4202496 859 0 0 0 124 0 0 0 22 0 1 0 657939049 1043271680 596 33554432000 134512640 135663704 4287301552 18446744073709551615 134544167 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17295/statm: 254705 596 113 282 0 254421 0
[pid=17296] ppid=17284 vsize=300592 CPUtime=0 cores=0,2,4,6
/proc/17296/stat : 17296 (python) S 17284 17284 16982 0 -1 4202560 733 0 0 0 0 0 0 0 18 0 1 0 657939049 307806208 5929 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 1427332673159823359 0 0 17 4 0 0 0
/proc/17296/statm: 75148 5929 347 364 0 11060 0
[pid=17300] ppid=17296 vsize=419160 CPUtime=0.51 cores=0,2,4,6
/proc/17300/stat : 17300 (adaptg2wsat2009) R 17296 17300 17300 0 -1 4202496 1253 0 0 0 51 0 0 0 25 0 1 0 657939051 429219840 990 33554432000 134512640 135107411 4288068816 18446744073709551615 134523314 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/17300/statm: 104790 990 49 146 0 103666 0
[pid=17298] ppid=17284 vsize=310848 CPUtime=0 cores=0,2,4,6
/proc/17298/stat : 17298 (python) S 17284 17284 16982 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 657939050 318308352 5932 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 15244703434641768447 0 0 17 4 0 0 0
/proc/17298/statm: 77712 5932 347 364 0 13621 0
[pid=17301] ppid=17298 vsize=419160 CPUtime=1.22 cores=0,2,4,6
/proc/17301/stat : 17301 (adaptg2wsat2009) R 17298 17301 17301 0 -1 4202496 1253 0 0 0 122 0 0 0 25 0 1 0 657939051 429219840 991 33554432000 134512640 135107411 4291235584 18446744073709551615 134523424 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17301/statm: 104790 991 49 146 0 103666 0
[pid=17284/tid=17291] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17291/stat : 17291 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 657939048 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17294] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17294/stat : 17294 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 23 0 5 0 657939048 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17297] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17297/stat : 17297 (python) S 17282 17284 16982 0 -1 4202560 39 934 0 0 0 0 0 0 25 0 5 0 657939049 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17299] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17299/stat : 17299 (python) S 17282 17284 16982 0 -1 4202560 16 934 0 0 0 0 0 0 25 0 5 0 657939051 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.28
Current children cumulated vsize (KiB) 4379236

[startup+6.30101 s]
/proc/loadavg: 8.98 8.21 6.06 11/483 17301
/proc/meminfo: memFree=24850640/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=321092 CPUtime=1.6 cores=0,2,4,6
/proc/17284/stat : 17284 (python) S 17282 17284 16982 0 -1 4202496 14601 934 1 0 150 10 0 0 25 0 5 0 657938864 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/17284/statm: 80273 7472 1883 364 0 16182 0
[pid=17290] ppid=17284 vsize=279924 CPUtime=0.01 cores=0,2,4,6
/proc/17290/stat : 17290 (python) S 17284 17284 16982 0 -1 4202560 881 0 0 0 0 1 0 0 18 0 1 0 657939047 286642176 5792 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535439 0 0 16781312 514 1297938296512413955 0 0 17 4 0 0 0
/proc/17290/statm: 69981 5792 378 364 0 5899 0
[pid=17292] ppid=17290 vsize=1018820 CPUtime=2.24 cores=0,2,4,6
/proc/17292/stat : 17292 (gnovelty+2) R 17290 17292 17292 0 -1 4202496 861 0 0 0 224 0 0 0 25 0 1 0 657939048 1043271680 596 33554432000 134512640 135663704 4286591632 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17292/statm: 254705 596 113 282 0 254421 0
[pid=17293] ppid=17284 vsize=290820 CPUtime=0.02 cores=0,2,4,6
/proc/17293/stat : 17293 (python) S 17284 17284 16982 0 -1 4202560 890 0 0 0 1 1 0 0 18 0 1 0 657939048 297799680 5815 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 5127656318189928705 0 0 17 2 0 0 0
/proc/17293/statm: 72705 5815 372 364 0 8620 0
[pid=17295] ppid=17293 vsize=1018820 CPUtime=2.99 cores=0,2,4,6
/proc/17295/stat : 17295 (gnovelty+2) R 17293 17295 17295 0 -1 4202496 859 0 0 0 299 0 0 0 25 0 1 0 657939049 1043271680 596 33554432000 134512640 135663704 4287301552 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17295/statm: 254705 596 113 282 0 254421 0
[pid=17296] ppid=17284 vsize=300592 CPUtime=0.01 cores=0,2,4,6
/proc/17296/stat : 17296 (python) S 17284 17284 16982 0 -1 4202560 900 0 0 0 0 1 0 0 18 0 1 0 657939049 307806208 5952 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 1427332673159790852 0 0 17 4 0 0 0
/proc/17296/statm: 75148 5952 370 364 0 11060 0
[pid=17300] ppid=17296 vsize=419428 CPUtime=3.56 cores=0,2,4,6
/proc/17300/stat : 17300 (adaptg2wsat2009) R 17296 17300 17300 0 -1 4202496 1438 0 0 0 356 0 0 0 25 0 1 0 657939051 429494272 1175 33554432000 134512640 135107411 4288068816 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/17300/statm: 104857 1175 59 146 0 103733 0
[pid=17298] ppid=17284 vsize=310848 CPUtime=0.01 cores=0,2,4,6
/proc/17298/stat : 17298 (python) S 17284 17284 16982 0 -1 4202560 911 0 0 0 0 1 0 0 18 0 1 0 657939050 318308352 5955 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 15244703434641735938 0 0 17 4 0 0 0
/proc/17298/statm: 77712 5955 370 364 0 13621 0
[pid=17301] ppid=17298 vsize=419428 CPUtime=4.36 cores=0,2,4,6
/proc/17301/stat : 17301 (adaptg2wsat2009) R 17298 17301 17301 0 -1 4202496 1438 0 0 0 436 0 0 0 25 0 1 0 657939051 429494272 1176 33554432000 134512640 135107411 4291235584 18446744073709551615 134531232 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17301/statm: 104857 1176 59 146 0 103733 0
[pid=17284/tid=17291] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17291/stat : 17291 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 657939048 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17294] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17294/stat : 17294 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 23 0 5 0 657939048 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17297] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17297/stat : 17297 (python) S 17282 17284 16982 0 -1 4202560 39 934 0 0 0 0 0 0 25 0 5 0 657939049 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17299] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17299/stat : 17299 (python) S 17282 17284 16982 0 -1 4202560 16 934 0 0 0 0 0 0 25 0 5 0 657939051 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 14.8
Current children cumulated vsize (KiB) 4379772

[startup+12.7009 s]
/proc/loadavg: 9.06 8.24 6.08 12/483 17301
/proc/meminfo: memFree=24848420/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=321092 CPUtime=1.6 cores=0,2,4,6
/proc/17284/stat : 17284 (python) S 17282 17284 16982 0 -1 4202496 14601 934 1 0 150 10 0 0 25 0 5 0 657938864 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/17284/statm: 80273 7472 1883 364 0 16182 0
[pid=17290] ppid=17284 vsize=279924 CPUtime=0.03 cores=0,2,4,6
/proc/17290/stat : 17290 (python) S 17284 17284 16982 0 -1 4202560 887 0 0 0 2 1 0 0 18 0 1 0 657939047 286642176 5792 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535439 0 0 16781312 514 18304440880796238083 0 0 17 4 0 0 0
/proc/17290/statm: 69981 5792 378 364 0 5899 0
[pid=17292] ppid=17290 vsize=1018820 CPUtime=5.42 cores=0,2,4,6
/proc/17292/stat : 17292 (gnovelty+2) R 17290 17292 17292 0 -1 4202496 861 0 0 0 542 0 0 0 25 0 1 0 657939048 1043271680 596 33554432000 134512640 135663704 4286591632 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17292/statm: 254705 596 113 282 0 254421 0
[pid=17293] ppid=17284 vsize=290820 CPUtime=0.03 cores=0,2,4,6
/proc/17293/stat : 17293 (python) S 17284 17284 16982 0 -1 4202560 893 0 0 0 2 1 0 0 18 0 1 0 657939048 297799680 5815 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 18304440880796238081 0 0 17 2 0 0 0
/proc/17293/statm: 72705 5815 372 364 0 8620 0
[pid=17295] ppid=17293 vsize=1018820 CPUtime=6.19 cores=0,2,4,6
/proc/17295/stat : 17295 (gnovelty+2) R 17293 17295 17295 0 -1 4202496 859 0 0 0 619 0 0 0 25 0 1 0 657939049 1043271680 596 33554432000 134512640 135663704 4287301552 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17295/statm: 254705 596 113 282 0 254421 0
[pid=17296] ppid=17284 vsize=300592 CPUtime=0.03 cores=0,2,4,6
/proc/17296/stat : 17296 (python) S 17284 17284 16982 0 -1 4202560 903 0 0 0 1 2 0 0 18 0 1 0 657939049 307806208 5952 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 18304440880796238084 0 0 17 4 0 0 0
/proc/17296/statm: 75148 5952 370 364 0 11060 0
[pid=17300] ppid=17296 vsize=419428 CPUtime=9.86 cores=0,2,4,6
/proc/17300/stat : 17300 (adaptg2wsat2009) R 17296 17300 17300 0 -1 4202496 1438 0 0 0 986 0 0 0 25 0 1 0 657939051 429494272 1175 33554432000 134512640 135107411 4288068816 18446744073709551615 134531332 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/17300/statm: 104857 1175 59 146 0 103733 0
[pid=17298] ppid=17284 vsize=310848 CPUtime=0.03 cores=0,2,4,6
/proc/17298/stat : 17298 (python) S 17284 17284 16982 0 -1 4202560 914 0 0 0 1 2 0 0 18 0 1 0 657939050 318308352 5955 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 18304440880796238082 0 0 17 4 0 0 0
/proc/17298/statm: 77712 5955 370 364 0 13621 0
[pid=17301] ppid=17298 vsize=419428 CPUtime=10.7 cores=0,2,4,6
/proc/17301/stat : 17301 (adaptg2wsat2009) R 17298 17301 17301 0 -1 4202496 1438 0 0 0 1070 0 0 0 25 0 1 0 657939051 429494272 1176 33554432000 134512640 135107411 4291235584 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17301/statm: 104857 1176 59 146 0 103733 0
[pid=17284/tid=17291] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17291/stat : 17291 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 657939048 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17294] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17294/stat : 17294 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 23 0 5 0 657939048 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17297] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17297/stat : 17297 (python) S 17282 17284 16982 0 -1 4202560 39 934 0 0 0 0 0 0 25 0 5 0 657939049 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17299] ppid=17282 vsize=321092 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17299/stat : 17299 (python) S 17282 17284 16982 0 -1 4202560 16 934 0 0 0 0 0 0 25 0 5 0 657939051 328798208 7472 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 33.89
Current children cumulated vsize (KiB) 4379772

[startup+25.5016 s]
/proc/loadavg: 9.34 8.34 6.15 11/483 17301
/proc/meminfo: memFree=24848672/32951124 swapFree=53501244/67111528
[pid=17284] ppid=17282 vsize=321092 CPUtime=1.6 cores=0,2,4,6

################
# More data... #
################

[pid=17301] ppid=17298 vsize=419428 CPUtime=112.62 cores=0,2,4,6
/proc/17301/stat : 17301 (adaptg2wsat2009) R 17298 17301 17301 0 -1 4202496 1438 0 0 0 11262 0 0 0 25 0 1 0 657939051 429494272 1176 33554432000 134512640 135107411 4291235584 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/17301/statm: 104857 1176 59 146 0 103733 0
[pid=17298/tid=17311] ppid=17284 vsize=310848 CPUtime=0 cores=0,2,4,6
/proc/17298/task/17311/stat : 17311 (python) S 17284 17284 16982 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657949088 318308352 5985 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604447872192768 0 0 -1 0 0 0 0
[pid=17312] ppid=17284 vsize=321104 CPUtime=0.35 cores=0,2,4,6
/proc/17312/stat : 17312 (python) S 17284 17284 16982 0 -1 4202560 1100 0 0 0 18 17 0 0 18 0 2 0 657949089 328810496 5989 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/17312/statm: 80276 5989 399 364 0 16182 0
[pid=17314] ppid=17312 vsize=419428 CPUtime=99.87 cores=0,2,4,6
/proc/17314/stat : 17314 (adaptg2wsat2009) T 17312 17314 17314 0 -1 4202496 1443 0 0 0 9987 0 0 0 25 0 1 0 657949089 429494272 1175 33554432000 134512640 135107411 4288367296 18446744073709551615 134532128 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/17314/statm: 104857 1175 59 146 0 103733 0
[pid=17312/tid=17339] ppid=17284 vsize=321104 CPUtime=0 cores=0,2,4,6
/proc/17312/task/17339/stat : 17339 (python) S 17284 17284 16982 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657959123 328810496 5989 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604443232208000 0 0 -1 0 0 0 0
[pid=17316] ppid=17284 vsize=331952 CPUtime=0.47 cores=0,2,4,6
/proc/17316/stat : 17316 (python) S 17284 17284 16982 0 -1 4202560 1115 0 0 0 26 21 0 0 19 0 2 0 657949095 339918848 5995 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17316/statm: 82988 5995 400 364 0 18891 0
[pid=17318] ppid=17316 vsize=1018820 CPUtime=98.4 cores=0,2,4,6
/proc/17318/stat : 17318 (gnovelty+2) T 17316 17318 17318 0 -1 4202496 857 0 0 0 9840 0 0 0 25 0 1 0 657949096 1043271680 595 33554432000 134512640 135663704 4292581936 18446744073709551615 134525243 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/17318/statm: 254705 595 113 282 0 254421 0
[pid=17316/tid=17344] ppid=17284 vsize=331952 CPUtime=0 cores=0,2,4,6
/proc/17316/task/17344/stat : 17344 (python) S 17284 17284 16982 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657962750 339918848 5995 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604450139292608 0 0 -1 0 0 0 0
[pid=17322] ppid=17284 vsize=341864 CPUtime=0.4 cores=0,2,4,6
/proc/17322/stat : 17322 (python) S 17284 17284 16982 0 -1 4202560 1096 0 0 0 21 19 0 0 19 0 2 0 657953906 350068736 6026 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17322/statm: 85466 6026 400 364 0 21366 0
[pid=17324] ppid=17322 vsize=249188 CPUtime=97.95 cores=0,2,4,6
/proc/17324/stat : 17324 (FH) T 17322 17324 17324 0 -1 4202496 1321 0 0 0 9795 0 0 0 25 0 1 0 657953907 255168512 1058 33554432000 134512640 134936988 4291223264 18446744073709551615 134552080 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/17324/statm: 62297 1058 41 104 0 61801 0
[pid=17322/tid=17349] ppid=17284 vsize=341864 CPUtime=0 cores=0,2,4,6
/proc/17322/task/17349/stat : 17349 (python) S 17284 17284 16982 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657965145 350068736 6026 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604443312822272 0 0 -1 6 0 0 0
[pid=17326] ppid=17284 vsize=352020 CPUtime=0.5 cores=0,2,4,6
/proc/17326/stat : 17326 (python) S 17284 17284 16982 0 -1 4202560 1153 0 0 0 27 23 0 0 19 0 2 0 657956317 360468480 6036 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17326/statm: 88005 6036 406 364 0 23902 0
[pid=17328] ppid=17326 vsize=419428 CPUtime=98.6 cores=0,2,4,6
/proc/17328/stat : 17328 (adaptg2wsat2009) T 17326 17328 17328 0 -1 4202496 1444 0 0 0 9860 0 0 0 25 0 1 0 657956317 429494272 1176 33554432000 134512640 135107411 4292773248 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/17328/statm: 104857 1176 59 146 0 103733 0
[pid=17326/tid=17410] ppid=17284 vsize=352020 CPUtime=0 cores=0,2,4,6
/proc/17326/task/17410/stat : 17410 (python) S 17284 17284 16982 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657970568 360468480 6036 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604453909256960 0 0 -1 6 0 0 0
[pid=17340] ppid=17284 vsize=362484 CPUtime=0.39 cores=0,2,4,6
/proc/17340/stat : 17340 (python) S 17284 17284 16982 0 -1 4202560 1147 0 0 0 20 19 0 0 19 0 2 0 657959126 371183616 6040 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17340/statm: 90621 6040 406 364 0 26515 0
[pid=17342] ppid=17340 vsize=419428 CPUtime=97.31 cores=0,2,4,6
/proc/17342/stat : 17342 (adaptg2wsat2009) T 17340 17342 17342 0 -1 4202496 1445 0 0 0 9730 1 0 0 25 0 1 0 657959127 429494272 1176 33554432000 134512640 135107411 4290281232 18446744073709551615 134531435 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/17342/statm: 104857 1176 59 146 0 103733 0
[pid=17340/tid=17406] ppid=17284 vsize=362484 CPUtime=0 cores=0,2,4,6
/proc/17340/task/17406/stat : 17406 (python) S 17284 17284 16982 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657970368 371183616 6040 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604445053257664 0 0 -1 0 0 0 0
[pid=17345] ppid=17284 vsize=372856 CPUtime=0.46 cores=0,2,4,6
/proc/17345/stat : 17345 (python) S 17284 17284 16982 0 -1 4202560 1122 0 0 0 24 22 0 0 18 0 2 0 657962751 381804544 6038 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17345/statm: 93214 6038 400 364 0 29105 0
[pid=17347] ppid=17345 vsize=249192 CPUtime=100.33 cores=0,2,4,6
/proc/17347/stat : 17347 (FH) T 17345 17347 17347 0 -1 4202496 1324 0 0 0 10032 1 0 0 25 0 1 0 657962751 255172608 1059 33554432000 134512640 134936988 4289111280 18446744073709551615 134552073 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/17347/statm: 62298 1059 41 104 0 61802 0
[pid=17345/tid=17415] ppid=17284 vsize=372856 CPUtime=0 cores=0,2,4,6
/proc/17345/task/17415/stat : 17415 (python) S 17284 17284 16982 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657975798 381804544 6038 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604444824865600 0 0 -1 0 0 0 0
[pid=17350] ppid=17284 vsize=383044 CPUtime=1.16 cores=0,2,4,6
/proc/17350/stat : 17350 (python) S 17284 17284 16982 0 -1 4202560 1174 0 0 0 67 49 0 0 18 0 2 0 657965147 392237056 6042 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17350/statm: 95761 6042 400 364 0 31649 0
[pid=17352] ppid=17350 vsize=268424 CPUtime=294.77 cores=0,2,4,6
/proc/17352/stat : 17352 (hybridGM3) T 17350 17352 17352 0 -1 4202496 5275 0 0 0 29474 3 0 0 25 0 1 0 657965147 274866176 5010 33554432000 134512640 135204385 4291638032 18446744073709551615 134538524 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/17352/statm: 67106 5010 63 169 0 66936 0
[pid=17350/tid=17473] ppid=17284 vsize=383044 CPUtime=0 cores=0,2,4,6
/proc/17350/task/17473/stat : 17473 (python) S 17284 17284 16982 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 657987833 392237056 6042 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604445053256768 0 0 -1 0 0 0 0
[pid=17407] ppid=17284 vsize=393428 CPUtime=0.56 cores=0,2,4,6
/proc/17407/stat : 17407 (python) S 17284 17284 16982 0 -1 4202560 1155 0 0 0 29 27 0 0 19 0 2 0 657970369 402870272 6046 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17407/statm: 98357 6046 400 364 0 34242 0
[pid=17409] ppid=17407 vsize=1018820 CPUtime=99.78 cores=0,2,4,6
/proc/17409/stat : 17409 (gnovelty+2) T 17407 17409 17409 0 -1 4202496 863 0 0 0 9978 0 0 0 25 0 1 0 657970369 1043271680 595 33554432000 134512640 135663704 4287278400 18446744073709551615 134525286 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/17409/statm: 254705 595 113 282 0 254421 0
[pid=17407/tid=17425] ppid=17284 vsize=393428 CPUtime=0 cores=0,2,4,6
/proc/17407/task/17425/stat : 17425 (python) S 17284 17284 16982 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657986427 402870272 6046 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604441099383168 0 0 -1 0 0 0 0
[pid=17411] ppid=17284 vsize=403600 CPUtime=1.27 cores=0,2,4,6
/proc/17411/stat : 17411 (python) S 17284 17284 16982 0 -1 4202560 1179 0 0 0 69 58 0 0 18 0 2 0 657970569 413286400 6050 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17411/statm: 100900 6050 400 364 0 36782 0
[pid=17413] ppid=17411 vsize=268428 CPUtime=243.31 cores=0,2,4,6
/proc/17413/stat : 17413 (hybridGM3) R 17411 17413 17413 0 -1 4202496 5279 0 0 0 24329 2 0 0 25 0 1 0 657970569 274870272 5010 33554432000 134512640 135204385 4290087296 18446744073709551615 134537753 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17413/statm: 67107 5010 63 169 0 66937 0
[pid=17411/tid=17484] ppid=17284 vsize=403600 CPUtime=0 cores=0,2,4,6
/proc/17411/task/17484/stat : 17484 (python) S 17284 17284 16982 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657999278 413286400 6050 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604441099384064 0 0 -1 0 0 0 0
[pid=17416] ppid=17284 vsize=413996 CPUtime=0.53 cores=0,2,4,6
/proc/17416/stat : 17416 (python) S 17284 17284 16982 0 -1 4202560 1163 0 0 0 27 26 0 0 19 0 2 0 657975801 423931904 6054 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/17416/statm: 103499 6054 400 364 0 39378 0
[pid=17418] ppid=17416 vsize=419428 CPUtime=99.34 cores=0,2,4,6
/proc/17418/stat : 17418 (adaptg2wsat2009) T 17416 17418 17418 0 -1 4202496 1445 0 0 0 9933 1 0 0 25 0 1 0 657975802 429494272 1175 33554432000 134512640 135107411 4287330560 18446744073709551615 134531246 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/17418/statm: 104857 1175 59 146 0 103733 0
[pid=17416/tid=17475] ppid=17284 vsize=413996 CPUtime=0 cores=0,2,4,6
/proc/17416/task/17475/stat : 17475 (python) S 17284 17284 16982 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657991263 423931904 6054 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604441039332672 0 0 -1 2 0 0 0
[pid=17426] ppid=17284 vsize=424224 CPUtime=0.73 cores=0,2,4,6
/proc/17426/stat : 17426 (python) S 17284 17284 16982 0 -1 4202560 1230 0 0 0 38 35 0 0 19 0 2 0 657986428 434405376 6066 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/17426/statm: 106056 6066 408 364 0 41932 0
[pid=17428] ppid=17426 vsize=268424 CPUtime=196.67 cores=0,2,4,6
/proc/17428/stat : 17428 (hybridGM3) T 17426 17428 17428 0 -1 4202496 5284 0 0 0 19663 4 0 0 25 0 1 0 657986429 274866176 5009 33554432000 134512640 135204385 4287995728 18446744073709551615 134538520 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/17428/statm: 67106 5009 63 169 0 66936 0
[pid=17426/tid=17491] ppid=17284 vsize=424224 CPUtime=0 cores=0,2,4,6
/proc/17426/task/17491/stat : 17491 (python) S 17284 17284 16982 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 658007311 434405376 6066 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604440892580224 0 0 -1 4 0 0 0
[pid=17476] ppid=17284 vsize=434868 CPUtime=0.53 cores=0,2,4,6
/proc/17476/stat : 17476 (python) S 17284 17284 16982 0 -1 4202560 1200 0 0 0 27 26 0 0 19 0 2 0 657991264 445304832 6069 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646342091 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/17476/statm: 108717 6069 407 364 0 44590 0
[pid=17478] ppid=17476 vsize=249192 CPUtime=102.69 cores=0,2,4,6
/proc/17478/stat : 17478 (FH) T 17476 17478 17478 0 -1 4202496 1333 0 0 0 10269 0 0 0 25 0 1 0 657991265 255172608 1059 33554432000 134512640 134936988 4292614288 18446744073709551615 134554109 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/17478/statm: 62298 1059 41 104 0 61802 0
[pid=17476/tid=17490] ppid=17284 vsize=434868 CPUtime=0 cores=0,2,4,6
/proc/17476/task/17490/stat : 17490 (python) S 17284 17284 16982 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658006115 445304832 6069 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 514 18446604445786712256 0 0 -1 0 0 0 0
[pid=17480] ppid=17284 vsize=445008 CPUtime=0.32 cores=0,2,4,6
/proc/17480/stat : 17480 (python) S 17284 17284 16982 0 -1 4202560 1035 0 0 0 18 14 0 0 18 0 1 0 657997970 455688192 6133 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 18304440880796238081 0 0 17 0 0 0 0
/proc/17480/statm: 111252 6133 379 364 0 47122 0
[pid=17482] ppid=17480 vsize=268420 CPUtime=61.93 cores=0,2,4,6
/proc/17482/stat : 17482 (hybridGM3) R 17480 17482 17482 0 -1 4202496 5286 0 0 0 6192 1 0 0 25 0 1 0 657997970 274862080 5008 33554432000 134512640 135204385 4287439200 18446744073709551615 134538041 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/17482/statm: 67105 5008 63 169 0 66935 0
[pid=17492] ppid=17284 vsize=455532 CPUtime=0 cores=0,2,4,6
/proc/17492/stat : 17492 (python) S 17284 17284 16982 0 -1 4202560 838 0 0 0 0 0 0 0 18 0 1 0 658007313 466464768 6122 33554432000 4194304 5683644 140734495140144 18446744073709551615 216634535494 0 0 16781312 514 2773919681981972479 0 0 17 0 0 0 0
/proc/17492/statm: 113883 6122 350 364 0 49750 0
[pid=17494] ppid=17492 vsize=1018820 CPUtime=0.41 cores=0,2,4,6
/proc/17494/stat : 17494 (gnovelty+2) R 17492 17494 17494 0 -1 4202496 872 0 0 0 41 0 0 0 21 0 1 0 658007313 1043271680 596 33554432000 134512640 135663704 4288312176 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/17494/statm: 254705 596 113 282 0 254421 0
[pid=17284/tid=17291] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17291/stat : 17291 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 19 0 657939048 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17294] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17294/stat : 17294 (python) S 17282 17284 16982 0 -1 4202560 24 934 0 0 0 0 0 0 23 0 19 0 657939048 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17297] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17297/stat : 17297 (python) S 17282 17284 16982 0 -1 4202560 39 934 0 0 0 0 0 0 25 0 19 0 657939049 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17299] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17299/stat : 17299 (python) S 17282 17284 16982 0 -1 4202560 23 934 0 0 0 0 0 0 18 0 19 0 657939051 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 4 0 0 0
[pid=17284/tid=17313] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17313/stat : 17313 (python) S 17282 17284 16982 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 657949089 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17317] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17317/stat : 17317 (python) S 17282 17284 16982 0 -1 4202560 29 934 0 0 0 0 0 0 21 0 19 0 657949095 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 6 0 0 0
[pid=17284/tid=17323] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17323/stat : 17323 (python) S 17282 17284 16982 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 657953906 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17327] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17327/stat : 17327 (python) S 17282 17284 16982 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 657956317 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17341] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17341/stat : 17341 (python) S 17282 17284 16982 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 19 0 657959127 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17346] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17346/stat : 17346 (python) S 17282 17284 16982 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 657962751 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17351] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17351/stat : 17351 (python) S 17282 17284 16982 0 -1 4202560 28 934 0 0 0 0 0 0 18 0 19 0 657965147 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17408] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17408/stat : 17408 (python) S 17282 17284 16982 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 657970369 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17412] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17412/stat : 17412 (python) S 17282 17284 16982 0 -1 4202560 35 934 0 0 0 0 0 0 18 0 19 0 657970569 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17417] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17417/stat : 17417 (python) S 17282 17284 16982 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 657975802 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17427] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17427/stat : 17427 (python) S 17282 17284 16982 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 657986429 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17477] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17477/stat : 17477 (python) S 17282 17284 16982 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 657991265 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17481] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17481/stat : 17481 (python) S 17282 17284 16982 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 657997970 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
[pid=17284/tid=17493] ppid=17282 vsize=465140 CPUtime=0 cores=0,2,4,6
/proc/17284/task/17493/stat : 17493 (python) S 17282 17284 16982 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 658007313 476303360 7687 33554432000 4194304 5683644 140734495140144 18446744073709551615 216646338993 0 0 16781312 2 18446604450399284288 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2109.71
Current children cumulated vsize (KiB) 16525484

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 17284 and gives
#  childrusage.ru_utime.tv_sec=5
#  childrusage.ru_utime.tv_usec=290195
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=330493
# CPU time returned by wait4() is 8.62069
# while last known CPU time is 2109.71
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 685.437
CPU time (s): 2109.71
CPU user time (s): 2105.02
CPU system time (s): 4.69
CPU usage (%): 307.79
Max. virtual memory (cumulated for all children) (KiB): 16525484

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 5.29019
system time used= 3.33049
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 44100
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 5336
involuntary context switches= 108

runsolver used 5.41718 second user time and 11.4843 second system time

The end

Launcher Data 

Begin job on node103 at 2011-05-15 14:42:42
IDJOB=3378172
IDBENCH=82581
IDSOLVER=1859
FILE ID=node103/3378172-1305463362
RUNJOBID= node103-1305462668-17000
PBS_JOBID= 13324971
Free space on /tmp= 72940 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v250-c21250-S317257086-003-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378172-1305463362/watcher-3378172-1305463362 -o /tmp/evaluation-result-3378172-1305463362/solver-3378172-1305463362 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378172-1305463362.cnf -seed 916887850 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 6f9c7c042784ec486e9e8bc570309805
RANDOM SEED=916887850

node103.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.71
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.858
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      24902568 kB
Buffers:        326728 kB
Cached:        4191868 kB
SwapCached:     340184 kB
Active:        2533988 kB
Inactive:      5212220 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      24902568 kB
SwapTotal:    67111528 kB
SwapFree:     53501244 kB
Dirty:           32264 kB
Writeback:           0 kB
AnonPages:     2898252 kB
Mapped:          26760 kB
Slab:           163752 kB
PageTables:      71124 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 136394980 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72904 MiB
End job on node103 at 2011-05-15 14:54:10