Trace number 3386528

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 3458.74 3446.34

General information on the benchmark

Namerandom/large/
unif-k7-r85-v200-c17000-S771170845-016-UNKNOWN.cnf
MD5SUM8a262b49dcf27b27fb405f8a8cdf125f
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2600.06
Satisfiable
(Un)Satisfiability was proved
Number of variables200
Number of clauses17000
Sum of the clauses size119000
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 517000

Solver Data 

0.38/0.65	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.38/0.65	c INFO: solving HOME/instance-3386528-1305586224.cnf
0.38/0.65	c INFO: computing task features
0.45/0.76	c INFO: computed variable-clause graph statistics
0.45/0.77	c INFO: computed clause balance statistics
1.05/1.36	c INFO: computed variable balance statistics
1.25/1.59	c INFO: computed small-clause counts
1.25/1.59	c INFO: computed Horn-clause counts
1.35/1.61	c INFO: computed variable graph statistics
1.35/1.61	c INFO: collected features for HOME/instance-3386528-1305586224.cnf in 0.95 s
1.35/1.64	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.35)
102.08/102.02	c INFO: running gnovelty+2@0 for 97 with 4898 remaining (b = 0.12)
201.80/201.42	c INFO: running adaptg2wsat2009++@0 for 97 with 4799 remaining (b = 0.08)
300.52/299.80	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.06)
400.15/399.20	c INFO: running adaptg2wsat2009++@0 for 97 with 4601 remaining (b = 0.05)
500.88/499.57	c INFO: running gnovelty+2@0 for 97 with 4501 remaining (b = 0.04)
601.62/599.98	c INFO: running FH@0 for 97 with 4400 remaining (b = 0.03)
702.37/700.37	c INFO: running adaptg2wsat2009++@0 for 97 with 4300 remaining (b = 0.03)
802.13/799.77	c INFO: running adaptg2wsat2009++@0 for 97 with 4201 remaining (b = 0.02)
900.84/898.18	c INFO: running FH@0 for 97 with 4102 remaining (b = 0.02)
998.74/995.61	c INFO: running hybridGM3@0 for 194 with 4005 remaining (b = 0.04)
1196.26/1192.49	c INFO: running gnovelty+2@0 for 97 with 3808 remaining (b = 0.02)
1299.14/1295.00	c INFO: running hybridGM3@0 for 194 with 3705 remaining (b = 0.03)
1499.77/1494.86	c INFO: running hybridGM3@0 for 194 with 3506 remaining (b = 0.03)
1700.42/1694.72	c INFO: running TNM@0 for 1552 with 3306 remaining (b = 0.20)
3259.84/3248.09	c INFO: running adaptg2wsat2009++@0 for 97 with 1752 remaining (b = 0.01)
3357.74/3345.53	c INFO: running FH@0 for 97 with 1655 remaining (b = 0.01)
3458.43/3445.91	c INFO: running hybridGM3@0 for 194 with 1555 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386528-1305586224/watcher-3386528-1305586224 -o /tmp/evaluation-result-3386528-1305586224/solver-3386528-1305586224 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386528-1305586224.cnf -seed 1217366361 -budget 5000 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.52 0.53 1.65 2/459 8701
/proc/meminfo: memFree=11521160/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=15756 CPUtime=0 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 839 0 0 0 0 0 0 0 25 0 1 0 651037938 16134144 723 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095449088 0 2147483391 16781312 2 18446744071699207358 0 0 17 1 0 0 0
/proc/8701/statm: 3939 723 334 364 0 426 0

[startup+0.02696 s]
/proc/loadavg: 1.52 0.53 1.65 2/459 8701
/proc/meminfo: memFree=11521160/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=16400 CPUtime=0.01 cores=1,3,5,7
/proc/8701/stat : 8701 (python) R 8699 8701 8584 0 -1 4202496 1117 0 0 0 1 0 0 0 17 0 1 0 651037938 16793600 985 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095449088 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/8701/statm: 4100 985 409 364 0 587 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16400

[startup+0.100931 s]
/proc/loadavg: 1.52 0.53 1.65 2/459 8701
/proc/meminfo: memFree=11521160/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=87708 CPUtime=0.05 cores=1,3,5,7
/proc/8701/stat : 8701 (python) R 8699 8701 8584 0 -1 4202496 2637 0 0 0 4 1 0 0 17 0 1 0 651037938 89812992 2354 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095446885 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/8701/statm: 21927 2354 717 364 0 1623 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 87708

[startup+0.300923 s]
/proc/loadavg: 1.52 0.53 1.65 2/459 8701
/proc/meminfo: memFree=11521160/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=135752 CPUtime=0.18 cores=1,3,5,7
/proc/8701/stat : 8701 (python) R 8699 8701 8584 0 -1 4202496 5975 934 1 0 13 5 0 0 17 0 1 0 651037938 139010048 4149 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095447045 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/8701/statm: 33938 4149 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135752

[startup+0.700817 s]
/proc/loadavg: 1.52 0.53 1.65 2/459 8701
/proc/meminfo: memFree=11521160/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=280084 CPUtime=0.45 cores=1,3,5,7
/proc/8701/stat : 8701 (python) R 8699 8701 8584 0 -1 4202496 9693 934 1 0 34 11 0 0 15 0 1 0 651037938 286806016 7361 33554432000 4194304 5683644 140733220042592 18446744073709551615 4602358 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/8701/statm: 70021 7361 1761 364 0 5945 0
Current children cumulated CPU time (s) 0.45
Current children cumulated vsize (KiB) 280084

[startup+1.50064 s]
/proc/loadavg: 1.52 0.53 1.65 3/460 8706
/proc/meminfo: memFree=11494236/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=282560 CPUtime=1.25 cores=1,3,5,7
/proc/8701/stat : 8701 (python) R 8699 8701 8584 0 -1 4202496 11222 934 1 0 114 11 0 0 16 0 1 0 651037938 289341440 8053 33554432000 4194304 5683644 140733220042592 18446744073709551615 46998442008941 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/8701/statm: 70640 8053 1827 364 0 6564 0
Current children cumulated CPU time (s) 1.25
Current children cumulated vsize (KiB) 282560

[startup+3.10035 s]
/proc/loadavg: 1.52 0.53 1.65 3/463 8709
/proc/meminfo: memFree=11493716/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=289440 CPUtime=1.4 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 11595 934 1 0 128 12 0 0 19 0 2 0 651037938 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8701/statm: 72360 7232 1877 364 0 8278 0
[pid=8707] ppid=8701 vsize=279196 CPUtime=0 cores=1,3,5,7
/proc/8707/stat : 8707 (python) S 8701 8701 8584 0 -1 4202560 730 0 0 0 0 0 0 0 18 0 1 0 651038102 285896704 5705 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095471119 0 0 16781312 514 15015590225302749447 0 0 17 1 0 0 0
/proc/8707/statm: 69799 5705 354 364 0 5717 0
[pid=8709] ppid=8707 vsize=1018616 CPUtime=1.45 cores=1,3,5,7
/proc/8709/stat : 8709 (gnovelty+2) R 8707 8709 8709 0 -1 4202496 767 0 0 0 145 0 0 0 23 0 1 0 651038102 1043062784 500 33554432000 134512640 135663704 4287978960 18446744073709551615 134525271 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8709/statm: 254654 500 113 282 0 254370 0
[pid=8701/tid=8708] ppid=8699 vsize=289440 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8708/stat : 8708 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 651038102 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.85
Current children cumulated vsize (KiB) 1587252

[startup+6.30076 s]
/proc/loadavg: 1.56 0.55 1.65 3/463 8709
/proc/meminfo: memFree=11493948/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=289440 CPUtime=1.4 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 11595 934 1 0 128 12 0 0 19 0 2 0 651037938 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8701/statm: 72360 7232 1877 364 0 8278 0
[pid=8707] ppid=8701 vsize=279196 CPUtime=0.01 cores=1,3,5,7
/proc/8707/stat : 8707 (python) S 8701 8701 8584 0 -1 4202560 873 0 0 0 1 0 0 0 18 0 1 0 651038102 285896704 5728 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095471119 0 0 16781312 514 15015590225302749447 0 0 17 1 0 0 0
/proc/8707/statm: 69799 5728 377 364 0 5717 0
[pid=8709] ppid=8707 vsize=1018616 CPUtime=4.65 cores=1,3,5,7
/proc/8709/stat : 8709 (gnovelty+2) R 8707 8709 8709 0 -1 4202496 767 0 0 0 465 0 0 0 25 0 1 0 651038102 1043062784 500 33554432000 134512640 135663704 4287978960 18446744073709551615 134524945 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8709/statm: 254654 500 113 282 0 254370 0
[pid=8701/tid=8708] ppid=8699 vsize=289440 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8708/stat : 8708 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 651038102 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 6.06
Current children cumulated vsize (KiB) 1587252

[startup+12.7006 s]
/proc/loadavg: 1.59 0.58 1.65 3/463 8710
/proc/meminfo: memFree=11493244/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=289440 CPUtime=1.4 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 11595 934 1 0 128 12 0 0 19 0 2 0 651037938 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8701/statm: 72360 7232 1877 364 0 8278 0
[pid=8707] ppid=8701 vsize=279196 CPUtime=0.03 cores=1,3,5,7
/proc/8707/stat : 8707 (python) S 8701 8701 8584 0 -1 4202560 876 0 0 0 2 1 0 0 18 0 1 0 651038102 285896704 5728 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095471119 0 0 16781312 514 13239791256097227015 0 0 17 1 0 0 0
/proc/8707/statm: 69799 5728 377 364 0 5717 0
[pid=8709] ppid=8707 vsize=1018616 CPUtime=11.05 cores=1,3,5,7
/proc/8709/stat : 8709 (gnovelty+2) R 8707 8709 8709 0 -1 4202496 767 0 0 0 1105 0 0 0 25 0 1 0 651038102 1043062784 500 33554432000 134512640 135663704 4287978960 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8709/statm: 254654 500 113 282 0 254370 0
[pid=8701/tid=8708] ppid=8699 vsize=289440 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8708/stat : 8708 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 651038102 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 12.48
Current children cumulated vsize (KiB) 1587252

[startup+25.5002 s]
/proc/loadavg: 1.68 0.64 1.65 3/467 8714
/proc/meminfo: memFree=11488640/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=289440 CPUtime=1.4 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 11595 934 1 0 128 12 0 0 19 0 2 0 651037938 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8701/statm: 72360 7232 1877 364 0 8278 0
[pid=8707] ppid=8701 vsize=279196 CPUtime=0.08 cores=1,3,5,7
/proc/8707/stat : 8707 (python) S 8701 8701 8584 0 -1 4202560 883 0 0 0 5 3 0 0 18 0 1 0 651038102 285896704 5728 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095471119 0 0 16781312 514 13239791256097227015 0 0 17 1 0 0 0
/proc/8707/statm: 69799 5728 377 364 0 5717 0
[pid=8709] ppid=8707 vsize=1018616 CPUtime=23.85 cores=1,3,5,7
/proc/8709/stat : 8709 (gnovelty+2) R 8707 8709 8709 0 -1 4202496 767 0 0 0 2385 0 0 0 25 0 1 0 651038102 1043062784 500 33554432000 134512640 135663704 4287978960 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8709/statm: 254654 500 113 282 0 254370 0
[pid=8701/tid=8708] ppid=8699 vsize=289440 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8708/stat : 8708 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 651038102 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 25.33
Current children cumulated vsize (KiB) 1587252

[startup+51.1005 s]
/proc/loadavg: 1.79 0.75 1.66 3/467 8715
/proc/meminfo: memFree=11487556/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=289440 CPUtime=1.4 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 11595 934 1 0 128 12 0 0 19 0 2 0 651037938 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8701/statm: 72360 7232 1877 364 0 8278 0
[pid=8707] ppid=8701 vsize=279196 CPUtime=0.15 cores=1,3,5,7
/proc/8707/stat : 8707 (python) S 8701 8701 8584 0 -1 4202560 894 0 0 0 8 7 0 0 18 0 1 0 651038102 285896704 5728 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095471119 0 0 16781312 514 13239791256097227015 0 0 17 1 0 0 0
/proc/8707/statm: 69799 5728 377 364 0 5717 0
[pid=8709] ppid=8707 vsize=1018616 CPUtime=49.45 cores=1,3,5,7
/proc/8709/stat : 8709 (gnovelty+2) R 8707 8709 8709 0 -1 4202496 767 0 0 0 4945 0 0 0 25 0 1 0 651038102 1043062784 500 33554432000 134512640 135663704 4287978960 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8709/statm: 254654 500 113 282 0 254370 0
[pid=8701/tid=8708] ppid=8699 vsize=289440 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8708/stat : 8708 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 2 0 651038102 296386560 7232 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 51
Current children cumulated vsize (KiB) 1587252

[startup+102.317 s]
/proc/loadavg: 1.91 0.94 1.68 3/471 8720
/proc/meminfo: memFree=11480148/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=299900 CPUtime=1.43 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 12443 934 1 0 131 12 0 0 19 0 3 0 651037938 307097600 7307 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8701/statm: 74975 7307 1880 364 0 10890 0
[pid=8707] ppid=8701 vsize=289440 CPUtime=0.33 cores=1,3,5,7
/proc/8707/stat : 8707 (python) S 8701 8701 8584 0 -1 4202560 1068 0 0 0 17 16 0 0 19 0 2 0 651038102 296386560 5766 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8707/statm: 72360 5766 410 364 0 8278 0
[pid=8709] ppid=8707 vsize=1018616 CPUtime=100.35 cores=1,3,5,7
/proc/8709/stat : 8709 (gnovelty+2) T 8707 8709 8709 0 -1 4202496 767 0 0 0 10035 0 0 0 25 0 1 0 651038102 1043062784 500 33554432000 134512640 135663704 4287978960 18446744073709551615 134525224 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8709/statm: 254654 500 113 282 0 254370 0
[pid=8707/tid=8717] ppid=8701 vsize=289440 CPUtime=0 cores=1,3,5,7
/proc/8707/task/8717/stat : 8717 (python) S 8701 8701 8584 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651048137 296386560 5766 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604469293684672 0 0 -1 7 0 0 0
[pid=8718] ppid=8701 vsize=289656 CPUtime=0 cores=1,3,5,7
/proc/8718/stat : 8718 (python) S 8701 8701 8584 0 -1 4202560 743 0 0 0 0 0 0 0 19 0 1 0 651048140 296607744 5771 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095471174 0 0 16781312 514 14945407848346025983 0 0 17 1 0 0 0
/proc/8718/statm: 72414 5771 348 364 0 8329 0
[pid=8720] ppid=8718 vsize=1018616 CPUtime=0.28 cores=1,3,5,7
/proc/8720/stat : 8720 (gnovelty+2) R 8718 8720 8720 0 -1 4202496 765 0 0 0 28 0 0 0 21 0 1 0 651048141 1043062784 501 33554432000 134512640 135663704 4294946816 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8720/statm: 254654 501 113 282 0 254370 0
[pid=8701/tid=8708] ppid=8699 vsize=299900 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8708/stat : 8708 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 3 0 651038102 307097600 7307 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8719] ppid=8699 vsize=299900 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8719/stat : 8719 (python) S 8699 8701 8584 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 3 0 651048141 307097600 7307 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 102.39
Current children cumulated vsize (KiB) 2916228

[startup+162.301 s]
/proc/loadavg: 2.03 1.16 1.70 3/475 8726
/proc/meminfo: memFree=11471400/32951124 swapFree=65980224/67111528
[pid=8701] ppid=8699 vsize=299900 CPUtime=1.43 cores=1,3,5,7
/proc/8701/stat : 8701 (python) S 8699 8701 8584 0 -1 4202496 12443 934 1 0 131 12 0 0 19 0 3 0 651037938 307097600 7307 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8701/statm: 74975 7307 1880 364 0 10890 0
[pid=8707] ppid=8701 vsize=289440 CPUtime=0.33 cores=1,3,5,7
/proc/8707/stat : 8707 (python) S 8701 8701 8584 0 -1 4202560 1068 0 0 0 17 16 0 0 19 0 2 0 651038102 296386560 5766 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8707/statm: 72360 5766 410 364 0 8278 0
[pid=8709] ppid=8707 vsize=1018616 CPUtime=100.35 cores=1,3,5,7
/proc/8709/stat : 8709 (gnovelty+2) T 8707 8709 8709 0 -1 4202496 767 0 0 0 10035 0 0 0 25 0 1 0 651038102 1043062784 500 33554432000 134512640 135663704 4287978960 18446744073709551615 134525224 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8709/statm: 254654 500 113 282 0 254370 0
[pid=8707/tid=8717] ppid=8701 vsize=289440 CPUtime=0 cores=1,3,5,7
/proc/8707/task/8717/stat : 8717 (python) S 8701 8701 8584 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651048137 296386560 5766 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604469293684672 0 0 -1 7 0 0 0
[pid=8718] ppid=8701 vsize=289656 CPUtime=0.19 cores=1,3,5,7

################
# More data... #
################

/proc/8741/statm: 104659 955 59 146 0 103535 0
[pid=8739/tid=8749] ppid=8701 vsize=310776 CPUtime=0 cores=1,3,5,7
/proc/8739/task/8749/stat : 8749 (python) S 8701 8701 8584 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 651077855 318234624 5967 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604469170770368 0 0 -1 7 0 0 0
[pid=8750] ppid=8701 vsize=321032 CPUtime=0.34 cores=1,3,5,7
/proc/8750/stat : 8750 (python) S 8701 8701 8584 0 -1 4202560 1108 0 0 0 17 17 0 0 19 0 2 0 651077858 328736768 5971 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8750/statm: 80258 5971 399 364 0 16164 0
[pid=8752] ppid=8750 vsize=418636 CPUtime=100.33 cores=1,3,5,7
/proc/8752/stat : 8752 (adaptg2wsat2009) T 8750 8752 8752 0 -1 4202496 1223 0 0 0 10033 0 0 0 25 0 1 0 651077858 428683264 955 33554432000 134512640 135107411 4294290752 18446744073709551615 134532347 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8752/statm: 104659 955 59 146 0 103535 0
[pid=8750/tid=8760] ppid=8701 vsize=321032 CPUtime=0 cores=1,3,5,7
/proc/8750/task/8760/stat : 8760 (python) S 8701 8701 8584 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 651087892 328736768 5971 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604455810792384 0 0 -1 7 0 0 0
[pid=8761] ppid=8701 vsize=331288 CPUtime=0.35 cores=1,3,5,7
/proc/8761/stat : 8761 (python) S 8701 8701 8584 0 -1 4202560 1127 0 0 0 19 16 0 0 19 0 2 0 651087895 339238912 5975 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/8761/statm: 82822 5975 400 364 0 18725 0
[pid=8763] ppid=8761 vsize=1018616 CPUtime=100.37 cores=1,3,5,7
/proc/8763/stat : 8763 (gnovelty+2) T 8761 8763 8763 0 -1 4202496 770 0 0 0 10037 0 0 0 25 0 1 0 651087896 1043062784 501 33554432000 134512640 135663704 4289111104 18446744073709551615 134525070 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8763/statm: 254654 501 113 282 0 254370 0
[pid=8761/tid=8770] ppid=8701 vsize=331288 CPUtime=0 cores=1,3,5,7
/proc/8761/task/8770/stat : 8770 (python) S 8701 8701 8584 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651097933 339238912 5975 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604467261773824 0 0 -1 3 0 0 0
[pid=8771] ppid=8701 vsize=341716 CPUtime=0.35 cores=1,3,5,7
/proc/8771/stat : 8771 (python) S 8701 8701 8584 0 -1 4202560 1191 0 0 0 18 17 0 0 19 0 2 0 651097936 349917184 6028 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8771/statm: 85429 6028 405 364 0 21329 0
[pid=8773] ppid=8771 vsize=248652 CPUtime=100.35 cores=1,3,5,7
/proc/8773/stat : 8773 (FH) T 8771 8773 8773 0 -1 4202496 1125 0 0 0 10035 0 0 0 25 0 1 0 651097937 254619648 857 33554432000 134512640 134936988 4288524752 18446744073709551615 134552326 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8773/statm: 62163 857 41 104 0 61667 0
[pid=8771/tid=8790] ppid=8701 vsize=341716 CPUtime=0 cores=1,3,5,7
/proc/8771/task/8790/stat : 8790 (python) S 8701 8701 8584 0 -1 4202560 101 0 0 0 0 0 0 0 19 0 2 0 651107972 349917184 6028 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604468335899072 0 0 -1 7 0 0 0
[pid=8791] ppid=8701 vsize=351972 CPUtime=0.35 cores=1,3,5,7
/proc/8791/stat : 8791 (python) S 8701 8701 8584 0 -1 4202560 1174 0 0 0 18 17 0 0 19 0 2 0 651107975 360419328 6032 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8791/statm: 87993 6032 406 364 0 23890 0
[pid=8793] ppid=8791 vsize=418636 CPUtime=99.35 cores=1,3,5,7
/proc/8793/stat : 8793 (adaptg2wsat2009) T 8791 8793 8793 0 -1 4202496 1229 0 0 0 9935 0 0 0 25 0 1 0 651107976 428683264 955 33554432000 134512640 135107411 4290835600 18446744073709551615 134531401 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/8793/statm: 104659 955 59 146 0 103535 0
[pid=8791/tid=8801] ppid=8701 vsize=351972 CPUtime=0 cores=1,3,5,7
/proc/8791/task/8801/stat : 8801 (python) S 8701 8701 8584 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 651117912 360419328 6032 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604458001719552 0 0 -1 7 0 0 0
[pid=8802] ppid=8701 vsize=362228 CPUtime=0.36 cores=1,3,5,7
/proc/8802/stat : 8802 (python) S 8701 8701 8584 0 -1 4202560 1140 0 0 0 19 17 0 0 16 0 2 0 651117915 370921472 6030 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8802/statm: 90557 6030 399 364 0 26451 0
[pid=8804] ppid=8802 vsize=418636 CPUtime=98.36 cores=1,3,5,7
/proc/8804/stat : 8804 (adaptg2wsat2009) T 8802 8804 8804 0 -1 4202496 1231 0 0 0 9836 0 0 0 25 0 1 0 651117916 428683264 955 33554432000 134512640 135107411 4294434064 18446744073709551615 134531830 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/8804/statm: 104659 955 59 146 0 103535 0
[pid=8802/tid=8811] ppid=8701 vsize=362228 CPUtime=0 cores=1,3,5,7
/proc/8802/task/8811/stat : 8811 (python) S 8701 8701 8584 0 -1 4202560 53 0 0 0 0 0 0 0 16 0 2 0 651127752 370921472 6030 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604459710344000 0 0 -1 7 0 0 0
[pid=8812] ppid=8701 vsize=372484 CPUtime=0.37 cores=1,3,5,7
/proc/8812/stat : 8812 (python) S 8701 8701 8584 0 -1 4202560 1136 0 0 0 19 18 0 0 19 0 2 0 651127756 381423616 6034 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8812/statm: 93121 6034 400 364 0 29012 0
[pid=8814] ppid=8812 vsize=248652 CPUtime=97.39 cores=1,3,5,7
/proc/8814/stat : 8814 (FH) T 8812 8814 8814 0 -1 4202496 1125 0 0 0 9739 0 0 0 25 0 1 0 651127756 254619648 857 33554432000 134512640 134936988 4287930288 18446744073709551615 134553267 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/8814/statm: 62163 857 41 104 0 61667 0
[pid=8812/tid=8822] ppid=8701 vsize=372484 CPUtime=0 cores=1,3,5,7
/proc/8812/task/8822/stat : 8822 (python) S 8701 8701 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651137496 381423616 6034 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604467261772928 0 0 -1 1 0 0 0
[pid=8823] ppid=8701 vsize=382896 CPUtime=0.74 cores=1,3,5,7
/proc/8823/stat : 8823 (python) S 8701 8701 8584 0 -1 4202560 1161 0 0 0 36 38 0 0 19 0 2 0 651137500 392085504 6076 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8823/statm: 95724 6076 399 364 0 31612 0
[pid=8825] ppid=8823 vsize=268428 CPUtime=196.83 cores=1,3,5,7
/proc/8825/stat : 8825 (hybridGM3) T 8823 8825 8825 0 -1 4202496 5169 0 0 0 19683 0 0 0 25 0 1 0 651137500 274870272 4897 33554432000 134512640 135204385 4294621600 18446744073709551615 134537792 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8825/statm: 67107 4897 63 169 0 66937 0
[pid=8823/tid=8835] ppid=8701 vsize=382896 CPUtime=0 cores=1,3,5,7
/proc/8823/task/8835/stat : 8835 (python) S 8701 8701 8584 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 651157184 392085504 6076 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604471158414400 0 0 -1 7 0 0 0
[pid=8836] ppid=8701 vsize=393152 CPUtime=0.38 cores=1,3,5,7
/proc/8836/stat : 8836 (python) S 8701 8701 8584 0 -1 4202560 1151 0 0 0 19 19 0 0 19 0 2 0 651157188 402587648 6081 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8836/statm: 98288 6081 400 364 0 34173 0
[pid=8838] ppid=8836 vsize=1018616 CPUtime=102.45 cores=1,3,5,7
/proc/8838/stat : 8838 (gnovelty+2) T 8836 8838 8838 0 -1 4202496 771 0 0 0 10245 0 0 0 25 0 1 0 651157188 1043062784 500 33554432000 134512640 135663704 4291656080 18446744073709551615 134524530 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8838/statm: 254654 500 113 282 0 254370 0
[pid=8836/tid=8846] ppid=8701 vsize=393152 CPUtime=0 cores=1,3,5,7
/proc/8836/task/8846/stat : 8846 (python) S 8701 8701 8584 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 651167434 402587648 6081 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604460560171968 0 0 -1 7 0 0 0
[pid=8847] ppid=8701 vsize=403408 CPUtime=0.75 cores=1,3,5,7
/proc/8847/stat : 8847 (python) S 8701 8701 8584 0 -1 4202560 1162 0 0 0 39 36 0 0 19 0 2 0 651167438 413089792 6096 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8847/statm: 100852 6096 399 364 0 36734 0
[pid=8849] ppid=8847 vsize=268420 CPUtime=199.82 cores=1,3,5,7
/proc/8849/stat : 8849 (hybridGM3) T 8847 8849 8849 0 -1 4202496 5166 0 0 0 19982 0 0 0 25 0 1 0 651167438 274862080 4895 33554432000 134512640 135204385 4291723376 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8849/statm: 67105 4895 63 169 0 66935 0
[pid=8847/tid=8859] ppid=8701 vsize=403408 CPUtime=0 cores=1,3,5,7
/proc/8847/task/8859/stat : 8859 (python) S 8701 8701 8584 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 651187421 413089792 6096 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604460560172864 0 0 -1 3 0 0 0
[pid=8860] ppid=8701 vsize=413664 CPUtime=0.75 cores=1,3,5,7
/proc/8860/stat : 8860 (python) S 8701 8701 8584 0 -1 4202560 1188 0 0 0 38 37 0 0 19 0 2 0 651187425 423591936 6101 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8860/statm: 103416 6101 400 364 0 39295 0
[pid=8862] ppid=8860 vsize=268424 CPUtime=199.81 cores=1,3,5,7
/proc/8862/stat : 8862 (hybridGM3) T 8860 8862 8862 0 -1 4202496 5166 0 0 0 19980 1 0 0 25 0 1 0 651187425 274866176 4895 33554432000 134512640 135204385 4289195360 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/8862/statm: 67106 4895 63 169 0 66936 0
[pid=8860/tid=8872] ppid=8701 vsize=413664 CPUtime=0 cores=1,3,5,7
/proc/8860/task/8872/stat : 8872 (python) S 8701 8701 8584 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 651207406 423591936 6101 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604464107996160 0 0 -1 7 0 0 0
[pid=8873] ppid=8701 vsize=424056 CPUtime=6.12 cores=1,3,5,7
/proc/8873/stat : 8873 (python) S 8701 8701 8584 0 -1 4202560 1237 0 0 0 314 298 0 0 19 0 2 0 651207410 434233344 6135 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8873/statm: 106014 6135 408 364 0 41890 0
[pid=8875] ppid=8873 vsize=457124 CPUtime=1553.38 cores=1,3,5,7
/proc/8875/stat : 8875 (TNM) T 8873 8875 8875 0 -1 4202496 1113 0 0 0 155338 0 0 0 25 0 1 0 651207410 468094976 837 33554432000 134512640 135008748 4287497536 18446744073709551615 134529176 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/8875/statm: 114281 837 46 122 0 113181 0
[pid=8873/tid=8925] ppid=8701 vsize=424056 CPUtime=0 cores=1,3,5,7
/proc/8873/task/8925/stat : 8925 (python) S 8701 8701 8584 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 651362745 434233344 6135 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604464107997056 0 0 -1 7 0 0 0
[pid=8926] ppid=8701 vsize=434312 CPUtime=0.39 cores=1,3,5,7
/proc/8926/stat : 8926 (python) S 8701 8701 8584 0 -1 4202560 1192 0 0 0 21 18 0 0 17 0 2 0 651362747 444735488 6138 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/8926/statm: 108578 6138 407 364 0 44451 0
[pid=8928] ppid=8926 vsize=418636 CPUtime=97.4 cores=1,3,5,7
/proc/8928/stat : 8928 (adaptg2wsat2009) T 8926 8928 8928 0 -1 4202496 1232 0 0 0 9740 0 0 0 25 0 1 0 651362748 428683264 957 33554432000 134512640 135107411 4287524960 18446744073709551615 134531942 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/8928/statm: 104659 957 59 146 0 103535 0
[pid=8926/tid=8935] ppid=8701 vsize=434312 CPUtime=0 cores=1,3,5,7
/proc/8926/task/8935/stat : 8935 (python) S 8701 8701 8584 0 -1 4202560 46 0 0 0 0 0 0 0 17 0 2 0 651372489 444735488 6138 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604463493109696 0 0 -1 3 0 0 0
[pid=8936] ppid=8701 vsize=444568 CPUtime=0.37 cores=1,3,5,7
/proc/8936/stat : 8936 (python) S 8701 8701 8584 0 -1 4202560 1169 0 0 0 18 19 0 0 16 0 2 0 651372491 455237632 6136 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107277771 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/8936/statm: 111142 6136 401 364 0 47012 0
[pid=8938] ppid=8936 vsize=248648 CPUtime=100.28 cores=1,3,5,7
/proc/8938/stat : 8938 (FH) T 8936 8938 8938 0 -1 4202496 1132 0 0 0 10028 0 0 0 25 0 1 0 651372491 254615552 857 33554432000 134512640 134936988 4294512416 18446744073709551615 134553579 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/8938/statm: 62162 857 41 104 0 61666 0
[pid=8936/tid=8982] ppid=8701 vsize=444568 CPUtime=0 cores=1,3,5,7
/proc/8936/task/8982/stat : 8982 (python) S 8701 8701 8584 0 -1 4202560 46 0 0 0 0 0 0 0 16 0 2 0 651382527 455237632 6136 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 514 18446604463493108800 0 0 -1 7 0 0 0
[pid=8983] ppid=8701 vsize=454824 CPUtime=0 cores=1,3,5,7
/proc/8983/stat : 8983 (python) S 8701 8701 8584 0 -1 4202560 843 0 0 0 0 0 0 0 17 0 1 0 651382530 465739776 6115 33554432000 4194304 5683644 140733220042592 18446744073709551615 228095471174 0 0 16781312 514 13549014062297251839 0 0 17 5 0 0 0
/proc/8983/statm: 113706 6115 349 364 0 49573 0
[pid=8985] ppid=8983 vsize=268424 CPUtime=0.29 cores=1,3,5,7
/proc/8985/stat : 8985 (hybridGM3) R 8983 8985 8985 0 -1 4202496 5173 0 0 0 29 0 0 0 19 0 1 0 651382530 274866176 4897 33554432000 134512640 135204385 4287424064 18446744073709551615 134537792 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/8985/statm: 67106 4897 63 169 0 66936 0
[pid=8701/tid=8708] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8708/stat : 8708 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 19 0 651038102 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8719] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8719/stat : 8719 (python) S 8699 8701 8584 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 651048141 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8730] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8730/stat : 8730 (python) S 8699 8701 8584 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 19 0 651058080 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8740] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8740/stat : 8740 (python) S 8699 8701 8584 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 651067918 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8751] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8751/stat : 8751 (python) S 8699 8701 8584 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 651077858 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8762] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8762/stat : 8762 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 651087895 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8772] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8772/stat : 8772 (python) S 8699 8701 8584 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 651097936 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 3 0 0 0
[pid=8701/tid=8792] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8792/stat : 8792 (python) S 8699 8701 8584 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 19 0 651107975 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 3 0 0 0
[pid=8701/tid=8803] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8803/stat : 8803 (python) S 8699 8701 8584 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 651117916 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 7 0 0 0
[pid=8701/tid=8813] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8813/stat : 8813 (python) S 8699 8701 8584 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 651127756 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 7 0 0 0
[pid=8701/tid=8824] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8824/stat : 8824 (python) S 8699 8701 8584 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 651137500 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8837] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8837/stat : 8837 (python) S 8699 8701 8584 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 651157188 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 1 0 0 0
[pid=8701/tid=8848] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8848/stat : 8848 (python) S 8699 8701 8584 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 651167438 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 7 0 0 0
[pid=8701/tid=8861] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8861/stat : 8861 (python) S 8699 8701 8584 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 651187425 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 3 0 0 0
[pid=8701/tid=8874] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8874/stat : 8874 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 651207410 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 3 0 0 0
[pid=8701/tid=8927] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8927/stat : 8927 (python) S 8699 8701 8584 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 19 0 651362748 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 5 0 0 0
[pid=8701/tid=8937] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8937/stat : 8937 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 651372491 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 5 0 0 0
[pid=8701/tid=8984] ppid=8699 vsize=465068 CPUtime=0 cores=1,3,5,7
/proc/8701/task/8984/stat : 8984 (python) S 8699 8701 8584 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 651382530 476229632 7682 33554432000 4194304 5683644 140733220042592 18446744073709551615 228107274673 0 0 16781312 2 18446604455810790592 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3458.74
Current children cumulated vsize (KiB) 15950112

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 8701 and gives
#  childrusage.ru_utime.tv_sec=1
#  childrusage.ru_utime.tv_usec=821723
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=170974
# CPU time returned by wait4() is 1.9927
# while last known CPU time is 3458.74
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3446.34
CPU time (s): 3458.74
CPU user time (s): 3452.3
CPU system time (s): 6.44
CPU usage (%): 100.36
Max. virtual memory (cumulated for all children) (KiB): 15950112

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1.82172
system time used= 0.170974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 28936
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2617
involuntary context switches= 45

runsolver used 25.2432 second user time and 57.7452 second system time

The end

Launcher Data 

Begin job on node142 at 2011-05-17 00:50:25
IDJOB=3386528
IDBENCH=82593
IDSOLVER=1872
FILE ID=node142/3386528-1305586224
RUNJOBID= node142-1305586143-8603
PBS_JOBID= 13350336
Free space on /tmp= 73540 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v200-c17000-S771170845-016-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386528-1305586224/watcher-3386528-1305586224 -o /tmp/evaluation-result-3386528-1305586224/solver-3386528-1305586224 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386528-1305586224.cnf -seed 1217366361 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8a262b49dcf27b27fb405f8a8cdf125f
RANDOM SEED=1217366361

node142.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5131.95
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11521316 kB
Buffers:        314832 kB
Cached:        4027548 kB
SwapCached:      11512 kB
Active:       17208156 kB
Inactive:      3851144 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11521316 kB
SwapTotal:    67111528 kB
SwapFree:     65980224 kB
Dirty:           32064 kB
Writeback:           0 kB
AnonPages:    16713856 kB
Mapped:          33088 kB
Slab:           235508 kB
PageTables:      67520 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 115599404 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73492 MiB
End job on node142 at 2011-05-17 01:47:53