Trace number 3386521

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-seq-random (fixed)? (MO) 3451.63 3440.93

General information on the benchmark

Namerandom/large/
unif-k7-r85-v250-c21250-S655589261-059-UNKNOWN.cnf
MD5SUM7ca36bcba61bdf9cc15c9739d715b3da
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables250
Number of clauses21250
Sum of the clauses size148750
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 521250

Solver Data 

0.32/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.46	c INFO: solving HOME/instance-3386521-1305586195.cnf
0.32/0.46	c INFO: computing task features
0.50/0.62	c INFO: computed variable-clause graph statistics
0.50/0.62	c INFO: computed clause balance statistics
1.30/1.46	c INFO: computed variable balance statistics
1.60/1.79	c INFO: computed small-clause counts
1.60/1.79	c INFO: computed Horn-clause counts
1.70/1.82	c INFO: computed variable graph statistics
1.70/1.82	c INFO: collected features for HOME/instance-3386521-1305586195.cnf in 1.35 s
1.70/1.85	c INFO: running gnovelty+2@0 for 97 with 4998 remaining (b = 0.36)
99.38/99.21	c INFO: running gnovelty+2@0 for 97 with 4901 remaining (b = 0.13)
198.93/198.57	c INFO: running adaptg2wsat2009++@0 for 97 with 4801 remaining (b = 0.08)
298.63/297.92	c INFO: running adaptg2wsat2009++@0 for 97 with 4702 remaining (b = 0.07)
398.25/397.25	c INFO: running adaptg2wsat2009++@0 for 97 with 4603 remaining (b = 0.05)
497.74/496.58	c INFO: running gnovelty+2@0 for 97 with 4504 remaining (b = 0.04)
598.45/596.95	c INFO: running FH@0 for 97 with 4403 remaining (b = 0.03)
698.04/696.31	c INFO: running adaptg2wsat2009++@0 for 97 with 4304 remaining (b = 0.03)
798.75/796.67	c INFO: running adaptg2wsat2009++@0 for 97 with 4204 remaining (b = 0.02)
898.48/896.05	c INFO: running FH@0 for 97 with 4104 remaining (b = 0.02)
999.19/996.42	c INFO: running hybridGM3@0 for 194 with 4004 remaining (b = 0.04)
1194.50/1191.14	c INFO: running gnovelty+2@0 for 97 with 3809 remaining (b = 0.02)
1295.22/1291.54	c INFO: running hybridGM3@0 for 194 with 3709 remaining (b = 0.03)
1491.51/1487.27	c INFO: running hybridGM3@0 for 194 with 3513 remaining (b = 0.03)
1686.98/1682.02	c INFO: running TNM@0 for 1552 with 3318 remaining (b = 0.20)
3250.48/3240.27	c INFO: running adaptg2wsat2009++@0 for 97 with 1760 remaining (b = 0.01)
3351.08/3340.64	c INFO: running FH@0 for 97 with 1660 remaining (b = 0.01)
3450.81/3440.01	c INFO: running hybridGM3@0 for 194 with 1561 remaining (b = 0.02)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386521-1305586195/watcher-3386521-1305586195 -o /tmp/evaluation-result-3386521-1305586195/solver-3386521-1305586195 -C 5000 -W 5100 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386521-1305586195.cnf -seed 412364273 -budget 5000 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.09 0.29 0.86 2/401 2592
/proc/meminfo: memFree=12523160/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 791 0 0 0 0 0 0 0 25 0 1 0 670219173 15749120 679 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080230400 0 2147483391 16781312 2 18446744071699207358 0 0 17 4 0 0 0
/proc/2592/statm: 3845 679 322 364 0 332 0

[startup+0.109591 s]
/proc/loadavg: 1.09 0.29 0.86 2/401 2592
/proc/meminfo: memFree=12523160/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=105228 CPUtime=0.08 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 3309 0 0 0 6 2 0 0 25 0 1 0 670219173 107753472 2974 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080230400 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/2592/statm: 26307 2974 868 364 0 2055 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 105228

[startup+0.206457 s]
/proc/loadavg: 1.09 0.29 0.86 2/401 2592
/proc/meminfo: memFree=12523160/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=131288 CPUtime=0.17 cores=0,2,4,6
/proc/2592/stat : 2592 (python) R 2590 2592 2509 0 -1 4202496 5711 935 1 0 13 4 0 0 19 0 1 0 670219173 134438912 4097 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080232192 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/2592/statm: 32822 4098 1015 364 0 3083 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 131288

[startup+0.300401 s]
/proc/loadavg: 1.09 0.29 0.86 2/401 2592
/proc/meminfo: memFree=12523160/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=192320 CPUtime=0.23 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 6901 935 1 0 17 6 0 0 18 0 1 0 670219173 196935680 4948 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080230400 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/2592/statm: 48080 4948 1425 364 0 3723 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 192320

[startup+0.700375 s]
/proc/loadavg: 1.09 0.29 0.86 2/401 2592
/proc/meminfo: memFree=12523160/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=283908 CPUtime=0.6 cores=0,2,4,6
/proc/2592/stat : 2592 (python) R 2590 2592 2509 0 -1 4202496 11852 935 1 0 52 8 0 0 18 0 1 0 670219173 290721792 8362 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055944 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/2592/statm: 70977 8362 1826 364 0 6901 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 283908

[startup+1.50024 s]
/proc/loadavg: 1.09 0.29 0.86 3/402 2597
/proc/meminfo: memFree=12495252/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=283908 CPUtime=1.4 cores=0,2,4,6
/proc/2592/stat : 2592 (python) R 2590 2592 2509 0 -1 4202496 11853 935 1 0 132 8 0 0 20 0 1 0 670219173 290721792 8363 33554432000 4194304 5683644 140735265715616 18446744073709551615 208083641013 0 0 16781312 2 0 0 0 17 0 0 0 0
/proc/2592/statm: 70977 8363 1827 364 0 6901 0
Current children cumulated CPU time (s) 1.4
Current children cumulated vsize (KiB) 283908

[startup+3.10094 s]
/proc/loadavg: 1.09 0.29 0.86 3/405 2600
/proc/meminfo: memFree=12494964/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=289692 CPUtime=1.75 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 12181 935 1 0 167 8 0 0 23 0 2 0 670219173 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/2592/statm: 72423 7296 1877 364 0 8341 0
[pid=2598] ppid=2592 vsize=279932 CPUtime=0 cores=0,2,4,6
/proc/2598/stat : 2598 (python) S 2592 2592 2509 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 670219358 286650368 5771 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080252431 0 0 16781312 514 9026041314923217153 0 0 17 6 0 0 0
/proc/2598/statm: 69983 5771 355 364 0 5901 0
[pid=2600] ppid=2598 vsize=1018820 CPUtime=1.24 cores=0,2,4,6
/proc/2600/stat : 2600 (gnovelty+2) R 2598 2600 2600 0 -1 4202496 862 0 0 0 124 0 0 0 25 0 1 0 670219359 1043271680 596 33554432000 134512640 135663704 4293301536 18446744073709551615 134525699 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2600/statm: 254705 596 113 282 0 254421 0
[pid=2592/tid=2599] ppid=2590 vsize=289692 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2599/stat : 2599 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 23 0 2 0 670219358 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.99
Current children cumulated vsize (KiB) 1588444

[startup+6.30025 s]
/proc/loadavg: 1.16 0.32 0.87 3/405 2600
/proc/meminfo: memFree=12495464/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=289692 CPUtime=1.75 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 12181 935 1 0 167 8 0 0 23 0 2 0 670219173 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/2592/statm: 72423 7296 1877 364 0 8341 0
[pid=2598] ppid=2592 vsize=279932 CPUtime=0.01 cores=0,2,4,6
/proc/2598/stat : 2598 (python) S 2592 2592 2509 0 -1 4202560 884 0 0 0 0 1 0 0 18 0 1 0 670219358 286650368 5794 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080252431 0 0 16781312 514 9026041314923217153 0 0 17 6 0 0 0
/proc/2598/statm: 69983 5794 378 364 0 5901 0
[pid=2600] ppid=2598 vsize=1018820 CPUtime=4.44 cores=0,2,4,6
/proc/2600/stat : 2600 (gnovelty+2) R 2598 2600 2600 0 -1 4202496 862 0 0 0 444 0 0 0 25 0 1 0 670219359 1043271680 596 33554432000 134512640 135663704 4293301536 18446744073709551615 134547176 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2600/statm: 254705 596 113 282 0 254421 0
[pid=2592/tid=2599] ppid=2590 vsize=289692 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2599/stat : 2599 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 23 0 2 0 670219358 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 6.2
Current children cumulated vsize (KiB) 1588444

[startup+12.701 s]
/proc/loadavg: 1.23 0.35 0.87 3/405 2600
/proc/meminfo: memFree=12494244/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=289692 CPUtime=1.75 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 12181 935 1 0 167 8 0 0 23 0 2 0 670219173 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/2592/statm: 72423 7296 1877 364 0 8341 0
[pid=2598] ppid=2592 vsize=279932 CPUtime=0.02 cores=0,2,4,6
/proc/2598/stat : 2598 (python) S 2592 2592 2509 0 -1 4202560 887 0 0 0 1 1 0 0 18 0 1 0 670219358 286650368 5794 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080252431 0 0 16781312 514 18172868921370050817 0 0 17 6 0 0 0
/proc/2598/statm: 69983 5794 378 364 0 5901 0
[pid=2600] ppid=2598 vsize=1018820 CPUtime=10.84 cores=0,2,4,6
/proc/2600/stat : 2600 (gnovelty+2) R 2598 2600 2600 0 -1 4202496 862 0 0 0 1084 0 0 0 25 0 1 0 670219359 1043271680 596 33554432000 134512640 135663704 4293301536 18446744073709551615 134525111 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2600/statm: 254705 596 113 282 0 254421 0
[pid=2592/tid=2599] ppid=2590 vsize=289692 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2599/stat : 2599 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 23 0 2 0 670219358 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.61
Current children cumulated vsize (KiB) 1588444

[startup+25.5005 s]
/proc/loadavg: 1.40 0.43 0.89 3/405 2600
/proc/meminfo: memFree=12494736/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=289692 CPUtime=1.75 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 12181 935 1 0 167 8 0 0 23 0 2 0 670219173 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/2592/statm: 72423 7296 1877 364 0 8341 0
[pid=2598] ppid=2592 vsize=279932 CPUtime=0.06 cores=0,2,4,6
/proc/2598/stat : 2598 (python) S 2592 2592 2509 0 -1 4202560 893 0 0 0 3 3 0 0 18 0 1 0 670219358 286650368 5794 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080252431 0 0 16781312 514 18172868921370050817 0 0 17 6 0 0 0
/proc/2598/statm: 69983 5794 378 364 0 5901 0
[pid=2600] ppid=2598 vsize=1018820 CPUtime=23.64 cores=0,2,4,6
/proc/2600/stat : 2600 (gnovelty+2) R 2598 2600 2600 0 -1 4202496 862 0 0 0 2364 0 0 0 25 0 1 0 670219359 1043271680 596 33554432000 134512640 135663704 4293301536 18446744073709551615 134524882 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2600/statm: 254705 596 113 282 0 254421 0
[pid=2592/tid=2599] ppid=2590 vsize=289692 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2599/stat : 2599 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 23 0 2 0 670219358 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.45
Current children cumulated vsize (KiB) 1588444

[startup+51.1007 s]
/proc/loadavg: 1.67 0.57 0.92 3/405 2601
/proc/meminfo: memFree=12493916/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=289692 CPUtime=1.75 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 12181 935 1 0 167 8 0 0 23 0 2 0 670219173 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/2592/statm: 72423 7296 1877 364 0 8341 0
[pid=2598] ppid=2592 vsize=279932 CPUtime=0.13 cores=0,2,4,6
/proc/2598/stat : 2598 (python) S 2592 2592 2509 0 -1 4202560 905 0 0 0 7 6 0 0 18 0 1 0 670219358 286650368 5794 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080252431 0 0 16781312 514 18172868921370050817 0 0 17 6 0 0 0
/proc/2598/statm: 69983 5794 378 364 0 5901 0
[pid=2600] ppid=2598 vsize=1018820 CPUtime=49.24 cores=0,2,4,6
/proc/2600/stat : 2600 (gnovelty+2) R 2598 2600 2600 0 -1 4202496 862 0 0 0 4924 0 0 0 25 0 1 0 670219359 1043271680 596 33554432000 134512640 135663704 4293301536 18446744073709551615 134544370 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2600/statm: 254705 596 113 282 0 254421 0
[pid=2592/tid=2599] ppid=2590 vsize=289692 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2599/stat : 2599 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 23 0 2 0 670219358 296644608 7296 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 51.12
Current children cumulated vsize (KiB) 1588444

[startup+102.301 s]
/proc/loadavg: 1.85 0.79 0.98 3/413 2610
/proc/meminfo: memFree=12482528/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=299948 CPUtime=1.79 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 13020 935 1 0 170 9 0 0 20 0 3 0 670219173 307146752 7328 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/2592/statm: 74987 7328 1880 364 0 10902 0
[pid=2598] ppid=2592 vsize=290176 CPUtime=0.28 cores=0,2,4,6
/proc/2598/stat : 2598 (python) S 2592 2592 2509 0 -1 4202560 1081 0 0 0 15 13 0 0 19 0 2 0 670219358 297140224 5831 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2598/statm: 72544 5831 410 364 0 8462 0
[pid=2600] ppid=2598 vsize=1018820 CPUtime=97.32 cores=0,2,4,6
/proc/2600/stat : 2600 (gnovelty+2) T 2598 2600 2600 0 -1 4202496 862 0 0 0 9732 0 0 0 25 0 1 0 670219359 1043271680 596 33554432000 134512640 135663704 4293301536 18446744073709551615 134524553 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2600/statm: 254705 596 113 282 0 254421 0
[pid=2598/tid=2607] ppid=2592 vsize=290176 CPUtime=0 cores=0,2,4,6
/proc/2598/task/2607/stat : 2607 (python) S 2592 2592 2509 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670229091 297140224 5831 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604446424297664 0 0 -1 6 0 0 0
[pid=2608] ppid=2592 vsize=290828 CPUtime=0 cores=0,2,4,6
/proc/2608/stat : 2608 (python) S 2592 2592 2509 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 670229094 297807872 5794 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080252486 0 0 16781312 514 10484701818842480896 0 0 17 0 0 0 0
/proc/2608/statm: 72707 5794 349 364 0 8622 0
[pid=2610] ppid=2608 vsize=1018820 CPUtime=3.08 cores=0,2,4,6
/proc/2610/stat : 2610 (gnovelty+2) R 2608 2610 2610 0 -1 4202496 858 0 0 0 308 0 0 0 25 0 1 0 670229094 1043271680 595 33554432000 134512640 135663704 4290896848 18446744073709551615 134524916 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/2610/statm: 254705 595 113 282 0 254421 0
[pid=2592/tid=2599] ppid=2590 vsize=299948 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2599/stat : 2599 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 23 0 3 0 670219358 307146752 7328 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2609] ppid=2590 vsize=299948 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2609/stat : 2609 (python) S 2590 2592 2509 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 3 0 670229094 307146752 7328 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 102.47
Current children cumulated vsize (KiB) 2918592

[startup+162.301 s]
/proc/loadavg: 1.94 1.01 1.04 3/417 2616
/proc/meminfo: memFree=12472528/32950928 swapFree=57376972/67111528
[pid=2592] ppid=2590 vsize=299948 CPUtime=1.79 cores=0,2,4,6
/proc/2592/stat : 2592 (python) S 2590 2592 2509 0 -1 4202496 13020 935 1 0 170 9 0 0 20 0 3 0 670219173 307146752 7328 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 0
/proc/2592/statm: 74987 7328 1880 364 0 10902 0
[pid=2598] ppid=2592 vsize=290176 CPUtime=0.28 cores=0,2,4,6
/proc/2598/stat : 2598 (python) S 2592 2592 2509 0 -1 4202560 1081 0 0 0 15 13 0 0 19 0 2 0 670219358 297140224 5831 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2598/statm: 72544 5831 410 364 0 8462 0
[pid=2600] ppid=2598 vsize=1018820 CPUtime=97.32 cores=0,2,4,6
/proc/2600/stat : 2600 (gnovelty+2) T 2598 2600 2600 0 -1 4202496 862 0 0 0 9732 0 0 0 25 0 1 0 670219359 1043271680 596 33554432000 134512640 135663704 4293301536 18446744073709551615 134524553 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2600/statm: 254705 596 113 282 0 254421 0
[pid=2598/tid=2607] ppid=2592 vsize=290176 CPUtime=0 cores=0,2,4,6
/proc/2598/task/2607/stat : 2607 (python) S 2592 2592 2509 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670229091 297140224 5831 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604446424297664 0 0 -1 6 0 0 0
[pid=2608] ppid=2592 vsize=290828 CPUtime=0.18 cores=0,2,4,6

################
# More data... #
################

[pid=2630] ppid=2592 vsize=311028 CPUtime=0.3 cores=0,2,4,6
/proc/2630/stat : 2630 (python) S 2592 2592 2509 0 -1 4202560 1102 0 0 0 17 13 0 0 19 0 2 0 670248965 318492672 5989 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2630/statm: 77757 5989 400 364 0 13666 0
[pid=2632] ppid=2630 vsize=419428 CPUtime=99.29 cores=0,2,4,6
/proc/2632/stat : 2632 (adaptg2wsat2009) T 2630 2632 2632 0 -1 4202496 1440 0 0 0 9929 0 0 0 25 0 1 0 670248965 429494272 1175 33554432000 134512640 135107411 4294347840 18446744073709551615 134531656 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/2632/statm: 104857 1175 59 146 0 103733 0
[pid=2630/tid=2639] ppid=2592 vsize=311028 CPUtime=0 cores=0,2,4,6
/proc/2630/task/2639/stat : 2639 (python) S 2592 2592 2509 0 -1 4202560 53 0 0 0 0 0 0 0 19 0 2 0 670258895 318492672 5989 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604441208660288 0 0 -1 4 0 0 0
[pid=2640] ppid=2592 vsize=321112 CPUtime=0.3 cores=0,2,4,6
/proc/2640/stat : 2640 (python) S 2592 2592 2509 0 -1 4202560 1120 0 0 0 17 13 0 0 19 0 2 0 670258898 328818688 5992 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2640/statm: 80278 5992 399 364 0 16184 0
[pid=2642] ppid=2640 vsize=419428 CPUtime=99.23 cores=0,2,4,6
/proc/2642/stat : 2642 (adaptg2wsat2009) T 2640 2642 2642 0 -1 4202496 1441 0 0 0 9923 0 0 0 25 0 1 0 670258898 429494272 1175 33554432000 134512640 135107411 4291255664 18446744073709551615 134531709 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/2642/statm: 104857 1175 59 146 0 103733 0
[pid=2640/tid=2650] ppid=2592 vsize=321112 CPUtime=0 cores=0,2,4,6
/proc/2640/task/2650/stat : 2650 (python) S 2592 2592 2509 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 670268828 328818688 5992 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604447236092352 0 0 -1 2 0 0 0
[pid=2651] ppid=2592 vsize=331556 CPUtime=0.3 cores=0,2,4,6
/proc/2651/stat : 2651 (python) S 2592 2592 2509 0 -1 4202560 1129 0 0 0 17 13 0 0 19 0 2 0 670268831 339513344 5997 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/2651/statm: 82889 5997 400 364 0 18792 0
[pid=2653] ppid=2651 vsize=1018820 CPUtime=100.32 cores=0,2,4,6
/proc/2653/stat : 2653 (gnovelty+2) T 2651 2653 2653 0 -1 4202496 863 0 0 0 10032 0 0 0 25 0 1 0 670268832 1043271680 595 33554432000 134512640 135663704 4290850304 18446744073709551615 134524509 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2653/statm: 254705 595 113 282 0 254421 0
[pid=2651/tid=2661] ppid=2592 vsize=331556 CPUtime=0 cores=0,2,4,6
/proc/2651/task/2661/stat : 2661 (python) S 2592 2592 2509 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 670278865 339513344 5997 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604442282035328 0 0 -1 6 0 0 0
[pid=2662] ppid=2592 vsize=342036 CPUtime=0.31 cores=0,2,4,6
/proc/2662/stat : 2662 (python) S 2592 2592 2509 0 -1 4202560 1179 0 0 0 16 15 0 0 19 0 2 0 670278869 350244864 6007 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2662/statm: 85509 6007 406 364 0 21409 0
[pid=2664] ppid=2662 vsize=249188 CPUtime=99.31 cores=0,2,4,6
/proc/2664/stat : 2664 (FH) T 2662 2664 2664 0 -1 4202496 1324 0 0 0 9931 0 0 0 25 0 1 0 670278869 255168512 1058 33554432000 134512640 134936988 4292873712 18446744073709551615 134553491 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/2664/statm: 62297 1058 41 104 0 61801 0
[pid=2662/tid=2680] ppid=2592 vsize=342036 CPUtime=0 cores=0,2,4,6
/proc/2662/task/2680/stat : 2680 (python) S 2592 2592 2509 0 -1 4202560 99 0 0 0 0 0 0 0 19 0 2 0 670288800 350244864 6007 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604446747636992 0 0 -1 6 0 0 0
[pid=2681] ppid=2592 vsize=352164 CPUtime=0.33 cores=0,2,4,6
/proc/2681/stat : 2681 (python) S 2592 2592 2509 0 -1 4202560 1164 0 0 0 17 16 0 0 19 0 2 0 670288804 360615936 6011 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2681/statm: 88041 6011 406 364 0 23938 0
[pid=2683] ppid=2681 vsize=419428 CPUtime=100.32 cores=0,2,4,6
/proc/2683/stat : 2683 (adaptg2wsat2009) T 2681 2683 2683 0 -1 4202496 1451 0 0 0 10032 0 0 0 25 0 1 0 670288804 429494272 1176 33554432000 134512640 135107411 4287476384 18446744073709551615 134532010 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/2683/statm: 104857 1176 59 146 0 103733 0
[pid=2681/tid=2691] ppid=2592 vsize=352164 CPUtime=0 cores=0,2,4,6
/proc/2681/task/2691/stat : 2691 (python) S 2592 2592 2509 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 670298837 360615936 6011 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604451857407040 0 0 -1 6 0 0 0
[pid=2692] ppid=2592 vsize=362676 CPUtime=0.31 cores=0,2,4,6
/proc/2692/stat : 2692 (python) S 2592 2592 2509 0 -1 4202560 1130 0 0 0 17 14 0 0 19 0 2 0 670298841 371380224 6009 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2692/statm: 90669 6009 400 364 0 26563 0
[pid=2694] ppid=2692 vsize=419428 CPUtime=99.32 cores=0,2,4,6
/proc/2694/stat : 2694 (adaptg2wsat2009) T 2692 2694 2694 0 -1 4202496 1449 0 0 0 9932 0 0 0 25 0 1 0 670298841 429494272 1176 33554432000 134512640 135107411 4287992640 18446744073709551615 134531232 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/2694/statm: 104857 1176 59 146 0 103733 0
[pid=2692/tid=2705] ppid=2592 vsize=362676 CPUtime=0 cores=0,2,4,6
/proc/2692/task/2705/stat : 2705 (python) S 2592 2592 2509 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670308774 371380224 6009 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604444028140224 0 0 -1 6 0 0 0
[pid=2706] ppid=2592 vsize=372744 CPUtime=0.32 cores=0,2,4,6
/proc/2706/stat : 2706 (python) S 2592 2592 2509 0 -1 4202560 1145 0 0 0 18 14 0 0 19 0 2 0 670308778 381689856 6027 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2706/statm: 93186 6027 400 364 0 29077 0
[pid=2708] ppid=2706 vsize=249196 CPUtime=100.32 cores=0,2,4,6
/proc/2708/stat : 2708 (FH) T 2706 2708 2708 0 -1 4202496 1328 0 0 0 10032 0 0 0 25 0 1 0 670308778 255176704 1058 33554432000 134512640 134936988 4286900752 18446744073709551615 134553498 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/2708/statm: 62299 1058 41 104 0 61803 0
[pid=2706/tid=2715] ppid=2592 vsize=372744 CPUtime=0 cores=0,2,4,6
/proc/2706/task/2715/stat : 2715 (python) S 2592 2592 2509 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670318811 381689856 6027 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604447990451264 0 0 -1 2 0 0 0
[pid=2716] ppid=2592 vsize=383144 CPUtime=0.62 cores=0,2,4,6
/proc/2716/stat : 2716 (python) S 2592 2592 2509 0 -1 4202560 1163 0 0 0 36 26 0 0 19 0 2 0 670318815 392339456 6031 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2716/statm: 95786 6031 400 364 0 31674 0
[pid=2718] ppid=2716 vsize=268420 CPUtime=194.68 cores=0,2,4,6
/proc/2718/stat : 2718 (hybridGM3) T 2716 2718 2718 0 -1 4202496 5279 0 0 0 19467 1 0 0 25 0 1 0 670318815 274862080 5008 33554432000 134512640 135204385 4289563712 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2718/statm: 67105 5008 63 169 0 66935 0
[pid=2716/tid=2728] ppid=2592 vsize=383144 CPUtime=0 cores=0,2,4,6
/proc/2716/task/2728/stat : 2728 (python) S 2592 2592 2509 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670338284 392339456 6031 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604446424300352 0 0 -1 6 0 0 0
[pid=2729] ppid=2592 vsize=393356 CPUtime=0.31 cores=0,2,4,6
/proc/2729/stat : 2729 (python) S 2592 2592 2509 0 -1 4202560 1149 0 0 0 18 13 0 0 19 0 2 0 670338288 402796544 6035 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2729/statm: 98339 6035 400 364 0 34224 0
[pid=2731] ppid=2729 vsize=1018820 CPUtime=100.35 cores=0,2,4,6
/proc/2731/stat : 2731 (gnovelty+2) T 2729 2731 2731 0 -1 4202496 869 0 0 0 10035 0 0 0 25 0 1 0 670338288 1043271680 596 33554432000 134512640 135663704 4288333392 18446744073709551615 134524548 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/2731/statm: 254705 596 113 282 0 254421 0
[pid=2729/tid=2739] ppid=2592 vsize=393356 CPUtime=0 cores=0,2,4,6
/proc/2729/task/2739/stat : 2739 (python) S 2592 2592 2509 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670348324 402796544 6035 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604438786091904 0 0 -1 6 0 0 0
[pid=2743] ppid=2592 vsize=403768 CPUtime=0.63 cores=0,2,4,6
/proc/2743/stat : 2743 (python) S 2592 2592 2509 0 -1 4202560 1177 0 0 0 35 28 0 0 19 0 2 0 670348328 413458432 6039 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2743/statm: 100942 6039 400 364 0 36824 0
[pid=2745] ppid=2743 vsize=268420 CPUtime=195.68 cores=0,2,4,6
/proc/2745/stat : 2745 (hybridGM3) T 2743 2745 2745 0 -1 4202496 5282 0 0 0 19568 0 0 0 25 0 1 0 670348328 274862080 5008 33554432000 134512640 135204385 4294004096 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/2745/statm: 67105 5008 63 169 0 66935 0
[pid=2743/tid=2756] ppid=2592 vsize=403768 CPUtime=0 cores=0,2,4,6
/proc/2743/task/2756/stat : 2756 (python) S 2592 2592 2509 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 670367897 413458432 6039 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604446246865664 0 0 -1 6 0 0 0
[pid=2757] ppid=2592 vsize=413988 CPUtime=0.65 cores=0,2,4,6
/proc/2757/stat : 2757 (python) S 2592 2592 2509 0 -1 4202560 1182 0 0 0 35 30 0 0 19 0 2 0 670367901 423923712 6060 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2757/statm: 103497 6060 400 364 0 39376 0
[pid=2759] ppid=2757 vsize=268420 CPUtime=194.71 cores=0,2,4,6
/proc/2759/stat : 2759 (hybridGM3) T 2757 2759 2759 0 -1 4202496 5282 0 0 0 19471 0 0 0 25 0 1 0 670367901 274862080 5009 33554432000 134512640 135204385 4289860688 18446744073709551615 134538215 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/2759/statm: 67105 5009 63 169 0 66935 0
[pid=2757/tid=2771] ppid=2592 vsize=413988 CPUtime=0 cores=0,2,4,6
/proc/2757/task/2771/stat : 2771 (python) S 2592 2592 2509 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 670387372 423923712 6060 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604453861129088 0 0 -1 6 0 0 0
[pid=2772] ppid=2592 vsize=424416 CPUtime=5.28 cores=0,2,4,6
/proc/2772/stat : 2772 (python) S 2592 2592 2509 0 -1 4202560 1220 0 0 0 304 224 0 0 19 0 2 0 670387376 434601984 6072 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2772/statm: 106104 6072 408 364 0 41980 0
[pid=2774] ppid=2772 vsize=457668 CPUtime=1558.27 cores=0,2,4,6
/proc/2774/stat : 2774 (TNM) T 2772 2774 2774 0 -1 4202496 1312 0 0 0 155826 1 0 0 25 0 1 0 670387376 468652032 1036 33554432000 134512640 135008748 4290529904 18446744073709551615 134529448 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2774/statm: 114417 1036 46 122 0 113317 0
[pid=2772/tid=2824] ppid=2592 vsize=424416 CPUtime=0 cores=0,2,4,6
/proc/2772/task/2824/stat : 2824 (python) S 2592 2592 2509 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 670543198 434601984 6072 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604442830196992 0 0 -1 6 0 0 0
[pid=2825] ppid=2592 vsize=434612 CPUtime=0.34 cores=0,2,4,6
/proc/2825/stat : 2825 (python) S 2592 2592 2509 0 -1 4202560 1172 0 0 0 19 15 0 0 19 0 2 0 670543200 445042688 6109 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2825/statm: 108653 6109 407 364 0 44526 0
[pid=2827] ppid=2825 vsize=419428 CPUtime=100.2 cores=0,2,4,6
/proc/2827/stat : 2827 (adaptg2wsat2009) T 2825 2827 2827 0 -1 4202496 1449 0 0 0 10020 0 0 0 25 0 1 0 670543201 429494272 1176 33554432000 134512640 135107411 4287186208 18446744073709551615 134531848 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/2827/statm: 104857 1176 59 146 0 103733 0
[pid=2825/tid=2835] ppid=2592 vsize=434612 CPUtime=0 cores=0,2,4,6
/proc/2825/task/2835/stat : 2835 (python) S 2592 2592 2509 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 670553235 445042688 6109 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604453861126400 0 0 -1 0 0 0 0
[pid=2836] ppid=2592 vsize=445056 CPUtime=0.34 cores=0,2,4,6
/proc/2836/stat : 2836 (python) S 2592 2592 2509 0 -1 4202560 1148 0 0 0 18 16 0 0 19 0 2 0 670553237 455737344 6107 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092059083 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/2836/statm: 111264 6107 401 364 0 47134 0
[pid=2838] ppid=2836 vsize=249196 CPUtime=99.34 cores=0,2,4,6
/proc/2838/stat : 2838 (FH) T 2836 2838 2838 0 -1 4202496 1332 0 0 0 9932 2 0 0 25 0 1 0 670553237 255176704 1058 33554432000 134512640 134936988 4291169024 18446744073709551615 134544771 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/2838/statm: 62299 1058 41 104 0 61803 0
[pid=2836/tid=2890] ppid=2592 vsize=445056 CPUtime=0 cores=0,2,4,6
/proc/2836/task/2890/stat : 2890 (python) S 2592 2592 2509 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 670563173 455737344 6107 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 514 18446604451065368768 0 0 -1 6 0 0 0
[pid=2891] ppid=2592 vsize=455480 CPUtime=0 cores=0,2,4,6
/proc/2891/stat : 2891 (python) S 2592 2592 2509 0 -1 4202560 847 0 0 0 0 0 0 0 19 0 1 0 670563175 466411520 6091 33554432000 4194304 5683644 140735265715616 18446744073709551615 208080252486 0 0 16781312 514 10254691408015261695 0 0 17 0 0 0 0
/proc/2891/statm: 113870 6091 350 364 0 49737 0
[pid=2893] ppid=2891 vsize=268424 CPUtime=0.79 cores=0,2,4,6
/proc/2893/stat : 2893 (hybridGM3) R 2891 2893 2893 0 -1 4202496 5283 0 0 0 79 0 0 0 22 0 1 0 670563175 274866176 5010 33554432000 134512640 135204385 4294150144 18446744073709551615 134538284 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/2893/statm: 67106 5010 63 169 0 66936 0
[pid=2592/tid=2599] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2599/stat : 2599 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 23 0 19 0 670219358 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2609] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2609/stat : 2609 (python) S 2590 2592 2509 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 670229094 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 6 0 0 0
[pid=2592/tid=2620] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2620/stat : 2620 (python) S 2590 2592 2509 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 670239031 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2631] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2631/stat : 2631 (python) S 2590 2592 2509 0 -1 4202560 21 935 0 0 0 0 0 0 20 0 19 0 670248965 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 6 0 0 0
[pid=2592/tid=2641] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2641/stat : 2641 (python) S 2590 2592 2509 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 19 0 670258898 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 6 0 0 0
[pid=2592/tid=2652] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2652/stat : 2652 (python) S 2590 2592 2509 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 670268832 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 6 0 0 0
[pid=2592/tid=2663] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2663/stat : 2663 (python) S 2590 2592 2509 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 670278869 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2682] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2682/stat : 2682 (python) S 2590 2592 2509 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 19 0 670288804 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2693] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2693/stat : 2693 (python) S 2590 2592 2509 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 19 0 670298841 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2707] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2707/stat : 2707 (python) S 2590 2592 2509 0 -1 4202560 29 935 0 0 0 0 0 0 20 0 19 0 670308778 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 6 0 0 0
[pid=2592/tid=2717] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2717/stat : 2717 (python) S 2590 2592 2509 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 19 0 670318815 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 2 0 0 0
[pid=2592/tid=2730] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2730/stat : 2730 (python) S 2590 2592 2509 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 19 0 670338288 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2744] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2744/stat : 2744 (python) S 2590 2592 2509 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 670348328 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2758] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2758/stat : 2758 (python) S 2590 2592 2509 0 -1 4202560 32 935 0 0 0 0 0 0 20 0 19 0 670367901 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 2 0 0 0
[pid=2592/tid=2773] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2773/stat : 2773 (python) S 2590 2592 2509 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 670387376 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 4 0 0 0
[pid=2592/tid=2826] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2826/stat : 2826 (python) S 2590 2592 2509 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 670543201 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 6 0 0 0
[pid=2592/tid=2837] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2837/stat : 2837 (python) S 2590 2592 2509 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 670553237 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
[pid=2592/tid=2892] ppid=2590 vsize=464944 CPUtime=0 cores=0,2,4,6
/proc/2592/task/2892/stat : 2892 (python) S 2590 2592 2509 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 19 0 670563175 476102656 7656 33554432000 4194304 5683644 140735265715616 18446744073709551615 208092055985 0 0 16781312 2 18446604444832253184 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3451.63
Current children cumulated vsize (KiB) 15964464

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 2592 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=232660
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=146977
# CPU time returned by wait4() is 2.37964
# while last known CPU time is 3451.63
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 3440.93
CPU time (s): 3451.63
CPU user time (s): 3446.55
CPU system time (s): 5.08
CPU usage (%): 100.311
Max. virtual memory (cumulated for all children) (KiB): 15964464

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.23266
system time used= 0.146977
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 29476
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1063
involuntary context switches= 51

runsolver used 25.2242 second user time and 50.5413 second system time

The end

Launcher Data 

Begin job on node127 at 2011-05-17 00:49:55
IDJOB=3386521
IDBENCH=82586
IDSOLVER=1872
FILE ID=node127/3386521-1305586195
RUNJOBID= node127-1305586146-2527
PBS_JOBID= 13350351
Free space on /tmp= 73352 MiB

SOLVER NAME= borg-sat 11.03.02-seq-random (fixed)
BENCH NAME= SAT11/random/large/unif-k7-r85-v250-c21250-S655589261-059-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386521-1305586195/watcher-3386521-1305586195 -o /tmp/evaluation-result-3386521-1305586195/solver-3386521-1305586195 -C 5000 -W 5100 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3386521-1305586195.cnf -seed 412364273 -budget 5000

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 7ca36bcba61bdf9cc15c9739d715b3da
RANDOM SEED=412364273

node127.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.64
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.60
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.823
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12523324 kB
Buffers:         87456 kB
Cached:         538364 kB
SwapCached:     117904 kB
Active:        5643308 kB
Inactive:     14560340 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12523324 kB
SwapTotal:    67111528 kB
SwapFree:     57376972 kB
Dirty:           32160 kB
Writeback:           0 kB
AnonPages:    19466160 kB
Mapped:          32308 kB
Slab:            74356 kB
PageTables:      82340 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 85708120 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73304 MiB
End job on node127 at 2011-05-17 01:47:18