Trace number 3317098

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 1984.32 500.517

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S926861876-013-UNKNOWN.cnf
MD5SUMaa5d2fab702f701bf63e2d26e76ab2b3
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.40/1.13	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.40/1.14	c INFO: solving HOME/instance-3317098-1304267308.cnf
0.40/1.14	c INFO: computing task features
0.57/1.32	c INFO: computed variable-clause graph statistics
0.57/1.32	c INFO: computed clause balance statistics
1.48/2.25	c INFO: computed variable balance statistics
1.88/2.60	c INFO: computed small-clause counts
1.88/2.60	c INFO: computed Horn-clause counts
1.88/2.63	c INFO: computed variable graph statistics
1.88/2.63	c INFO: collected features for HOME/instance-3317098-1304267308.cnf in 1.49 s
1.88/2.66	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.37)
1.88/2.69	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.13)
1.97/2.72	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.08)
1.97/2.75	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.07)
401.74/102.85	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.05)
401.74/102.89	c INFO: running gnovelty+2@0 for 97 with 4698 remaining (b = 0.04)
402.07/102.92	c INFO: running FH@0 for 97 with 4698 remaining (b = 0.03)
402.07/102.96	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
796.90/202.03	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.02)
801.26/203.10	c INFO: running FH@0 for 97 with 4598 remaining (b = 0.02)
801.26/203.14	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
801.26/203.19	c INFO: running gnovelty+2@0 for 97 with 4597 remaining (b = 0.02)
1195.92/302.22	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1199.88/303.29	c INFO: running adaptg2wsat2009++@0 for 97 with 4497 remaining (b = 0.01)
1200.58/303.42	c INFO: running hybridGM3@0 for 194 with 4497 remaining (b = 0.03)
1579.37/398.56	c INFO: running hybridGM3@100 for 97 with 4402 remaining (b = 0.01)
1598.89/403.50	c INFO: running hybridGM3@0 for 194 with 4397 remaining (b = 0.02)
1981.52/499.76	c INFO: running TNM@0 for 1552 with 4301 remaining (b = 0.19)
1981.87/499.84	c INFO: running FH@0 for 97 with 4301 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317098-1304267308/watcher-3317098-1304267308 -o /tmp/evaluation-result-3317098-1304267308/solver-3317098-1304267308 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317098-1304267308.cnf -seed 1621517985 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.59 0.13 0.04 5/189 9941
/proc/meminfo: memFree=30714728/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=15188 CPUtime=0 cores=1,3,5,7
/proc/9941/stat : 9941 (python) R 9939 9941 9843 0 -1 4202496 716 0 0 0 0 0 0 0 25 0 1 0 519147122 15552512 608 33554432000 4194304 5683644 140736025783840 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 5 0 0 0
/proc/9941/statm: 3797 609 319 364 0 284 0

[startup+0.0277831 s]
/proc/loadavg: 0.59 0.13 0.04 5/189 9941
/proc/meminfo: memFree=30714728/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=15776 CPUtime=0 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 895 0 0 0 0 0 0 0 24 0 1 0 519147122 16154624 777 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/9941/statm: 3944 777 343 364 0 431 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 15776

[startup+0.100938 s]
/proc/loadavg: 0.59 0.13 0.04 5/189 9941
/proc/meminfo: memFree=30714728/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=21104 CPUtime=0.02 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 1315 0 0 0 2 0 0 0 24 0 1 0 519147122 21610496 1166 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/9941/statm: 5276 1166 428 364 0 723 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 21104

[startup+0.300739 s]
/proc/loadavg: 0.59 0.13 0.04 5/189 9941
/proc/meminfo: memFree=30714728/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=104640 CPUtime=0.07 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 3174 0 0 0 5 2 0 0 17 0 1 0 519147122 107151360 2846 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039653376 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/9941/statm: 26160 2846 847 364 0 1908 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104640

[startup+0.700625 s]
/proc/loadavg: 0.59 0.13 0.04 5/189 9941
/proc/meminfo: memFree=30714728/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=135752 CPUtime=0.21 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 5964 934 1 0 14 7 0 0 16 0 1 0 519147122 139010048 4152 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039651333 0 2147483391 16781312 2 18446744071699199166 0 0 17 7 0 0 0
/proc/9941/statm: 33938 4152 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 135752

[startup+1.50051 s]
/proc/loadavg: 0.94 0.21 0.07 6/190 9946
/proc/meminfo: memFree=30695296/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=285636 CPUtime=0.78 cores=1,3,5,7
/proc/9941/stat : 9941 (python) R 9939 9941 9843 0 -1 4202496 12500 934 1 0 64 14 0 0 16 0 1 0 519147122 292491264 8794 33554432000 4194304 5683644 140736025783840 18446744073709551615 47760132772096 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/9941/statm: 71409 8794 1826 364 0 7333 0
Current children cumulated CPU time (s) 0.78
Current children cumulated vsize (KiB) 285636

[startup+3.10019 s]
/proc/loadavg: 0.94 0.21 0.07 6/190 9946
/proc/meminfo: memFree=30683268/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=321288 CPUtime=2.01 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 15357 934 1 0 187 14 0 0 22 0 5 0 519147122 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9941/statm: 80322 7538 1883 364 0 16231 0
Current children cumulated CPU time (s) 2.01
Current children cumulated vsize (KiB) 321288
heavy processes:

[startup+6.30062 s]
/proc/loadavg: 1.51 0.34 0.11 9/202 9959
/proc/meminfo: memFree=30657776/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=321288 CPUtime=2.01 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 15357 934 1 0 187 14 0 0 22 0 5 0 519147122 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9941/statm: 80322 7538 1883 364 0 16231 0
[pid=9947] ppid=9941 vsize=280760 CPUtime=0 cores=1,3,5,7
/proc/9947/stat : 9947 (python) S 9941 9941 9843 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 519147389 287498240 5978 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675407 0 0 16781312 514 4435068446295687432 0 0 17 7 0 0 0
/proc/9947/statm: 70190 5978 355 364 0 6108 0
[pid=9949] ppid=9947 vsize=1019708 CPUtime=3.62 cores=1,3,5,7
/proc/9949/stat : 9949 (gnovelty+2) R 9947 9949 9949 0 -1 4202496 952 0 0 0 362 0 0 0 25 0 1 0 519147389 1044180992 690 33554432000 134512640 135663704 4290384048 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/9949/statm: 254927 690 113 282 0 254643 0
[pid=9950] ppid=9941 vsize=291624 CPUtime=0 cores=1,3,5,7
/proc/9950/stat : 9950 (python) S 9941 9941 9843 0 -1 4202560 738 0 0 0 0 0 0 0 18 0 1 0 519147391 298622976 5993 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 16652266059805458695 0 0 17 7 0 0 0
/proc/9950/statm: 72906 5993 349 364 0 8821 0
[pid=9952] ppid=9950 vsize=1019708 CPUtime=3.6 cores=1,3,5,7
/proc/9952/stat : 9952 (gnovelty+2) R 9950 9952 9952 0 -1 4202496 949 0 0 0 360 0 0 0 25 0 1 0 519147392 1044180992 689 33554432000 134512640 135663704 4288130304 18446744073709551615 134525101 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9952/statm: 254927 689 113 282 0 254643 0
[pid=9953] ppid=9941 vsize=301340 CPUtime=0 cores=1,3,5,7
/proc/9953/stat : 9953 (python) S 9941 9941 9843 0 -1 4202560 746 0 0 0 0 0 0 0 18 0 1 0 519147394 308572160 5997 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 5629749952281280776 0 0 17 3 0 0 0
/proc/9953/statm: 75335 5997 349 364 0 11247 0
[pid=9955] ppid=9953 vsize=419820 CPUtime=3.57 cores=1,3,5,7
/proc/9955/stat : 9955 (adaptg2wsat2009) R 9953 9955 9955 0 -1 4202496 1437 0 0 0 357 0 0 0 25 0 1 0 519147395 429895680 1172 33554432000 134512640 135107411 4291479456 18446744073709551615 134523319 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/9955/statm: 104955 1172 49 146 0 103831 0
[pid=9956] ppid=9941 vsize=311664 CPUtime=0 cores=1,3,5,7
/proc/9956/stat : 9956 (python) S 9941 9941 9843 0 -1 4202560 745 0 0 0 0 0 0 0 18 0 1 0 519147397 319143936 6001 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 5629757648862675208 0 0 17 7 0 0 0
/proc/9956/statm: 77916 6001 349 364 0 13825 0
[pid=9958] ppid=9956 vsize=419820 CPUtime=3.51 cores=1,3,5,7
/proc/9958/stat : 9958 (adaptg2wsat2009) R 9956 9958 9958 0 -1 4202496 1436 0 0 0 351 0 0 0 25 0 1 0 519147397 429895680 1172 33554432000 134512640 135107411 4289965328 18446744073709551615 134523319 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/9958/statm: 104955 1172 49 146 0 103831 0
[pid=9941/tid=9948] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9948/stat : 9948 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 519147389 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
[pid=9941/tid=9951] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9951/stat : 9951 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 519147392 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 5 0 0 0
[pid=9941/tid=9954] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9954/stat : 9954 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 5 0 519147394 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=9957] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9957/stat : 9957 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 5 0 519147397 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 16.31
Current children cumulated vsize (KiB) 4385732

[startup+12.7015 s]
/proc/loadavg: 2.03 0.47 0.15 9/202 9959
/proc/meminfo: memFree=30654520/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=321288 CPUtime=2.01 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 15357 934 1 0 187 14 0 0 22 0 5 0 519147122 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9941/statm: 80322 7538 1883 364 0 16231 0
[pid=9947] ppid=9941 vsize=280760 CPUtime=0 cores=1,3,5,7
/proc/9947/stat : 9947 (python) S 9941 9941 9843 0 -1 4202560 907 0 0 0 0 0 0 0 18 0 1 0 519147389 287498240 6001 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675407 0 0 16781312 514 4435068446295687432 0 0 17 7 0 0 0
/proc/9947/statm: 70190 6001 378 364 0 6108 0
[pid=9949] ppid=9947 vsize=1019708 CPUtime=10.03 cores=1,3,5,7
/proc/9949/stat : 9949 (gnovelty+2) R 9947 9949 9949 0 -1 4202496 952 0 0 0 1003 0 0 0 25 0 1 0 519147389 1044180992 690 33554432000 134512640 135663704 4290384048 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/9949/statm: 254927 690 113 282 0 254643 0
[pid=9950] ppid=9941 vsize=291624 CPUtime=0 cores=1,3,5,7
/proc/9950/stat : 9950 (python) S 9941 9941 9843 0 -1 4202560 915 0 0 0 0 0 0 0 18 0 1 0 519147391 298622976 6016 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 16652266059805458695 0 0 17 7 0 0 0
/proc/9950/statm: 72906 6016 372 364 0 8821 0
[pid=9952] ppid=9950 vsize=1019708 CPUtime=9.99 cores=1,3,5,7
/proc/9952/stat : 9952 (gnovelty+2) R 9950 9952 9952 0 -1 4202496 949 0 0 0 999 0 0 0 25 0 1 0 519147392 1044180992 689 33554432000 134512640 135663704 4288130304 18446744073709551615 134545631 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9952/statm: 254927 689 113 282 0 254643 0
[pid=9953] ppid=9941 vsize=301340 CPUtime=0 cores=1,3,5,7
/proc/9953/stat : 9953 (python) S 9941 9941 9843 0 -1 4202560 918 0 0 0 0 0 0 0 18 0 1 0 519147394 308572160 6019 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 5629749952281280776 0 0 17 3 0 0 0
/proc/9953/statm: 75335 6019 371 364 0 11247 0
[pid=9955] ppid=9953 vsize=420220 CPUtime=9.97 cores=1,3,5,7
/proc/9955/stat : 9955 (adaptg2wsat2009) R 9953 9955 9955 0 -1 4202496 1665 0 0 0 997 0 0 0 25 0 1 0 519147395 430305280 1400 33554432000 134512640 135107411 4291479456 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/9955/statm: 105055 1400 59 146 0 103931 0
[pid=9956] ppid=9941 vsize=311664 CPUtime=0 cores=1,3,5,7
/proc/9956/stat : 9956 (python) S 9941 9941 9843 0 -1 4202560 920 0 0 0 0 0 0 0 18 0 1 0 519147397 319143936 6024 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 5629757648862675208 0 0 17 7 0 0 0
/proc/9956/statm: 77916 6024 372 364 0 13825 0
[pid=9958] ppid=9956 vsize=420220 CPUtime=9.85 cores=1,3,5,7
/proc/9958/stat : 9958 (adaptg2wsat2009) R 9956 9958 9958 0 -1 4202496 1663 0 0 0 985 0 0 0 25 0 1 0 519147397 430305280 1399 33554432000 134512640 135107411 4289965328 18446744073709551615 134532264 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/9958/statm: 105055 1399 59 146 0 103931 0
[pid=9941/tid=9948] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9948/stat : 9948 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 519147389 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
[pid=9941/tid=9951] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9951/stat : 9951 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 519147392 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 5 0 0 0
[pid=9941/tid=9954] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9954/stat : 9954 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 5 0 519147394 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=9957] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9957/stat : 9957 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 5 0 519147397 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 41.85
Current children cumulated vsize (KiB) 4386532

[startup+25.5012 s]
/proc/loadavg: 2.95 0.71 0.24 9/202 9959
/proc/meminfo: memFree=30655504/32951124 swapFree=67106388/67111528
[pid=9941] ppid=9939 vsize=321288 CPUtime=2.01 cores=1,3,5,7
/proc/9941/stat : 9941 (python) S 9939 9941 9843 0 -1 4202496 15357 934 1 0 187 14 0 0 22 0 5 0 519147122 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/9941/statm: 80322 7538 1883 364 0 16231 0
[pid=9947] ppid=9941 vsize=280760 CPUtime=0.02 cores=1,3,5,7
/proc/9947/stat : 9947 (python) S 9941 9941 9843 0 -1 4202560 919 0 0 0 1 1 0 0 18 0 1 0 519147389 287498240 6001 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675407 0 0 16781312 514 195026974488363272 0 0 17 7 0 0 0
/proc/9947/statm: 70190 6001 378 364 0 6108 0
[pid=9949] ppid=9947 vsize=1019708 CPUtime=22.83 cores=1,3,5,7
/proc/9949/stat : 9949 (gnovelty+2) R 9947 9949 9949 0 -1 4202496 952 0 0 0 2283 0 0 0 25 0 1 0 519147389 1044180992 690 33554432000 134512640 135663704 4290384048 18446744073709551615 134524566 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/9949/statm: 254927 690 113 282 0 254643 0
[pid=9950] ppid=9941 vsize=291624 CPUtime=0.02 cores=1,3,5,7
/proc/9950/stat : 9950 (python) S 9941 9941 9843 0 -1 4202560 921 0 0 0 1 1 0 0 18 0 1 0 519147391 298622976 6016 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 195026974488363271 0 0 17 7 0 0 0
/proc/9950/statm: 72906 6016 372 364 0 8821 0
[pid=9952] ppid=9950 vsize=1019708 CPUtime=22.77 cores=1,3,5,7
/proc/9952/stat : 9952 (gnovelty+2) R 9950 9952 9952 0 -1 4202496 949 0 0 0 2277 0 0 0 25 0 1 0 519147392 1044180992 689 33554432000 134512640 135663704 4288130304 18446744073709551615 134525271 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/9952/statm: 254927 689 113 282 0 254643 0
[pid=9953] ppid=9941 vsize=301340 CPUtime=0.02 cores=1,3,5,7
/proc/9953/stat : 9953 (python) S 9941 9941 9843 0 -1 4202560 925 0 0 0 1 1 0 0 18 0 1 0 519147394 308572160 6020 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 195026974488363272 0 0 17 3 0 0 0
/proc/9953/statm: 75335 6020 372 364 0 11247 0
[pid=9955] ppid=9953 vsize=420220 CPUtime=22.77 cores=1,3,5,7
/proc/9955/stat : 9955 (adaptg2wsat2009) R 9953 9955 9955 0 -1 4202496 1665 0 0 0 2277 0 0 0 25 0 1 0 519147395 430305280 1400 33554432000 134512640 135107411 4291479456 18446744073709551615 134532293 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/9955/statm: 105055 1400 59 146 0 103931 0
[pid=9956] ppid=9941 vsize=311664 CPUtime=0.03 cores=1,3,5,7
/proc/9956/stat : 9956 (python) S 9941 9941 9843 0 -1 4202560 928 0 0 0 2 1 0 0 18 0 1 0 519147397 319143936 6024 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 195026974488363272 0 0 17 7 0 0 0
/proc/9956/statm: 77916 6024 372 364 0 13825 0
[pid=9958] ppid=9956 vsize=420220 CPUtime=22.5 cores=1,3,5,7
/proc/9958/stat : 9958 (adaptg2wsat2009) R 9956 9958 9958 0 -1 4202496 1663 0 0 0 2250 0 0 0 25 0 1 0 519147397 430305280 1399 33554432000 134512640 135107411 4289965328 18446744073709551615 134531817 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/9958/statm: 105055 1399 59 146 0 103931 0
[pid=9941/tid=9948] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9948/stat : 9948 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 5 0 519147389 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
[pid=9941/tid=9951] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9951/stat : 9951 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 5 0 519147392 328998912 7538 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 5 0 0 0
[pid=9941/tid=9954] ppid=9939 vsize=321288 CPUtime=0 cores=1,3,5,7

################
# More data... #
################

/proc/9953/task/9979/stat : 9979 (python) S 9941 9941 9843 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 519157407 308572160 6049 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604454223471552 0 0 -1 3 0 0 0
[pid=9956] ppid=9941 vsize=311664 CPUtime=0.13 cores=1,3,5,7
/proc/9956/stat : 9956 (python) S 9941 9941 9843 0 -1 4202560 1107 0 0 0 8 5 0 0 19 0 2 0 519147397 319143936 6053 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9956/statm: 77916 6053 400 364 0 13825 0
[pid=9958] ppid=9956 vsize=420220 CPUtime=99.17 cores=1,3,5,7
/proc/9958/stat : 9958 (adaptg2wsat2009) T 9956 9958 9958 0 -1 4202496 1663 0 0 0 9917 0 0 0 25 0 1 0 519147397 430305280 1399 33554432000 134512640 135107411 4289965328 18446744073709551615 134539338 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/9958/statm: 105055 1399 59 146 0 103931 0
[pid=9956/tid=9986] ppid=9941 vsize=311664 CPUtime=0 cores=1,3,5,7
/proc/9956/task/9986/stat : 9986 (python) S 9941 9941 9843 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519157412 319143936 6053 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604454223473344 0 0 -1 7 0 0 0
[pid=9980] ppid=9941 vsize=321784 CPUtime=0.14 cores=1,3,5,7
/proc/9980/stat : 9980 (python) S 9941 9941 9843 0 -1 4202560 1122 0 0 0 7 7 0 0 19 0 2 0 519157408 329506816 6074 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9980/statm: 80446 6074 400 364 0 16352 0
[pid=9983] ppid=9980 vsize=420220 CPUtime=98.99 cores=1,3,5,7
/proc/9983/stat : 9983 (adaptg2wsat2009) T 9980 9983 9983 0 -1 4202496 1665 0 0 0 9899 0 0 0 25 0 1 0 519157409 430305280 1399 33554432000 134512640 135107411 4288392272 18446744073709551615 134531935 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/9983/statm: 105055 1399 59 146 0 103931 0
[pid=9980/tid=10013] ppid=9941 vsize=321784 CPUtime=0 cores=1,3,5,7
/proc/9980/task/10013/stat : 10013 (python) S 9941 9941 9843 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 519167322 329506816 6074 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604471134303232 0 0 -1 7 0 0 0
[pid=9984] ppid=9941 vsize=332648 CPUtime=0.13 cores=1,3,5,7
/proc/9984/stat : 9984 (python) S 9941 9941 9843 0 -1 4202560 1132 0 0 0 6 7 0 0 19 0 2 0 519157412 340631552 6078 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/9984/statm: 83162 6078 400 364 0 19065 0
[pid=9987] ppid=9984 vsize=1019708 CPUtime=98.36 cores=1,3,5,7
/proc/9987/stat : 9987 (gnovelty+2) T 9984 9987 9987 0 -1 4202496 953 0 0 0 9836 0 0 0 25 0 1 0 519157412 1044180992 689 33554432000 134512640 135663704 4290503104 18446744073709551615 134545583 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/9987/statm: 254927 689 113 282 0 254643 0
[pid=9984/tid=10018] ppid=9941 vsize=332648 CPUtime=0 cores=1,3,5,7
/proc/9984/task/10018/stat : 10018 (python) S 9941 9941 9843 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519167430 340631552 6078 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604462625885184 0 0 -1 1 0 0 0
[pid=9988] ppid=9941 vsize=342420 CPUtime=0.13 cores=1,3,5,7
/proc/9988/stat : 9988 (python) S 9941 9941 9843 0 -1 4202560 1179 0 0 0 6 7 0 0 19 0 2 0 519157415 350638080 6205 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/9988/statm: 85605 6205 405 364 0 21505 0
[pid=9990] ppid=9988 vsize=249872 CPUtime=99.99 cores=1,3,5,7
/proc/9990/stat : 9990 (FH) T 9988 9990 9990 0 -1 4202496 1529 0 0 0 9999 0 0 0 25 0 1 0 519157416 255868928 1263 33554432000 134512640 134936988 4286913344 18446744073709551615 134554083 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/9990/statm: 62468 1263 41 104 0 61972 0
[pid=9988/tid=10017] ppid=9941 vsize=342420 CPUtime=0 cores=1,3,5,7
/proc/9988/task/10017/stat : 10017 (python) S 9941 9941 9843 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519167429 350638080 6205 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604462625884288 0 0 -1 7 0 0 0
[pid=9991] ppid=9941 vsize=352888 CPUtime=0.14 cores=1,3,5,7
/proc/9991/stat : 9991 (python) S 9941 9941 9843 0 -1 4202560 1167 0 0 0 8 6 0 0 19 0 2 0 519157419 361357312 6210 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/9991/statm: 88222 6210 406 364 0 24119 0
[pid=9993] ppid=9991 vsize=420220 CPUtime=100.12 cores=1,3,5,7
/proc/9993/stat : 9993 (adaptg2wsat2009) T 9991 9993 9993 0 -1 4202496 1669 0 0 0 10012 0 0 0 25 0 1 0 519157419 430305280 1399 33554432000 134512640 135107411 4289739440 18446744073709551615 134531942 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/9993/statm: 105055 1399 59 146 0 103931 0
[pid=9991/tid=10019] ppid=9941 vsize=352888 CPUtime=0 cores=1,3,5,7
/proc/9991/task/10019/stat : 10019 (python) S 9941 9941 9843 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519167432 361357312 6210 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604462625883392 0 0 -1 1 0 0 0
[pid=10014] ppid=9941 vsize=363252 CPUtime=0.15 cores=1,3,5,7
/proc/10014/stat : 10014 (python) S 9941 9941 9843 0 -1 4202560 1158 0 0 0 7 8 0 0 19 0 2 0 519167326 371970048 6208 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10014/statm: 90813 6208 400 364 0 26707 0
[pid=10016] ppid=10014 vsize=420220 CPUtime=99.68 cores=1,3,5,7
/proc/10016/stat : 10016 (adaptg2wsat2009) T 10014 10016 10016 0 -1 4202496 1669 0 0 0 9968 0 0 0 25 0 1 0 519167326 430305280 1399 33554432000 134512640 135107411 4290563456 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10016/statm: 105055 1399 59 146 0 103931 0
[pid=10014/tid=10043] ppid=9941 vsize=363252 CPUtime=0 cores=1,3,5,7
/proc/10014/task/10043/stat : 10043 (python) S 9941 9941 9843 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519177340 371970048 6208 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604462625882496 0 0 -1 3 0 0 0
[pid=10020] ppid=9941 vsize=373468 CPUtime=0.15 cores=1,3,5,7
/proc/10020/stat : 10020 (python) S 9941 9941 9843 0 -1 4202560 1171 0 0 0 6 9 0 0 19 0 2 0 519167433 382431232 6212 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10020/statm: 93367 6212 400 364 0 29258 0
[pid=10022] ppid=10020 vsize=249868 CPUtime=98.35 cores=1,3,5,7
/proc/10022/stat : 10022 (FH) T 10020 10022 10022 0 -1 4202496 1533 0 0 0 9835 0 0 0 25 0 1 0 519167433 255864832 1263 33554432000 134512640 134936988 4290416384 18446744073709551615 134552080 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/10022/statm: 62467 1263 41 104 0 61971 0
[pid=10020/tid=10047] ppid=9941 vsize=373468 CPUtime=0 cores=1,3,5,7
/proc/10020/task/10047/stat : 10047 (python) S 9941 9941 9843 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519177447 382431232 6212 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604471160826752 0 0 -1 7 0 0 0
[pid=10023] ppid=9941 vsize=383964 CPUtime=0.48 cores=1,3,5,7
/proc/10023/stat : 10023 (python) S 9941 9941 9843 0 -1 4202560 1188 0 0 0 24 24 0 0 18 0 2 0 519167437 393179136 6216 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/10023/statm: 95991 6216 400 364 0 31879 0
[pid=10025] ppid=10023 vsize=268420 CPUtime=296.49 cores=1,3,5,7
/proc/10025/stat : 10025 (hybridGM3) T 10023 10025 10025 0 -1 4202496 5395 0 0 0 29649 0 0 0 25 0 1 0 519167437 274862080 5126 33554432000 134512640 135204385 4288166720 18446744073709551615 134538041 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10025/statm: 67105 5126 63 169 0 66935 0
[pid=10023/tid=10063] ppid=9941 vsize=383964 CPUtime=0 cores=1,3,5,7
/proc/10023/task/10063/stat : 10063 (python) S 9941 9941 9843 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 519186973 393179136 6216 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604467924822784 0 0 -1 7 0 0 0
[pid=10026] ppid=9941 vsize=393844 CPUtime=0.15 cores=1,3,5,7
/proc/10026/stat : 10026 (python) S 9941 9941 9843 0 -1 4202560 1175 0 0 0 8 7 0 0 19 0 2 0 519167442 403296256 6229 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10026/statm: 98461 6229 400 364 0 34346 0
[pid=10028] ppid=10026 vsize=1019708 CPUtime=100.16 cores=1,3,5,7
/proc/10028/stat : 10028 (gnovelty+2) T 10026 10028 10028 0 -1 4202496 964 0 0 0 10016 0 0 0 25 0 1 0 519167442 1044180992 689 33554432000 134512640 135663704 4287691456 18446744073709551615 134525157 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/10028/statm: 254927 689 113 282 0 254643 0
[pid=10026/tid=10051] ppid=9941 vsize=393844 CPUtime=0 cores=1,3,5,7
/proc/10026/task/10051/stat : 10051 (python) S 9941 9941 9843 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519177460 403296256 6229 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604464450978944 0 0 -1 3 0 0 0
[pid=10044] ppid=9941 vsize=404708 CPUtime=0.32 cores=1,3,5,7
/proc/10044/stat : 10044 (python) S 9941 9941 9843 0 -1 4202560 1030 0 0 0 18 14 0 0 18 0 1 0 519177344 414420992 6204 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 195026974488363272 0 0 17 3 0 0 0
/proc/10044/statm: 101177 6204 372 364 0 37059 0
[pid=10046] ppid=10044 vsize=268420 CPUtime=193.49 cores=1,3,5,7
/proc/10046/stat : 10046 (hybridGM3) R 10044 10046 10046 0 -1 4202496 5399 0 0 0 19349 0 0 0 25 0 1 0 519177345 274862080 5125 33554432000 134512640 135204385 4292345888 18446744073709551615 134537717 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/10046/statm: 67105 5125 63 169 0 66935 0
[pid=10048] ppid=9941 vsize=414772 CPUtime=0.16 cores=1,3,5,7
/proc/10048/stat : 10048 (python) S 9941 9941 9843 0 -1 4202560 1180 0 0 0 9 7 0 0 19 0 2 0 519177452 424726528 6295 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10048/statm: 103693 6295 400 364 0 39572 0
[pid=10050] ppid=10048 vsize=420220 CPUtime=99.67 cores=1,3,5,7
/proc/10050/stat : 10050 (adaptg2wsat2009) T 10048 10050 10050 0 -1 4202496 1680 0 0 0 9967 0 0 0 25 0 1 0 519177452 430305280 1399 33554432000 134512640 135107411 4291766944 18446744073709551615 134532135 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/10050/statm: 105055 1399 59 146 0 103931 0
[pid=10048/tid=10064] ppid=9941 vsize=414772 CPUtime=0 cores=1,3,5,7
/proc/10048/task/10064/stat : 10064 (python) S 9941 9941 9843 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519187468 424726528 6295 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604471138062528 0 0 -1 1 0 0 0
[pid=10052] ppid=9941 vsize=424920 CPUtime=0.32 cores=1,3,5,7
/proc/10052/stat : 10052 (python) S 9941 9941 9843 0 -1 4202560 1234 0 0 0 16 16 0 0 19 0 2 0 519177465 435118080 6307 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051482059 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/10052/statm: 106230 6307 408 364 0 42106 0
[pid=10054] ppid=10052 vsize=268424 CPUtime=196.36 cores=1,3,5,7
/proc/10054/stat : 10054 (hybridGM3) T 10052 10054 10054 0 -1 4202496 5405 0 0 0 19636 0 0 0 25 0 1 0 519177465 274866176 5127 33554432000 134512640 135204385 4289069568 18446744073709551615 134537776 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/10054/statm: 67106 5127 63 169 0 66936 0
[pid=10052/tid=10117] ppid=9941 vsize=424920 CPUtime=0 cores=1,3,5,7
/proc/10052/task/10117/stat : 10117 (python) S 9941 9941 9843 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519197101 435118080 6307 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 514 18446604460672145216 0 0 -1 3 0 0 0
[pid=10065] ppid=9941 vsize=435272 CPUtime=0.15 cores=1,3,5,7
/proc/10065/stat : 10065 (python) S 9941 9941 9843 0 -1 4202560 1074 0 0 0 8 7 0 0 18 0 1 0 519187472 445718528 6282 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 195026974488363270 0 0 17 3 0 0 0
/proc/10065/statm: 108818 6282 380 364 0 44691 0
[pid=10067] ppid=10065 vsize=268420 CPUtime=96.42 cores=1,3,5,7
/proc/10067/stat : 10067 (hybridGM3) R 10065 10067 10067 0 -1 4202496 5404 0 0 0 9642 0 0 0 25 0 1 0 519187473 274862080 5126 33554432000 134512640 135204385 4291072464 18446744073709551615 134538264 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/10067/statm: 67105 5126 63 169 0 66935 0
[pid=10114] ppid=9941 vsize=445948 CPUtime=0 cores=1,3,5,7
/proc/10114/stat : 10114 (python) S 9941 9941 9843 0 -1 4202560 821 0 0 0 0 0 0 0 19 0 1 0 519197099 456650752 6253 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 9136365487020572671 0 0 17 3 0 0 0
/proc/10114/statm: 111487 6253 347 364 0 47357 0
[pid=10116] ppid=10114 vsize=454088 CPUtime=0.63 cores=1,3,5,7
/proc/10116/stat : 10116 (TNM) R 10114 10116 10116 0 -1 4202496 1272 0 0 0 63 0 0 0 22 0 1 0 519197099 464986112 996 33554432000 134512640 135008748 4292646464 18446744073709551615 134523511 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/10116/statm: 113522 996 45 122 0 112422 0
[pid=10118] ppid=9941 vsize=456212 CPUtime=0 cores=1,3,5,7
/proc/10118/stat : 10118 (python) S 9941 9941 9843 0 -1 4202560 824 0 0 0 0 0 0 0 20 0 1 0 519197106 467161088 6256 33554432000 4194304 5683644 140736025783840 18446744073709551615 212039675462 0 0 16781312 514 9136369335311269887 0 0 17 3 0 0 0
/proc/10118/statm: 114053 6256 346 364 0 49920 0
[pid=10120] ppid=10118 vsize=247956 CPUtime=0.56 cores=1,3,5,7
/proc/10120/stat : 10120 (FH) R 10118 10120 10120 0 -1 4202496 1259 0 0 0 56 0 0 0 23 0 1 0 519197107 253906944 987 33554432000 134512640 134936988 4290721888 18446744073709551615 134523434 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/10120/statm: 61989 987 37 104 0 61493 0
[pid=9941/tid=9948] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9948/stat : 9948 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 19 0 19 0 519147389 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
[pid=9941/tid=9951] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9951/stat : 9951 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 19 0 519147392 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 5 0 0 0
[pid=9941/tid=9954] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9954/stat : 9954 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 19 0 519147394 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=9957] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9957/stat : 9957 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 22 0 19 0 519147397 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
[pid=9941/tid=9981] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9981/stat : 9981 (python) S 9939 9941 9843 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 519157408 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=9985] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9985/stat : 9985 (python) S 9939 9941 9843 0 -1 4202560 23 934 0 0 0 0 0 0 22 0 19 0 519157412 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 1 0 0 0
[pid=9941/tid=9989] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9989/stat : 9989 (python) S 9939 9941 9843 0 -1 4202560 24 934 0 0 0 0 0 0 20 0 19 0 519157415 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
[pid=9941/tid=9992] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/9992/stat : 9992 (python) S 9939 9941 9843 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 519157419 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10015] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10015/stat : 10015 (python) S 9939 9941 9843 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 19 0 519167326 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10021] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10021/stat : 10021 (python) S 9939 9941 9843 0 -1 4202560 23 934 0 0 0 0 0 0 20 0 19 0 519167433 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10024] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10024/stat : 10024 (python) S 9939 9941 9843 0 -1 4202560 29 934 0 0 0 0 0 0 18 0 19 0 519167437 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 7 0 0 0
[pid=9941/tid=10027] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10027/stat : 10027 (python) S 9939 9941 9843 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 519167442 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10045] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10045/stat : 10045 (python) S 9939 9941 9843 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 19 0 519177345 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10049] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10049/stat : 10049 (python) S 9939 9941 9843 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 19 0 519177452 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10053] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10053/stat : 10053 (python) S 9939 9941 9843 0 -1 4202560 27 934 0 0 0 0 0 0 21 0 19 0 519177465 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10066] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10066/stat : 10066 (python) S 9939 9941 9843 0 -1 4202560 29 934 0 0 0 0 0 0 20 0 19 0 519187473 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10115] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10115/stat : 10115 (python) S 9939 9941 9843 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 519197099 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
[pid=9941/tid=10119] ppid=9939 vsize=465748 CPUtime=0 cores=1,3,5,7
/proc/9941/task/10119/stat : 10119 (python) S 9939 9941 9843 0 -1 4202560 25 934 0 0 0 0 0 0 21 0 19 0 519197107 476925952 7825 33554432000 4194304 5683644 140736025783840 18446744073709551615 212051478961 0 0 16781312 2 18446604471134305024 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 1984.32
Current children cumulated vsize (KiB) 15983100

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 9941 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=511618
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=190970
# CPU time returned by wait4() is 2.70259
# while last known CPU time is 1984.32
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 500.517
CPU time (s): 1984.32
CPU user time (s): 1982.7
CPU system time (s): 1.62
CPU usage (%): 396.454
Max. virtual memory (cumulated for all children) (KiB): 15983100

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.51162
system time used= 0.19097
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30207
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2905
involuntary context switches= 66

runsolver used 2.50562 second user time and 5.56615 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-01 18:28:28
IDJOB=3317098
IDBENCH=82563
IDSOLVER=1705
FILE ID=node146/3317098-1304267308
RUNJOBID= node146-1304267279-9862
PBS_JOBID= 13173933
Free space on /tmp= 73340 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S926861876-013-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317098-1304267308/watcher-3317098-1304267308 -o /tmp/evaluation-result-3317098-1304267308/solver-3317098-1304267308 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317098-1304267308.cnf -seed 1621517985 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= aa5d2fab702f701bf63e2d26e76ab2b3
RANDOM SEED=1621517985

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      30715008 kB
Buffers:        461004 kB
Cached:        1275500 kB
SwapCached:        888 kB
Active:         333856 kB
Inactive:      1519960 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      30715008 kB
SwapTotal:    67111528 kB
SwapFree:     67106388 kB
Dirty:           66120 kB
Writeback:           0 kB
AnonPages:      116424 kB
Mapped:          21188 kB
Slab:           317056 kB
PageTables:       7516 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  3325396 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73300 MiB
End job on node146 at 2011-05-01 18:36:51