Trace number 3317195

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 1999.61 503.92

General information on the benchmark

Namerandom/large/
unif-k7-r85-v300-c25500-S69399346-065-UNKNOWN.cnf
MD5SUM98486271f2f72889c040eabfd36ebd13
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of clauses25500
Sum of the clauses size178500
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 525500

Solver Data 

0.35/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.67	c INFO: solving HOME/instance-3317195-1304268093.cnf
0.35/0.67	c INFO: computing task features
0.52/0.84	c INFO: computed variable-clause graph statistics
0.52/0.85	c INFO: computed clause balance statistics
1.42/1.76	c INFO: computed variable balance statistics
1.82/2.11	c INFO: computed small-clause counts
1.82/2.12	c INFO: computed Horn-clause counts
1.82/2.15	c INFO: computed variable graph statistics
1.82/2.15	c INFO: collected features for HOME/instance-3317195-1304268093.cnf in 1.47 s
1.82/2.18	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.37)
1.95/2.21	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.13)
1.95/2.24	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.08)
1.95/2.26	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.07)
397.89/101.45	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.05)
401.51/102.38	c INFO: running gnovelty+2@0 for 97 with 4698 remaining (b = 0.04)
401.88/102.43	c INFO: running FH@0 for 97 with 4698 remaining (b = 0.03)
401.88/102.47	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
793.01/200.63	c INFO: running adaptg2wsat2009++@0 for 97 with 4600 remaining (b = 0.02)
796.99/201.61	c INFO: running FH@0 for 97 with 4599 remaining (b = 0.02)
796.99/201.66	c INFO: running hybridGM3@0 for 194 with 4599 remaining (b = 0.04)
800.96/202.63	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1192.13/300.82	c INFO: running hybridGM3@0 for 194 with 4499 remaining (b = 0.03)
1196.08/301.81	c INFO: running adaptg2wsat2009++@0 for 97 with 4498 remaining (b = 0.01)
1196.08/301.89	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1579.15/398.05	c INFO: running hybridGM3@100 for 97 with 4402 remaining (b = 0.01)
1595.07/402.02	c INFO: running hybridGM3@0 for 194 with 4398 remaining (b = 0.02)
1957.43/493.26	c INFO: running TNM@0 for 1552 with 4307 remaining (b = 0.19)
1969.40/496.26	c INFO: running hybridGM3@99 for 97 with 4304 remaining (b = 0.01)
1997.58/503.33	c INFO: running FH@0 for 97 with 4297 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317195-1304268093/watcher-3317195-1304268093 -o /tmp/evaluation-result-3317195-1304268093/solver-3317195-1304268093 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317195-1304268093.cnf -seed 1264031527 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.77 7.51 4.62 5/242 7491
/proc/meminfo: memFree=29517864/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=15380 CPUtime=0 cores=1,3,5,7
/proc/7491/stat : 7491 (python) S 7489 7491 7090 0 -1 4202496 792 0 0 0 0 0 0 0 17 0 1 0 617039927 15749120 679 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020393984 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/7491/statm: 3845 679 322 364 0 332 0

[startup+0.102109 s]
/proc/loadavg: 7.77 7.51 4.62 5/242 7491
/proc/meminfo: memFree=29517864/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=91832 CPUtime=0.06 cores=1,3,5,7
/proc/7491/stat : 7491 (python) R 7489 7491 7090 0 -1 4202496 2675 0 0 0 5 1 0 0 17 0 1 0 617039927 94035968 2390 33554432000 4194304 5683644 140737050332144 18446744073709551615 260015472311 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7491/statm: 22958 2392 753 364 0 1623 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 91832

[startup+0.201105 s]
/proc/loadavg: 7.77 7.51 4.62 5/242 7491
/proc/meminfo: memFree=29517864/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=124236 CPUtime=0.12 cores=1,3,5,7
/proc/7491/stat : 7491 (python) S 7489 7491 7090 0 -1 4202496 4265 0 1 0 9 3 0 0 17 0 1 0 617039927 127217664 3872 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020393984 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/7491/statm: 31059 3872 996 364 0 2870 0
Current children cumulated CPU time (s) 0.12
Current children cumulated vsize (KiB) 124236

[startup+0.301091 s]
/proc/loadavg: 7.77 7.51 4.62 5/242 7491
/proc/meminfo: memFree=29517864/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=135752 CPUtime=0.18 cores=1,3,5,7
/proc/7491/stat : 7491 (python) S 7489 7491 7090 0 -1 4202496 5966 934 1 0 14 4 0 0 17 0 1 0 617039927 139010048 4151 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020391941 0 2147483391 16781312 2 18446744071699199166 0 0 17 3 0 0 0
/proc/7491/statm: 33938 4151 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.18
Current children cumulated vsize (KiB) 135752

[startup+0.701005 s]
/proc/loadavg: 7.77 7.51 4.62 5/242 7491
/proc/meminfo: memFree=29517864/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=279712 CPUtime=0.42 cores=1,3,5,7
/proc/7491/stat : 7491 (python) R 7489 7491 7090 0 -1 4202496 9583 934 1 0 32 10 0 0 15 0 1 0 617039927 286425088 7264 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020082387 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7491/statm: 69928 7264 1761 364 0 5852 0
Current children cumulated CPU time (s) 0.42
Current children cumulated vsize (KiB) 279712

[startup+1.50086 s]
/proc/loadavg: 7.77 7.51 4.62 6/243 7496
/proc/meminfo: memFree=29488220/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=285636 CPUtime=1.22 cores=1,3,5,7
/proc/7491/stat : 7491 (python) R 7489 7491 7090 0 -1 4202496 12502 934 1 0 112 10 0 0 16 0 1 0 617039927 292491264 8793 33554432000 4194304 5683644 140737050332144 18446744073709551615 47504960679668 0 0 16781312 2 0 0 0 17 3 0 0 0
/proc/7491/statm: 71409 8793 1826 364 0 7333 0
Current children cumulated CPU time (s) 1.22
Current children cumulated vsize (KiB) 285636

[startup+3.10053 s]
/proc/loadavg: 7.77 7.51 4.62 7/246 7499
/proc/meminfo: memFree=29489808/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=321292 CPUtime=1.98 cores=1,3,5,7
/proc/7491/stat : 7491 (python) S 7489 7491 7090 0 -1 4202496 15370 934 1 0 187 11 0 0 19 0 5 0 617039927 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7491/statm: 80323 7537 1883 364 0 16232 0
[pid=7497] ppid=7491 vsize=280764 CPUtime=0 cores=1,3,5,7
/proc/7497/stat : 7497 (python) S 7491 7491 7090 0 -1 4202560 735 0 0 0 0 0 0 0 18 0 1 0 617040145 287502336 5978 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416015 0 0 16781312 514 4411100467199737855 0 0 17 3 0 0 0
/proc/7497/statm: 70191 5978 355 364 0 6109 0
[pid=7499] ppid=7497 vsize=1019708 CPUtime=0.91 cores=1,3,5,7
/proc/7499/stat : 7499 (gnovelty+2) R 7497 7499 7499 0 -1 4202496 954 0 0 0 91 0 0 0 21 0 1 0 617040145 1044180992 689 33554432000 134512640 135663704 4288951120 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7499/statm: 254927 689 113 282 0 254643 0
[pid=7491/tid=7498] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7498/stat : 7498 (python) S 7489 7491 7090 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 5 0 617040145 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2.89
Current children cumulated vsize (KiB) 1621764

[startup+6.30095 s]
/proc/loadavg: 7.87 7.53 4.64 9/255 7508
/proc/meminfo: memFree=29464512/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=321292 CPUtime=1.98 cores=1,3,5,7
/proc/7491/stat : 7491 (python) S 7489 7491 7090 0 -1 4202496 15370 934 1 0 187 11 0 0 19 0 5 0 617039927 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7491/statm: 80323 7537 1883 364 0 16232 0
[pid=7497] ppid=7491 vsize=280764 CPUtime=0 cores=1,3,5,7
/proc/7497/stat : 7497 (python) S 7491 7491 7090 0 -1 4202560 897 0 0 0 0 0 0 0 18 0 1 0 617040145 287502336 6001 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416015 0 0 16781312 514 4411100467199705348 0 0 17 3 0 0 0
/proc/7497/statm: 70191 6001 378 364 0 6109 0
[pid=7499] ppid=7497 vsize=1019708 CPUtime=4.1 cores=1,3,5,7
/proc/7499/stat : 7499 (gnovelty+2) R 7497 7499 7499 0 -1 4202496 954 0 0 0 410 0 0 0 25 0 1 0 617040145 1044180992 689 33554432000 134512640 135663704 4288951120 18446744073709551615 134525678 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7499/statm: 254927 689 113 282 0 254643 0
[pid=7500] ppid=7491 vsize=291084 CPUtime=0 cores=1,3,5,7
/proc/7500/stat : 7500 (python) S 7491 7491 7090 0 -1 4202560 902 0 0 0 0 0 0 0 18 0 1 0 617040148 298070016 5999 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 4444811768585224456 0 0 17 7 0 0 0
/proc/7500/statm: 72771 5999 372 364 0 8686 0
[pid=7502] ppid=7500 vsize=1019708 CPUtime=4.04 cores=1,3,5,7
/proc/7502/stat : 7502 (gnovelty+2) R 7500 7502 7502 0 -1 4202496 953 0 0 0 404 0 0 0 25 0 1 0 617040148 1044180992 690 33554432000 134512640 135663704 4294198944 18446744073709551615 134525286 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7502/statm: 254927 690 113 282 0 254643 0
[pid=7503] ppid=7491 vsize=301276 CPUtime=0 cores=1,3,5,7
/proc/7503/stat : 7503 (python) S 7491 7491 7090 0 -1 4202560 897 0 0 0 0 0 0 0 18 0 1 0 617040151 308506624 6019 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 1692078030698021126 0 0 17 3 0 0 0
/proc/7503/statm: 75319 6019 372 364 0 11231 0
[pid=7505] ppid=7503 vsize=420220 CPUtime=4.03 cores=1,3,5,7
/proc/7505/stat : 7505 (adaptg2wsat2009) R 7503 7505 7505 0 -1 4202496 1666 0 0 0 403 0 0 0 25 0 1 0 617040151 430305280 1400 33554432000 134512640 135107411 4288772144 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7505/statm: 105055 1400 59 146 0 103931 0
[pid=7506] ppid=7491 vsize=311596 CPUtime=0 cores=1,3,5,7
/proc/7506/stat : 7506 (python) S 7491 7491 7090 0 -1 4202560 916 0 0 0 0 0 0 0 18 0 1 0 617040154 319074304 6022 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 7870154152578351366 0 0 17 7 0 0 0
/proc/7506/statm: 77899 6022 371 364 0 13808 0
[pid=7508] ppid=7506 vsize=420220 CPUtime=4.02 cores=1,3,5,7
/proc/7508/stat : 7508 (adaptg2wsat2009) R 7506 7508 7508 0 -1 4202496 1666 0 0 0 402 0 0 0 25 0 1 0 617040154 430305280 1400 33554432000 134512640 135107411 4287260464 18446744073709551615 134531582 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7508/statm: 105055 1400 59 146 0 103931 0
[pid=7491/tid=7498] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7498/stat : 7498 (python) S 7489 7491 7090 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 5 0 617040145 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 3 0 0 0
[pid=7491/tid=7501] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7501/stat : 7501 (python) S 7489 7491 7090 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 5 0 617040148 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
[pid=7491/tid=7504] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7504/stat : 7504 (python) S 7489 7491 7090 0 -1 4202560 22 934 0 0 0 0 0 0 23 0 5 0 617040151 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 3 0 0 0
[pid=7491/tid=7507] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7507/stat : 7507 (python) S 7489 7491 7090 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 5 0 617040154 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 18.17
Current children cumulated vsize (KiB) 4385868

[startup+12.7008 s]
/proc/loadavg: 7.88 7.54 4.66 9/255 7508
/proc/meminfo: memFree=29461048/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=321292 CPUtime=1.98 cores=1,3,5,7
/proc/7491/stat : 7491 (python) S 7489 7491 7090 0 -1 4202496 15370 934 1 0 187 11 0 0 19 0 5 0 617039927 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7491/statm: 80323 7537 1883 364 0 16232 0
[pid=7497] ppid=7491 vsize=280764 CPUtime=0 cores=1,3,5,7
/proc/7497/stat : 7497 (python) S 7491 7491 7090 0 -1 4202560 901 0 0 0 0 0 0 0 18 0 1 0 617040145 287502336 6001 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416015 0 0 16781312 514 11031954315336974596 0 0 17 3 0 0 0
/proc/7497/statm: 70191 6001 378 364 0 6109 0
[pid=7499] ppid=7497 vsize=1019708 CPUtime=10.5 cores=1,3,5,7
/proc/7499/stat : 7499 (gnovelty+2) R 7497 7499 7499 0 -1 4202496 954 0 0 0 1050 0 0 0 25 0 1 0 617040145 1044180992 689 33554432000 134512640 135663704 4288951120 18446744073709551615 134525678 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7499/statm: 254927 689 113 282 0 254643 0
[pid=7500] ppid=7491 vsize=291084 CPUtime=0.01 cores=1,3,5,7
/proc/7500/stat : 7500 (python) S 7491 7491 7090 0 -1 4202560 907 0 0 0 1 0 0 0 18 0 1 0 617040148 298070016 5999 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974600 0 0 17 7 0 0 0
/proc/7500/statm: 72771 5999 372 364 0 8686 0
[pid=7502] ppid=7500 vsize=1019708 CPUtime=10.37 cores=1,3,5,7
/proc/7502/stat : 7502 (gnovelty+2) R 7500 7502 7502 0 -1 4202496 953 0 0 0 1037 0 0 0 25 0 1 0 617040148 1044180992 690 33554432000 134512640 135663704 4294198944 18446744073709551615 134525302 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7502/statm: 254927 690 113 282 0 254643 0
[pid=7503] ppid=7491 vsize=301276 CPUtime=0.01 cores=1,3,5,7
/proc/7503/stat : 7503 (python) S 7491 7491 7090 0 -1 4202560 899 0 0 0 1 0 0 0 18 0 1 0 617040151 308506624 6019 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974598 0 0 17 3 0 0 0
/proc/7503/statm: 75319 6019 372 364 0 11231 0
[pid=7505] ppid=7503 vsize=420220 CPUtime=10.41 cores=1,3,5,7
/proc/7505/stat : 7505 (adaptg2wsat2009) R 7503 7505 7505 0 -1 4202496 1666 0 0 0 1041 0 0 0 25 0 1 0 617040151 430305280 1400 33554432000 134512640 135107411 4288772144 18446744073709551615 134531392 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7505/statm: 105055 1400 59 146 0 103931 0
[pid=7506] ppid=7491 vsize=311596 CPUtime=0.01 cores=1,3,5,7
/proc/7506/stat : 7506 (python) S 7491 7491 7090 0 -1 4202560 919 0 0 0 0 1 0 0 18 0 1 0 617040154 319074304 6023 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974598 0 0 17 7 0 0 0
/proc/7506/statm: 77899 6023 372 364 0 13808 0
[pid=7508] ppid=7506 vsize=420220 CPUtime=10.42 cores=1,3,5,7
/proc/7508/stat : 7508 (adaptg2wsat2009) R 7506 7508 7508 0 -1 4202496 1666 0 0 0 1042 0 0 0 25 0 1 0 617040154 430305280 1400 33554432000 134512640 135107411 4287260464 18446744073709551615 134532277 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7508/statm: 105055 1400 59 146 0 103931 0
[pid=7491/tid=7498] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7498/stat : 7498 (python) S 7489 7491 7090 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 5 0 617040145 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 3 0 0 0
[pid=7491/tid=7501] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7501/stat : 7501 (python) S 7489 7491 7090 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 5 0 617040148 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
[pid=7491/tid=7504] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7504/stat : 7504 (python) S 7489 7491 7090 0 -1 4202560 22 934 0 0 0 0 0 0 23 0 5 0 617040151 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 3 0 0 0
[pid=7491/tid=7507] ppid=7489 vsize=321292 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7507/stat : 7507 (python) S 7489 7491 7090 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 5 0 617040154 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 43.71
Current children cumulated vsize (KiB) 4385868

[startup+25.5014 s]
/proc/loadavg: 7.98 7.58 4.72 9/260 7514
/proc/meminfo: memFree=29417604/32951124 swapFree=67111464/67111528
[pid=7491] ppid=7489 vsize=321292 CPUtime=1.98 cores=1,3,5,7
/proc/7491/stat : 7491 (python) S 7489 7491 7090 0 -1 4202496 15370 934 1 0 187 11 0 0 19 0 5 0 617039927 329003008 7537 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/7491/statm: 80323 7537 1883 364 0 16232 0
[pid=7497] ppid=7491 vsize=280764 CPUtime=0.02 cores=1,3,5,7
/proc/7497/stat : 7497 (python) S 7491 7491 7090 0 -1 4202560 907 0 0 0 1 1 0 0 18 0 1 0 617040145 287502336 6001 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416015 0 0 16781312 514 11031954315336974596 0 0 17 3 0 0 0
/proc/7497/statm: 70191 6001 378 364 0 6109 0
[pid=7499] ppid=7497 vsize=1019708 CPUtime=23.3 cores=1,3,5,7
/proc/7499/stat : 7499 (gnovelty+2) R 7497 7499 7499 0 -1 4202496 954 0 0 0 2330 0 0 0 25 0 1 0 617040145 1044180992 689 33554432000 134512640 135663704 4288951120 18446744073709551615 134524461 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7499/statm: 254927 689 113 282 0 254643 0
[pid=7500] ppid=7491 vsize=291084 CPUtime=0.03 cores=1,3,5,7
/proc/7500/stat : 7500 (python) S 7491 7491 7090 0 -1 4202560 912 0 0 0 2 1 0 0 18 0 1 0 617040148 298070016 5999 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974600 0 0 17 7 0 0 0
/proc/7500/statm: 72771 5999 372 364 0 8686 0
[pid=7502] ppid=7500 vsize=1019708 CPUtime=23.04 cores=1,3,5,7
/proc/7502/stat : 7502 (gnovelty+2) R 7500 7502 7502 0 -1 4202496 953 0 0 0 2304 0 0 0 25 0 1 0 617040148 1044180992 690 33554432000 134512640 135663704 4294198944 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7502/statm: 254927 690 113 282 0 254643 0
[pid=7503] ppid=7491 vsize=301276 CPUtime=0.03 cores=1,3,5,7
/proc/7503/stat : 7503 (python) S 7491 7491 7090 0 -1 4202560 904 0 0 0 2 1 0 0 18 0 1 0 617040151 308506624 6019 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974598 0 0 17 3 0 0 0
/proc/7503/statm: 75319 6019 372 364 0 11231 0
[pid=7505] ppid=7503 vsize=420220 CPUtime=23.18 cores=1,3,5,7
/proc/7505/stat : 7505 (adaptg2wsat2009) R 7503 7505 7505 0 -1 4202496 1666 0 0 0 2318 0 0 0 25 0 1 0 617040151 430305280 1400 33554432000 134512640 135107411 4288772144 18446744073709551615 134531582 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7505/statm: 105055 1400 59 146 0 103931 0
[pid=7506] ppid=7491 vsize=311596 CPUtime=0.03 cores=1,3,5,7
/proc/7506/stat : 7506 (python) S 7491 7491 7090 0 -1 4202560 925 0 0 0 1 2 0 0 18 0 1 0 617040154 319074304 6023 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974598 0 0 17 7 0 0 0
/proc/7506/statm: 77899 6023 372 364 0 13808 0
[pid=7508] ppid=7506 vsize=420220 CPUtime=23.22 cores=1,3,5,7

################
# More data... #
################

/proc/7505/stat : 7505 (adaptg2wsat2009) T 7503 7505 7505 0 -1 4202496 1666 0 0 0 9986 0 0 0 25 0 1 0 617040151 430305280 1400 33554432000 134512640 135107411 4288772144 18446744073709551615 134531232 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7505/statm: 105055 1400 59 146 0 103931 0
[pid=7503/tid=7568] ppid=7491 vsize=301276 CPUtime=0 cores=1,3,5,7
/proc/7503/task/7568/stat : 7568 (python) S 7491 7491 7090 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 617050166 308506624 6048 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604466191562752 0 0 -1 3 0 0 0
[pid=7506] ppid=7491 vsize=311596 CPUtime=0.14 cores=1,3,5,7
/proc/7506/stat : 7506 (python) S 7491 7491 7090 0 -1 4202560 1105 0 0 0 7 7 0 0 19 0 2 0 617040154 319074304 6052 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/7506/statm: 77899 6052 400 364 0 13808 0
[pid=7508] ppid=7506 vsize=420220 CPUtime=100.12 cores=1,3,5,7
/proc/7508/stat : 7508 (adaptg2wsat2009) T 7506 7508 7508 0 -1 4202496 1666 0 0 0 10011 1 0 0 25 0 1 0 617040154 430305280 1400 33554432000 134512640 135107411 4287260464 18446744073709551615 134532371 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7508/statm: 105055 1400 59 146 0 103931 0
[pid=7506/tid=7569] ppid=7491 vsize=311596 CPUtime=0 cores=1,3,5,7
/proc/7506/task/7569/stat : 7569 (python) S 7491 7491 7090 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 617050167 319074304 6052 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604467630008192 0 0 -1 7 0 0 0
[pid=7561] ppid=7491 vsize=321928 CPUtime=0.14 cores=1,3,5,7
/proc/7561/stat : 7561 (python) S 7491 7491 7090 0 -1 4202560 1116 0 0 0 8 6 0 0 19 0 2 0 617050072 329654272 6056 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/7561/statm: 80482 6056 400 364 0 16388 0
[pid=7563] ppid=7561 vsize=420220 CPUtime=99.12 cores=1,3,5,7
/proc/7563/stat : 7563 (adaptg2wsat2009) T 7561 7563 7563 0 -1 4202496 1667 0 0 0 9912 0 0 0 25 0 1 0 617050072 430305280 1399 33554432000 134512640 135107411 4287977616 18446744073709551615 134532191 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/7563/statm: 105055 1399 59 146 0 103931 0
[pid=7561/tid=7583] ppid=7491 vsize=321928 CPUtime=0 cores=1,3,5,7
/proc/7561/task/7583/stat : 7583 (python) S 7491 7491 7090 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 617059986 329654272 6056 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604462036758848 0 0 -1 7 0 0 0
[pid=7565] ppid=7491 vsize=332348 CPUtime=0.14 cores=1,3,5,7
/proc/7565/stat : 7565 (python) S 7491 7491 7090 0 -1 4202560 1131 0 0 0 7 7 0 0 19 0 2 0 617050165 340324352 6079 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/7565/statm: 83087 6079 400 364 0 18990 0
[pid=7567] ppid=7565 vsize=1019708 CPUtime=99.03 cores=1,3,5,7
/proc/7567/stat : 7567 (gnovelty+2) T 7565 7567 7567 0 -1 4202496 954 0 0 0 9903 0 0 0 25 0 1 0 617050166 1044180992 690 33554432000 134512640 135663704 4291572592 18446744073709551615 134524882 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/7567/statm: 254927 690 113 282 0 254643 0
[pid=7565/tid=7587] ppid=7491 vsize=332348 CPUtime=0 cores=1,3,5,7
/proc/7565/task/7587/stat : 7587 (python) S 7491 7491 7090 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 617060084 340324352 6079 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604462036759744 0 0 -1 7 0 0 0
[pid=7570] ppid=7491 vsize=342524 CPUtime=0.15 cores=1,3,5,7
/proc/7570/stat : 7570 (python) S 7491 7491 7090 0 -1 4202560 1179 0 0 0 8 7 0 0 19 0 2 0 617050171 350744576 6130 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 7 0 0 0
/proc/7570/statm: 85631 6130 406 364 0 21531 0
[pid=7572] ppid=7570 vsize=249872 CPUtime=100.14 cores=1,3,5,7
/proc/7572/stat : 7572 (FH) T 7570 7572 7572 0 -1 4202496 1530 0 0 0 10014 0 0 0 25 0 1 0 617050171 255868928 1264 33554432000 134512640 134936988 4289573296 18446744073709551615 134553579 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/7572/statm: 62468 1264 41 104 0 61972 0
[pid=7570/tid=7595] ppid=7491 vsize=342524 CPUtime=0 cores=1,3,5,7
/proc/7570/task/7595/stat : 7595 (python) S 7491 7491 7090 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 617060186 350744576 6130 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604471164544320 0 0 -1 7 0 0 0
[pid=7573] ppid=7491 vsize=352876 CPUtime=0.15 cores=1,3,5,7
/proc/7573/stat : 7573 (python) S 7491 7491 7090 0 -1 4202560 1169 0 0 0 7 8 0 0 19 0 2 0 617050174 361345024 6134 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7573/statm: 88219 6134 406 364 0 24116 0
[pid=7575] ppid=7573 vsize=420220 CPUtime=97.12 cores=1,3,5,7
/proc/7575/stat : 7575 (adaptg2wsat2009) T 7573 7575 7575 0 -1 4202496 1672 0 0 0 9712 0 0 0 25 0 1 0 617050175 430305280 1400 33554432000 134512640 135107411 4293035344 18446744073709551615 134531739 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7575/statm: 105055 1400 59 146 0 103931 0
[pid=7573/tid=7591] ppid=7491 vsize=352876 CPUtime=0 cores=1,3,5,7
/proc/7573/task/7591/stat : 7591 (python) S 7491 7491 7090 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 617060089 361345024 6134 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604471164546112 0 0 -1 5 0 0 0
[pid=7584] ppid=7491 vsize=363344 CPUtime=0.15 cores=1,3,5,7
/proc/7584/stat : 7584 (python) S 7491 7491 7090 0 -1 4202560 1145 0 0 0 7 8 0 0 19 0 2 0 617059990 372064256 6136 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 3 0 0 0
/proc/7584/statm: 90836 6136 400 364 0 26730 0
[pid=7586] ppid=7584 vsize=420220 CPUtime=100.13 cores=1,3,5,7
/proc/7586/stat : 7586 (adaptg2wsat2009) T 7584 7586 7586 0 -1 4202496 1672 0 0 0 10013 0 0 0 25 0 1 0 617059990 430305280 1401 33554432000 134512640 135107411 4293804880 18446744073709551615 134531942 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/7586/statm: 105055 1401 59 146 0 103931 0
[pid=7584/tid=7604] ppid=7491 vsize=363344 CPUtime=0 cores=1,3,5,7
/proc/7584/task/7604/stat : 7604 (python) S 7491 7491 7090 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 617070004 372064256 6136 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604471094449984 0 0 -1 5 0 0 0
[pid=7588] ppid=7491 vsize=373380 CPUtime=0.14 cores=1,3,5,7
/proc/7588/stat : 7588 (python) S 7491 7491 7090 0 -1 4202560 1152 0 0 0 7 7 0 0 19 0 2 0 617060088 382341120 6193 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7588/statm: 93345 6193 400 364 0 29236 0
[pid=7590] ppid=7588 vsize=249876 CPUtime=99.56 cores=1,3,5,7
/proc/7590/stat : 7590 (FH) T 7588 7590 7590 0 -1 4202496 1534 0 0 0 9956 0 0 0 25 0 1 0 617060088 255873024 1264 33554432000 134512640 134936988 4291877008 18446744073709551615 134554261 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/7590/statm: 62469 1264 41 104 0 61973 0
[pid=7588/tid=7608] ppid=7491 vsize=373380 CPUtime=0 cores=1,3,5,7
/proc/7588/task/7608/stat : 7608 (python) S 7491 7491 7090 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 617070104 382341120 6193 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604471094448192 0 0 -1 3 0 0 0
[pid=7592] ppid=7491 vsize=383856 CPUtime=0.46 cores=1,3,5,7
/proc/7592/stat : 7592 (python) S 7491 7491 7090 0 -1 4202560 1191 0 0 0 25 21 0 0 18 0 2 0 617060093 393068544 6197 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7592/statm: 95964 6197 400 364 0 31852 0
[pid=7594] ppid=7592 vsize=268424 CPUtime=291.44 cores=1,3,5,7
/proc/7594/stat : 7594 (hybridGM3) T 7592 7594 7594 0 -1 4202496 5396 0 0 0 29144 0 0 0 25 0 1 0 617060094 274866176 5127 33554432000 134512640 135204385 4287874128 18446744073709551615 134538284 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/7594/statm: 67106 5127 63 169 0 66936 0
[pid=7592/tid=7632] ppid=7491 vsize=383856 CPUtime=0 cores=1,3,5,7
/proc/7592/task/7632/stat : 7632 (python) S 7491 7491 7090 0 -1 4202560 47 0 0 0 0 0 0 0 18 0 2 0 617079728 393068544 6197 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604471119459392 0 0 -1 1 0 0 0
[pid=7596] ppid=7491 vsize=394240 CPUtime=0.15 cores=1,3,5,7
/proc/7596/stat : 7596 (python) S 7491 7491 7090 0 -1 4202560 1159 0 0 0 8 7 0 0 19 0 2 0 617060190 403701760 6201 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7596/statm: 98560 6201 400 364 0 34445 0
[pid=7598] ppid=7596 vsize=1019708 CPUtime=97.56 cores=1,3,5,7
/proc/7598/stat : 7598 (gnovelty+2) T 7596 7598 7598 0 -1 4202496 962 0 0 0 9756 0 0 0 25 0 1 0 617060190 1044180992 689 33554432000 134512640 135663704 4293474944 18446744073709551615 134525255 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/7598/statm: 254927 689 113 282 0 254643 0
[pid=7596/tid=7612] ppid=7491 vsize=394240 CPUtime=0 cores=1,3,5,7
/proc/7596/task/7612/stat : 7612 (python) S 7491 7491 7090 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 617070110 403701760 6201 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604466191560960 0 0 -1 5 0 0 0
[pid=7605] ppid=7491 vsize=404328 CPUtime=0.32 cores=1,3,5,7
/proc/7605/stat : 7605 (python) S 7491 7491 7090 0 -1 4202560 1204 0 0 0 16 16 0 0 18 0 2 0 617070009 414031872 6237 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7605/statm: 101082 6237 400 364 0 36964 0
[pid=7607] ppid=7605 vsize=268420 CPUtime=202.86 cores=1,3,5,7
/proc/7607/stat : 7607 (hybridGM3) R 7605 7607 7607 0 -1 4202496 5397 0 0 0 20286 0 0 0 25 0 1 0 617070009 274862080 5125 33554432000 134512640 135204385 4290813088 18446744073709551615 134537792 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/7607/statm: 67105 5125 63 169 0 66935 0
[pid=7605/tid=7647] ppid=7491 vsize=404328 CPUtime=0 cores=1,3,5,7
/proc/7605/task/7647/stat : 7647 (python) S 7491 7491 7090 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 617089548 414031872 6237 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604471093441792 0 0 -1 5 0 0 0
[pid=7609] ppid=7491 vsize=414872 CPUtime=0.16 cores=1,3,5,7
/proc/7609/stat : 7609 (python) S 7491 7491 7090 0 -1 4202560 1171 0 0 0 9 7 0 0 19 0 2 0 617070108 424828928 6241 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7609/statm: 103718 6241 400 364 0 39597 0
[pid=7611] ppid=7609 vsize=420220 CPUtime=99.53 cores=1,3,5,7
/proc/7611/stat : 7611 (adaptg2wsat2009) T 7609 7611 7611 0 -1 4202496 1676 0 0 0 9953 0 0 0 25 0 1 0 617070109 430305280 1399 33554432000 134512640 135107411 4292311888 18446744073709551615 134531239 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/7611/statm: 105055 1399 59 146 0 103931 0
[pid=7609/tid=7633] ppid=7491 vsize=414872 CPUtime=0 cores=1,3,5,7
/proc/7609/task/7633/stat : 7633 (python) S 7491 7491 7090 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 617080125 424828928 6241 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604471093443584 0 0 -1 1 0 0 0
[pid=7613] ppid=7491 vsize=424872 CPUtime=0.33 cores=1,3,5,7
/proc/7613/stat : 7613 (python) S 7491 7491 7090 0 -1 4202560 1223 0 0 0 16 17 0 0 19 0 2 0 617070116 435068928 6283 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032222667 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/7613/statm: 106218 6283 408 364 0 42094 0
[pid=7615] ppid=7613 vsize=268424 CPUtime=196.94 cores=1,3,5,7
/proc/7615/stat : 7615 (hybridGM3) T 7613 7615 7615 0 -1 4202496 5402 0 0 0 19694 0 0 0 25 0 1 0 617070116 274866176 5127 33554432000 134512640 135204385 4291597264 18446744073709551615 134537939 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/7615/statm: 67106 5127 63 169 0 66936 0
[pid=7613/tid=7648] ppid=7491 vsize=424872 CPUtime=0 cores=1,3,5,7
/proc/7613/task/7648/stat : 7648 (python) S 7491 7491 7090 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 617090255 435068928 6283 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 514 18446604469240072128 0 0 -1 7 0 0 0
[pid=7634] ppid=7491 vsize=435144 CPUtime=0.17 cores=1,3,5,7
/proc/7634/stat : 7634 (python) S 7491 7491 7090 0 -1 4202560 1076 0 0 0 9 8 0 0 18 0 1 0 617080129 445587456 6273 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974597 0 0 17 5 0 0 0
/proc/7634/statm: 108786 6273 380 364 0 44659 0
[pid=7636] ppid=7634 vsize=268420 CPUtime=101.15 cores=1,3,5,7
/proc/7636/stat : 7636 (hybridGM3) R 7634 7636 7636 0 -1 4202496 5404 0 0 0 10115 0 0 0 25 0 1 0 617080130 274862080 5126 33554432000 134512640 135204385 4289315696 18446744073709551615 134538395 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/7636/statm: 67105 5126 63 169 0 66935 0
[pid=7643] ppid=7491 vsize=445728 CPUtime=0.02 cores=1,3,5,7
/proc/7643/stat : 7643 (python) S 7491 7491 7090 0 -1 4202560 1018 0 0 0 1 1 0 0 18 0 1 0 617089253 456425472 6271 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 11031954315336974598 0 0 17 7 0 0 0
/proc/7643/statm: 111432 6271 374 364 0 47302 0
[pid=7645] ppid=7643 vsize=458344 CPUtime=10.53 cores=1,3,5,7
/proc/7645/stat : 7645 (TNM) R 7643 7645 7645 0 -1 4202496 1513 0 0 0 1053 0 0 0 25 0 1 0 617089253 469344256 1239 33554432000 134512640 135008748 4293976512 18446744073709551615 134527320 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/7645/statm: 114586 1239 46 122 0 113486 0
[pid=7649] ppid=7491 vsize=456112 CPUtime=0 cores=1,3,5,7
/proc/7649/stat : 7649 (python) S 7491 7491 7090 0 -1 4202560 826 0 0 0 0 0 0 0 19 0 1 0 617090260 467058688 6255 33554432000 4194304 5683644 140737050332144 18446744073709551615 260020416070 0 0 16781312 514 7996497659336589311 0 0 17 5 0 0 0
/proc/7649/statm: 114028 6255 347 364 0 49895 0
[pid=7651] ppid=7649 vsize=247956 CPUtime=0.45 cores=1,3,5,7
/proc/7651/stat : 7651 (FH) R 7649 7651 7651 0 -1 4202496 1261 0 0 0 45 0 0 0 21 0 1 0 617090260 253906944 987 33554432000 134512640 134936988 4294313936 18446744073709551615 134523447 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/7651/statm: 61989 987 37 104 0 61493 0
[pid=7491/tid=7498] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7498/stat : 7498 (python) S 7489 7491 7090 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 19 0 617040145 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 3 0 0 0
[pid=7491/tid=7501] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7501/stat : 7501 (python) S 7489 7491 7090 0 -1 4202560 23 934 0 0 0 0 0 0 21 0 19 0 617040148 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
[pid=7491/tid=7504] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7504/stat : 7504 (python) S 7489 7491 7090 0 -1 4202560 22 934 0 0 0 0 0 0 23 0 19 0 617040151 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 3 0 0 0
[pid=7491/tid=7507] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7507/stat : 7507 (python) S 7489 7491 7090 0 -1 4202560 27 934 0 0 0 0 0 0 20 0 19 0 617040154 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
[pid=7491/tid=7562] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7562/stat : 7562 (python) S 7489 7491 7090 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 617050072 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
[pid=7491/tid=7566] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7566/stat : 7566 (python) S 7489 7491 7090 0 -1 4202560 33 934 0 0 0 0 0 0 20 0 19 0 617050165 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
[pid=7491/tid=7571] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7571/stat : 7571 (python) S 7489 7491 7090 0 -1 4202560 26 934 0 0 0 0 0 0 21 0 19 0 617050171 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 7 0 0 0
[pid=7491/tid=7574] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7574/stat : 7574 (python) S 7489 7491 7090 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 617050175 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7585] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7585/stat : 7585 (python) S 7489 7491 7090 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 19 0 617059990 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7589] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7589/stat : 7589 (python) S 7489 7491 7090 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 617060088 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7593] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7593/stat : 7593 (python) S 7489 7491 7090 0 -1 4202560 33 934 0 0 0 0 0 0 18 0 19 0 617060094 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7597] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7597/stat : 7597 (python) S 7489 7491 7090 0 -1 4202560 22 934 0 0 0 0 0 0 20 0 19 0 617060190 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7606] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7606/stat : 7606 (python) S 7489 7491 7090 0 -1 4202560 32 934 0 0 0 0 0 0 18 0 19 0 617070009 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7610] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7610/stat : 7610 (python) S 7489 7491 7090 0 -1 4202560 31 934 0 0 0 0 0 0 20 0 19 0 617070109 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7614] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7614/stat : 7614 (python) S 7489 7491 7090 0 -1 4202560 26 934 0 0 0 0 0 0 21 0 19 0 617070116 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7635] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7635/stat : 7635 (python) S 7489 7491 7090 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 617080130 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7644] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7644/stat : 7644 (python) S 7489 7491 7090 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 617089253 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
[pid=7491/tid=7650] ppid=7489 vsize=465656 CPUtime=0 cores=1,3,5,7
/proc/7491/task/7650/stat : 7650 (python) S 7489 7491 7090 0 -1 4202560 28 934 0 0 0 0 0 0 20 0 19 0 617090260 476831744 7823 33554432000 4194304 5683644 140737050332144 18446744073709551615 260032219569 0 0 16781312 2 18446604462509591616 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1999.61
Current children cumulated vsize (KiB) 15986060

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 7491 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=544613
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=159975
# CPU time returned by wait4() is 2.70459
# while last known CPU time is 1999.61
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 503.92
CPU time (s): 1999.61
CPU user time (s): 1997.99
CPU system time (s): 1.62
CPU usage (%): 396.811
Max. virtual memory (cumulated for all children) (KiB): 15986060

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.54461
system time used= 0.159975
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30233
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3113
involuntary context switches= 73

runsolver used 2.24566 second user time and 6.11707 second system time

The end

Launcher Data 

Begin job on node110 at 2011-05-01 18:41:33
IDJOB=3317195
IDBENCH=82660
IDSOLVER=1705
FILE ID=node110/3317195-1304268093
RUNJOBID= node110-1304267293-7109
PBS_JOBID= 13173968
Free space on /tmp= 73468 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v300-c25500-S69399346-065-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317195-1304268093/watcher-3317195-1304268093 -o /tmp/evaluation-result-3317195-1304268093/solver-3317195-1304268093 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317195-1304268093.cnf -seed 1264031527 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 98486271f2f72889c040eabfd36ebd13
RANDOM SEED=1264031527

node110.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29518152 kB
Buffers:       1027492 kB
Cached:        1226972 kB
SwapCached:         64 kB
Active:        1135360 kB
Inactive:      1861492 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29518152 kB
SwapTotal:    67111528 kB
SwapFree:     67111464 kB
Dirty:           53112 kB
Writeback:           0 kB
AnonPages:      742460 kB
Mapped:          22344 kB
Slab:           356964 kB
PageTables:      18588 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 14399584 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73476 MiB
End job on node110 at 2011-05-01 18:49:59