Trace number 3317174

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-random? (MO) 2790.52 702.925

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1500-c30000-S377999798-061-UNKNOWN.cnf
MD5SUMa4ba28fb1550ba8a4eca95d91f4669c6
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark872.659
Satisfiable
(Un)Satisfiability was proved
Number of variables1500
Number of clauses30000
Sum of the clauses size150000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 530000
Number of clauses of size over 50

Solver Data 

0.32/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.67	c INFO: solving HOME/instance-3317174-1304267975.cnf
0.32/0.67	c INFO: computing task features
0.48/0.83	c INFO: computed variable-clause graph statistics
0.48/0.83	c INFO: computed clause balance statistics
1.18/1.59	c INFO: computed variable balance statistics
1.68/2.00	c INFO: computed small-clause counts
1.68/2.00	c INFO: computed Horn-clause counts
1.68/2.03	c INFO: computed variable graph statistics
1.68/2.03	c INFO: collected features for HOME/instance-3317174-1304267975.cnf in 1.36 s
1.68/2.06	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.38)
1.68/2.09	c INFO: running gnovelty+2@0 for 97 with 4798 remaining (b = 0.14)
1.82/2.12	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.09)
1.82/2.15	c INFO: running adaptg2wsat2009++@0 for 97 with 4798 remaining (b = 0.07)
397.88/101.35	c INFO: running adaptg2wsat2009++@0 for 97 with 4699 remaining (b = 0.06)
401.48/102.26	c INFO: running gnovelty+2@0 for 97 with 4698 remaining (b = 0.05)
401.48/102.30	c INFO: running FH@0 for 97 with 4698 remaining (b = 0.04)
401.79/102.33	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
796.94/201.53	c INFO: running adaptg2wsat2009++@0 for 97 with 4599 remaining (b = 0.03)
800.53/202.48	c INFO: running FH@0 for 97 with 4598 remaining (b = 0.02)
800.85/202.52	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
800.85/202.57	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1195.96/301.73	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1196.30/301.82	c INFO: running hybridGM3@0 for 194 with 4498 remaining (b = 0.03)
1199.51/302.69	c INFO: running TNM@0 for 1552 with 4498 remaining (b = 0.21)
1582.97/398.94	c INFO: running adaptg2wsat2009++@0 for 97 with 4401 remaining (b = 0.02)
1977.14/498.18	c INFO: running hybridGM3@99 for 97 with 4302 remaining (b = 0.01)
1993.44/502.26	c INFO: running hybridGM3@99 for 97 with 4298 remaining (b = 0.01)
2004.95/505.16	c INFO: running hybridGM3@100 for 97 with 4295 remaining (b = 0.01)
2363.36/595.37	c INFO: running hybridGM3@297 for 97 with 4205 remaining (b = 0.01)
2391.59/602.48	c INFO: running FH@0 for 97 with 4198 remaining (b = 0.01)
2399.11/604.37	c INFO: running adaptg2wsat2009++@0 for 97 with 4196 remaining (b = 0.01)
2765.45/696.57	c INFO: running hybridGM3@397 for 97 with 4104 remaining (b = 0.01)
2777.80/699.63	c INFO: running adaptg2wsat2009++@99 for 97 with 4101 remaining (b = 0.01)
2789.69/702.70	c INFO: running gnovelty+2@0 for 97 with 4098 remaining (b = 0.01)

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317174-1304267975/watcher-3317174-1304267975 -o /tmp/evaluation-result-3317174-1304267975/solver-3317174-1304267975 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317174-1304267975.cnf -seed 1317240943 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.13 7.27 4.16 5/223 22438
/proc/meminfo: memFree=29786908/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=15380 CPUtime=0 cores=0,2,4,6
/proc/22438/stat : 22438 (python) S 22436 22438 22144 0 -1 4202496 794 0 0 0 0 0 0 0 25 0 1 0 519213394 15749120 681 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816234496 0 2147483391 16781312 2 18446744071699207358 0 0 17 4 0 0 0
/proc/22438/statm: 3845 681 322 364 0 332 0

[startup+0.027201 s]
/proc/loadavg: 8.13 7.27 4.16 5/223 22438
/proc/meminfo: memFree=29786908/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=16396 CPUtime=0.01 cores=0,2,4,6
/proc/22438/stat : 22438 (python) S 22436 22438 22144 0 -1 4202496 1117 0 0 0 1 0 0 0 25 0 1 0 519213394 16789504 985 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816232293 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/22438/statm: 4099 985 409 364 0 586 0
Current children cumulated CPU time (s) 0.01
Current children cumulated vsize (KiB) 16396

[startup+0.100202 s]
/proc/loadavg: 8.13 7.27 4.16 5/223 22438
/proc/meminfo: memFree=29786908/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=87704 CPUtime=0.06 cores=0,2,4,6
/proc/22438/stat : 22438 (python) R 22436 22438 22144 0 -1 4202496 2638 0 0 0 4 2 0 0 25 0 1 0 519213394 89808896 2354 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816234496 0 2147483391 16781312 2 0 0 0 17 6 0 0 0
/proc/22438/statm: 21926 2354 717 364 0 1622 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 87704

[startup+0.300209 s]
/proc/loadavg: 8.13 7.27 4.16 5/223 22438
/proc/meminfo: memFree=29786908/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=135748 CPUtime=0.16 cores=0,2,4,6
/proc/22438/stat : 22438 (python) S 22436 22438 22144 0 -1 4202496 5964 936 1 0 12 4 0 0 17 0 1 0 519213394 139005952 4151 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816232453 0 2147483391 16781312 2 18446744071699207358 0 0 17 6 0 0 0
/proc/22438/statm: 33937 4151 1032 364 0 3123 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 135748

[startup+0.705129 s]
/proc/loadavg: 8.13 7.27 4.16 5/223 22438
/proc/meminfo: memFree=29786908/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=280560 CPUtime=0.39 cores=0,2,4,6
/proc/22438/stat : 22438 (python) R 22436 22438 22144 0 -1 4202496 9797 936 1 0 29 10 0 0 16 0 1 0 519213394 287293440 7480 33554432000 4194304 5683644 140734686083376 18446744073709551615 4539201 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/22438/statm: 70140 7480 1761 364 0 6064 0
Current children cumulated CPU time (s) 0.39
Current children cumulated vsize (KiB) 280560

[startup+1.50099 s]
/proc/loadavg: 8.13 7.27 4.16 6/224 22443
/proc/meminfo: memFree=29756644/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=287584 CPUtime=1.18 cores=0,2,4,6
/proc/22438/stat : 22438 (python) R 22436 22438 22144 0 -1 4202496 12680 936 1 0 108 10 0 0 17 0 1 0 519213394 294486016 9281 33554432000 4194304 5683644 140734686083376 18446744073709551615 47846255445038 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/22438/statm: 71896 9281 1826 364 0 7820 0
Current children cumulated CPU time (s) 1.18
Current children cumulated vsize (KiB) 287584

[startup+3.10063 s]
/proc/loadavg: 8.13 7.27 4.16 9/230 22449
/proc/meminfo: memFree=29751636/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=324040 CPUtime=1.83 cores=0,2,4,6
/proc/22438/stat : 22438 (python) S 22436 22438 22144 0 -1 4202496 15587 936 1 0 172 11 0 0 20 0 5 0 519213394 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/22438/statm: 81010 8209 1883 364 0 16919 0
[pid=22444] ppid=22438 vsize=283512 CPUtime=0 cores=0,2,4,6
/proc/22444/stat : 22444 (python) S 22438 22438 22144 0 -1 4202560 740 0 0 0 0 0 0 0 19 0 1 0 519213601 290316288 6583 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256527 0 0 16781312 514 4402721913718177791 0 0 17 4 0 0 0
/proc/22444/statm: 70878 6583 355 364 0 6796 0
[pid=22446] ppid=22444 vsize=1018996 CPUtime=1.02 cores=0,2,4,6
/proc/22446/stat : 22446 (gnovelty+2) R 22444 22446 22446 0 -1 4202496 961 0 0 0 102 0 0 0 24 0 1 0 519213601 1043451904 698 33554432000 134512640 135663704 4293506208 18446744073709551615 134545644 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22446/statm: 254749 698 113 282 0 254465 0
[pid=22447] ppid=22438 vsize=293700 CPUtime=0 cores=0,2,4,6
/proc/22447/stat : 22447 (python) S 22438 22438 22144 0 -1 4202560 742 0 0 0 0 0 0 0 21 0 1 0 519213604 300748800 6664 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 4466439712059424767 0 0 17 6 0 0 0
/proc/22447/statm: 73425 6664 349 364 0 9340 0
[pid=22449] ppid=22447 vsize=1018996 CPUtime=0.99 cores=0,2,4,6
/proc/22449/stat : 22449 (gnovelty+2) R 22447 22449 22449 0 -1 4202496 959 0 0 0 99 0 0 0 25 0 1 0 519213604 1043451904 697 33554432000 134512640 135663704 4292640640 18446744073709551615 134524509 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22449/statm: 254749 697 113 282 0 254465 0
[pid=22438/tid=22445] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22445/stat : 22445 (python) S 22436 22438 22144 0 -1 4202560 26 936 0 0 0 0 0 0 20 0 5 0 519213601 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
[pid=22438/tid=22448] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22448/stat : 22448 (python) S 22436 22438 22144 0 -1 4202560 23 936 0 0 0 0 0 0 22 0 5 0 519213604 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 3.84
Current children cumulated vsize (KiB) 2939244

[startup+6.30104 s]
/proc/loadavg: 8.12 7.28 4.18 9/236 22456
/proc/meminfo: memFree=29725920/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=324040 CPUtime=1.83 cores=0,2,4,6
/proc/22438/stat : 22438 (python) S 22436 22438 22144 0 -1 4202496 15587 936 1 0 172 11 0 0 20 0 5 0 519213394 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/22438/statm: 81010 8209 1883 364 0 16919 0
[pid=22444] ppid=22438 vsize=283512 CPUtime=0 cores=0,2,4,6
/proc/22444/stat : 22444 (python) S 22438 22438 22144 0 -1 4202560 902 0 0 0 0 0 0 0 18 0 1 0 519213601 290316288 6606 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256527 0 0 16781312 514 4402721913718145284 0 0 17 4 0 0 0
/proc/22444/statm: 70878 6606 378 364 0 6796 0
[pid=22446] ppid=22444 vsize=1018996 CPUtime=4.18 cores=0,2,4,6
/proc/22446/stat : 22446 (gnovelty+2) R 22444 22446 22446 0 -1 4202496 961 0 0 0 418 0 0 0 25 0 1 0 519213601 1043451904 698 33554432000 134512640 135663704 4293506208 18446744073709551615 134545619 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22446/statm: 254749 698 113 282 0 254465 0
[pid=22447] ppid=22438 vsize=293700 CPUtime=0 cores=0,2,4,6
/proc/22447/stat : 22447 (python) S 22438 22438 22144 0 -1 4202560 909 0 0 0 0 0 0 0 18 0 1 0 519213604 300748800 6687 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 4466439712059392260 0 0 17 6 0 0 0
/proc/22447/statm: 73425 6687 372 364 0 9340 0
[pid=22449] ppid=22447 vsize=1018996 CPUtime=4.18 cores=0,2,4,6
/proc/22449/stat : 22449 (gnovelty+2) R 22447 22449 22449 0 -1 4202496 959 0 0 0 418 0 0 0 25 0 1 0 519213604 1043451904 697 33554432000 134512640 135663704 4292640640 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22449/statm: 254749 697 113 282 0 254465 0
[pid=22450] ppid=22438 vsize=303956 CPUtime=0 cores=0,2,4,6
/proc/22450/stat : 22450 (python) S 22438 22438 22144 0 -1 4202560 898 0 0 0 0 0 0 0 18 0 1 0 519213607 311250944 6690 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 4398157291195433218 0 0 17 6 0 0 0
/proc/22450/statm: 75989 6690 371 364 0 11901 0
[pid=22452] ppid=22450 vsize=421428 CPUtime=4.16 cores=0,2,4,6
/proc/22452/stat : 22452 (adaptg2wsat2009) R 22450 22452 22452 0 -1 4202496 2047 0 0 0 416 0 0 0 25 0 1 0 519213607 431542272 1781 33554432000 134512640 135107411 4289023184 18446744073709551615 134531392 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/22452/statm: 105357 1781 59 146 0 104233 0
[pid=22453] ppid=22438 vsize=314212 CPUtime=0 cores=0,2,4,6
/proc/22453/stat : 22453 (python) S 22438 22438 22144 0 -1 4202560 940 0 0 0 0 0 0 0 18 0 1 0 519213610 321753088 6695 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 4460117520199680260 0 0 17 2 0 0 0
/proc/22453/statm: 78553 6695 372 364 0 14462 0
[pid=22455] ppid=22453 vsize=421428 CPUtime=4.12 cores=0,2,4,6
/proc/22455/stat : 22455 (adaptg2wsat2009) R 22453 22455 22455 0 -1 4202496 2045 0 0 0 412 0 0 0 25 0 1 0 519213610 431542272 1781 33554432000 134512640 135107411 4291021920 18446744073709551615 134531496 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22455/statm: 105357 1781 59 146 0 104233 0
[pid=22438/tid=22445] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22445/stat : 22445 (python) S 22436 22438 22144 0 -1 4202560 26 936 0 0 0 0 0 0 20 0 5 0 519213601 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
[pid=22438/tid=22448] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22448/stat : 22448 (python) S 22436 22438 22144 0 -1 4202560 23 936 0 0 0 0 0 0 22 0 5 0 519213604 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
[pid=22438/tid=22451] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22451/stat : 22451 (python) S 22436 22438 22144 0 -1 4202560 20 936 0 0 0 0 0 0 20 0 5 0 519213607 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22454] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22454/stat : 22454 (python) S 22436 22438 22144 0 -1 4202560 28 936 0 0 0 0 0 0 20 0 5 0 519213610 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 18.47
Current children cumulated vsize (KiB) 4400268

[startup+12.7009 s]
/proc/loadavg: 8.11 7.29 4.20 9/236 22456
/proc/meminfo: memFree=29724584/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=324040 CPUtime=1.83 cores=0,2,4,6
/proc/22438/stat : 22438 (python) S 22436 22438 22144 0 -1 4202496 15587 936 1 0 172 11 0 0 20 0 5 0 519213394 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/22438/statm: 81010 8209 1883 364 0 16919 0
[pid=22444] ppid=22438 vsize=283512 CPUtime=0.01 cores=0,2,4,6
/proc/22444/stat : 22444 (python) S 22438 22438 22144 0 -1 4202560 905 0 0 0 1 0 0 0 18 0 1 0 519213601 290316288 6606 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256527 0 0 16781312 514 1142665260142919940 0 0 17 4 0 0 0
/proc/22444/statm: 70878 6606 378 364 0 6796 0
[pid=22446] ppid=22444 vsize=1018996 CPUtime=10.52 cores=0,2,4,6
/proc/22446/stat : 22446 (gnovelty+2) R 22444 22446 22446 0 -1 4202496 961 0 0 0 1052 0 0 0 25 0 1 0 519213601 1043451904 698 33554432000 134512640 135663704 4293506208 18446744073709551615 134545772 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22446/statm: 254749 698 113 282 0 254465 0
[pid=22447] ppid=22438 vsize=293700 CPUtime=0.01 cores=0,2,4,6
/proc/22447/stat : 22447 (python) S 22438 22438 22144 0 -1 4202560 912 0 0 0 0 1 0 0 18 0 1 0 519213604 300748800 6687 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 1142665260142919940 0 0 17 6 0 0 0
/proc/22447/statm: 73425 6687 372 364 0 9340 0
[pid=22449] ppid=22447 vsize=1018996 CPUtime=10.57 cores=0,2,4,6
/proc/22449/stat : 22449 (gnovelty+2) R 22447 22449 22449 0 -1 4202496 959 0 0 0 1057 0 0 0 25 0 1 0 519213604 1043451904 697 33554432000 134512640 135663704 4292640640 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22449/statm: 254749 697 113 282 0 254465 0
[pid=22450] ppid=22438 vsize=303956 CPUtime=0 cores=0,2,4,6
/proc/22450/stat : 22450 (python) S 22438 22438 22144 0 -1 4202560 900 0 0 0 0 0 0 0 18 0 1 0 519213607 311250944 6691 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 1142665260142919938 0 0 17 6 0 0 0
/proc/22450/statm: 75989 6691 372 364 0 11901 0
[pid=22452] ppid=22450 vsize=421428 CPUtime=10.56 cores=0,2,4,6
/proc/22452/stat : 22452 (adaptg2wsat2009) R 22450 22452 22452 0 -1 4202496 2047 0 0 0 1056 0 0 0 25 0 1 0 519213607 431542272 1781 33554432000 134512640 135107411 4289023184 18446744073709551615 134532088 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/22452/statm: 105357 1781 59 146 0 104233 0
[pid=22453] ppid=22438 vsize=314212 CPUtime=0.01 cores=0,2,4,6
/proc/22453/stat : 22453 (python) S 22438 22438 22144 0 -1 4202560 941 0 0 0 0 1 0 0 18 0 1 0 519213610 321753088 6695 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 1142665260142919940 0 0 17 2 0 0 0
/proc/22453/statm: 78553 6695 372 364 0 14462 0
[pid=22455] ppid=22453 vsize=421428 CPUtime=10.52 cores=0,2,4,6
/proc/22455/stat : 22455 (adaptg2wsat2009) R 22453 22455 22455 0 -1 4202496 2045 0 0 0 1052 0 0 0 25 0 1 0 519213610 431542272 1781 33554432000 134512640 135107411 4291021920 18446744073709551615 134532264 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22455/statm: 105357 1781 59 146 0 104233 0
[pid=22438/tid=22445] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22445/stat : 22445 (python) S 22436 22438 22144 0 -1 4202560 26 936 0 0 0 0 0 0 20 0 5 0 519213601 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
[pid=22438/tid=22448] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22448/stat : 22448 (python) S 22436 22438 22144 0 -1 4202560 23 936 0 0 0 0 0 0 22 0 5 0 519213604 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
[pid=22438/tid=22451] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22451/stat : 22451 (python) S 22436 22438 22144 0 -1 4202560 20 936 0 0 0 0 0 0 20 0 5 0 519213607 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22454] ppid=22436 vsize=324040 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22454/stat : 22454 (python) S 22436 22438 22144 0 -1 4202560 28 936 0 0 0 0 0 0 20 0 5 0 519213610 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 44.03
Current children cumulated vsize (KiB) 4400268

[startup+25.5015 s]
/proc/loadavg: 8.16 7.34 4.27 9/236 22456
/proc/meminfo: memFree=29724972/32951124 swapFree=67105204/67111528
[pid=22438] ppid=22436 vsize=324040 CPUtime=1.83 cores=0,2,4,6
/proc/22438/stat : 22438 (python) S 22436 22438 22144 0 -1 4202496 15587 936 1 0 172 11 0 0 20 0 5 0 519213394 331816960 8209 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 2 18446744073709551615 0 0 17 2 0 0 0
/proc/22438/statm: 81010 8209 1883 364 0 16919 0
[pid=22444] ppid=22438 vsize=283512 CPUtime=0.02 cores=0,2,4,6
/proc/22444/stat : 22444 (python) S 22438 22438 22144 0 -1 4202560 909 0 0 0 1 1 0 0 18 0 1 0 519213601 290316288 6606 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256527 0 0 16781312 514 1142665260142919940 0 0 17 4 0 0 0
/proc/22444/statm: 70878 6606 378 364 0 6796 0
[pid=22446] ppid=22444 vsize=1018996 CPUtime=23.21 cores=0,2,4,6
/proc/22446/stat : 22446 (gnovelty+2) R 22444 22446 22446 0 -1 4202496 961 0 0 0 2321 0 0 0 25 0 1 0 519213601 1043451904 698 33554432000 134512640 135663704 4293506208 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22446/statm: 254749 698 113 282 0 254465 0
[pid=22447] ppid=22438 vsize=293700 CPUtime=0.02 cores=0,2,4,6
/proc/22447/stat : 22447 (python) S 22438 22438 22144 0 -1 4202560 917 0 0 0 1 1 0 0 18 0 1 0 519213604 300748800 6687 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 1142665260142919940 0 0 17 6 0 0 0
/proc/22447/statm: 73425 6687 372 364 0 9340 0
[pid=22449] ppid=22447 vsize=1018996 CPUtime=23.36 cores=0,2,4,6
/proc/22449/stat : 22449 (gnovelty+2) R 22447 22449 22449 0 -1 4202496 959 0 0 0 2336 0 0 0 25 0 1 0 519213604 1043451904 697 33554432000 134512640 135663704 4292640640 18446744073709551615 134525111 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22449/statm: 254749 697 113 282 0 254465 0
[pid=22450] ppid=22438 vsize=303956 CPUtime=0.03 cores=0,2,4,6
/proc/22450/stat : 22450 (python) S 22438 22438 22144 0 -1 4202560 904 0 0 0 1 2 0 0 18 0 1 0 519213607 311250944 6691 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 1142665260142919938 0 0 17 6 0 0 0

################
# More data... #
################

[pid=22455] ppid=22453 vsize=421428 CPUtime=103.14 cores=0,2,4,6
/proc/22455/stat : 22455 (adaptg2wsat2009) R 22453 22455 22455 0 -1 4202496 2045 0 0 0 10314 0 0 0 25 0 1 0 519213610 431542272 1781 33554432000 134512640 135107411 4291021920 18446744073709551615 134532376 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/22455/statm: 105357 1781 59 146 0 104233 0
[pid=22453/tid=22476] ppid=22438 vsize=314212 CPUtime=0 cores=0,2,4,6
/proc/22453/task/22476/stat : 22476 (python) S 22438 22438 22144 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 519223623 321753088 6724 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453951145280 0 0 -1 2 0 0 0
[pid=22468] ppid=22438 vsize=324468 CPUtime=0.15 cores=0,2,4,6
/proc/22468/stat : 22468 (python) S 22438 22438 22144 0 -1 4202560 1122 0 0 0 7 8 0 0 19 0 2 0 519223529 332255232 6728 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/22468/statm: 81117 6728 400 364 0 17023 0
[pid=22470] ppid=22468 vsize=421428 CPUtime=98.36 cores=0,2,4,6
/proc/22470/stat : 22470 (adaptg2wsat2009) T 22468 22470 22470 0 -1 4202496 2050 0 0 0 9836 0 0 0 25 0 1 0 519223530 431542272 1781 33554432000 134512640 135107411 4287225616 18446744073709551615 134531817 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/22470/statm: 105357 1781 59 146 0 104233 0
[pid=22468/tid=22493] ppid=22438 vsize=324468 CPUtime=0 cores=0,2,4,6
/proc/22468/task/22493/stat : 22493 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519233544 332255232 6728 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453918018432 0 0 -1 2 0 0 0
[pid=22473] ppid=22438 vsize=334724 CPUtime=0.15 cores=0,2,4,6
/proc/22473/stat : 22473 (python) S 22438 22438 22144 0 -1 4202560 1142 0 0 0 9 6 0 0 19 0 2 0 519223621 342757376 6732 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/22473/statm: 83681 6732 400 364 0 19584 0
[pid=22475] ppid=22473 vsize=1018996 CPUtime=100.17 cores=0,2,4,6
/proc/22475/stat : 22475 (gnovelty+2) T 22473 22475 22475 0 -1 4202496 964 0 0 0 10017 0 0 0 25 0 1 0 519223621 1043451904 697 33554432000 134512640 135663704 4289336480 18446744073709551615 134545619 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/22475/statm: 254749 697 113 282 0 254465 0
[pid=22473/tid=22497] ppid=22438 vsize=334724 CPUtime=0 cores=0,2,4,6
/proc/22473/task/22497/stat : 22497 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519233639 342757376 6732 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453918015744 0 0 -1 6 0 0 0
[pid=22477] ppid=22438 vsize=344980 CPUtime=0.15 cores=0,2,4,6
/proc/22477/stat : 22477 (python) S 22438 22438 22144 0 -1 4202560 1184 0 0 0 7 8 0 0 18 0 2 0 519223624 353259520 6742 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22477/statm: 86245 6742 406 364 0 22145 0
[pid=22479] ppid=22477 vsize=251292 CPUtime=100.13 cores=0,2,4,6
/proc/22479/stat : 22479 (FH) T 22477 22479 22479 0 -1 4202496 1950 0 0 0 10013 0 0 0 25 0 1 0 519223624 257323008 1682 33554432000 134512640 134936988 4287167376 18446744073709551615 134554229 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/22479/statm: 62823 1682 41 104 0 62327 0
[pid=22477/tid=22498] ppid=22438 vsize=344980 CPUtime=0 cores=0,2,4,6
/proc/22477/task/22498/stat : 22498 (python) S 22438 22438 22144 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 519233639 353259520 6742 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453956798272 0 0 -1 6 0 0 0
[pid=22480] ppid=22438 vsize=355332 CPUtime=0.15 cores=0,2,4,6
/proc/22480/stat : 22480 (python) S 22438 22438 22144 0 -1 4202560 1169 0 0 0 6 9 0 0 19 0 2 0 519223628 363859968 6746 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/22480/statm: 88833 6746 406 364 0 24730 0
[pid=22482] ppid=22480 vsize=421428 CPUtime=99.68 cores=0,2,4,6
/proc/22482/stat : 22482 (adaptg2wsat2009) T 22480 22482 22482 0 -1 4202496 2051 0 0 0 9968 0 0 0 25 0 1 0 519223628 431542272 1783 33554432000 134512640 135107411 4293378592 18446744073709551615 134537891 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/22482/statm: 105357 1783 59 146 0 104233 0
[pid=22480/tid=22502] ppid=22438 vsize=355332 CPUtime=0 cores=0,2,4,6
/proc/22480/task/22502/stat : 22502 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519233644 363859968 6746 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453946065280 0 0 -1 2 0 0 0
[pid=22494] ppid=22438 vsize=365696 CPUtime=0.16 cores=0,2,4,6
/proc/22494/stat : 22494 (python) S 22438 22438 22144 0 -1 4202560 1161 0 0 0 8 8 0 0 19 0 2 0 519233547 374472704 6751 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/22494/statm: 91424 6751 400 364 0 27318 0
[pid=22496] ppid=22494 vsize=421428 CPUtime=98.33 cores=0,2,4,6
/proc/22496/stat : 22496 (adaptg2wsat2009) T 22494 22496 22496 0 -1 4202496 2055 0 0 0 9833 0 0 0 25 0 1 0 519233548 431542272 1783 33554432000 134512640 135107411 4287849456 18446744073709551615 134531537 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/22496/statm: 105357 1783 59 146 0 104233 0
[pid=22494/tid=22520] ppid=22438 vsize=365696 CPUtime=0 cores=0,2,4,6
/proc/22494/task/22520/stat : 22520 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519243563 374472704 6751 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453966755776 0 0 -1 2 0 0 0
[pid=22499] ppid=22438 vsize=376064 CPUtime=0.16 cores=0,2,4,6
/proc/22499/stat : 22499 (python) S 22438 22438 22144 0 -1 4202560 1180 0 0 0 10 6 0 0 19 0 2 0 519233642 385089536 6782 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/22499/statm: 94016 6782 400 364 0 29907 0
[pid=22501] ppid=22499 vsize=251292 CPUtime=100.15 cores=0,2,4,6
/proc/22501/stat : 22501 (FH) T 22499 22501 22501 0 -1 4202496 1954 0 0 0 10015 0 0 0 25 0 1 0 519233643 257323008 1683 33554432000 134512640 134936988 4292402432 18446744073709551615 134555012 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/22501/statm: 62823 1683 41 104 0 62327 0
[pid=22499/tid=22528] ppid=22438 vsize=376064 CPUtime=0 cores=0,2,4,6
/proc/22499/task/22528/stat : 22528 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519243658 385089536 6782 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453952614592 0 0 -1 2 0 0 0
[pid=22503] ppid=22438 vsize=386440 CPUtime=0.66 cores=0,2,4,6
/proc/22503/stat : 22503 (python) S 22438 22438 22144 0 -1 4202560 1203 0 0 0 35 31 0 0 18 0 2 0 519233647 395714560 6814 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/22503/statm: 96610 6814 400 364 0 32498 0
[pid=22505] ppid=22503 vsize=268420 CPUtime=399.72 cores=0,2,4,6
/proc/22505/stat : 22505 (hybridGM3) R 22503 22505 22505 0 -1 4202496 5814 0 0 0 39971 1 0 0 25 0 1 0 519233647 274862080 5543 33554432000 134512640 135204385 4289207072 18446744073709551615 134537792 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/22505/statm: 67105 5543 63 169 0 66935 0
[pid=22503/tid=22538] ppid=22438 vsize=386440 CPUtime=0 cores=0,2,4,6
/proc/22503/task/22538/stat : 22538 (python) S 22438 22438 22144 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 519253283 395714560 6814 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604440278879104 0 0 -1 2 0 0 0
[pid=22506] ppid=22438 vsize=396516 CPUtime=0.16 cores=0,2,4,6
/proc/22506/stat : 22506 (python) S 22438 22438 22144 0 -1 4202560 1189 0 0 0 9 7 0 0 19 0 2 0 519233651 406032384 6848 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22506/statm: 99129 6848 400 364 0 35014 0
[pid=22508] ppid=22506 vsize=1018996 CPUtime=99.19 cores=0,2,4,6
/proc/22508/stat : 22508 (gnovelty+2) T 22506 22508 22508 0 -1 4202496 970 0 0 0 9919 0 0 0 25 0 1 0 519233652 1043451904 697 33554432000 134512640 135663704 4291378432 18446744073709551615 134544204 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/22508/statm: 254749 697 113 282 0 254465 0
[pid=22506/tid=22524] ppid=22438 vsize=396516 CPUtime=0 cores=0,2,4,6
/proc/22506/task/22524/stat : 22524 (python) S 22438 22438 22144 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 519243571 406032384 6848 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604440278878208 0 0 -1 6 0 0 0
[pid=22521] ppid=22438 vsize=406884 CPUtime=0.5 cores=0,2,4,6
/proc/22521/stat : 22521 (python) S 22438 22438 22144 0 -1 4202560 1218 0 0 0 28 22 0 0 18 0 2 0 519243568 416649216 6852 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22521/statm: 101721 6852 400 364 0 37603 0
[pid=22523] ppid=22521 vsize=268424 CPUtime=296.51 cores=0,2,4,6
/proc/22523/stat : 22523 (hybridGM3) T 22521 22523 22523 0 -1 4202496 5813 0 0 0 29649 2 0 0 25 0 1 0 519243568 274866176 5543 33554432000 134512640 135204385 4288055216 18446744073709551615 134538323 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/22523/statm: 67106 5543 63 169 0 66936 0
[pid=22521/tid=22550] ppid=22438 vsize=406884 CPUtime=0 cores=0,2,4,6
/proc/22521/task/22550/stat : 22550 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 519263207 416649216 6852 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453928700800 0 0 -1 6 0 0 0
[pid=22525] ppid=22438 vsize=417284 CPUtime=0.5 cores=0,2,4,6
/proc/22525/stat : 22525 (python) S 22438 22438 22144 0 -1 4202560 1217 0 0 0 26 24 0 0 18 0 2 0 519243576 427298816 6856 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22525/statm: 104321 6856 400 364 0 40200 0
[pid=22527] ppid=22525 vsize=268424 CPUtime=294.4 cores=0,2,4,6
/proc/22527/stat : 22527 (hybridGM3) T 22525 22527 22527 0 -1 4202496 5814 0 0 0 29440 0 0 0 25 0 1 0 519243577 274866176 5543 33554432000 134512640 135204385 4288809904 18446744073709551615 134538480 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/22527/statm: 67106 5543 63 169 0 66936 0
[pid=22525/tid=22551] ppid=22438 vsize=417284 CPUtime=0 cores=0,2,4,6
/proc/22525/task/22551/stat : 22551 (python) S 22438 22438 22144 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 519263616 427298816 6856 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604440278214848 0 0 -1 4 0 0 0
[pid=22529] ppid=22438 vsize=427688 CPUtime=0.65 cores=0,2,4,6
/proc/22529/stat : 22529 (python) S 22438 22438 22144 0 -1 4202560 1100 0 0 0 36 29 0 0 18 0 1 0 519243663 437952512 6857 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 1142665260142919939 0 0 17 6 0 0 0
/proc/22529/statm: 106922 6857 379 364 0 42798 0
[pid=22531] ppid=22529 vsize=459768 CPUtime=397.21 cores=0,2,4,6
/proc/22531/stat : 22531 (TNM) R 22529 22531 22531 0 -1 4202496 1928 0 0 0 39720 1 0 0 25 0 1 0 519243663 470802432 1653 33554432000 134512640 135008748 4291973936 18446744073709551615 134527546 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/22531/statm: 114942 1653 46 122 0 113842 0
[pid=22539] ppid=22438 vsize=437796 CPUtime=0.17 cores=0,2,4,6
/proc/22539/stat : 22539 (python) S 22438 22438 22144 0 -1 4202560 1199 0 0 0 9 8 0 0 19 0 2 0 519253289 448303104 6927 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22539/statm: 109449 6927 407 364 0 45322 0
[pid=22541] ppid=22539 vsize=421428 CPUtime=102.77 cores=0,2,4,6
/proc/22541/stat : 22541 (adaptg2wsat2009) T 22539 22541 22541 0 -1 4202496 2062 0 0 0 10277 0 0 0 25 0 1 0 519253289 431542272 1783 33554432000 134512640 135107411 4288520544 18446744073709551615 134532277 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/22541/statm: 105357 1783 59 146 0 104233 0
[pid=22539/tid=22552] ppid=22438 vsize=437796 CPUtime=0 cores=0,2,4,6
/proc/22539/task/22552/stat : 22552 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519263906 448303104 6927 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453912230016 0 0 -1 4 0 0 0
[pid=22605] ppid=22438 vsize=448308 CPUtime=0.16 cores=0,2,4,6
/proc/22605/stat : 22605 (python) S 22438 22438 22144 0 -1 4202560 1214 0 0 0 9 7 0 0 19 0 2 0 519273642 459067392 6930 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828063179 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/22605/statm: 112077 6930 402 364 0 47947 0
[pid=22607] ppid=22605 vsize=251292 CPUtime=99.85 cores=0,2,4,6
/proc/22607/stat : 22607 (FH) T 22605 22607 22607 0 -1 4202496 1955 0 0 0 9985 0 0 0 25 0 1 0 519273643 257323008 1682 33554432000 134512640 134936988 4289318352 18446744073709551615 134546138 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/22607/statm: 62823 1682 41 104 0 62327 0
[pid=22605/tid=22627] ppid=22438 vsize=448308 CPUtime=0 cores=0,2,4,6
/proc/22605/task/22627/stat : 22627 (python) S 22438 22438 22144 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 519283660 459067392 6930 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 514 18446604453928145344 0 0 -1 4 0 0 0
[pid=22628] ppid=22438 vsize=458736 CPUtime=0 cores=0,2,4,6
/proc/22628/stat : 22628 (python) S 22438 22438 22144 0 -1 4202560 855 0 0 0 0 0 0 0 19 0 1 0 519283664 469745664 6926 33554432000 4194304 5683644 140734686083376 18446744073709551615 220816256582 0 0 16781312 514 2924845794836086783 0 0 17 4 0 0 0
/proc/22628/statm: 114684 6926 350 364 0 50551 0
[pid=22630] ppid=22628 vsize=1018996 CPUtime=0.1 cores=0,2,4,6
/proc/22630/stat : 22630 (gnovelty+2) R 22628 22630 22630 0 -1 4202496 972 0 0 0 10 0 0 0 21 0 1 0 519283665 1043451904 697 33554432000 134512640 135663704 4292119072 18446744073709551615 134524897 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/22630/statm: 254749 697 113 282 0 254465 0
[pid=22438/tid=22445] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22445/stat : 22445 (python) S 22436 22438 22144 0 -1 4202560 26 936 0 0 0 0 0 0 20 0 19 0 519213601 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
[pid=22438/tid=22448] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22448/stat : 22448 (python) S 22436 22438 22144 0 -1 4202560 23 936 0 0 0 0 0 0 22 0 19 0 519213604 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 4 0 0 0
[pid=22438/tid=22451] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22451/stat : 22451 (python) S 22436 22438 22144 0 -1 4202560 23 936 0 0 0 0 0 0 18 0 19 0 519213607 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22454] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22454/stat : 22454 (python) S 22436 22438 22144 0 -1 4202560 33 936 0 0 0 0 0 0 18 0 19 0 519213610 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
[pid=22438/tid=22469] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22469/stat : 22469 (python) S 22436 22438 22144 0 -1 4202560 29 936 0 0 0 0 0 0 20 0 19 0 519223530 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
[pid=22438/tid=22474] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22474/stat : 22474 (python) S 22436 22438 22144 0 -1 4202560 21 936 0 0 0 0 0 0 20 0 19 0 519223621 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
[pid=22438/tid=22478] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22478/stat : 22478 (python) S 22436 22438 22144 0 -1 4202560 24 936 0 0 0 0 0 0 22 0 19 0 519223624 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22481] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22481/stat : 22481 (python) S 22436 22438 22144 0 -1 4202560 28 936 0 0 0 0 0 0 24 0 19 0 519223628 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
[pid=22438/tid=22495] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22495/stat : 22495 (python) S 22436 22438 22144 0 -1 4202560 31 936 0 0 0 0 0 0 20 0 19 0 519233547 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
[pid=22438/tid=22500] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22500/stat : 22500 (python) S 22436 22438 22144 0 -1 4202560 24 936 0 0 0 0 0 0 20 0 19 0 519233643 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
[pid=22438/tid=22504] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22504/stat : 22504 (python) S 22436 22438 22144 0 -1 4202560 31 936 0 0 0 0 0 0 18 0 19 0 519233647 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 2 0 0 0
[pid=22438/tid=22507] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22507/stat : 22507 (python) S 22436 22438 22144 0 -1 4202560 26 936 0 0 0 0 0 0 20 0 19 0 519233652 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22522] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22522/stat : 22522 (python) S 22436 22438 22144 0 -1 4202560 34 936 0 0 0 0 0 0 18 0 19 0 519243568 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22526] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22526/stat : 22526 (python) S 22436 22438 22144 0 -1 4202560 30 936 0 0 0 0 0 0 18 0 19 0 519243576 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22530] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22530/stat : 22530 (python) S 22436 22438 22144 0 -1 4202560 25 936 0 0 0 0 0 0 20 0 19 0 519243663 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22540] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22540/stat : 22540 (python) S 22436 22438 22144 0 -1 4202560 27 936 0 0 0 0 0 0 20 0 19 0 519253289 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22606] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22606/stat : 22606 (python) S 22436 22438 22144 0 -1 4202560 28 936 0 0 0 0 0 0 20 0 19 0 519273643 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
[pid=22438/tid=22629] ppid=22436 vsize=468412 CPUtime=0 cores=0,2,4,6
/proc/22438/task/22629/stat : 22629 (python) S 22436 22438 22144 0 -1 4202560 29 936 0 0 0 0 0 0 20 0 19 0 519283665 479653888 8491 33554432000 4194304 5683644 140734686083376 18446744073709551615 220828060081 0 0 16781312 2 18446604453940312000 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2790.52
Current children cumulated vsize (KiB) 16797412

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 22438 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=649597
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=165974
# CPU time returned by wait4() is 2.81557
# while last known CPU time is 2790.52
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 702.925
CPU time (s): 2790.52
CPU user time (s): 2788.27
CPU system time (s): 2.25
CPU usage (%): 396.987
Max. virtual memory (cumulated for all children) (KiB): 16797412

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.6496
system time used= 0.165974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 30840
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3442
involuntary context switches= 64

runsolver used 4.33634 second user time and 9.34058 second system time

The end

Launcher Data 

Begin job on node147 at 2011-05-01 18:39:35
IDJOB=3317174
IDBENCH=82639
IDSOLVER=1705
FILE ID=node147/3317174-1304267975
RUNJOBID= node147-1304267279-22162
PBS_JOBID= 13173932
Free space on /tmp= 73240 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k5-r20-v1500-c30000-S377999798-061-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317174-1304267975/watcher-3317174-1304267975 -o /tmp/evaluation-result-3317174-1304267975/solver-3317174-1304267975 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317174-1304267975.cnf -seed 1317240943 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= a4ba28fb1550ba8a4eca95d91f4669c6
RANDOM SEED=1317240943

node147.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.836
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29787064 kB
Buffers:        349196 kB
Cached:        1920808 kB
SwapCached:       3088 kB
Active:         856288 kB
Inactive:      1955828 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29787064 kB
SwapTotal:    67111528 kB
SwapFree:     67105204 kB
Dirty:           32664 kB
Writeback:           0 kB
AnonPages:      541004 kB
Mapped:          21100 kB
Slab:           276748 kB
PageTables:      15524 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  5873084 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73208 MiB
End job on node147 at 2011-05-01 18:51:20