Trace number 3378107

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)? (MO) 1988.2 501.768

General information on the benchmark

Namerandom/large/
unif-k5-r20-v1500-c30000-S976428817-077-UNKNOWN.cnf
MD5SUM4d2cfd51677809bb15d90c916352e4e2
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark353.72
Satisfiable
(Un)Satisfiability was proved
Number of variables1500
Number of clauses30000
Sum of the clauses size150000
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 530000
Number of clauses of size over 50

Solver Data 

0.35/0.95	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.35/0.95	c INFO: solving HOME/instance-3378107-1305462672.cnf
0.35/0.95	c INFO: computing task features
0.54/1.11	c INFO: computed variable-clause graph statistics
0.54/1.12	c INFO: computed clause balance statistics
1.33/1.90	c INFO: computed variable balance statistics
1.74/2.35	c INFO: computed small-clause counts
1.74/2.35	c INFO: computed Horn-clause counts
1.74/2.38	c INFO: computed variable graph statistics
1.74/2.38	c INFO: collected features for HOME/instance-3378107-1305462672.cnf in 1.43 s
1.74/2.39	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.38)
1.83/2.40	c INFO: running gnovelty+2@0 for 97 with 1198 remaining (b = 0.14)
1.83/2.41	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.09)
1.83/2.42	c INFO: running adaptg2wsat2009++@0 for 97 with 1198 remaining (b = 0.07)
397.51/101.68	c INFO: running adaptg2wsat2009++@0 for 97 with 1099 remaining (b = 0.06)
401.49/102.66	c INFO: running gnovelty+2@0 for 97 with 1098 remaining (b = 0.05)
401.49/102.67	c INFO: running FH@0 for 97 with 1098 remaining (b = 0.04)
401.49/102.69	c INFO: running adaptg2wsat2009++@0 for 97 with 1098 remaining (b = 0.03)
788.75/199.94	c INFO: running adaptg2wsat2009++@0 for 97 with 1001 remaining (b = 0.03)
792.73/200.96	c INFO: running FH@0 for 97 with 999 remaining (b = 0.02)
800.68/202.91	c INFO: running hybridGM3@0 for 194 with 998 remaining (b = 0.04)
800.68/202.94	c INFO: running gnovelty+2@0 for 97 with 998 remaining (b = 0.02)
1187.46/300.18	c INFO: running hybridGM3@0 for 194 with 900 remaining (b = 0.03)
1191.83/301.21	c INFO: running adaptg2wsat2009++@0 for 97 with 899 remaining (b = 0.02)
1195.80/302.24	c INFO: running hybridGM3@0 for 194 with 898 remaining (b = 0.03)
1582.49/399.46	c INFO: running hybridGM3@99 for 97 with 801 remaining (b = 0.01)
1590.39/401.48	c INFO: running FH@0 for 97 with 799 remaining (b = 0.01)
1960.76/494.70	c INFO: running hybridGM3@0 for 194 with 706 remaining (b = 0.02)
1964.72/495.78	c INFO: running hybridGM3@99 for 97 with 705 remaining (b = 0.01)
1984.54/500.75	c INFO: running TNM@0 for 700 with 700 remaining (b = 0.08)
1988.20/501.70	c INFO: running adaptg2wsat2009++@98 for 97 with 699 remaining (b = 0.01)
1988.20/501.70	Traceback (most recent call last):
1988.20/501.70	  File "HOME/solve", line 33, in <module>
1988.20/501.70	    plac.call(main)
1988.20/501.70	  File "/home/evaluation/evaluation/tools64/lib/python2.7/site-packages/plac-0.8.1-py2.7.egg/plac_core.py", line 311, in call

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378107-1305462672/watcher-3378107-1305462672 -o /tmp/evaluation-result-3378107-1305462672/solver-3378107-1305462672 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378107-1305462672.cnf -seed 1373343021 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.96 3.23 3.16 5/314 31160
/proc/meminfo: memFree=11606408/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=15908 CPUtime=0 cores=0,2,4,6
/proc/31160/stat : 31160 (python) R 31158 31160 30970 0 -1 4202496 957 0 0 0 0 0 0 0 25 0 1 0 657865113 16289792 833 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377297253 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31160/statm: 3977 833 357 364 0 464 0

[startup+0.101211 s]
/proc/loadavg: 3.96 3.23 3.16 5/314 31160
/proc/meminfo: memFree=11606408/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=104232 CPUtime=0.07 cores=0,2,4,6
/proc/31160/stat : 31160 (python) R 31158 31160 30970 0 -1 4202496 3086 0 0 0 5 2 0 0 17 0 1 0 657865113 106733568 2767 33554432000 4194304 5683644 140736837937840 18446744073709551615 5116756 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31160/statm: 26058 2767 847 364 0 1806 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104232

[startup+0.200199 s]
/proc/loadavg: 3.96 3.23 3.16 5/314 31160
/proc/meminfo: memFree=11606408/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=128872 CPUtime=0.16 cores=0,2,4,6
/proc/31160/stat : 31160 (python) R 31158 31160 30970 0 -1 4202496 5550 933 1 0 12 4 0 0 18 0 1 0 657865113 131964928 4019 33554432000 4194304 5683644 140736837937840 18446744073709551615 4502173 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/31160/statm: 32218 4019 1011 364 0 2997 0
Current children cumulated CPU time (s) 0.16
Current children cumulated vsize (KiB) 128872

[startup+0.30017 s]
/proc/loadavg: 3.96 3.23 3.16 5/314 31160
/proc/meminfo: memFree=11606408/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=157384 CPUtime=0.23 cores=0,2,4,6
/proc/31160/stat : 31160 (python) D 31158 31160 30970 0 -1 4202496 6584 933 1 0 17 6 0 0 17 0 1 0 657865113 161161216 4695 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377307191 0 0 16781312 2 18446744071562154983 0 0 17 6 0 0 0
/proc/31160/statm: 39346 4695 1257 364 0 3454 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 157384

[startup+0.701091 s]
/proc/loadavg: 3.96 3.23 3.16 5/314 31160
/proc/meminfo: memFree=11606408/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=185092 CPUtime=0.24 cores=0,2,4,6
/proc/31160/stat : 31160 (python) D 31158 31160 30970 0 -1 4202496 6759 933 1 0 17 7 0 0 17 0 1 0 657865113 189534208 4827 33554432000 4194304 5683644 140736837937840 18446744073709551615 255372377735 0 0 16781312 2 18446744071696489144 0 0 17 6 0 0 9
/proc/31160/statm: 46273 4827 1349 364 0 3724 0
Current children cumulated CPU time (s) 0.24
Current children cumulated vsize (KiB) 185092

[startup+1.50094 s]
/proc/loadavg: 4.04 3.26 3.17 7/315 31165
/proc/meminfo: memFree=11575772/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=287588 CPUtime=0.93 cores=0,2,4,6
/proc/31160/stat : 31160 (python) R 31158 31160 30970 0 -1 4202496 12678 933 1 0 83 10 0 0 18 0 1 0 657865113 294490112 9281 33554432000 4194304 5683644 140736837937840 18446744073709551615 46981596384493 0 0 16781312 2 0 0 0 17 0 0 0 9
/proc/31160/statm: 71897 9281 1826 364 0 7821 0
Current children cumulated CPU time (s) 0.93
Current children cumulated vsize (KiB) 287588

[startup+3.10061 s]
/proc/loadavg: 4.04 3.26 3.17 6/315 31165
/proc/meminfo: memFree=11574348/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=324044 CPUtime=1.86 cores=0,2,4,6
/proc/31160/stat : 31160 (python) S 31158 31160 30970 0 -1 4202496 15484 933 1 0 175 11 0 0 24 0 5 0 657865113 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 9
/proc/31160/statm: 81011 8209 1883 364 0 16920 0
Current children cumulated CPU time (s) 1.86
Current children cumulated vsize (KiB) 324044
heavy processes:

[startup+6.30101 s]
/proc/loadavg: 4.04 3.26 3.17 9/327 31177
/proc/meminfo: memFree=11546156/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=324044 CPUtime=1.86 cores=0,2,4,6
/proc/31160/stat : 31160 (python) S 31158 31160 30970 0 -1 4202496 15484 933 1 0 175 11 0 0 24 0 5 0 657865113 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 9
/proc/31160/statm: 81011 8209 1883 364 0 16920 0
[pid=31166] ppid=31160 vsize=283516 CPUtime=0 cores=0,2,4,6
/proc/31166/stat : 31166 (python) S 31160 31160 30970 0 -1 4202560 721 0 0 0 0 0 0 0 18 0 1 0 657865352 290320384 6583 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321487 0 0 16781312 514 575888185918193923 0 0 17 0 0 0 0
/proc/31166/statm: 70879 6583 355 364 0 6797 0
[pid=31168] ppid=31166 vsize=1018996 CPUtime=3.9 cores=0,2,4,6
/proc/31168/stat : 31168 (gnovelty+2) R 31166 31168 31168 0 -1 4202496 959 0 0 0 390 0 0 0 25 0 1 0 657865353 1043451904 697 33554432000 134512640 135663704 4289592848 18446744073709551615 134525260 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31168/statm: 254749 697 113 282 0 254465 0
[pid=31169] ppid=31160 vsize=293704 CPUtime=0 cores=0,2,4,6
/proc/31169/stat : 31169 (python) S 31160 31160 30970 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 657865353 300752896 6664 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 575884337627496706 0 0 17 4 0 0 0
/proc/31169/statm: 73426 6664 349 364 0 9341 0
[pid=31171] ppid=31169 vsize=1018996 CPUtime=3.88 cores=0,2,4,6
/proc/31171/stat : 31171 (gnovelty+2) R 31169 31171 31171 0 -1 4202496 959 0 0 0 388 0 0 0 25 0 1 0 657865354 1043451904 697 33554432000 134512640 135663704 4289573584 18446744073709551615 134524534 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31171/statm: 254749 697 113 282 0 254465 0
[pid=31172] ppid=31160 vsize=303960 CPUtime=0 cores=0,2,4,6
/proc/31172/stat : 31172 (python) S 31160 31160 30970 0 -1 4202560 741 0 0 0 0 0 0 0 18 0 1 0 657865354 311255040 6668 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 575892034208891138 0 0 17 0 0 0 0
/proc/31172/statm: 75990 6668 349 364 0 11902 0
[pid=31174] ppid=31172 vsize=421428 CPUtime=3.87 cores=0,2,4,6
/proc/31174/stat : 31174 (adaptg2wsat2009) R 31172 31174 31174 0 -1 4202496 2046 0 0 0 387 0 0 0 25 0 1 0 657865355 431542272 1781 33554432000 134512640 135107411 4292515024 18446744073709551615 134531428 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31174/statm: 105357 1781 59 146 0 104233 0
[pid=31175] ppid=31160 vsize=314216 CPUtime=0 cores=0,2,4,6
/proc/31175/stat : 31175 (python) S 31160 31160 30970 0 -1 4202560 740 0 0 0 0 0 0 0 18 0 1 0 657865355 321757184 6672 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 3102253370367246594 0 0 17 6 0 0 0
/proc/31175/statm: 78554 6672 349 364 0 14463 0
[pid=31177] ppid=31175 vsize=421428 CPUtime=3.82 cores=0,2,4,6
/proc/31177/stat : 31177 (adaptg2wsat2009) R 31175 31177 31177 0 -1 4202496 2046 0 0 0 382 0 0 0 25 0 1 0 657865356 431542272 1781 33554432000 134512640 135107411 4293282768 18446744073709551615 134531392 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31177/statm: 105357 1781 59 146 0 104233 0
[pid=31160/tid=31167] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31167/stat : 31167 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 5 0 657865353 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 0 0 0 0
[pid=31160/tid=31170] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31170/stat : 31170 (python) S 31158 31160 30970 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 5 0 657865353 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
[pid=31160/tid=31173] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31173/stat : 31173 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 5 0 657865354 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 0 0 0 0
[pid=31160/tid=31176] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31176/stat : 31176 (python) S 31158 31160 30970 0 -1 4202560 27 933 0 0 0 0 0 0 24 0 5 0 657865356 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 17.33
Current children cumulated vsize (KiB) 4400288

[startup+12.7008 s]
/proc/loadavg: 4.65 3.42 3.22 9/327 31178
/proc/meminfo: memFree=11543940/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=324044 CPUtime=1.86 cores=0,2,4,6
/proc/31160/stat : 31160 (python) S 31158 31160 30970 0 -1 4202496 15484 933 1 0 175 11 0 0 24 0 5 0 657865113 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 9
/proc/31160/statm: 81011 8209 1883 364 0 16920 0
[pid=31166] ppid=31160 vsize=283516 CPUtime=0.02 cores=0,2,4,6
/proc/31166/stat : 31166 (python) S 31160 31160 30970 0 -1 4202560 885 0 0 0 1 1 0 0 18 0 1 0 657865352 290320384 6606 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321487 0 0 16781312 514 13574218712801575171 0 0 17 0 0 0 0
/proc/31166/statm: 70879 6606 378 364 0 6797 0
[pid=31168] ppid=31166 vsize=1018996 CPUtime=10.29 cores=0,2,4,6
/proc/31168/stat : 31168 (gnovelty+2) R 31166 31168 31168 0 -1 4202496 959 0 0 0 1029 0 0 0 25 0 1 0 657865353 1043451904 697 33554432000 134512640 135663704 4289592848 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31168/statm: 254749 697 113 282 0 254465 0
[pid=31169] ppid=31160 vsize=293704 CPUtime=0.01 cores=0,2,4,6
/proc/31169/stat : 31169 (python) S 31160 31160 30970 0 -1 4202560 907 0 0 0 1 0 0 0 18 0 1 0 657865353 300752896 6687 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 13574218712801575170 0 0 17 4 0 0 0
/proc/31169/statm: 73426 6687 372 364 0 9341 0
[pid=31171] ppid=31169 vsize=1018996 CPUtime=10.26 cores=0,2,4,6
/proc/31171/stat : 31171 (gnovelty+2) R 31169 31171 31171 0 -1 4202496 959 0 0 0 1026 0 0 0 25 0 1 0 657865354 1043451904 697 33554432000 134512640 135663704 4289573584 18446744073709551615 134545772 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31171/statm: 254749 697 113 282 0 254465 0
[pid=31172] ppid=31160 vsize=303960 CPUtime=0.02 cores=0,2,4,6
/proc/31172/stat : 31172 (python) S 31160 31160 30970 0 -1 4202560 910 0 0 0 1 1 0 0 18 0 1 0 657865354 311255040 6691 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 13574218712801575170 0 0 17 0 0 0 0
/proc/31172/statm: 75990 6691 372 364 0 11902 0
[pid=31174] ppid=31172 vsize=421428 CPUtime=10.22 cores=0,2,4,6
/proc/31174/stat : 31174 (adaptg2wsat2009) R 31172 31174 31174 0 -1 4202496 2046 0 0 0 1022 0 0 0 25 0 1 0 657865355 431542272 1781 33554432000 134512640 135107411 4292515024 18446744073709551615 134532107 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31174/statm: 105357 1781 59 146 0 104233 0
[pid=31175] ppid=31160 vsize=314216 CPUtime=0.01 cores=0,2,4,6
/proc/31175/stat : 31175 (python) S 31160 31160 30970 0 -1 4202560 912 0 0 0 0 1 0 0 18 0 1 0 657865355 321757184 6695 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 13574218712801575170 0 0 17 6 0 0 0
/proc/31175/statm: 78554 6695 372 364 0 14463 0
[pid=31177] ppid=31175 vsize=421428 CPUtime=10.13 cores=0,2,4,6
/proc/31177/stat : 31177 (adaptg2wsat2009) R 31175 31177 31177 0 -1 4202496 2046 0 0 0 1013 0 0 0 25 0 1 0 657865356 431542272 1781 33554432000 134512640 135107411 4293282768 18446744073709551615 134532224 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31177/statm: 105357 1781 59 146 0 104233 0
[pid=31160/tid=31167] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31167/stat : 31167 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 5 0 657865353 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 0 0 0 0
[pid=31160/tid=31170] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31170/stat : 31170 (python) S 31158 31160 30970 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 5 0 657865353 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
[pid=31160/tid=31173] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31173/stat : 31173 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 5 0 657865354 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 0 0 0 0
[pid=31160/tid=31176] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31176/stat : 31176 (python) S 31158 31160 30970 0 -1 4202560 27 933 0 0 0 0 0 0 24 0 5 0 657865356 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 42.82
Current children cumulated vsize (KiB) 4400288

[startup+25.5015 s]
/proc/loadavg: 5.17 3.57 3.27 9/327 31178
/proc/meminfo: memFree=11544236/32951124 swapFree=66050672/67111528
[pid=31160] ppid=31158 vsize=324044 CPUtime=1.86 cores=0,2,4,6
/proc/31160/stat : 31160 (python) S 31158 31160 30970 0 -1 4202496 15484 933 1 0 175 11 0 0 24 0 5 0 657865113 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 2 18446744073709551615 0 0 17 6 0 0 9
/proc/31160/statm: 81011 8209 1883 364 0 16920 0
[pid=31166] ppid=31160 vsize=283516 CPUtime=0.05 cores=0,2,4,6
/proc/31166/stat : 31166 (python) S 31160 31160 30970 0 -1 4202560 894 0 0 0 3 2 0 0 18 0 1 0 657865352 290320384 6606 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321487 0 0 16781312 514 13574218712801575171 0 0 17 0 0 0 0
/proc/31166/statm: 70879 6606 378 364 0 6797 0
[pid=31168] ppid=31166 vsize=1018996 CPUtime=23.09 cores=0,2,4,6
/proc/31168/stat : 31168 (gnovelty+2) R 31166 31168 31168 0 -1 4202496 959 0 0 0 2309 0 0 0 25 0 1 0 657865353 1043451904 697 33554432000 134512640 135663704 4289592848 18446744073709551615 134546050 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31168/statm: 254749 697 113 282 0 254465 0
[pid=31169] ppid=31160 vsize=293704 CPUtime=0.04 cores=0,2,4,6
/proc/31169/stat : 31169 (python) S 31160 31160 30970 0 -1 4202560 912 0 0 0 3 1 0 0 18 0 1 0 657865353 300752896 6687 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 13574218712801575170 0 0 17 4 0 0 0
/proc/31169/statm: 73426 6687 372 364 0 9341 0
[pid=31171] ppid=31169 vsize=1018996 CPUtime=23.03 cores=0,2,4,6
/proc/31171/stat : 31171 (gnovelty+2) R 31169 31171 31171 0 -1 4202496 959 0 0 0 2303 0 0 0 25 0 1 0 657865354 1043451904 697 33554432000 134512640 135663704 4289573584 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31171/statm: 254749 697 113 282 0 254465 0
[pid=31172] ppid=31160 vsize=303960 CPUtime=0.05 cores=0,2,4,6
/proc/31172/stat : 31172 (python) S 31160 31160 30970 0 -1 4202560 916 0 0 0 3 2 0 0 18 0 1 0 657865354 311255040 6691 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 13574218712801575170 0 0 17 0 0 0 0
/proc/31172/statm: 75990 6691 372 364 0 11902 0
[pid=31174] ppid=31172 vsize=421428 CPUtime=22.96 cores=0,2,4,6
/proc/31174/stat : 31174 (adaptg2wsat2009) R 31172 31174 31174 0 -1 4202496 2046 0 0 0 2296 0 0 0 25 0 1 0 657865355 431542272 1781 33554432000 134512640 135107411 4292515024 18446744073709551615 134539283 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31174/statm: 105357 1781 59 146 0 104233 0
[pid=31175] ppid=31160 vsize=314216 CPUtime=0.05 cores=0,2,4,6
/proc/31175/stat : 31175 (python) S 31160 31160 30970 0 -1 4202560 916 0 0 0 2 3 0 0 18 0 1 0 657865355 321757184 6695 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 13574218712801575170 0 0 17 6 0 0 0
/proc/31175/statm: 78554 6695 372 364 0 14463 0
[pid=31177] ppid=31175 vsize=421428 CPUtime=22.77 cores=0,2,4,6
/proc/31177/stat : 31177 (adaptg2wsat2009) R 31175 31177 31177 0 -1 4202496 2046 0 0 0 2277 0 0 0 25 0 1 0 657865356 431542272 1781 33554432000 134512640 135107411 4293282768 18446744073709551615 134531613 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31177/statm: 105357 1781 59 146 0 104233 0
[pid=31160/tid=31167] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31167/stat : 31167 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 5 0 657865353 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 0 0 0 0
[pid=31160/tid=31170] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31170/stat : 31170 (python) S 31158 31160 30970 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 5 0 657865353 331821056 8209 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
[pid=31160/tid=31173] ppid=31158 vsize=324044 CPUtime=0 cores=0,2,4,6

################
# More data... #
################

/proc/31174/stat : 31174 (adaptg2wsat2009) T 31172 31174 31174 0 -1 4202496 2046 0 0 0 9976 0 0 0 25 0 1 0 657865355 431542272 1781 33554432000 134512640 135107411 4292515024 18446744073709551615 134532313 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31174/statm: 105357 1781 59 146 0 104233 0
[pid=31172/tid=31229] ppid=31160 vsize=303960 CPUtime=0 cores=0,2,4,6
/proc/31172/task/31229/stat : 31229 (python) S 31160 31160 30970 0 -1 4202560 45 0 0 0 0 0 0 0 19 0 2 0 657875377 311255040 6720 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604444456843968 0 0 -1 4 0 0 0
[pid=31175] ppid=31160 vsize=314216 CPUtime=0.23 cores=0,2,4,6
/proc/31175/stat : 31175 (python) S 31160 31160 30970 0 -1 4202560 1095 0 0 0 12 11 0 0 19 0 2 0 657865355 321757184 6724 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/31175/statm: 78554 6724 400 364 0 14463 0
[pid=31177] ppid=31175 vsize=421428 CPUtime=98 cores=0,2,4,6
/proc/31177/stat : 31177 (adaptg2wsat2009) T 31175 31177 31177 0 -1 4202496 2046 0 0 0 9800 0 0 0 25 0 1 0 657865356 431542272 1781 33554432000 134512640 135107411 4293282768 18446744073709551615 134531613 0 0 16781312 0 18446744071562691360 0 0 17 6 0 0 0
/proc/31177/statm: 105357 1781 59 146 0 104233 0
[pid=31175/tid=31225] ppid=31160 vsize=314216 CPUtime=0 cores=0,2,4,6
/proc/31175/task/31225/stat : 31225 (python) S 31160 31160 30970 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657875281 321757184 6724 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604453634992896 0 0 -1 6 0 0 0
[pid=31226] ppid=31160 vsize=324472 CPUtime=0.23 cores=0,2,4,6
/proc/31226/stat : 31226 (python) S 31160 31160 30970 0 -1 4202560 1102 0 0 0 12 11 0 0 19 0 2 0 657875282 332259328 6728 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/31226/statm: 81118 6728 400 364 0 17024 0
[pid=31228] ppid=31226 vsize=421428 CPUtime=98.22 cores=0,2,4,6
/proc/31228/stat : 31228 (adaptg2wsat2009) T 31226 31228 31228 0 -1 4202496 2044 0 0 0 9822 0 0 0 25 0 1 0 657875282 431542272 1781 33554432000 134512640 135107411 4288727520 18446744073709551615 134531781 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31228/statm: 105357 1781 59 146 0 104233 0
[pid=31226/tid=31247] ppid=31160 vsize=324472 CPUtime=0 cores=0,2,4,6
/proc/31226/task/31247/stat : 31247 (python) S 31160 31160 30970 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657885106 332259328 6728 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604440278457024 0 0 -1 6 0 0 0
[pid=31232] ppid=31160 vsize=334728 CPUtime=0.23 cores=0,2,4,6
/proc/31232/stat : 31232 (python) S 31160 31160 30970 0 -1 4202560 1103 0 0 0 13 10 0 0 19 0 2 0 657875379 342761472 6732 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/31232/statm: 83682 6732 400 364 0 19585 0
[pid=31234] ppid=31232 vsize=1018996 CPUtime=98.26 cores=0,2,4,6
/proc/31234/stat : 31234 (gnovelty+2) T 31232 31234 31234 0 -1 4202496 960 0 0 0 9826 0 0 0 25 0 1 0 657875380 1043451904 698 33554432000 134512640 135663704 4292460240 18446744073709551615 134525213 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/31234/statm: 254749 698 113 282 0 254465 0
[pid=31232/tid=31251] ppid=31160 vsize=334728 CPUtime=0 cores=0,2,4,6
/proc/31232/task/31251/stat : 31251 (python) S 31160 31160 30970 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657885208 342761472 6732 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604443732280320 0 0 -1 6 0 0 0
[pid=31235] ppid=31160 vsize=345164 CPUtime=0.23 cores=0,2,4,6
/proc/31235/stat : 31235 (python) S 31160 31160 30970 0 -1 4202560 1103 0 0 0 13 10 0 0 19 0 2 0 657875381 353447936 6736 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/31235/statm: 86291 6736 400 364 0 22191 0
[pid=31237] ppid=31235 vsize=251288 CPUtime=97.92 cores=0,2,4,6
/proc/31237/stat : 31237 (FH) T 31235 31237 31237 0 -1 4202496 1945 0 0 0 9791 1 0 0 25 0 1 0 657875381 257318912 1682 33554432000 134512640 134936988 4286755472 18446744073709551615 134545187 0 0 16781312 0 18446744071562691360 0 0 17 2 0 0 0
/proc/31237/statm: 62822 1682 41 104 0 62326 0
[pid=31235/tid=31255] ppid=31160 vsize=345164 CPUtime=0 cores=0,2,4,6
/proc/31235/task/31255/stat : 31255 (python) S 31160 31160 30970 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 657885403 353447936 6736 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604453634990208 0 0 -1 4 0 0 0
[pid=31238] ppid=31160 vsize=355240 CPUtime=0.23 cores=0,2,4,6
/proc/31238/stat : 31238 (python) S 31160 31160 30970 0 -1 4202560 1128 0 0 0 11 12 0 0 19 0 2 0 657875382 363765760 6774 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31238/statm: 88810 6774 406 364 0 24707 0
[pid=31240] ppid=31238 vsize=421428 CPUtime=99.79 cores=0,2,4,6
/proc/31240/stat : 31240 (adaptg2wsat2009) T 31238 31240 31240 0 -1 4202496 2053 0 0 0 9979 0 0 0 25 0 1 0 657875382 431542272 1781 33554432000 134512640 135107411 4291288800 18446744073709551615 134532371 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31240/statm: 105357 1781 59 146 0 104233 0
[pid=31238/tid=31259] ppid=31160 vsize=355240 CPUtime=0 cores=0,2,4,6
/proc/31238/task/31259/stat : 31259 (python) S 31160 31160 30970 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 657885406 363765760 6774 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604444456841280 0 0 -1 4 0 0 0
[pid=31248] ppid=31160 vsize=365600 CPUtime=0.23 cores=0,2,4,6
/proc/31248/stat : 31248 (python) S 31160 31160 30970 0 -1 4202560 1153 0 0 0 11 12 0 0 19 0 2 0 657885107 374374400 6778 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31248/statm: 91400 6778 406 364 0 27294 0
[pid=31250] ppid=31248 vsize=421428 CPUtime=99.97 cores=0,2,4,6
/proc/31250/stat : 31250 (adaptg2wsat2009) T 31248 31250 31250 0 -1 4202496 2050 0 0 0 9995 2 0 0 25 0 1 0 657885107 431542272 1781 33554432000 134512640 135107411 4292727968 18446744073709551615 134531435 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31250/statm: 105357 1781 59 146 0 104233 0
[pid=31248/tid=31269] ppid=31160 vsize=365600 CPUtime=0 cores=0,2,4,6
/proc/31248/task/31269/stat : 31269 (python) S 31160 31160 30970 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657895130 374374400 6778 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604445307468736 0 0 -1 4 0 0 0
[pid=31252] ppid=31160 vsize=376080 CPUtime=0.24 cores=0,2,4,6
/proc/31252/stat : 31252 (python) S 31160 31160 30970 0 -1 4202560 1119 0 0 0 13 11 0 0 19 0 2 0 657885209 385105920 6776 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31252/statm: 94020 6776 400 364 0 29911 0
[pid=31254] ppid=31252 vsize=251292 CPUtime=99.47 cores=0,2,4,6
/proc/31254/stat : 31254 (FH) T 31252 31254 31254 0 -1 4202496 1950 0 0 0 9947 0 0 0 25 0 1 0 657885210 257323008 1682 33554432000 134512640 134936988 4293414016 18446744073709551615 134553554 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31254/statm: 62823 1682 41 104 0 62327 0
[pid=31252/tid=31273] ppid=31160 vsize=376080 CPUtime=0 cores=0,2,4,6
/proc/31252/task/31273/stat : 31273 (python) S 31160 31160 30970 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657895233 385105920 6776 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604443606205504 0 0 -1 0 0 0 0
[pid=31256] ppid=31160 vsize=386128 CPUtime=0.71 cores=0,2,4,6
/proc/31256/stat : 31256 (python) S 31160 31160 30970 0 -1 4202560 1182 0 0 0 39 32 0 0 18 0 2 0 657885404 395395072 6817 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31256/statm: 96532 6817 400 364 0 32420 0
[pid=31258] ppid=31256 vsize=268424 CPUtime=291.46 cores=0,2,4,6
/proc/31258/stat : 31258 (hybridGM3) T 31256 31258 31258 0 -1 4202496 5812 0 0 0 29145 1 0 0 25 0 1 0 657885405 274866176 5544 33554432000 134512640 135204385 4292377216 18446744073709551615 134538282 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31258/statm: 67106 5544 63 169 0 66936 0
[pid=31256/tid=31294] ppid=31160 vsize=386128 CPUtime=0 cores=0,2,4,6
/proc/31256/task/31294/stat : 31294 (python) S 31160 31160 30970 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 657905058 395395072 6817 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604440278456128 0 0 -1 0 0 0 0
[pid=31260] ppid=31160 vsize=396640 CPUtime=0.24 cores=0,2,4,6
/proc/31260/stat : 31260 (python) S 31160 31160 30970 0 -1 4202560 1134 0 0 0 12 12 0 0 19 0 2 0 657885408 406159360 6821 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31260/statm: 99160 6821 400 364 0 35045 0
[pid=31262] ppid=31260 vsize=1018996 CPUtime=97.32 cores=0,2,4,6
/proc/31262/stat : 31262 (gnovelty+2) T 31260 31262 31262 0 -1 4202496 964 0 0 0 9732 0 0 0 25 0 1 0 657885408 1043451904 697 33554432000 134512640 135663704 4289098240 18446744073709551615 134525260 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31262/statm: 254749 697 113 282 0 254465 0
[pid=31260/tid=31277] ppid=31160 vsize=396640 CPUtime=0 cores=0,2,4,6
/proc/31260/task/31277/stat : 31277 (python) S 31160 31160 30970 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 657895336 406159360 6821 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604443484519232 0 0 -1 0 0 0 0
[pid=31270] ppid=31160 vsize=407028 CPUtime=0.51 cores=0,2,4,6
/proc/31270/stat : 31270 (python) S 31160 31160 30970 0 -1 4202560 1178 0 0 0 26 25 0 0 18 0 2 0 657895131 416796672 6837 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 18446744073709551615 0 0 17 6 0 0 0
/proc/31270/statm: 101757 6837 400 364 0 37639 0
[pid=31272] ppid=31270 vsize=268424 CPUtime=200.82 cores=0,2,4,6
/proc/31272/stat : 31272 (hybridGM3) R 31270 31272 31272 0 -1 4202496 5815 0 0 0 20080 2 0 0 25 0 1 0 657895131 274866176 5543 33554432000 134512640 135204385 4291525888 18446744073709551615 134538284 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/31272/statm: 67106 5543 63 169 0 66936 0
[pid=31270/tid=31306] ppid=31160 vsize=407028 CPUtime=0 cores=0,2,4,6
/proc/31270/task/31306/stat : 31306 (python) S 31160 31160 30970 0 -1 4202560 48 0 0 0 0 0 0 0 19 0 2 0 657914690 416796672 6837 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604449469901888 0 0 -1 6 0 0 0
[pid=31274] ppid=31160 vsize=417196 CPUtime=0.25 cores=0,2,4,6
/proc/31274/stat : 31274 (python) S 31160 31160 30970 0 -1 4202560 1150 0 0 0 12 13 0 0 19 0 2 0 657895234 427208704 6874 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 0 0 0 0
/proc/31274/statm: 104299 6874 400 364 0 40178 0
[pid=31276] ppid=31274 vsize=421428 CPUtime=99.96 cores=0,2,4,6
/proc/31276/stat : 31276 (adaptg2wsat2009) T 31274 31276 31276 0 -1 4202496 2050 0 0 0 9996 0 0 0 25 0 1 0 657895235 431542272 1781 33554432000 134512640 135107411 4291099936 18446744073709551615 134531656 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31276/statm: 105357 1781 59 146 0 104233 0
[pid=31274/tid=31295] ppid=31160 vsize=417196 CPUtime=0 cores=0,2,4,6
/proc/31274/task/31295/stat : 31295 (python) S 31160 31160 30970 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 657905260 427208704 6874 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604437821637696 0 0 -1 0 0 0 0
[pid=31278] ppid=31160 vsize=427600 CPUtime=0.49 cores=0,2,4,6
/proc/31278/stat : 31278 (python) S 31160 31160 30970 0 -1 4202560 1095 0 0 0 25 24 0 0 18 0 1 0 657895337 437862400 6857 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 13574218712801575169 0 0 17 4 0 0 0
/proc/31278/statm: 106900 6857 380 364 0 42776 0
[pid=31280] ppid=31278 vsize=268420 CPUtime=193.89 cores=0,2,4,6
/proc/31280/stat : 31280 (hybridGM3) R 31278 31280 31280 0 -1 4202496 5818 0 0 0 19389 0 0 0 25 0 1 0 657895338 274862080 5543 33554432000 134512640 135204385 4293008240 18446744073709551615 134538432 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/31280/statm: 67105 5543 63 169 0 66935 0
[pid=31296] ppid=31160 vsize=437912 CPUtime=0.25 cores=0,2,4,6
/proc/31296/stat : 31296 (python) S 31160 31160 30970 0 -1 4202560 1207 0 0 0 12 13 0 0 19 0 2 0 657905261 448421888 6905 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389128139 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/31296/statm: 109478 6905 408 364 0 45351 0
[pid=31298] ppid=31296 vsize=251296 CPUtime=98.78 cores=0,2,4,6
/proc/31298/stat : 31298 (FH) T 31296 31298 31298 0 -1 4202496 1963 0 0 0 9878 0 0 0 25 0 1 0 657905261 257327104 1682 33554432000 134512640 134936988 4288846688 18446744073709551615 134552304 0 0 16781312 0 18446744071562691360 0 0 17 0 0 0 0
/proc/31298/statm: 62824 1682 41 104 0 62328 0
[pid=31296/tid=31307] ppid=31160 vsize=437912 CPUtime=0 cores=0,2,4,6
/proc/31296/task/31307/stat : 31307 (python) S 31160 31160 30970 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 657915187 448421888 6905 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 514 18446604453329154240 0 0 -1 4 0 0 0
[pid=31303] ppid=31160 vsize=448032 CPUtime=0.01 cores=0,2,4,6
/proc/31303/stat : 31303 (python) S 31160 31160 30970 0 -1 4202560 1026 0 0 0 1 0 0 0 18 0 1 0 657914583 458784768 6894 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 7166009039086649600 0 0 17 4 0 0 0
/proc/31303/statm: 112008 6894 380 364 0 47878 0
[pid=31305] ppid=31303 vsize=268420 CPUtime=6.73 cores=0,2,4,6
/proc/31305/stat : 31305 (hybridGM3) R 31303 31305 31305 0 -1 4202496 5825 0 0 0 673 0 0 0 25 0 1 0 657914586 274862080 5543 33554432000 134512640 135204385 4288037808 18446744073709551615 134538068 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/31305/statm: 67105 5543 63 169 0 66935 0
[pid=31308] ppid=31160 vsize=458572 CPUtime=0 cores=0,2,4,6
/proc/31308/stat : 31308 (python) S 31160 31160 30970 0 -1 4202560 824 0 0 0 0 0 0 0 19 0 1 0 657915188 469577728 6873 33554432000 4194304 5683644 140736837937840 18446744073709551615 255377321542 0 0 16781312 514 7166001342505287679 0 0 17 4 0 0 0
/proc/31308/statm: 114643 6873 346 364 0 50510 0
[pid=31310] ppid=31308 vsize=459772 CPUtime=0.81 cores=0,2,4,6
/proc/31310/stat : 31310 (TNM) R 31308 31310 31310 0 -1 4202496 1925 0 0 0 81 0 0 0 23 0 1 0 657915188 470806528 1652 33554432000 134512640 135008748 4294711504 18446744073709551615 134529183 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/31310/statm: 114943 1652 46 122 0 113843 0
[pid=31160/tid=31167] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31167/stat : 31167 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 657865353 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 0 0 0 0
[pid=31160/tid=31170] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31170/stat : 31170 (python) S 31158 31160 30970 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 657865353 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
[pid=31160/tid=31173] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31173/stat : 31173 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 19 0 657865354 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 0 0 0 0
[pid=31160/tid=31176] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31176/stat : 31176 (python) S 31158 31160 30970 0 -1 4202560 27 933 0 0 0 0 0 0 24 0 19 0 657865356 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
[pid=31160/tid=31227] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31227/stat : 31227 (python) S 31158 31160 30970 0 -1 4202560 28 933 0 0 0 0 0 0 20 0 19 0 657875282 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 6 0 0 0
[pid=31160/tid=31233] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31233/stat : 31233 (python) S 31158 31160 30970 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 19 0 657875380 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 2 0 0 0
[pid=31160/tid=31236] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31236/stat : 31236 (python) S 31158 31160 30970 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 657875381 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31239] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31239/stat : 31239 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 22 0 19 0 657875382 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31249] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31249/stat : 31249 (python) S 31158 31160 30970 0 -1 4202560 24 933 0 0 0 0 0 0 20 0 19 0 657885107 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31253] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31253/stat : 31253 (python) S 31158 31160 30970 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 19 0 657885210 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31257] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31257/stat : 31257 (python) S 31158 31160 30970 0 -1 4202560 26 933 0 0 0 0 0 0 18 0 19 0 657885405 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31261] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31261/stat : 31261 (python) S 31158 31160 30970 0 -1 4202560 25 933 0 0 0 0 0 0 21 0 19 0 657885408 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31271] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31271/stat : 31271 (python) S 31158 31160 30970 0 -1 4202560 30 933 0 0 0 0 0 0 18 0 19 0 657895131 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31275] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31275/stat : 31275 (python) S 31158 31160 30970 0 -1 4202560 32 933 0 0 0 0 0 0 20 0 19 0 657895234 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31279] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31279/stat : 31279 (python) S 31158 31160 30970 0 -1 4202560 25 933 0 0 0 0 0 0 20 0 19 0 657895338 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31297] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31297/stat : 31297 (python) S 31158 31160 30970 0 -1 4202560 29 933 0 0 0 0 0 0 20 0 19 0 657905261 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31304] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31304/stat : 31304 (python) S 31158 31160 30970 0 -1 4202560 27 933 0 0 0 0 0 0 20 0 19 0 657914584 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
[pid=31160/tid=31309] ppid=31158 vsize=468220 CPUtime=0 cores=0,2,4,6
/proc/31160/task/31309/stat : 31309 (python) S 31158 31160 30970 0 -1 4202560 26 933 0 0 0 0 0 0 20 0 19 0 657915188 479457280 8442 33554432000 4194304 5683644 140736837937840 18446744073709551615 255389125041 0 0 16781312 2 18446604453528310144 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1988.2
Current children cumulated vsize (KiB) 16046140

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 31160 and gives
#  childrusage.ru_utime.tv_sec=2
#  childrusage.ru_utime.tv_usec=66685
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=274958
# CPU time returned by wait4() is 2.34164
# while last known CPU time is 1988.2
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 501.768
CPU time (s): 1988.2
CPU user time (s): 1985.74
CPU system time (s): 2.46
CPU usage (%): 396.239
Max. virtual memory (cumulated for all children) (KiB): 16046140

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2.06669
system time used= 0.274958
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 31200
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1357
involuntary context switches= 72

runsolver used 3.52446 second user time and 6.549 second system time

The end

Launcher Data 

Begin job on node135 at 2011-05-15 14:31:12
IDJOB=3378107
IDBENCH=82516
IDSOLVER=1859
FILE ID=node135/3378107-1305462672
RUNJOBID= node135-1305462647-30988
PBS_JOBID= 13324970
Free space on /tmp= 73340 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/large/unif-k5-r20-v1500-c30000-S976428817-077-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378107-1305462672/watcher-3378107-1305462672 -o /tmp/evaluation-result-3378107-1305462672/solver-3378107-1305462672 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378107-1305462672.cnf -seed 1373343021 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4d2cfd51677809bb15d90c916352e4e2
RANDOM SEED=1373343021

node135.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6640.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11606688 kB
Buffers:        296060 kB
Cached:        6328684 kB
SwapCached:      50684 kB
Active:       10231744 kB
Inactive:     10854920 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11606688 kB
SwapTotal:    67111528 kB
SwapFree:     66050672 kB
Dirty:           66132 kB
Writeback:           8 kB
AnonPages:    14416932 kB
Mapped:          25148 kB
Slab:           144020 kB
PageTables:      50028 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 59615872 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73304 MiB
End job on node135 at 2011-05-15 14:39:36