Trace number 3322060

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-randomUNSAT 2901.56 412.134

General information on the benchmark

Namerandom/medium/
unif-k7-r89-v60-c5340-S642174893-006.UNKNOWN.cnf
MD5SUM6df48a49b3cf6574a0f4cdf651f82d66
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark377.49
Satisfiable
(Un)Satisfiability was proved
Number of variables60
Number of clauses5340
Sum of the clauses size37380
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 55340

Solver Data 

0.32/0.46	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.46	c INFO: solving HOME/instance-3322060-1304371974.cnf
0.32/0.46	c INFO: computing task features
0.41/0.50	c INFO: computed variable-clause graph statistics
0.41/0.50	c INFO: computed clause balance statistics
0.51/0.68	c INFO: computed variable balance statistics
0.61/0.76	c INFO: computed small-clause counts
0.61/0.76	c INFO: computed Horn-clause counts
0.61/0.77	c INFO: computed variable graph statistics
0.61/0.77	c INFO: collected features for HOME/instance-3322060-1304371974.cnf in 0.30 s
0.71/0.80	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.56)
0.71/0.84	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.10)
0.71/0.88	c INFO: running adaptg2wsat2009++@0 for 97 with 39999 remaining (b = 0.05)
0.81/0.92	c INFO: running adaptg2wsat2009++@0 for 97 with 39999 remaining (b = 0.03)
0.81/0.95	c INFO: running adaptg2wsat2009++@0 for 97 with 39999 remaining (b = 0.03)
0.81/0.99	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.02)
1.47/1.03	c INFO: running FH@0 for 97 with 39999 remaining (b = 0.02)
1.47/1.10	c INFO: running march_hi@0 for 2134 with 39999 remaining (b = 0.32)
694.85/100.23	c INFO: running adaptg2wsat2009++@0 for 97 with 39900 remaining (b = 0.01)
699.39/101.00	c INFO: running march_hi@0 for 2134 with 39899 remaining (b = 0.31)
700.00/101.05	c INFO: running march_hi@0 for 388 with 39899 remaining (b = 0.12)
700.00/101.10	c INFO: running march_hi@0 for 2134 with 39899 remaining (b = 0.28)
701.16/101.16	c INFO: running march_hi@0 for 388 with 39899 remaining (b = 0.10)
1036.94/149.21	c INFO: running march_hi@0 for 388 with 39851 remaining (b = 0.08)
1065.54/153.38	c INFO: running adaptg2wsat2009++@0 for 97 with 39847 remaining (b = 0.01)
1394.45/200.43	c INFO: running adaptg2wsat2009++@0 for 97 with 39800 remaining (b = 0.01)
1962.45/279.64	c INFO: running adaptg2wsat2009++@0 for 97 with 39720 remaining (b = 0.01)
2213.87/315.67	c INFO: running adaptg2wsat2009++@100 for 97 with 39684 remaining (b = 0.01)
2774.43/393.89	c INFO: running FH@0 for 97 with 39606 remaining (b = 0.01)
2592.11/412.11	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322060-1304371974/watcher-3322060-1304371974 -o /tmp/evaluation-result-3322060-1304371974/solver-3322060-1304371974 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3322060-1304371974.cnf -seed 1809150463 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.90 8.96 8.99 3/207 29182
/proc/meminfo: memFree=26988524/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=15752 CPUtime=0 cores=0-7
/proc/29182/stat : 29182 (python) R 29180 29182 29100 0 -1 4202496 839 0 0 0 0 0 0 0 17 0 1 0 548795262 16130048 722 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550535680 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29182/statm: 3939 723 334 364 0 426 0

[startup+0.048982 s]
/proc/loadavg: 8.90 8.96 8.99 3/207 29182
/proc/meminfo: memFree=26988524/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=29788 CPUtime=0.03 cores=0-7
/proc/29182/stat : 29182 (python) R 29180 29182 29100 0 -1 4202496 1452 0 0 0 3 0 0 0 17 0 1 0 548795262 30502912 1293 33554432000 4194304 5683644 140734608028304 18446744073709551615 5114087 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29182/statm: 7447 1293 443 364 0 817 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29788

[startup+0.100973 s]
/proc/loadavg: 8.90 8.96 8.99 3/207 29182
/proc/meminfo: memFree=26988524/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=104228 CPUtime=0.06 cores=0-7
/proc/29182/stat : 29182 (python) R 29180 29182 29100 0 -1 4202496 3087 0 0 0 5 1 0 0 17 0 1 0 548795262 106729472 2767 33554432000 4194304 5683644 140734608028304 18446744073709551615 5114063 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29182/statm: 26057 2767 847 364 0 1805 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 104228

[startup+0.300932 s]
/proc/loadavg: 8.90 8.96 8.99 3/207 29182
/proc/meminfo: memFree=26988524/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=194412 CPUtime=0.23 cores=0-7
/proc/29182/stat : 29182 (python) S 29180 29182 29100 0 -1 4202496 6905 935 1 0 17 6 0 0 17 0 1 0 548795262 199077888 4959 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550535680 0 2147483391 16781312 2 18446744071699469502 0 0 17 2 0 0 0
/proc/29182/statm: 48603 4959 1431 364 0 3722 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 194412

[startup+0.700861 s]
/proc/loadavg: 8.90 8.96 8.99 3/207 29182
/proc/meminfo: memFree=26988524/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=279552 CPUtime=0.61 cores=0-7
/proc/29182/stat : 29182 (python) R 29180 29182 29100 0 -1 4202496 9751 935 1 0 53 8 0 0 17 0 1 0 548795262 286261248 7274 33554432000 4194304 5683644 140734608028304 18446744073709551615 47729191558620 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/29182/statm: 69888 7274 1827 364 0 5812 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 279552

[startup+1.50068 s]
/proc/loadavg: 8.90 8.96 8.99 10/225 29204
/proc/meminfo: memFree=26938656/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=361332 CPUtime=0.98 cores=0-7
/proc/29182/stat : 29182 (python) S 29180 29182 29100 0 -1 4202496 16285 935 1 0 88 10 0 0 25 0 9 0 548795262 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/29182/statm: 90333 7300 1891 364 0 26230 0
[pid=29188] ppid=29182 vsize=278700 CPUtime=0 cores=0-7
/proc/29188/stat : 29188 (python) S 29182 29182 29100 0 -1 4202560 737 0 0 0 0 0 0 0 19 0 1 0 548795343 285388800 5614 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557711 0 0 16781312 514 69195844443701247 0 0 17 2 0 0 0
/proc/29188/statm: 69675 5614 354 364 0 5593 0
[pid=29190] ppid=29188 vsize=1017932 CPUtime=0.68 cores=0-7
/proc/29190/stat : 29190 (gnovelty+2) R 29188 29190 29190 0 -1 4202496 514 0 0 0 68 0 0 0 22 0 1 0 548795343 1042362368 248 33554432000 134512640 135663704 4286985584 18446744073709551615 134524534 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29190/statm: 254483 248 113 282 0 254199 0
[pid=29191] ppid=29182 vsize=289352 CPUtime=0 cores=0-7
/proc/29191/stat : 29191 (python) S 29182 29182 29100 0 -1 4202560 737 0 0 0 0 0 0 0 21 0 1 0 548795347 296296448 5696 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 11026469955632627711 0 0 17 0 0 0 0
/proc/29191/statm: 72338 5696 348 364 0 8253 0
[pid=29193] ppid=29191 vsize=1017932 CPUtime=0.64 cores=0-7
/proc/29193/stat : 29193 (gnovelty+2) R 29191 29193 29193 0 -1 4202496 510 0 0 0 64 0 0 0 25 0 1 0 548795347 1042362368 247 33554432000 134512640 135663704 4288384368 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29193/statm: 254483 247 113 282 0 254199 0
[pid=29194] ppid=29182 vsize=299400 CPUtime=0 cores=0-7
/proc/29194/stat : 29194 (python) S 29182 29182 29100 0 -1 4202560 720 0 0 0 0 0 0 0 19 0 1 0 548795350 306585600 5627 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 6959851158634495999 0 0 17 4 0 0 0
/proc/29194/statm: 74850 5627 344 364 0 10762 0
[pid=29196] ppid=29194 vsize=416652 CPUtime=0.61 cores=0-7
/proc/29196/stat : 29196 (adaptg2wsat2009) R 29194 29196 29196 0 -1 4202496 570 0 0 0 61 0 0 0 22 0 1 0 548795351 426651648 303 33554432000 134512640 135107411 4286758784 18446744073709551615 134523442 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29196/statm: 104163 303 49 146 0 103039 0
[pid=29197] ppid=29182 vsize=310000 CPUtime=0 cores=0-7
/proc/29197/stat : 29197 (python) S 29182 29182 29100 0 -1 4202560 730 0 0 0 0 0 0 0 19 0 1 0 548795354 317440000 5717 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 17610838163964559359 0 0 17 1 0 0 0
/proc/29197/statm: 77500 5717 344 364 0 13409 0
[pid=29199] ppid=29197 vsize=416652 CPUtime=0.57 cores=0-7
/proc/29199/stat : 29199 (adaptg2wsat2009) R 29197 29199 29199 0 -1 4202496 568 0 0 0 57 0 0 0 22 0 1 0 548795355 426651648 302 33554432000 134512640 135107411 4289059104 18446744073709551615 134523503 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29199/statm: 104163 302 49 146 0 103039 0
[pid=29200] ppid=29182 vsize=320256 CPUtime=0 cores=0-7
/proc/29200/stat : 29200 (python) S 29182 29182 29100 0 -1 4202560 725 0 0 0 0 0 0 0 20 0 1 0 548795358 327942144 5721 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 4432293278947213311 0 0 17 2 0 0 0
/proc/29200/statm: 80064 5721 344 364 0 15970 0
[pid=29202] ppid=29200 vsize=416652 CPUtime=0.31 cores=0-7
/proc/29202/stat : 29202 (adaptg2wsat2009) R 29200 29202 29202 0 -1 4202496 568 0 0 0 31 0 0 0 22 0 1 0 548795358 426651648 302 33554432000 134512640 135107411 4287748800 18446744073709551615 134523457 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29202/statm: 104163 302 49 146 0 103039 0
[pid=29203] ppid=29182 vsize=330036 CPUtime=0 cores=0-7
/proc/29203/stat : 29203 (python) S 29182 29182 29100 0 -1 4202560 759 0 0 0 0 0 0 0 22 0 1 0 548795362 337956864 5627 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 6959843462053101567 0 0 17 7 0 0 0
/proc/29203/statm: 82509 5627 348 364 0 18412 0
[pid=29182/tid=29189] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29189/stat : 29189 (python) S 29180 29182 29100 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 9 0 548795343 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29192] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29192/stat : 29192 (python) S 29180 29182 29100 0 -1 4202560 21 935 0 0 0 0 0 0 22 0 9 0 548795347 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 6 0 0 0
[pid=29182/tid=29195] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29195/stat : 29195 (python) S 29180 29182 29100 0 -1 4202560 22 935 0 0 0 0 0 0 20 0 9 0 548795351 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29198] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29198/stat : 29198 (python) S 29180 29182 29100 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 9 0 548795354 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29201] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29201/stat : 29201 (python) S 29180 29182 29100 0 -1 4202560 22 935 0 0 0 0 0 0 21 0 9 0 548795358 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 2 0 0 0
[pid=29182/tid=29204] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29204/stat : 29204 (python) S 29180 29182 29100 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 9 0 548795362 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3.79
Current children cumulated vsize (KiB) 5474896

[startup+3.10139 s]
/proc/loadavg: 8.90 8.96 8.99 10/232 29211
/proc/meminfo: memFree=26925712/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=361332 CPUtime=0.98 cores=0-7
/proc/29182/stat : 29182 (python) S 29180 29182 29100 0 -1 4202496 16285 935 1 0 88 10 0 0 25 0 9 0 548795262 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/29182/statm: 90333 7300 1891 364 0 26230 0
[pid=29188] ppid=29182 vsize=278700 CPUtime=0 cores=0-7
/proc/29188/stat : 29188 (python) S 29182 29182 29100 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 548795343 285388800 5614 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557711 0 0 16781312 514 69195844443668740 0 0 17 2 0 0 0
/proc/29188/statm: 69675 5614 354 364 0 5593 0
[pid=29190] ppid=29188 vsize=1017932 CPUtime=2.28 cores=0-7
/proc/29190/stat : 29190 (gnovelty+2) R 29188 29190 29190 0 -1 4202496 514 0 0 0 228 0 0 0 25 0 1 0 548795343 1042362368 248 33554432000 134512640 135663704 4286985584 18446744073709551615 134524548 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29190/statm: 254483 248 113 282 0 254199 0
[pid=29191] ppid=29182 vsize=289352 CPUtime=0 cores=0-7
/proc/29191/stat : 29191 (python) S 29182 29182 29100 0 -1 4202560 737 0 0 0 0 0 0 0 18 0 1 0 548795347 296296448 5696 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 11026469955632595201 0 0 17 0 0 0 0
/proc/29191/statm: 72338 5696 348 364 0 8253 0
[pid=29193] ppid=29191 vsize=1017932 CPUtime=2.24 cores=0-7
/proc/29193/stat : 29193 (gnovelty+2) R 29191 29193 29193 0 -1 4202496 510 0 0 0 224 0 0 0 25 0 1 0 548795347 1042362368 247 33554432000 134512640 135663704 4288384368 18446744073709551615 134525276 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29193/statm: 254483 247 113 282 0 254199 0
[pid=29194] ppid=29182 vsize=299400 CPUtime=0 cores=0-7
/proc/29194/stat : 29194 (python) S 29182 29182 29100 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 548795350 306585600 5631 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 6959851158634463489 0 0 17 4 0 0 0
/proc/29194/statm: 74850 5631 348 364 0 10762 0
[pid=29196] ppid=29194 vsize=416788 CPUtime=2.21 cores=0-7
/proc/29196/stat : 29196 (adaptg2wsat2009) R 29194 29196 29196 0 -1 4202496 630 0 0 0 221 0 0 0 25 0 1 0 548795351 426790912 363 33554432000 134512640 135107411 4286758784 18446744073709551615 134531239 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29196/statm: 104197 363 59 146 0 103073 0
[pid=29197] ppid=29182 vsize=310000 CPUtime=0 cores=0-7
/proc/29197/stat : 29197 (python) S 29182 29182 29100 0 -1 4202560 748 0 0 0 0 0 0 0 18 0 1 0 548795354 317440000 5721 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 17610838163964526852 0 0 17 1 0 0 0
/proc/29197/statm: 77500 5721 348 364 0 13409 0
[pid=29199] ppid=29197 vsize=416788 CPUtime=2.17 cores=0-7
/proc/29199/stat : 29199 (adaptg2wsat2009) R 29197 29199 29199 0 -1 4202496 629 0 0 0 217 0 0 0 25 0 1 0 548795355 426790912 363 33554432000 134512640 135107411 4289059104 18446744073709551615 134531674 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29199/statm: 104197 363 59 146 0 103073 0
[pid=29200] ppid=29182 vsize=320256 CPUtime=0 cores=0-7
/proc/29200/stat : 29200 (python) S 29182 29182 29100 0 -1 4202560 744 0 0 0 0 0 0 0 18 0 1 0 548795358 327942144 5725 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 4432293278947180804 0 0 17 2 0 0 0
/proc/29200/statm: 80064 5725 348 364 0 15970 0
[pid=29202] ppid=29200 vsize=416788 CPUtime=1.11 cores=0-7
/proc/29202/stat : 29202 (adaptg2wsat2009) R 29200 29202 29202 0 -1 4202496 629 0 0 0 111 0 0 0 25 0 1 0 548795358 426790912 363 33554432000 134512640 135107411 4287748800 18446744073709551615 134532371 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29202/statm: 104197 363 59 146 0 103073 0
[pid=29203] ppid=29182 vsize=330036 CPUtime=0 cores=0-7
/proc/29203/stat : 29203 (python) S 29182 29182 29100 0 -1 4202560 759 0 0 0 0 0 0 0 18 0 1 0 548795362 337956864 5627 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 6959843462053069057 0 0 17 7 0 0 0
/proc/29203/statm: 82509 5627 348 364 0 18412 0
[pid=29205] ppid=29203 vsize=1017932 CPUtime=2.04 cores=0-7
/proc/29205/stat : 29205 (gnovelty+2) R 29203 29205 29205 0 -1 4202496 513 0 0 0 204 0 0 0 25 0 1 0 548795364 1042362368 248 33554432000 134512640 135663704 4288386928 18446744073709551615 134524509 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/29205/statm: 254483 248 113 282 0 254199 0
[pid=29206] ppid=29182 vsize=340832 CPUtime=0 cores=0-7
/proc/29206/stat : 29206 (python) S 29182 29182 29100 0 -1 4202560 758 0 0 0 0 0 0 0 18 0 1 0 548795366 349011968 5749 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 16431989900296814849 0 0 17 7 0 0 0
/proc/29206/statm: 85208 5749 348 364 0 21108 0
[pid=29208] ppid=29206 vsize=246936 CPUtime=1.01 cores=0-7
/proc/29208/stat : 29208 (FH) R 29206 29208 29208 0 -1 4202496 590 0 0 0 101 0 0 0 25 0 1 0 548795368 252862464 326 33554432000 134512640 134936988 4286671840 18446744073709551615 134545312 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29208/statm: 61734 326 41 104 0 61238 0
[pid=29209] ppid=29182 vsize=351088 CPUtime=0 cores=0-7
/proc/29209/stat : 29209 (python) S 29182 29182 29100 0 -1 4202560 813 0 0 0 0 0 0 0 15 0 1 0 548795373 359514112 5759 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 14863097033501147400 0 0 17 7 0 0 0
/proc/29209/statm: 87772 5759 354 364 0 23669 0
[pid=29211] ppid=29209 vsize=2364 CPUtime=1.97 cores=0-7
/proc/29211/stat : 29211 (march_hi) R 29209 29211 29211 0 -1 4202496 1263 0 0 0 197 0 0 0 25 0 1 0 548795373 2420736 407 33554432000 4194304 4955518 140734628501520 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/29211/statm: 591 407 106 186 0 404 0
[pid=29182/tid=29189] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29189/stat : 29189 (python) S 29180 29182 29100 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 9 0 548795343 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29192] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29192/stat : 29192 (python) S 29180 29182 29100 0 -1 4202560 21 935 0 0 0 0 0 0 22 0 9 0 548795347 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 6 0 0 0
[pid=29182/tid=29195] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29195/stat : 29195 (python) S 29180 29182 29100 0 -1 4202560 22 935 0 0 0 0 0 0 20 0 9 0 548795351 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29198] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29198/stat : 29198 (python) S 29180 29182 29100 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 9 0 548795354 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29201] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29201/stat : 29201 (python) S 29180 29182 29100 0 -1 4202560 22 935 0 0 0 0 0 0 21 0 9 0 548795358 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 2 0 0 0
[pid=29182/tid=29204] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29204/stat : 29204 (python) S 29180 29182 29100 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 9 0 548795362 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29207] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29207/stat : 29207 (python) S 29180 29182 29100 0 -1 4202560 29 935 0 0 0 0 0 0 23 0 9 0 548795366 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 7 0 0 0
[pid=29182/tid=29210] ppid=29180 vsize=361332 CPUtime=0 cores=0-7
/proc/29182/task/29210/stat : 29210 (python) S 29180 29182 29100 0 -1 4202560 29 935 0 0 0 0 0 0 25 0 9 0 548795373 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 16.01
Current children cumulated vsize (KiB) 7434456

[startup+6.30181 s]
/proc/loadavg: 8.90 8.96 8.99 10/232 29211
/proc/meminfo: memFree=26922736/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=361332 CPUtime=0.98 cores=0-7
/proc/29182/stat : 29182 (python) S 29180 29182 29100 0 -1 4202496 16285 935 1 0 88 10 0 0 25 0 9 0 548795262 370003968 7300 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562364363 0 0 16781312 2 18446744073709551615 0 0 17 7 0 0 0
/proc/29182/statm: 90333 7300 1891 364 0 26230 0
[pid=29188] ppid=29182 vsize=278700 CPUtime=0 cores=0-7
/proc/29188/stat : 29188 (python) S 29182 29182 29100 0 -1 4202560 897 0 0 0 0 0 0 0 18 0 1 0 548795343 285388800 5637 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557711 0 0 16781312 514 69195844443668740 0 0 17 2 0 0 0
/proc/29188/statm: 69675 5637 377 364 0 5593 0

################
# More data... #
################

[pid=29220] ppid=29182 vsize=437136 CPUtime=0.59 cores=0-7
/proc/29220/stat : 29220 (python) S 29182 29182 29100 0 -1 4202560 842 0 0 0 30 29 0 0 15 0 1 0 548805362 447627264 5957 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 15885242625100316932 0 0 17 1 0 0 0
/proc/29220/statm: 109284 5957 371 364 0 45175 0
[pid=29223] ppid=29220 vsize=4692 CPUtime=258.69 cores=0-7
/proc/29223/stat : 29223 (march_hi) R 29220 29223 29223 0 -1 4202496 1743 0 0 0 25869 0 0 0 25 0 1 0 548805363 4804608 853 33554432000 4194304 4955518 140736686970352 18446744073709551615 4297472 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/29223/statm: 1173 853 106 186 0 986 0
[pid=29225] ppid=29182 vsize=447392 CPUtime=0.56 cores=0-7
/proc/29225/stat : 29225 (python) S 29182 29182 29100 0 -1 4202560 849 0 0 0 31 25 0 0 15 0 1 0 548805368 458129408 5953 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 15849363357367763204 0 0 17 4 0 0 0
/proc/29225/statm: 111848 5953 371 364 0 47736 0
[pid=29228] ppid=29225 vsize=4692 CPUtime=252.16 cores=0-7
/proc/29228/stat : 29228 (march_hi) R 29225 29228 29228 0 -1 4202496 1735 0 0 0 25216 0 0 0 25 0 1 0 548805369 4804608 841 33554432000 4194304 4955518 140734378868880 18446744073709551615 4297477 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29228/statm: 1173 841 106 186 0 986 0
[pid=29229] ppid=29182 vsize=457648 CPUtime=0.54 cores=0-7
/proc/29229/stat : 29229 (python) S 29182 29182 29100 0 -1 4202560 865 0 0 0 31 23 0 0 15 0 1 0 548805373 468631552 5957 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 4757846302425841924 0 0 17 6 0 0 0
/proc/29229/statm: 114412 5957 371 364 0 50297 0
[pid=29231] ppid=29229 vsize=4692 CPUtime=309.63 cores=0-7
/proc/29231/stat : 29231 (march_hi) R 29229 29231 29231 0 -1 4202496 1860 0 0 0 30963 0 0 0 25 0 1 0 548805375 4804608 964 33554432000 4194304 4955518 140733360831424 18446744073709551615 4224468 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29231/statm: 1173 964 106 186 0 986 0
[pid=29232] ppid=29182 vsize=467904 CPUtime=0.58 cores=0-7
/proc/29232/stat : 29232 (python) S 29182 29182 29100 0 -1 4202560 1042 0 0 0 31 27 0 0 15 0 1 0 548805378 479133696 5961 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 3403498665656942849 0 0 17 0 0 0 0
/proc/29232/statm: 116976 5961 371 364 0 52858 0
[pid=29234] ppid=29232 vsize=4692 CPUtime=283.2 cores=0-7
/proc/29234/stat : 29234 (march_hi) R 29232 29234 29234 0 -1 4202496 1806 0 0 0 28318 2 0 0 25 0 1 0 548805386 4804608 905 33554432000 4194304 4955518 140735342233360 18446744073709551615 4208638 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29234/statm: 1173 905 106 186 0 986 0
[pid=29237] ppid=29182 vsize=478160 CPUtime=0.46 cores=0-7
/proc/29237/stat : 29237 (python) S 29182 29182 29100 0 -1 4202560 1049 0 0 0 27 19 0 0 15 0 1 0 548810184 489635840 5986 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 3438054117094686979 0 0 17 0 0 0 0
/proc/29237/statm: 119540 5986 371 364 0 55419 0
[pid=29239] ppid=29237 vsize=4692 CPUtime=234.18 cores=0-7
/proc/29239/stat : 29239 (march_hi) R 29237 29239 29239 0 -1 4202496 1710 0 0 0 23418 0 0 0 25 0 1 0 548810184 4804608 805 33554432000 4194304 4955518 140737000872912 18446744073709551615 4238144 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/29239/statm: 1173 805 106 186 0 986 0
[pid=29249] ppid=29182 vsize=488704 CPUtime=0.02 cores=0-7
/proc/29249/stat : 29249 (python) S 29182 29182 29100 0 -1 4202560 1047 0 0 0 1 1 0 0 18 0 1 0 548834651 500432896 6076 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 15849363357367763204 0 0 17 1 0 0 0
/proc/29249/statm: 122176 6076 377 364 0 58052 0
[pid=29251] ppid=29249 vsize=246936 CPUtime=9.63 cores=0-7
/proc/29251/stat : 29251 (FH) R 29249 29251 29251 0 -1 4202496 604 0 0 0 963 0 0 0 25 0 1 0 548834652 252862464 326 33554432000 134512640 134936988 4289752160 18446744073709551615 134544807 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29251/statm: 61734 326 41 104 0 61238 0
[pid=29182/tid=29189] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29189/stat : 29189 (python) S 29180 29182 29100 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 16 0 548795343 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29192] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29192/stat : 29192 (python) S 29180 29182 29100 0 -1 4202560 21 935 0 0 0 0 0 0 22 0 16 0 548795347 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 6 0 0 0
[pid=29182/tid=29195] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29195/stat : 29195 (python) S 29180 29182 29100 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 16 0 548795351 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29198] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29198/stat : 29198 (python) S 29180 29182 29100 0 -1 4202560 31 935 0 0 0 0 0 0 18 0 16 0 548795354 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29201] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29201/stat : 29201 (python) S 29180 29182 29100 0 -1 4202560 27 935 0 0 0 0 0 0 18 0 16 0 548795358 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 2 0 0 0
[pid=29182/tid=29204] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29204/stat : 29204 (python) S 29180 29182 29100 0 -1 4202560 23 935 0 0 0 0 0 0 22 0 16 0 548795362 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29207] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29207/stat : 29207 (python) S 29180 29182 29100 0 -1 4202560 29 935 0 0 0 0 0 0 23 0 16 0 548795366 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 7 0 0 0
[pid=29182/tid=29210] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29210/stat : 29210 (python) S 29180 29182 29100 0 -1 4202560 29 935 0 0 0 0 0 0 25 0 16 0 548795373 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 7 0 0 0
[pid=29182/tid=29217] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29217/stat : 29217 (python) S 29180 29182 29100 0 -1 4202560 34 935 0 0 0 0 0 0 18 0 16 0 548805285 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29221] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29221/stat : 29221 (python) S 29180 29182 29100 0 -1 4202560 85 935 0 0 0 0 0 0 19 0 16 0 548805362 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29226] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29226/stat : 29226 (python) S 29180 29182 29100 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 16 0 548805368 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 1 0 0 0
[pid=29182/tid=29230] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29230/stat : 29230 (python) S 29180 29182 29100 0 -1 4202560 30 935 0 0 0 0 0 0 22 0 16 0 548805373 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29233] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29233/stat : 29233 (python) S 29180 29182 29100 0 -1 4202560 27 935 0 0 0 0 0 0 23 0 16 0 548805379 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29238] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29238/stat : 29238 (python) S 29180 29182 29100 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 16 0 548810184 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29250] ppid=29180 vsize=498948 CPUtime=0 cores=0-7
/proc/29182/task/29250/stat : 29250 (python) S 29180 29182 29100 0 -1 4202560 32 935 0 0 0 0 0 0 19 0 16 0 548834651 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2900.87
Current children cumulated vsize (KiB) 11470960

[startup+412.102 s]
/proc/loadavg: 9.07 9.08 9.02 12/261 29251
/proc/meminfo: memFree=26871424/32951124 swapFree=67111336/67111528
[pid=29182] ppid=29180 vsize=498948 CPUtime=992.98 cores=0-7
/proc/29182/stat : 29182 (python) S 29180 29182 29100 0 -1 4202496 22452 13051 1 0 143 12 99056 87 17 0 16 0 548795262 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562367807 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/29182/statm: 124737 7609 1906 364 0 60613 0
[pid=29209] ppid=29182 vsize=0 CPUtime=390.64 cores=0-7
/proc/29209/stat : 29209 (python) Z 29182 29182 29100 0 -1 4202588 1286 2081 0 0 264 67 38732 1 20 0 1 0 548795373 0 0 33554432000 0 0 0 0 0 0 0 16781824 2 18446744073709551615 0 0 17 7 0 0 0
/proc/29209/statm: 0 0 0 0 0 0 0
[pid=29216] ppid=29182 vsize=361344 CPUtime=0.3 cores=0-7
/proc/29216/stat : 29216 (python) S 29182 29182 29100 0 -1 4202560 1197 0 0 0 16 14 0 0 18 0 2 0 548805285 370016256 5810 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/29216/statm: 90336 5810 400 364 0 26230 0
[pid=29218] ppid=29216 vsize=416788 CPUtime=167.22 cores=0-7
/proc/29218/stat : 29218 (adaptg2wsat2009) R 29216 29218 29218 0 -1 4202496 634 0 0 0 16722 0 0 0 25 0 1 0 548805286 426790912 363 33554432000 134512640 135107411 4287043184 18446744073709551615 134531795 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/29218/statm: 104197 363 59 146 0 103073 0
[pid=29216/tid=29242] ppid=29182 vsize=361344 CPUtime=0 cores=0-7
/proc/29216/task/29242/stat : 29242 (python) S 29182 29182 29100 0 -1 4202560 51 0 0 0 0 0 0 0 19 0 2 0 548815300 370016256 5810 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 514 18446604463712010368 0 0 -1 2 0 0 0
[pid=29220] ppid=29182 vsize=437136 CPUtime=0.59 cores=0-7
/proc/29220/stat : 29220 (python) S 29182 29182 29100 0 -1 4202560 842 0 0 0 30 29 0 0 15 0 1 0 548805362 447627264 5957 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 15885242625100316932 0 0 17 1 0 0 0
/proc/29220/statm: 109284 5957 371 364 0 45175 0
[pid=29223] ppid=29220 vsize=4692 CPUtime=258.89 cores=0-7
/proc/29223/stat : 29223 (march_hi) R 29220 29223 29223 0 -1 4202496 1744 0 0 0 25889 0 0 0 25 0 1 0 548805363 4804608 854 33554432000 4194304 4955518 140736686970352 18446744073709551615 4235634 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/29223/statm: 1173 854 106 186 0 986 0
[pid=29225] ppid=29182 vsize=447392 CPUtime=0.56 cores=0-7
/proc/29225/stat : 29225 (python) S 29182 29182 29100 0 -1 4202560 849 0 0 0 31 25 0 0 15 0 1 0 548805368 458129408 5953 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 15849363357367763204 0 0 17 4 0 0 0
/proc/29225/statm: 111848 5953 371 364 0 47736 0
[pid=29228] ppid=29225 vsize=4692 CPUtime=252.26 cores=0-7
/proc/29228/stat : 29228 (march_hi) R 29225 29228 29228 0 -1 4202496 1735 0 0 0 25226 0 0 0 25 0 1 0 548805369 4804608 841 33554432000 4194304 4955518 140734378868880 18446744073709551615 4237804 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/29228/statm: 1173 841 106 186 0 986 0
[pid=29232] ppid=29182 vsize=467904 CPUtime=0.58 cores=0-7
/proc/29232/stat : 29232 (python) S 29182 29182 29100 0 -1 4202560 1042 0 0 0 31 27 0 0 15 0 1 0 548805378 479133696 5961 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 3403498665656942849 0 0 17 0 0 0 0
/proc/29232/statm: 116976 5961 371 364 0 52858 0
[pid=29234] ppid=29232 vsize=4692 CPUtime=283.4 cores=0-7
/proc/29234/stat : 29234 (march_hi) R 29232 29234 29234 0 -1 4202496 1807 0 0 0 28338 2 0 0 25 0 1 0 548805386 4804608 906 33554432000 4194304 4955518 140735342233360 18446744073709551615 4224471 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/29234/statm: 1173 906 106 186 0 986 0
[pid=29237] ppid=29182 vsize=478160 CPUtime=0.46 cores=0-7
/proc/29237/stat : 29237 (python) S 29182 29182 29100 0 -1 4202560 1049 0 0 0 27 19 0 0 15 0 1 0 548810184 489635840 5986 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 3438054117094686979 0 0 17 0 0 0 0
/proc/29237/statm: 119540 5986 371 364 0 55419 0
[pid=29239] ppid=29237 vsize=4692 CPUtime=234.38 cores=0-7
/proc/29239/stat : 29239 (march_hi) R 29237 29239 29239 0 -1 4202496 1710 0 0 0 23438 0 0 0 25 0 1 0 548810184 4804608 805 33554432000 4194304 4955518 140737000872912 18446744073709551615 4208667 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/29239/statm: 1173 805 106 186 0 986 0
[pid=29249] ppid=29182 vsize=488704 CPUtime=0.02 cores=0-7
/proc/29249/stat : 29249 (python) S 29182 29182 29100 0 -1 4202560 1047 0 0 0 1 1 0 0 18 0 1 0 548834651 500432896 6076 33554432000 4194304 5683644 140734608028304 18446744073709551615 271550557766 0 0 16781312 514 15849363357367763204 0 0 17 1 0 0 0
/proc/29249/statm: 122176 6076 377 364 0 58052 0
[pid=29251] ppid=29249 vsize=246936 CPUtime=9.83 cores=0-7
/proc/29251/stat : 29251 (FH) R 29249 29251 29251 0 -1 4202496 604 0 0 0 983 0 0 0 25 0 1 0 548834652 252862464 326 33554432000 134512640 134936988 4289752160 18446744073709551615 134554242 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/29251/statm: 61734 326 41 104 0 61238 0
[pid=29182/tid=29189] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29189/stat : 29189 (python) S 29180 29182 29100 0 -1 4202560 26 15824 0 0 0 0 130061 112 20 0 16 0 548795343 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29192] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29192/stat : 29192 (python) S 29180 29182 29100 0 -1 4202560 21 15824 0 0 0 0 130061 112 22 0 16 0 548795347 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 6 0 0 0
[pid=29182/tid=29195] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29195/stat : 29195 (python) S 29180 29182 29100 0 -1 4202560 27 15824 0 0 0 0 130061 112 18 0 16 0 548795351 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29198] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29198/stat : 29198 (python) S 29180 29182 29100 0 -1 4202560 31 15824 0 0 0 0 130061 112 18 0 16 0 548795354 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29201] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29201/stat : 29201 (python) S 29180 29182 29100 0 -1 4202560 27 15824 0 0 0 0 130061 112 18 0 16 0 548795358 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 2 0 0 0
[pid=29182/tid=29204] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29204/stat : 29204 (python) S 29180 29182 29100 0 -1 4202560 23 15824 0 0 0 0 130061 112 22 0 16 0 548795362 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29207] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29207/stat : 29207 (python) S 29180 29182 29100 0 -1 4202560 29 15824 0 0 0 0 130061 112 23 0 16 0 548795366 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 7 0 0 0
[pid=29182/tid=29210] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29210/stat : 29210 (python) S 29180 29182 29100 0 -1 4202560 29 15824 0 0 0 0 130061 112 25 0 16 0 548795373 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 7 0 0 0
[pid=29182/tid=29217] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29217/stat : 29217 (python) S 29180 29182 29100 0 -1 4202560 34 15824 0 0 0 0 130061 112 18 0 16 0 548805285 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29221] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29221/stat : 29221 (python) S 29180 29182 29100 0 -1 4202560 85 15824 0 0 0 0 130061 112 19 0 16 0 548805362 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29226] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29226/stat : 29226 (python) S 29180 29182 29100 0 -1 4202560 30 15824 0 0 0 0 130061 112 20 0 16 0 548805368 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 1 0 0 0
[pid=29182/tid=29230] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29230/stat : 29230 (python) S 29180 29182 29100 0 -1 4202560 30 15824 0 0 0 0 130061 112 22 0 16 0 548805373 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29233] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29233/stat : 29233 (python) S 29180 29182 29100 0 -1 4202560 27 15824 0 0 0 0 130061 112 23 0 16 0 548805379 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29238] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29238/stat : 29238 (python) S 29180 29182 29100 0 -1 4202560 26 15824 0 0 0 0 130061 112 20 0 16 0 548810184 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
[pid=29182/tid=29250] ppid=29180 vsize=498948 CPUtime=1301.73 cores=0-7
/proc/29182/task/29250/stat : 29250 (python) S 29180 29182 29100 0 -1 4202560 32 15824 0 0 0 0 130061 112 19 0 16 0 548834651 510922752 7609 33554432000 4194304 5683644 140734608028304 18446744073709551615 271562361265 0 0 16781312 2 18446604448148627648 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2592.11
Current children cumulated vsize (KiB) 3862080

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 29182 and gives
#  childrusage.ru_utime.tv_sec=2899
#  childrusage.ru_utime.tv_usec=451216
#  childrusage.ru_stime.tv_sec=3
#  childrusage.ru_stime.tv_usec=184515
# CPU time returned by wait4() is 2902.64
# while last known CPU time is 2592.11
#
# Using CPU time of the last sample as value...


# WARNING:
# The CPU time of some children was not reported to their father
# (probably because of a missing or aborted wait()).
# This 'lost CPU time' is added to the watched process CPU time.
#  lost CPU time (s): 309.45
#  lost CPU user time (s): 309.29
#  lost CPU system time (s): 0.16

Real time (s): 412.134
CPU time (s): 2901.56
CPU user time (s): 2898.56
CPU system time (s): 3
CPU usage (%): 704.034
Max. virtual memory (cumulated for all children) (KiB): 11476804

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2899.45
system time used= 3.18452
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 60588
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7069
involuntary context switches= 40256

runsolver used 1.89371 second user time and 4.19936 second system time

The end

Launcher Data 

Begin job on node133 at 2011-05-02 23:32:54
IDJOB=3322060
IDBENCH=82838
IDSOLVER=1712
FILE ID=node133/3322060-1304371974
RUNJOBID= node133-1304371961-29114
PBS_JOBID= 13181778
Free space on /tmp= 73648 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/medium/unif-k7-r89-v60-c5340-S642174893-006.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322060-1304371974/watcher-3322060-1304371974 -o /tmp/evaluation-result-3322060-1304371974/solver-3322060-1304371974 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3322060-1304371974.cnf -seed 1809150463 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 6df48a49b3cf6574a0f4cdf651f82d66
RANDOM SEED=1809150463

node133.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6741.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.834
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      26988680 kB
Buffers:        608748 kB
Cached:        1731228 kB
SwapCached:        124 kB
Active:        3734240 kB
Inactive:      1860728 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      26988680 kB
SwapTotal:    67111528 kB
SwapFree:     67111336 kB
Dirty:           32648 kB
Writeback:           0 kB
AnonPages:     3255164 kB
Mapped:          20200 kB
Slab:           293888 kB
PageTables:      12632 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  3757760 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73616 MiB
End job on node133 at 2011-05-02 23:39:46