Trace number 3317187

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-randomSAT 2493.72 627.13

General information on the benchmark

Namerandom/large/
unif-k7-r85-v150-c12750-S1101422472-003-UNKNOWN.cnf
MD5SUM1a4e1674582e40801444509cae2068da
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark27.5668
Satisfiable
(Un)Satisfiability was proved
Number of variables150
Number of clauses12750
Sum of the clauses size89250
Maximum clause length7
Minimum clause length7
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 512750

Solver Data 

0.31/0.67	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.31/0.67	c INFO: solving HOME/instance-3317187-1304268019.cnf
0.31/0.67	c INFO: computing task features
0.38/0.75	c INFO: computed variable-clause graph statistics
0.38/0.76	c INFO: computed clause balance statistics
0.88/1.21	c INFO: computed variable balance statistics
0.98/1.39	c INFO: computed small-clause counts
0.98/1.40	c INFO: computed Horn-clause counts
1.08/1.41	c INFO: computed variable graph statistics
1.08/1.41	c INFO: collected features for HOME/instance-3317187-1304268019.cnf in 0.74 s
1.08/1.44	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.37)
1.08/1.47	c INFO: running gnovelty+2@0 for 97 with 4799 remaining (b = 0.12)
1.18/1.50	c INFO: running adaptg2wsat2009++@0 for 97 with 4799 remaining (b = 0.08)
1.18/1.53	c INFO: running adaptg2wsat2009++@0 for 97 with 4799 remaining (b = 0.06)
396.93/100.66	c INFO: running adaptg2wsat2009++@0 for 97 with 4700 remaining (b = 0.05)
400.91/101.63	c INFO: running gnovelty+2@0 for 97 with 4699 remaining (b = 0.04)
400.91/101.68	c INFO: running FH@0 for 97 with 4699 remaining (b = 0.03)
401.27/101.73	c INFO: running adaptg2wsat2009++@0 for 97 with 4698 remaining (b = 0.03)
792.89/199.87	c INFO: running adaptg2wsat2009++@0 for 97 with 4600 remaining (b = 0.02)
796.88/200.86	c INFO: running FH@0 for 97 with 4599 remaining (b = 0.02)
801.24/201.92	c INFO: running hybridGM3@0 for 194 with 4598 remaining (b = 0.04)
801.24/201.97	c INFO: running gnovelty+2@0 for 97 with 4598 remaining (b = 0.02)
1192.30/300.10	c INFO: running hybridGM3@0 for 194 with 4500 remaining (b = 0.03)
1195.88/301.08	c INFO: running hybridGM3@0 for 194 with 4499 remaining (b = 0.03)
1200.63/302.20	c INFO: running TNM@0 for 1552 with 4498 remaining (b = 0.19)
1583.31/398.34	c INFO: running hybridGM3@99 for 97 with 4402 remaining (b = 0.01)
1970.40/495.51	c INFO: running hybridGM3@99 for 97 with 4305 remaining (b = 0.01)
1977.97/497.48	c INFO: running hybridGM3@98 for 97 with 4303 remaining (b = 0.01)
1986.33/499.54	c INFO: running adaptg2wsat2009++@0 for 97 with 4301 remaining (b = 0.01)
2373.30/596.76	c INFO: running hybridGM3@294 for 97 with 4203 remaining (b = 0.01)
2380.86/598.69	c INFO: running hybridGM3@295 for 97 with 4202 remaining (b = 0.01)
2385.23/599.77	c INFO: running FH@0 for 97 with 4200 remaining (b = 0.01)
2394.64/627.10	s SATISFIABLE
2394.64/627.10	v -1 -2 3 4 -5 6 7 8 9 -10 -11 -12 13 14 15 16 -17 18 -19 -20 -21 22 -23 24 25 -26 27 28 -29 -30 31 -32 33 34 -35 36 37 -38 39 -40 41 -42 -43 44 -45 46 47 48 49 50 51 -52 -53 -54 55 -56 -57 58 59 -60 -61 62 63 -64 -65 66 67 -68 69 70 -71 72 -73 -74 75 76 -77 78 79 -80 -81 82 83 84 -85 -86 87 88 -89 90 91 -92 -93 -94 95 96 97 -98 -99 -100 -101 102 103 -104 105 106 107 -108 -109 110 -111 112 113 -114 115 -116 -117 -118 119 -120 -121 122 -123 124 -125 -126 127 -128 -129 -130 131 132 -133 -134 -135 136 -137 138 -139 -140 141 142 -143 144 -145 146 -147 148 -149 150 0

Verifier Data 

OK

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3317187-1304268019/watcher-3317187-1304268019 -o /tmp/evaluation-result-3317187-1304268019/solver-3317187-1304268019 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317187-1304268019.cnf -seed 1760787075 -budget 4800 -cores 4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.10 7.37 4.33 5/233 11345
/proc/meminfo: memFree=22960008/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=15384 CPUtime=0 cores=0,2,4,6
/proc/11345/stat : 11345 (python) R 11343 11345 10986 0 -1 4202496 808 0 0 0 0 0 0 0 17 0 1 0 617032570 15753216 694 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735178384 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/11345/statm: 3846 694 332 364 0 333 0

[startup+0.0618221 s]
/proc/loadavg: 8.10 7.37 4.33 5/233 11345
/proc/meminfo: memFree=22960008/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=29792 CPUtime=0.03 cores=0,2,4,6
/proc/11345/stat : 11345 (python) R 11343 11345 10986 0 -1 4202496 1458 0 0 0 3 0 0 0 17 0 1 0 617032570 30507008 1299 33554432000 4194304 5683644 140733274636288 18446744073709551615 4540219 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/11345/statm: 7447 1298 443 364 0 817 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29792

[startup+0.100793 s]
/proc/loadavg: 8.10 7.37 4.33 5/233 11345
/proc/meminfo: memFree=22960008/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=89768 CPUtime=0.05 cores=0,2,4,6
/proc/11345/stat : 11345 (python) R 11343 11345 10986 0 -1 4202496 2672 0 0 0 4 1 0 0 17 0 1 0 617032570 91922432 2388 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735176704 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/11345/statm: 22442 2388 750 364 0 1622 0
Current children cumulated CPU time (s) 0.05
Current children cumulated vsize (KiB) 89768

[startup+0.30077 s]
/proc/loadavg: 8.10 7.37 4.33 5/233 11345
/proc/meminfo: memFree=22960008/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=135752 CPUtime=0.17 cores=0,2,4,6
/proc/11345/stat : 11345 (python) R 11343 11345 10986 0 -1 4202496 5962 935 1 0 13 4 0 0 17 0 1 0 617032570 139010048 4152 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735174661 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/11345/statm: 33938 4152 1032 364 0 3124 0
Current children cumulated CPU time (s) 0.17
Current children cumulated vsize (KiB) 135752

[startup+0.700731 s]
/proc/loadavg: 8.10 7.37 4.33 5/233 11345
/proc/meminfo: memFree=22960008/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=279548 CPUtime=0.38 cores=0,2,4,6
/proc/11345/stat : 11345 (python) R 11343 11345 10986 0 -1 4202496 9541 935 1 0 30 8 0 0 16 0 1 0 617032570 286257152 7225 33554432000 4194304 5683644 140733274636288 18446744073709551615 216734539524 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/11345/statm: 69887 7226 1761 364 0 5811 0
Current children cumulated CPU time (s) 0.38
Current children cumulated vsize (KiB) 279548

[startup+1.50058 s]
/proc/loadavg: 8.10 7.37 4.33 6/234 11350
/proc/meminfo: memFree=22934704/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=300104 CPUtime=1.18 cores=0,2,4,6
/proc/11345/stat : 11345 (python) R 11343 11345 10986 0 -1 4202496 12608 935 1 0 109 9 0 0 21 0 3 0 617032570 307306496 7337 33554432000 4194304 5683644 140733274636288 18446744073709551615 4509181 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/11345/statm: 75026 7337 1878 364 0 10938 0
Current children cumulated CPU time (s) 1.18
Current children cumulated vsize (KiB) 300104

[startup+3.10126 s]
/proc/loadavg: 8.09 7.38 4.35 9/246 11362
/proc/meminfo: memFree=22915016/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=320604 CPUtime=1.21 cores=0,2,4,6
/proc/11345/stat : 11345 (python) S 11343 11345 10986 0 -1 4202496 13716 935 1 0 112 9 0 0 25 0 5 0 617032570 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/11345/statm: 80151 7350 1883 364 0 16060 0
[pid=11351] ppid=11345 vsize=278984 CPUtime=0 cores=0,2,4,6
/proc/11351/stat : 11351 (python) S 11345 11345 10986 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 617032715 285679616 5670 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198735 0 0 16781312 514 9697652978860654852 0 0 17 4 0 0 0
/proc/11351/statm: 69746 5670 354 364 0 5664 0
[pid=11353] ppid=11351 vsize=1018364 CPUtime=1.65 cores=0,2,4,6
/proc/11353/stat : 11353 (gnovelty+2) R 11351 11353 11353 0 -1 4202496 670 0 0 0 165 0 0 0 25 0 1 0 617032715 1042804736 409 33554432000 134512640 135663704 4289359568 18446744073709551615 134524509 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11353/statm: 254591 409 113 282 0 254307 0
[pid=11354] ppid=11345 vsize=289848 CPUtime=0 cores=0,2,4,6
/proc/11354/stat : 11354 (python) S 11345 11345 10986 0 -1 4202560 739 0 0 0 0 0 0 0 18 0 1 0 617032718 296804352 5803 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 13790889852855615746 0 0 17 6 0 0 0
/proc/11354/statm: 72462 5803 348 364 0 8377 0
[pid=11356] ppid=11354 vsize=1018364 CPUtime=1.62 cores=0,2,4,6
/proc/11356/stat : 11356 (gnovelty+2) R 11354 11356 11356 0 -1 4202496 672 0 0 0 162 0 0 0 25 0 1 0 617032718 1042804736 409 33554432000 134512640 135663704 4291565424 18446744073709551615 134525213 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11356/statm: 254591 409 113 282 0 254307 0
[pid=11357] ppid=11345 vsize=300104 CPUtime=0 cores=0,2,4,6
/proc/11357/stat : 11357 (python) S 11345 11345 10986 0 -1 4202560 736 0 0 0 0 0 0 0 18 0 1 0 617032720 307306496 5807 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 4580939829319041023 0 0 17 4 0 0 0
/proc/11357/statm: 75026 5807 348 364 0 10938 0
[pid=11359] ppid=11357 vsize=417840 CPUtime=1.58 cores=0,2,4,6
/proc/11359/stat : 11359 (adaptg2wsat2009) R 11357 11359 11359 0 -1 4202496 887 0 0 0 158 0 0 0 25 0 1 0 617032721 427868160 624 33554432000 134512640 135107411 4292210256 18446744073709551615 134523306 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11359/statm: 104460 624 49 146 0 103336 0
[pid=11360] ppid=11345 vsize=310360 CPUtime=0 cores=0,2,4,6
/proc/11360/stat : 11360 (python) S 11345 11345 10986 0 -1 4202560 745 0 0 0 0 0 0 0 18 0 1 0 617032724 317808640 5811 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 5412863587121102847 0 0 17 4 0 0 0
/proc/11360/statm: 77590 5811 348 364 0 13499 0
[pid=11362] ppid=11360 vsize=417840 CPUtime=1.56 cores=0,2,4,6
/proc/11362/stat : 11362 (adaptg2wsat2009) R 11360 11362 11362 0 -1 4202496 892 0 0 0 156 0 0 0 25 0 1 0 617032724 427868160 624 33554432000 134512640 135107411 4292111072 18446744073709551615 134523432 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11362/statm: 104460 624 49 146 0 103336 0
[pid=11345/tid=11352] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11352/stat : 11352 (python) S 11343 11345 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 5 0 617032715 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11355] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11355/stat : 11355 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 21 0 5 0 617032718 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11358] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11358/stat : 11358 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 5 0 617032721 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11361] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11361/stat : 11361 (python) S 11343 11345 10986 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 5 0 617032724 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7.62
Current children cumulated vsize (KiB) 4372308

[startup+6.3007 s]
/proc/loadavg: 8.09 7.38 4.35 9/246 11362
/proc/meminfo: memFree=22912668/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=320604 CPUtime=1.21 cores=0,2,4,6
/proc/11345/stat : 11345 (python) S 11343 11345 10986 0 -1 4202496 13716 935 1 0 112 9 0 0 25 0 5 0 617032570 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/11345/statm: 80151 7350 1883 364 0 16060 0
[pid=11351] ppid=11345 vsize=278984 CPUtime=0 cores=0,2,4,6
/proc/11351/stat : 11351 (python) S 11345 11345 10986 0 -1 4202560 893 0 0 0 0 0 0 0 18 0 1 0 617032715 285679616 5693 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198735 0 0 16781312 514 9697652978860654852 0 0 17 4 0 0 0
/proc/11351/statm: 69746 5693 377 364 0 5664 0
[pid=11353] ppid=11351 vsize=1018364 CPUtime=4.85 cores=0,2,4,6
/proc/11353/stat : 11353 (gnovelty+2) R 11351 11353 11353 0 -1 4202496 670 0 0 0 485 0 0 0 25 0 1 0 617032715 1042804736 409 33554432000 134512640 135663704 4289359568 18446744073709551615 134524534 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11353/statm: 254591 409 113 282 0 254307 0
[pid=11354] ppid=11345 vsize=289848 CPUtime=0 cores=0,2,4,6
/proc/11354/stat : 11354 (python) S 11345 11345 10986 0 -1 4202560 897 0 0 0 0 0 0 0 18 0 1 0 617032718 296804352 5826 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 13790889852855615746 0 0 17 6 0 0 0
/proc/11354/statm: 72462 5826 371 364 0 8377 0
[pid=11356] ppid=11354 vsize=1018364 CPUtime=4.82 cores=0,2,4,6
/proc/11356/stat : 11356 (gnovelty+2) R 11354 11356 11356 0 -1 4202496 672 0 0 0 482 0 0 0 25 0 1 0 617032718 1042804736 409 33554432000 134512640 135663704 4291565424 18446744073709551615 134524923 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11356/statm: 254591 409 113 282 0 254307 0
[pid=11357] ppid=11345 vsize=300104 CPUtime=0 cores=0,2,4,6
/proc/11357/stat : 11357 (python) S 11345 11345 10986 0 -1 4202560 906 0 0 0 0 0 0 0 18 0 1 0 617032720 307306496 5830 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 4580939829319008515 0 0 17 4 0 0 0
/proc/11357/statm: 75026 5830 371 364 0 10938 0
[pid=11359] ppid=11357 vsize=417976 CPUtime=4.74 cores=0,2,4,6
/proc/11359/stat : 11359 (adaptg2wsat2009) R 11357 11359 11359 0 -1 4202496 998 0 0 0 474 0 0 0 25 0 1 0 617032721 428007424 735 33554432000 134512640 135107411 4292210256 18446744073709551615 134531905 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11359/statm: 104494 735 59 146 0 103370 0
[pid=11360] ppid=11345 vsize=310360 CPUtime=0 cores=0,2,4,6
/proc/11360/stat : 11360 (python) S 11345 11345 10986 0 -1 4202560 920 0 0 0 0 0 0 0 18 0 1 0 617032724 317808640 5833 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 5412863587121102847 0 0 17 4 0 0 0
/proc/11360/statm: 77590 5833 370 364 0 13499 0
[pid=11362] ppid=11360 vsize=417976 CPUtime=4.73 cores=0,2,4,6
/proc/11362/stat : 11362 (adaptg2wsat2009) R 11360 11362 11362 0 -1 4202496 1003 0 0 0 473 0 0 0 25 0 1 0 617032724 428007424 735 33554432000 134512640 135107411 4292111072 18446744073709551615 134531768 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11362/statm: 104494 735 59 146 0 103370 0
[pid=11345/tid=11352] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11352/stat : 11352 (python) S 11343 11345 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 5 0 617032715 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11355] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11355/stat : 11355 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 21 0 5 0 617032718 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11358] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11358/stat : 11358 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 5 0 617032721 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11361] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11361/stat : 11361 (python) S 11343 11345 10986 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 5 0 617032724 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 20.35
Current children cumulated vsize (KiB) 4372580

[startup+12.7006 s]
/proc/loadavg: 8.07 7.40 4.39 9/246 11363
/proc/meminfo: memFree=22911452/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=320604 CPUtime=1.21 cores=0,2,4,6
/proc/11345/stat : 11345 (python) S 11343 11345 10986 0 -1 4202496 13716 935 1 0 112 9 0 0 25 0 5 0 617032570 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747005387 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/11345/statm: 80151 7350 1883 364 0 16060 0
[pid=11351] ppid=11345 vsize=278984 CPUtime=0.01 cores=0,2,4,6
/proc/11351/stat : 11351 (python) S 11345 11345 10986 0 -1 4202560 895 0 0 0 1 0 0 0 18 0 1 0 617032715 285679616 5693 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198735 0 0 16781312 514 12952528450177958148 0 0 17 4 0 0 0
/proc/11351/statm: 69746 5693 377 364 0 5664 0
[pid=11353] ppid=11351 vsize=1018364 CPUtime=11.25 cores=0,2,4,6
/proc/11353/stat : 11353 (gnovelty+2) R 11351 11353 11353 0 -1 4202496 670 0 0 0 1125 0 0 0 25 0 1 0 617032715 1042804736 409 33554432000 134512640 135663704 4289359568 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11353/statm: 254591 409 113 282 0 254307 0
[pid=11354] ppid=11345 vsize=289848 CPUtime=0 cores=0,2,4,6
/proc/11354/stat : 11354 (python) S 11345 11345 10986 0 -1 4202560 900 0 0 0 0 0 0 0 18 0 1 0 617032718 296804352 5826 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 12952528450177958146 0 0 17 6 0 0 0
/proc/11354/statm: 72462 5826 371 364 0 8377 0
[pid=11356] ppid=11354 vsize=1018364 CPUtime=11.22 cores=0,2,4,6
/proc/11356/stat : 11356 (gnovelty+2) R 11354 11356 11356 0 -1 4202496 672 0 0 0 1122 0 0 0 25 0 1 0 617032718 1042804736 409 33554432000 134512640 135663704 4291565424 18446744073709551615 134524916 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11356/statm: 254591 409 113 282 0 254307 0
[pid=11357] ppid=11345 vsize=300104 CPUtime=0.01 cores=0,2,4,6
/proc/11357/stat : 11357 (python) S 11345 11345 10986 0 -1 4202560 908 0 0 0 1 0 0 0 18 0 1 0 617032720 307306496 5830 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 12952528450177958147 0 0 17 4 0 0 0
/proc/11357/statm: 75026 5830 371 364 0 10938 0
[pid=11359] ppid=11357 vsize=417976 CPUtime=11.09 cores=0,2,4,6
/proc/11359/stat : 11359 (adaptg2wsat2009) R 11357 11359 11359 0 -1 4202496 998 0 0 0 1109 0 0 0 25 0 1 0 617032721 428007424 735 33554432000 134512640 135107411 4292210256 18446744073709551615 134531817 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11359/statm: 104494 735 59 146 0 103370 0
[pid=11360] ppid=11345 vsize=310360 CPUtime=0.01 cores=0,2,4,6
/proc/11360/stat : 11360 (python) S 11345 11345 10986 0 -1 4202560 923 0 0 0 0 1 0 0 18 0 1 0 617032724 317808640 5834 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 5412863587121070339 0 0 17 4 0 0 0
/proc/11360/statm: 77590 5834 371 364 0 13499 0
[pid=11362] ppid=11360 vsize=417976 CPUtime=11.1 cores=0,2,4,6
/proc/11362/stat : 11362 (adaptg2wsat2009) R 11360 11362 11362 0 -1 4202496 1003 0 0 0 1110 0 0 0 25 0 1 0 617032724 428007424 735 33554432000 134512640 135107411 4292111072 18446744073709551615 134531810 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11362/statm: 104494 735 59 146 0 103370 0
[pid=11345/tid=11352] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11352/stat : 11352 (python) S 11343 11345 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 5 0 617032715 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11355] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11355/stat : 11355 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 21 0 5 0 617032718 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11358] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11358/stat : 11358 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 5 0 617032721 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11361] ppid=11343 vsize=320604 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11361/stat : 11361 (python) S 11343 11345 10986 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 5 0 617032724 328298496 7350 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 45.9
Current children cumulated vsize (KiB) 4372580

[startup+25.5013 s]
/proc/loadavg: 8.06 7.42 4.43 9/246 11363
/proc/meminfo: memFree=22911708/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=320604 CPUtime=1.21 cores=0,2,4,6

################
# More data... #
################

[pid=11422/tid=11550] ppid=11345 vsize=382504 CPUtime=0 cores=0,2,4,6
/proc/11422/task/11550/stat : 11550 (python) S 11345 11345 10986 0 -1 4202560 48 0 0 0 0 0 0 0 18 0 2 0 617072400 391684096 5980 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 514 18446604445666970688 0 0 -1 0 0 0 0
[pid=11426] ppid=11345 vsize=393032 CPUtime=0.17 cores=0,2,4,6
/proc/11426/stat : 11426 (python) S 11345 11345 10986 0 -1 4202560 1160 0 0 0 10 7 0 0 19 0 2 0 617052768 402464768 6052 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11426/statm: 98258 6052 400 364 0 34143 0
[pid=11428] ppid=11426 vsize=1018364 CPUtime=98.44 cores=0,2,4,6
/proc/11428/stat : 11428 (gnovelty+2) T 11426 11428 11428 0 -1 4202496 678 0 0 0 9844 0 0 0 25 0 1 0 617052768 1042804736 408 33554432000 134512640 135663704 4288200512 18446744073709551615 134525247 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11428/statm: 254591 408 113 282 0 254307 0
[pid=11426/tid=11444] ppid=11345 vsize=393032 CPUtime=0 cores=0,2,4,6
/proc/11426/task/11444/stat : 11444 (python) S 11345 11345 10986 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 617062786 402464768 6052 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 514 18446604445666968000 0 0 -1 4 0 0 0
[pid=11437] ppid=11345 vsize=403288 CPUtime=0.57 cores=0,2,4,6
/proc/11437/stat : 11437 (python) S 11345 11345 10986 0 -1 4202560 1192 0 0 0 28 29 0 0 18 0 2 0 617062581 412966912 6056 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11437/statm: 100822 6056 400 364 0 36704 0
[pid=11439] ppid=11437 vsize=268424 CPUtime=324.29 cores=0,2,4,6
/proc/11439/stat : 11439 (hybridGM3) R 11437 11439 11439 0 -1 4202496 5058 0 0 0 32428 1 0 0 25 0 1 0 617062581 274866176 4788 33554432000 134512640 135204385 4293781312 18446744073709551615 134538494 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/11439/statm: 67106 4788 63 169 0 66936 0
[pid=11437/tid=11563] ppid=11345 vsize=403288 CPUtime=0 cores=0,2,4,6
/proc/11437/task/11563/stat : 11563 (python) S 11345 11345 10986 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 617082118 412966912 6056 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 514 18446604446820055104 0 0 -1 2 0 0 0
[pid=11441] ppid=11345 vsize=413544 CPUtime=0.52 cores=0,2,4,6
/proc/11441/stat : 11441 (python) S 11345 11345 10986 0 -1 4202560 1203 0 0 0 24 28 0 0 18 0 2 0 617062679 423469056 6060 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11441/statm: 103386 6060 400 364 0 39265 0
[pid=11443] ppid=11441 vsize=268428 CPUtime=296.78 cores=0,2,4,6
/proc/11443/stat : 11443 (hybridGM3) T 11441 11443 11443 0 -1 4202496 5056 0 0 0 29677 1 0 0 25 0 1 0 617062679 274870272 4787 33554432000 134512640 135204385 4289931808 18446744073709551615 134538355 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11443/statm: 67107 4787 63 169 0 66937 0
[pid=11441/tid=11571] ppid=11345 vsize=413544 CPUtime=0 cores=0,2,4,6
/proc/11441/task/11571/stat : 11571 (python) S 11345 11345 10986 0 -1 4202560 49 0 0 0 0 0 0 0 18 0 2 0 617082314 423469056 6060 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 514 18446604447533701312 0 0 -1 0 0 0 0
[pid=11445] ppid=11345 vsize=423800 CPUtime=0.57 cores=0,2,4,6
/proc/11445/stat : 11445 (python) S 11345 11345 10986 0 -1 4202560 1073 0 0 0 32 25 0 0 18 0 1 0 617062791 433971200 6041 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 12952528450177958147 0 0 17 4 0 0 0
/proc/11445/statm: 105950 6041 378 364 0 41826 0
[pid=11447] ppid=11445 vsize=456468 CPUtime=318.4 cores=0,2,4,6
/proc/11447/stat : 11447 (TNM) R 11445 11447 11447 0 -1 4202496 919 0 0 0 31839 1 0 0 25 0 1 0 617062791 467423232 644 33554432000 134512640 135008748 4292511312 18446744073709551615 134529367 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11447/statm: 114117 644 46 122 0 113017 0
[pid=11572] ppid=11345 vsize=434056 CPUtime=0.19 cores=0,2,4,6
/proc/11572/stat : 11572 (python) S 11345 11345 10986 0 -1 4202560 1197 0 0 0 11 8 0 0 19 0 2 0 617082525 444473344 6076 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747005387 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11572/statm: 108514 6076 408 364 0 44387 0
[pid=11574] ppid=11572 vsize=417976 CPUtime=99.42 cores=0,2,4,6
/proc/11574/stat : 11574 (adaptg2wsat2009) T 11572 11574 11574 0 -1 4202496 1014 0 0 0 9942 0 0 0 25 0 1 0 617082525 428007424 736 33554432000 134512640 135107411 4287465024 18446744073709551615 134532358 0 0 16781312 0 18446744071562691360 0 0 17 4 0 0 0
/proc/11574/statm: 104494 736 59 146 0 103370 0
[pid=11572/tid=11593] ppid=11345 vsize=434056 CPUtime=0 cores=0,2,4,6
/proc/11572/task/11593/stat : 11593 (python) S 11345 11345 10986 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 617092544 444473344 6076 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 514 18446604450203746624 0 0 -1 4 0 0 0
[pid=11594] ppid=11345 vsize=444312 CPUtime=0.04 cores=0,2,4,6
/proc/11594/stat : 11594 (python) S 11345 11345 10986 0 -1 4202560 1007 0 0 0 2 2 0 0 18 0 1 0 617092548 454975488 6068 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 12952528450177958145 0 0 17 4 0 0 0
/proc/11594/statm: 111078 6068 373 364 0 46948 0
[pid=11596] ppid=11594 vsize=247988 CPUtime=26.78 cores=0,2,4,6
/proc/11596/stat : 11596 (FH) R 11594 11596 11596 0 -1 4202496 934 0 0 0 2678 0 0 0 25 0 1 0 617092548 253939712 658 33554432000 134512640 134936988 4288933328 18446744073709551615 134553238 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/11596/statm: 61997 658 41 104 0 61501 0
[pid=11345/tid=11352] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11352/stat : 11352 (python) S 11343 11345 10986 0 -1 4202560 27 935 0 0 0 0 0 0 19 0 18 0 617032715 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11355] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11355/stat : 11355 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 21 0 18 0 617032718 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11358] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11358/stat : 11358 (python) S 11343 11345 10986 0 -1 4202560 23 935 0 0 0 0 0 0 23 0 18 0 617032721 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11361] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11361/stat : 11361 (python) S 11343 11345 10986 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 18 0 617032724 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11384] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11384/stat : 11384 (python) S 11343 11345 10986 0 -1 4202560 31 935 0 0 0 0 0 0 20 0 18 0 617042637 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11388] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11388/stat : 11388 (python) S 11343 11345 10986 0 -1 4202560 21 935 0 0 0 0 0 0 20 0 18 0 617042735 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11392] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11392/stat : 11392 (python) S 11343 11345 10986 0 -1 4202560 21 935 0 0 0 0 0 0 20 0 18 0 617042739 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 2 0 0 0
[pid=11345/tid=11396] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11396/stat : 11396 (python) S 11343 11345 10986 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 18 0 617042744 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11415] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11415/stat : 11415 (python) S 11343 11345 10986 0 -1 4202560 30 935 0 0 0 0 0 0 20 0 18 0 617052558 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11419] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11419/stat : 11419 (python) S 11343 11345 10986 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 18 0 617052657 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11423] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11423/stat : 11423 (python) S 11343 11345 10986 0 -1 4202560 30 935 0 0 0 0 0 0 18 0 18 0 617052763 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 0 0 0 0
[pid=11345/tid=11427] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11427/stat : 11427 (python) S 11343 11345 10986 0 -1 4202560 27 935 0 0 0 0 0 0 21 0 18 0 617052768 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11438] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11438/stat : 11438 (python) S 11343 11345 10986 0 -1 4202560 34 935 0 0 0 0 0 0 18 0 18 0 617062581 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11442] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11442/stat : 11442 (python) S 11343 11345 10986 0 -1 4202560 32 935 0 0 0 0 0 0 18 0 18 0 617062679 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11446] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11446/stat : 11446 (python) S 11343 11345 10986 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 18 0 617062791 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11573] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11573/stat : 11573 (python) S 11343 11345 10986 0 -1 4202560 26 935 0 0 0 0 0 0 20 0 18 0 617082525 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11595] ppid=11343 vsize=454556 CPUtime=0 cores=0,2,4,6
/proc/11345/task/11595/stat : 11595 (python) S 11343 11345 10986 0 -1 4202560 25 935 0 0 0 0 0 0 20 0 18 0 617092548 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2492.48
Current children cumulated vsize (KiB) 15204964

[startup+627.103 s]
/proc/loadavg: 8.28 8.04 6.19 9/330 11597
/proc/meminfo: memFree=22592144/32951124 swapFree=67102140/67111528
[pid=11345] ppid=11343 vsize=454556 CPUtime=1394.9 cores=0,2,4,6
/proc/11345/stat : 11345 (python) S 11343 11345 10986 0 -1 4202496 26147 29512 1 0 185 13 139169 123 17 0 18 0 617032570 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747008831 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11345/statm: 113639 7609 1910 364 0 49509 0
[pid=11422] ppid=11345 vsize=0 CPUtime=327.93 cores=0,2,4,6
/proc/11422/stat : 11422 (python) Z 11345 11345 10986 0 -1 4202588 1263 5068 0 0 33 26 32733 1 16 0 1 0 617052763 0 0 33554432000 0 0 0 0 0 0 0 16781824 2 18446744073709551615 0 0 17 0 0 0 0
/proc/11422/statm: 0 0 0 0 0 0 0
[pid=11437] ppid=11345 vsize=403288 CPUtime=0.58 cores=0,2,4,6
/proc/11437/stat : 11437 (python) S 11345 11345 10986 0 -1 4202560 1192 0 0 0 29 29 0 0 18 0 2 0 617062581 412966912 6056 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/11437/statm: 100822 6056 400 364 0 36704 0
[pid=11439] ppid=11437 vsize=268424 CPUtime=324.67 cores=0,2,4,6
/proc/11439/stat : 11439 (hybridGM3) R 11437 11439 11439 0 -1 4202496 5058 0 0 0 32466 1 0 0 25 0 1 0 617062581 274866176 4788 33554432000 134512640 135204385 4293781312 18446744073709551615 134538549 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/11439/statm: 67106 4788 63 169 0 66936 0
[pid=11437/tid=11563] ppid=11345 vsize=403288 CPUtime=0 cores=0,2,4,6
/proc/11437/task/11563/stat : 11563 (python) S 11345 11345 10986 0 -1 4202560 46 0 0 0 0 0 0 0 18 0 2 0 617082118 412966912 6056 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 514 18446604446820055104 0 0 -1 2 0 0 0
[pid=11445] ppid=11345 vsize=423800 CPUtime=0.57 cores=0,2,4,6
/proc/11445/stat : 11445 (python) S 11345 11345 10986 0 -1 4202560 1092 0 0 0 32 25 0 0 15 0 1 0 617062791 433971200 6046 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747008831 0 0 16781824 2 18446744071562232103 0 0 17 4 0 0 0
/proc/11445/statm: 105950 6046 383 364 0 41826 0
[pid=11447] ppid=11445 vsize=0 CPUtime=318.78 cores=0,2,4,6
/proc/11447/stat : 11447 (TNM) R 11445 11447 11447 0 -1 4203524 919 0 0 0 31877 1 0 0 25 0 1 0 617062791 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 0 0 0 0
/proc/11447/statm: 0 0 0 0 0 0 0
[pid=11594] ppid=11345 vsize=444312 CPUtime=0.04 cores=0,2,4,6
/proc/11594/stat : 11594 (python) S 11345 11345 10986 0 -1 4202560 1007 0 0 0 2 2 0 0 18 0 1 0 617092548 454975488 6068 33554432000 4194304 5683644 140733274636288 18446744073709551615 216735198790 0 0 16781312 514 12952528450177958145 0 0 17 4 0 0 0
/proc/11594/statm: 111078 6068 373 364 0 46948 0
[pid=11596] ppid=11594 vsize=247988 CPUtime=27.17 cores=0,2,4,6
/proc/11596/stat : 11596 (FH) R 11594 11596 11596 0 -1 4202496 934 0 0 0 2717 0 0 0 25 0 1 0 617092548 253939712 658 33554432000 134512640 134936988 4288933328 18446744073709551615 134552411 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/11596/statm: 61997 658 41 104 0 61501 0
[pid=11345/tid=11352] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11352/stat : 11352 (python) S 11343 11345 10986 0 -1 4202560 27 31771 0 0 0 0 149123 132 19 0 18 0 617032715 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11355] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11355/stat : 11355 (python) S 11343 11345 10986 0 -1 4202560 23 31771 0 0 0 0 149123 132 21 0 18 0 617032718 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11358] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11358/stat : 11358 (python) S 11343 11345 10986 0 -1 4202560 23 31771 0 0 0 0 149123 132 23 0 18 0 617032721 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11361] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11361/stat : 11361 (python) S 11343 11345 10986 0 -1 4202560 26 31771 0 0 0 0 149123 132 25 0 18 0 617032724 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11384] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11384/stat : 11384 (python) S 11343 11345 10986 0 -1 4202560 31 31771 0 0 0 0 149123 132 20 0 18 0 617042637 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11388] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11388/stat : 11388 (python) S 11343 11345 10986 0 -1 4202560 21 31771 0 0 0 0 149123 132 20 0 18 0 617042735 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 6 0 0 0
[pid=11345/tid=11392] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11392/stat : 11392 (python) S 11343 11345 10986 0 -1 4202560 21 31771 0 0 0 0 149123 132 20 0 18 0 617042739 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 2 0 0 0
[pid=11345/tid=11396] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11396/stat : 11396 (python) S 11343 11345 10986 0 -1 4202560 30 31771 0 0 0 0 149123 132 20 0 18 0 617042744 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11415] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11415/stat : 11415 (python) S 11343 11345 10986 0 -1 4202560 30 31771 0 0 0 0 149123 132 20 0 18 0 617052558 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11419] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11419/stat : 11419 (python) S 11343 11345 10986 0 -1 4202560 25 31771 0 0 0 0 149123 132 20 0 18 0 617052657 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11423] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11423/stat : 11423 (python) S 11343 11345 10986 0 -1 4202560 30 31771 0 0 0 0 149123 132 18 0 18 0 617052763 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 0 0 0 0
[pid=11345/tid=11427] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11427/stat : 11427 (python) S 11343 11345 10986 0 -1 4202560 27 31771 0 0 0 0 149123 132 21 0 18 0 617052768 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11438] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11438/stat : 11438 (python) S 11343 11345 10986 0 -1 4202560 34 31771 0 0 0 0 149123 132 18 0 18 0 617062581 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11442] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11442/stat : 11442 (python) S 11343 11345 10986 0 -1 4202560 32 31771 0 0 0 0 149123 132 18 0 18 0 617062679 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11446] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11446/stat : 11446 (python) S 11343 11345 10986 0 -1 4202560 27 31771 0 0 0 0 149123 132 20 0 18 0 617062791 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11573] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11573/stat : 11573 (python) S 11343 11345 10986 0 -1 4202560 26 31771 0 0 0 0 149123 132 20 0 18 0 617082525 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
[pid=11345/tid=11595] ppid=11343 vsize=454556 CPUtime=1492.55 cores=0,2,4,6
/proc/11345/task/11595/stat : 11595 (python) S 11343 11345 10986 0 -1 4202560 25 31771 0 0 0 0 149123 132 20 0 18 0 617092548 465465344 7609 33554432000 4194304 5683644 140733274636288 18446744073709551615 216747002289 0 0 16781312 2 18446604440279247552 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2394.64
Current children cumulated vsize (KiB) 2242368

Child status: 0

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 11345 and gives
#  childrusage.ru_utime.tv_sec=2492
#  childrusage.ru_utime.tv_usec=31153
#  childrusage.ru_stime.tv_sec=2
#  childrusage.ru_stime.tv_usec=364640
# CPU time returned by wait4() is 2494.4
# while last known CPU time is 2394.64
#
# Using CPU time of the last sample as value...


# WARNING:
# The CPU time of some children was not reported to their father
# (probably because of a missing or aborted wait()).
# This 'lost CPU time' is added to the watched process CPU time.
#  lost CPU time (s): 99.0801
#  lost CPU user time (s): 99.0901
#  lost CPU system time (s): -0.00999975

Real time (s): 627.13
CPU time (s): 2493.72
CPU user time (s): 2491.52
CPU system time (s): 2.2
CPU usage (%): 397.64
Max. virtual memory (cumulated for all children) (KiB): 15204968

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2492.03
system time used= 2.36464
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 79007
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6439
involuntary context switches= 14311

runsolver used 2.89656 second user time and 6.35603 second system time

The end

Launcher Data 

Begin job on node119 at 2011-05-01 18:40:19
IDJOB=3317187
IDBENCH=82652
IDSOLVER=1705
FILE ID=node119/3317187-1304268019
RUNJOBID= node119-1304267283-11004
PBS_JOBID= 13173960
Free space on /tmp= 73004 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/large/unif-k7-r85-v150-c12750-S1101422472-003-UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3317187-1304268019/watcher-3317187-1304268019 -o /tmp/evaluation-result-3317187-1304268019/solver-3317187-1304268019 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3317187-1304268019.cnf -seed 1760787075 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 1a4e1674582e40801444509cae2068da
RANDOM SEED=1760787075

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      22960172 kB
Buffers:       1964632 kB
Cached:        7096756 kB
SwapCached:       3688 kB
Active:         937412 kB
Inactive:      8462916 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      22960172 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:           31940 kB
Writeback:           0 kB
AnonPages:      338400 kB
Mapped:          23432 kB
Slab:           517160 kB
PageTables:      14716 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 21098956 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 72964 MiB
End job on node119 at 2011-05-01 18:50:46