Trace number 3322238

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-randomUNSAT 2164.56 306.275

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v450-c1917-S1879031150-062.UNKNOWN.cnf
MD5SUMdea41a8eae52b60f8e05be01aec1734d
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark300.017
Satisfiable
(Un)Satisfiability was proved
Number of variables450
Number of clauses1917
Sum of the clauses size5751
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 31917
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.31/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.31/0.47	c INFO: solving HOME/instance-3322238-1304376529.cnf
0.31/0.47	c INFO: computing task features
0.31/0.48	c INFO: computed variable-clause graph statistics
0.31/0.48	c INFO: computed clause balance statistics
0.41/0.51	c INFO: computed variable balance statistics
0.41/0.54	c INFO: computed small-clause counts
0.41/0.54	c INFO: computed Horn-clause counts
0.41/0.54	c INFO: computed variable graph statistics
0.41/0.54	c INFO: collected features for HOME/instance-3322238-1304376529.cnf in 0.07 s
0.41/0.58	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.56)
0.50/0.62	c INFO: running march_hi@0 for 388 with 39999 remaining (b = 0.37)
0.50/0.66	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.37)
0.60/0.70	c INFO: running march_hi@0 for 97 with 39999 remaining (b = 0.12)
0.60/0.74	c INFO: running gnovelty+2@0 for 97 with 39999 remaining (b = 0.07)
0.60/0.78	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.26)
0.70/0.82	c INFO: running march_hi@0 for 388 with 39999 remaining (b = 0.19)
0.70/0.86	c INFO: running march_hi@0 for 485 with 39999 remaining (b = 0.21)
724.85/100.02	c INFO: running march_hi@0 for 485 with 39900 remaining (b = 0.19)
728.03/100.53	c INFO: running march_hi@0 for 97 with 39899 remaining (b = 0.05)
729.94/100.85	c INFO: running kcnfs-2006@0 for 582 with 39899 remaining (b = 0.19)
1835.26/259.18	c INFO: running march_hi@100 for 97 with 39741 remaining (b = 0.04)
2164.03/306.25	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3322238-1304376529/watcher-3322238-1304376529 -o /tmp/evaluation-result-3322238-1304376529/solver-3322238-1304376529 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3322238-1304376529.cnf -seed 1576083386 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 5.82 8.03 8.76 2/326 20213
/proc/meminfo: memFree=21488336/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=15904 CPUtime=0 cores=0-7
/proc/20213/stat : 20213 (python) R 20211 20213 18999 0 -1 4202496 960 0 0 0 0 0 0 0 25 0 1 0 627883605 16285696 834 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415434887 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/20213/statm: 3976 834 357 364 0 463 0

[startup+0.0946419 s]
/proc/loadavg: 5.82 8.03 8.76 2/326 20213
/proc/meminfo: memFree=21488336/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=99372 CPUtime=0.07 cores=0-7
/proc/20213/stat : 20213 (python) S 20211 20213 18999 0 -1 4202496 2891 0 0 0 5 2 0 0 25 0 1 0 627883605 101756928 2587 33554432000 4194304 5683644 140737200111600 18446744073709551615 256410467975 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/20213/statm: 25394 2589 833 364 0 1694 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 99372

[startup+0.100651 s]
/proc/loadavg: 5.82 8.03 8.76 2/326 20213
/proc/meminfo: memFree=21488336/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=104216 CPUtime=0.07 cores=0-7
/proc/20213/stat : 20213 (python) R 20211 20213 18999 0 -1 4202496 3047 0 0 0 5 2 0 0 25 0 1 0 627883605 106717184 2731 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415389696 0 0 16781312 2 0 0 0 17 4 0 0 0
/proc/20213/statm: 26054 2731 847 364 0 1802 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104216

[startup+0.300625 s]
/proc/loadavg: 5.82 8.03 8.76 2/326 20213
/proc/meminfo: memFree=21488336/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=168672 CPUtime=0.23 cores=0-7
/proc/20213/stat : 20213 (python) R 20211 20213 18999 0 -1 4202496 6723 934 1 0 16 7 0 0 17 0 1 0 627883605 172720128 4798 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415389696 0 2147483391 16781312 2 18446744071699199166 0 0 17 4 0 0 0
/proc/20213/statm: 42168 4798 1331 364 0 3453 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 168672

[startup+0.700453 s]
/proc/loadavg: 5.82 8.03 8.76 2/326 20213
/proc/meminfo: memFree=21488336/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=309828 CPUtime=0.6 cores=0-7
/proc/20213/stat : 20213 (python) R 20211 20213 18999 0 -1 4202496 11877 934 1 0 51 9 0 0 20 0 4 0 627883605 317263872 7228 33554432000 4194304 5683644 140737200111600 18446744073709551615 256419031954 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/20213/statm: 77457 7228 1878 364 0 13369 0
Current children cumulated CPU time (s) 0.6
Current children cumulated vsize (KiB) 309828

[startup+1.5013 s]
/proc/loadavg: 5.82 8.03 8.76 10/351 20242
/proc/meminfo: memFree=21428484/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=361280 CPUtime=0.76 cores=0-7
/proc/20213/stat : 20213 (python) S 20211 20213 18999 0 -1 4202496 15788 934 1 0 66 10 0 0 20 0 9 0 627883605 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/20213/statm: 90320 7294 1889 364 0 26217 0
[pid=20219] ppid=20213 vsize=278596 CPUtime=0 cores=0-7
/proc/20219/stat : 20219 (python) S 20213 20213 18999 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 627883663 285282304 5573 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411727 0 0 16781312 514 1191175167699582975 0 0 17 4 0 0 0
/proc/20219/statm: 69649 5573 354 364 0 5567 0
[pid=20221] ppid=20219 vsize=1017800 CPUtime=0.91 cores=0-7
/proc/20221/stat : 20221 (gnovelty+2) R 20219 20221 20221 0 -1 4202496 437 0 0 0 91 0 0 0 21 0 1 0 627883664 1042227200 173 33554432000 134512640 135663704 4292521440 18446744073709551615 134525187 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/20221/statm: 254450 173 113 282 0 254166 0
[pid=20222] ppid=20213 vsize=289328 CPUtime=0 cores=0-7
/proc/20222/stat : 20222 (python) S 20213 20213 18999 0 -1 4202560 738 0 0 0 0 0 0 0 15 0 1 0 627883667 296271872 5690 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 8423925932686803201 0 0 17 0 0 0 0
/proc/20222/statm: 72332 5690 348 364 0 8247 0
[pid=20224] ppid=20222 vsize=2028 CPUtime=0.85 cores=0-7
/proc/20224/stat : 20224 (march_hi) R 20222 20224 20224 0 -1 4202496 592 0 0 0 85 0 0 0 25 0 1 0 627883668 2076672 331 33554432000 4194304 4955518 140736461013904 18446744073709551615 4239004 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/20224/statm: 507 331 104 186 0 320 0
[pid=20225] ppid=20213 vsize=299584 CPUtime=0 cores=0-7
/proc/20225/stat : 20225 (python) S 20213 20213 18999 0 -1 4202560 747 0 0 0 0 0 0 0 16 0 1 0 627883671 306774016 5694 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 13298324861408477443 0 0 17 4 0 0 0
/proc/20225/statm: 74896 5694 348 364 0 10808 0
[pid=20227] ppid=20225 vsize=2028 CPUtime=0.83 cores=0-7
/proc/20227/stat : 20227 (march_hi) R 20225 20227 20227 0 -1 4202496 592 0 0 0 83 0 0 0 23 0 1 0 627883672 2076672 330 33554432000 4194304 4955518 140737228553984 18446744073709551615 4238449 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/20227/statm: 507 330 104 186 0 320 0
[pid=20228] ppid=20213 vsize=309840 CPUtime=0 cores=0-7
/proc/20228/stat : 20228 (python) S 20213 20213 18999 0 -1 4202560 754 0 0 0 0 0 0 0 19 0 1 0 627883675 317276160 5698 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 11382139976985575681 0 0 17 2 0 0 0
/proc/20228/statm: 77460 5698 348 364 0 13369 0
[pid=20230] ppid=20228 vsize=2028 CPUtime=0.79 cores=0-7
/proc/20230/stat : 20230 (march_hi) R 20228 20230 20230 0 -1 4202496 593 0 0 0 79 0 0 0 25 0 1 0 627883676 2076672 330 33554432000 4194304 4955518 140734863126448 18446744073709551615 4227500 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/20230/statm: 507 330 104 186 0 320 0
[pid=20231] ppid=20213 vsize=320096 CPUtime=0 cores=0-7
/proc/20231/stat : 20231 (python) S 20213 20213 18999 0 -1 4202560 763 0 0 0 0 0 0 0 19 0 1 0 627883679 327778304 5711 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 12429377798320160767 0 0 17 2 0 0 0
/proc/20231/statm: 80024 5711 348 364 0 15930 0
[pid=20233] ppid=20231 vsize=1017800 CPUtime=0.63 cores=0-7
/proc/20233/stat : 20233 (gnovelty+2) R 20231 20233 20233 0 -1 4202496 434 0 0 0 63 0 0 0 22 0 1 0 627883680 1042227200 173 33554432000 134512640 135663704 4292836944 18446744073709551615 135008531 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/20233/statm: 254450 173 113 282 0 254166 0
[pid=20234] ppid=20213 vsize=330524 CPUtime=0 cores=0-7
/proc/20234/stat : 20234 (python) S 20213 20213 18999 0 -1 4202560 770 0 0 0 0 0 0 0 15 0 1 0 627883683 338456576 5732 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 11382126782846042371 0 0 17 0 0 0 0
/proc/20234/statm: 82631 5732 348 364 0 18534 0
[pid=20236] ppid=20234 vsize=2028 CPUtime=0.32 cores=0-7
/proc/20236/stat : 20236 (march_hi) R 20234 20236 20236 0 -1 4202496 580 0 0 0 32 0 0 0 21 0 1 0 627883684 2076672 316 33554432000 4194304 4955518 140735863778944 18446744073709551615 4209733 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/20236/statm: 507 316 104 186 0 320 0
[pid=20237] ppid=20213 vsize=340780 CPUtime=0 cores=0-7
/proc/20237/stat : 20237 (python) S 20213 20213 18999 0 -1 4202560 781 0 0 0 0 0 0 0 18 0 1 0 627883687 348958720 5736 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 2113239362647851268 0 0 17 2 0 0 0
/proc/20237/statm: 85195 5736 348 364 0 21095 0
[pid=20239] ppid=20237 vsize=2028 CPUtime=0.67 cores=0-7
/proc/20239/stat : 20239 (march_hi) R 20237 20239 20239 0 -1 4202496 587 0 0 0 67 0 0 0 25 0 1 0 627883688 2076672 324 33554432000 4194304 4955518 140733201215776 18446744073709551615 4209635 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/20239/statm: 507 324 104 186 0 320 0
[pid=20240] ppid=20213 vsize=351036 CPUtime=0 cores=0-7
/proc/20240/stat : 20240 (python) S 20213 20213 18999 0 -1 4202560 814 0 0 0 0 0 0 0 15 0 1 0 627883691 359460864 5755 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 7620178534736036097 0 0 17 0 0 0 0
/proc/20240/statm: 87759 5755 354 364 0 23656 0
[pid=20242] ppid=20240 vsize=2028 CPUtime=0.49 cores=0-7
/proc/20242/stat : 20242 (march_hi) R 20240 20242 20242 0 -1 4202496 591 0 0 0 49 0 0 0 21 0 1 0 627883692 2076672 324 33554432000 4194304 4955518 140735390915856 18446744073709551615 4209643 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/20242/statm: 507 324 104 186 0 320 0
[pid=20213/tid=20220] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20220/stat : 20220 (python) S 20211 20213 18999 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 9 0 627883663 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20223] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20223/stat : 20223 (python) S 20211 20213 18999 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 9 0 627883668 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20226] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20226/stat : 20226 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 9 0 627883671 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20229] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20229/stat : 20229 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 22 0 9 0 627883676 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20232] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20232/stat : 20232 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 9 0 627883679 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20235] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20235/stat : 20235 (python) S 20211 20213 18999 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 9 0 627883684 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20238] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20238/stat : 20238 (python) S 20211 20213 18999 0 -1 4202560 24 934 0 0 0 0 0 0 22 0 9 0 627883687 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20241] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20241/stat : 20241 (python) S 20211 20213 18999 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 9 0 627883692 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 6.25
Current children cumulated vsize (KiB) 4928832

[startup+3.101 s]
/proc/loadavg: 5.82 8.03 8.76 11/351 20242
/proc/meminfo: memFree=21427244/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=361280 CPUtime=0.76 cores=0-7
/proc/20213/stat : 20213 (python) S 20211 20213 18999 0 -1 4202496 15788 934 1 0 66 10 0 0 20 0 9 0 627883605 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/20213/statm: 90320 7294 1889 364 0 26217 0
[pid=20219] ppid=20213 vsize=278596 CPUtime=0 cores=0-7
/proc/20219/stat : 20219 (python) S 20213 20213 18999 0 -1 4202560 734 0 0 0 0 0 0 0 18 0 1 0 627883663 285282304 5573 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411727 0 0 16781312 514 1191175167699550465 0 0 17 4 0 0 0
/proc/20219/statm: 69649 5573 354 364 0 5567 0
[pid=20221] ppid=20219 vsize=1017800 CPUtime=2.51 cores=0-7
/proc/20221/stat : 20221 (gnovelty+2) R 20219 20221 20221 0 -1 4202496 437 0 0 0 251 0 0 0 25 0 1 0 627883664 1042227200 173 33554432000 134512640 135663704 4292521440 18446744073709551615 134544204 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/20221/statm: 254450 173 113 282 0 254166 0
[pid=20222] ppid=20213 vsize=289328 CPUtime=0 cores=0-7
/proc/20222/stat : 20222 (python) S 20213 20213 18999 0 -1 4202560 744 0 0 0 0 0 0 0 15 0 1 0 627883667 296271872 5690 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 8423925932686803201 0 0 17 0 0 0 0
/proc/20222/statm: 72332 5690 348 364 0 8247 0
[pid=20224] ppid=20222 vsize=2160 CPUtime=2.44 cores=0-7
/proc/20224/stat : 20224 (march_hi) R 20222 20224 20224 0 -1 4202496 612 0 0 0 244 0 0 0 25 0 1 0 627883668 2211840 351 33554432000 4194304 4955518 140736461013904 18446744073709551615 4238494 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/20224/statm: 540 351 104 186 0 353 0
[pid=20225] ppid=20213 vsize=299584 CPUtime=0 cores=0-7
/proc/20225/stat : 20225 (python) S 20213 20213 18999 0 -1 4202560 752 0 0 0 0 0 0 0 15 0 1 0 627883671 306774016 5694 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 13298324861408477443 0 0 17 4 0 0 0
/proc/20225/statm: 74896 5694 348 364 0 10808 0
[pid=20227] ppid=20225 vsize=2160 CPUtime=2.43 cores=0-7
/proc/20227/stat : 20227 (march_hi) R 20225 20227 20227 0 -1 4202496 612 0 0 0 243 0 0 0 25 0 1 0 627883672 2211840 350 33554432000 4194304 4955518 140737228553984 18446744073709551615 4315021 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/20227/statm: 540 350 104 186 0 353 0
[pid=20228] ppid=20213 vsize=309840 CPUtime=0 cores=0-7
/proc/20228/stat : 20228 (python) S 20213 20213 18999 0 -1 4202560 757 0 0 0 0 0 0 0 15 0 1 0 627883675 317276160 5698 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 11382139976985575681 0 0 17 2 0 0 0
/proc/20228/statm: 77460 5698 348 364 0 13369 0
[pid=20230] ppid=20228 vsize=2160 CPUtime=2.38 cores=0-7
/proc/20230/stat : 20230 (march_hi) R 20228 20230 20230 0 -1 4202496 613 0 0 0 238 0 0 0 25 0 1 0 627883676 2211840 350 33554432000 4194304 4955518 140734863126448 18446744073709551615 4224704 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/20230/statm: 540 350 104 186 0 353 0
[pid=20231] ppid=20213 vsize=320096 CPUtime=0 cores=0-7
/proc/20231/stat : 20231 (python) S 20213 20213 18999 0 -1 4202560 763 0 0 0 0 0 0 0 18 0 1 0 627883679 327778304 5711 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 12429377798320128259 0 0 17 2 0 0 0
/proc/20231/statm: 80024 5711 348 364 0 15930 0
[pid=20233] ppid=20231 vsize=1017800 CPUtime=2.23 cores=0-7
/proc/20233/stat : 20233 (gnovelty+2) R 20231 20233 20233 0 -1 4202496 434 0 0 0 223 0 0 0 25 0 1 0 627883680 1042227200 173 33554432000 134512640 135663704 4292836944 18446744073709551615 134545786 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/20233/statm: 254450 173 113 282 0 254166 0
[pid=20234] ppid=20213 vsize=330524 CPUtime=0 cores=0-7
/proc/20234/stat : 20234 (python) S 20213 20213 18999 0 -1 4202560 773 0 0 0 0 0 0 0 15 0 1 0 627883683 338456576 5732 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 11382126782846042371 0 0 17 0 0 0 0
/proc/20234/statm: 82631 5732 348 364 0 18534 0
[pid=20236] ppid=20234 vsize=2160 CPUtime=1.12 cores=0-7
/proc/20236/stat : 20236 (march_hi) R 20234 20236 20236 0 -1 4202496 597 0 0 0 112 0 0 0 25 0 1 0 627883684 2211840 333 33554432000 4194304 4955518 140735863778944 18446744073709551615 4224648 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/20236/statm: 540 333 104 186 0 353 0
[pid=20237] ppid=20213 vsize=340780 CPUtime=0 cores=0-7
/proc/20237/stat : 20237 (python) S 20213 20213 18999 0 -1 4202560 790 0 0 0 0 0 0 0 15 0 1 0 627883687 348958720 5736 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 2113239362647851268 0 0 17 4 0 0 0
/proc/20237/statm: 85195 5736 348 364 0 21095 0
[pid=20239] ppid=20237 vsize=2160 CPUtime=2.27 cores=0-7
/proc/20239/stat : 20239 (march_hi) R 20237 20239 20239 0 -1 4202496 612 0 0 0 227 0 0 0 25 0 1 0 627883688 2211840 349 33554432000 4194304 4955518 140733201215776 18446744073709551615 4224539 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/20239/statm: 540 349 104 186 0 353 0
[pid=20240] ppid=20213 vsize=351036 CPUtime=0 cores=0-7
/proc/20240/stat : 20240 (python) S 20213 20213 18999 0 -1 4202560 816 0 0 0 0 0 0 0 15 0 1 0 627883691 359460864 5755 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 7620178534736036097 0 0 17 0 0 0 0
/proc/20240/statm: 87759 5755 354 364 0 23656 0
[pid=20242] ppid=20240 vsize=2160 CPUtime=2.09 cores=0-7
/proc/20242/stat : 20242 (march_hi) R 20240 20242 20242 0 -1 4202496 609 0 0 0 209 0 0 0 25 0 1 0 627883692 2211840 342 33554432000 4194304 4955518 140735390915856 18446744073709551615 4224773 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/20242/statm: 540 342 104 186 0 353 0
[pid=20213/tid=20220] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20220/stat : 20220 (python) S 20211 20213 18999 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 9 0 627883663 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20223] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20223/stat : 20223 (python) S 20211 20213 18999 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 9 0 627883668 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20226] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20226/stat : 20226 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 9 0 627883671 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20229] ppid=20211 vsize=361280 CPUtime=0 cores=0-7
/proc/20213/task/20229/stat : 20229 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 22 0 9 0 627883676 369950720 7294 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20232] ppid=20211 vsize=361280 CPUtime=0 cores=0-7

################
# More data... #
################

/proc/20224/statm: 1581 1299 104 186 0 1394 0
[pid=20225] ppid=20213 vsize=299584 CPUtime=0.86 cores=0-7
/proc/20225/stat : 20225 (python) S 20213 20213 18999 0 -1 4202560 1001 0 0 0 45 41 0 0 15 0 1 0 627883671 306774016 5717 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 13298324861408477443 0 0 17 2 0 0 0
/proc/20225/statm: 74896 5717 371 364 0 10808 0
[pid=20227] ppid=20225 vsize=11060 CPUtime=282.98 cores=0-7
/proc/20227/stat : 20227 (march_hi) R 20225 20227 20227 0 -1 4202496 2340 0 0 0 28298 0 0 0 25 0 1 0 627883672 11325440 2078 33554432000 4194304 4955518 140737228553984 18446744073709551615 4314200 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/20227/statm: 2765 2078 104 186 0 2578 0
[pid=20228] ppid=20213 vsize=309840 CPUtime=0.84 cores=0-7
/proc/20228/stat : 20228 (python) S 20213 20213 18999 0 -1 4202560 1154 0 0 0 45 39 0 0 15 0 2 0 627883675 317276160 5751 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/20228/statm: 77460 5751 400 364 0 13369 0
[pid=20230] ppid=20228 vsize=8048 CPUtime=229.34 cores=0-7
/proc/20230/stat : 20230 (march_hi) R 20228 20230 20230 0 -1 4202496 1748 0 0 0 22933 1 0 0 25 0 1 0 627883676 8241152 1485 33554432000 4194304 4955518 140734863126448 18446744073709551615 4289873 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/20230/statm: 2012 1485 104 186 0 1825 0
[pid=20228/tid=20249] ppid=20213 vsize=309840 CPUtime=0 cores=0-7
/proc/20228/task/20249/stat : 20249 (python) S 20213 20213 18999 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 627893654 317276160 5751 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 514 18446604440305132544 0 0 -1 2 0 0 0
[pid=20231] ppid=20213 vsize=385632 CPUtime=0.23 cores=0-7
/proc/20231/stat : 20231 (python) S 20213 20213 18999 0 -1 4202560 1131 0 0 0 11 12 0 0 18 0 2 0 627883679 394887168 5765 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/20231/statm: 96408 5765 400 364 0 32314 0
[pid=20233] ppid=20231 vsize=1017800 CPUtime=99.12 cores=0-7
/proc/20233/stat : 20233 (gnovelty+2) T 20231 20233 20233 0 -1 4202496 434 0 0 0 9912 0 0 0 25 0 1 0 627883680 1042227200 173 33554432000 134512640 135663704 4292836944 18446744073709551615 134525286 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/20233/statm: 254450 173 113 282 0 254166 0
[pid=20231/tid=20245] ppid=20213 vsize=385632 CPUtime=0 cores=0-7
/proc/20231/task/20245/stat : 20245 (python) S 20213 20213 18999 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 627893604 394887168 5765 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 514 18446604447216698688 0 0 -1 1 0 0 0
[pid=20234] ppid=20213 vsize=330524 CPUtime=0.84 cores=0-7
/proc/20234/stat : 20234 (python) S 20213 20213 18999 0 -1 4202560 1031 0 0 0 46 38 0 0 15 0 1 0 627883683 338456576 5756 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 14930083675616805123 0 0 17 7 0 0 0
/proc/20234/statm: 82631 5756 372 364 0 18534 0
[pid=20236] ppid=20234 vsize=8472 CPUtime=258.92 cores=0-7
/proc/20236/stat : 20236 (march_hi) R 20234 20236 20236 0 -1 4202496 2032 0 0 0 25892 0 0 0 25 0 1 0 627883684 8675328 1768 33554432000 4194304 4955518 140735863778944 18446744073709551615 4239032 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/20236/statm: 2118 1768 104 186 0 1931 0
[pid=20237] ppid=20213 vsize=340780 CPUtime=1.35 cores=0-7
/proc/20237/stat : 20237 (python) R 20213 20213 18999 0 -1 4202560 1042 0 0 0 87 48 0 0 16 0 1 0 627883687 348958720 5759 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427221823 0 0 16781312 514 0 0 0 17 4 0 0 0
/proc/20237/statm: 85195 5759 371 364 0 21095 0
[pid=20239] ppid=20237 vsize=0 CPUtime=303.61 cores=0-7
/proc/20239/stat : 20239 (march_hi) R 20237 20239 20239 0 -1 4202500 2688 0 0 0 30360 1 0 0 25 0 1 0 627883688 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 4 0 0 0
/proc/20239/statm: 0 0 0 0 0 0 0
[pid=20240] ppid=20213 vsize=351036 CPUtime=0.83 cores=0-7
/proc/20240/stat : 20240 (python) S 20213 20213 18999 0 -1 4202560 1079 0 0 0 44 39 0 0 15 0 1 0 627883691 359460864 5778 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 7620178534736036097 0 0 17 0 0 0 0
/proc/20240/statm: 87759 5778 377 364 0 23656 0
[pid=20242] ppid=20240 vsize=11060 CPUtime=292.84 cores=0-7
/proc/20242/stat : 20242 (march_hi) R 20240 20242 20242 0 -1 4202496 2505 0 0 0 29280 4 0 0 25 0 1 0 627883692 11325440 2238 33554432000 4194304 4955518 140735390915856 18446744073709551615 4235347 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/20242/statm: 2765 2238 104 186 0 2578 0
[pid=20246] ppid=20213 vsize=361292 CPUtime=0.57 cores=0-7
/proc/20246/stat : 20246 (python) S 20213 20213 18999 0 -1 4202560 1048 0 0 0 30 27 0 0 15 0 1 0 627893608 369963008 5776 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 15389930228973273347 0 0 17 1 0 0 0
/proc/20246/statm: 90323 5776 371 364 0 26217 0
[pid=20248] ppid=20246 vsize=6112 CPUtime=190.34 cores=0-7
/proc/20248/stat : 20248 (march_hi) R 20246 20248 20248 0 -1 4202496 1449 0 0 0 19034 0 0 0 25 0 1 0 627893608 6258688 1181 33554432000 4194304 4955518 140734556999200 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/20248/statm: 1528 1181 104 186 0 1341 0
[pid=20251] ppid=20213 vsize=371680 CPUtime=0.51 cores=0-7
/proc/20251/stat : 20251 (python) S 20213 20213 18999 0 -1 4202560 1038 0 0 0 27 24 0 0 18 0 1 0 627893690 380600320 5804 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 14930083675616805121 0 0 17 1 0 0 0
/proc/20251/statm: 92920 5804 371 364 0 28811 0
[pid=20253] ppid=20251 vsize=182452 CPUtime=190.47 cores=0-7
/proc/20253/stat : 20253 (kcnfs-2006) R 20251 20253 20253 0 -1 4202496 571 0 0 0 19047 0 0 0 25 0 1 0 627893691 186830848 307 33554432000 134512640 135163436 4288748592 18446744073709551615 134569416 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/20253/statm: 45613 307 77 159 0 45453 0
[pid=20213/tid=20220] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20220/stat : 20220 (python) S 20211 20213 18999 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 11 0 627883663 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20223] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20223/stat : 20223 (python) S 20211 20213 18999 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 11 0 627883668 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20226] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20226/stat : 20226 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 11 0 627883671 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20229] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20229/stat : 20229 (python) S 20211 20213 18999 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 11 0 627883676 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20232] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20232/stat : 20232 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 11 0 627883679 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20235] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20235/stat : 20235 (python) S 20211 20213 18999 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 11 0 627883684 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20238] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20238/stat : 20238 (python) S 20211 20213 18999 0 -1 4202560 24 934 0 0 0 0 0 0 22 0 11 0 627883687 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20241] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20241/stat : 20241 (python) S 20211 20213 18999 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 11 0 627883692 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20247] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20247/stat : 20247 (python) S 20211 20213 18999 0 -1 4202560 28 934 0 0 0 0 0 0 19 0 11 0 627893608 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20252] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20252/stat : 20252 (python) S 20211 20213 18999 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 11 0 627893691 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2163.33
Current children cumulated vsize (KiB) 5979588

[startup+306.202 s]
/proc/loadavg: 9.13 8.75 8.86 11/360 20259
/proc/meminfo: memFree=21382020/32950928 swapFree=67111364/67111528
[pid=20213] ppid=20211 vsize=381924 CPUtime=0.94 cores=0-7
/proc/20213/stat : 20213 (python) S 20211 20213 18999 0 -1 4202496 18253 934 1 0 83 11 0 0 18 0 11 0 627883605 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427218379 0 0 16781312 2 18446744073709551615 0 0 17 0 0 0 0
/proc/20213/statm: 95481 7352 1894 364 0 31372 0
[pid=20219] ppid=20213 vsize=288840 CPUtime=0.23 cores=0-7
/proc/20219/stat : 20219 (python) S 20213 20213 18999 0 -1 4202560 1080 0 0 0 13 10 0 0 19 0 2 0 627883663 295772160 5634 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 4 0 0 0
/proc/20219/statm: 72210 5634 410 364 0 8128 0
[pid=20221] ppid=20219 vsize=1017800 CPUtime=100.22 cores=0-7
/proc/20221/stat : 20221 (gnovelty+2) T 20219 20221 20221 0 -1 4202496 437 0 0 0 10022 0 0 0 25 0 1 0 627883664 1042227200 173 33554432000 134512640 135663704 4292521440 18446744073709551615 134545631 0 0 16781312 0 18446744071562691360 0 0 17 5 0 0 0
/proc/20221/statm: 254450 173 113 282 0 254166 0
[pid=20219/tid=20250] ppid=20213 vsize=288840 CPUtime=0 cores=0-7
/proc/20219/task/20250/stat : 20250 (python) S 20213 20213 18999 0 -1 4202560 50 0 0 0 0 0 0 0 19 0 2 0 627893686 295772160 5634 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 514 18446604440305133440 0 0 -1 4 0 0 0
[pid=20222] ppid=20213 vsize=289328 CPUtime=0.84 cores=0-7
/proc/20222/stat : 20222 (python) S 20213 20213 18999 0 -1 4202560 987 0 0 0 43 41 0 0 15 0 1 0 627883667 296271872 5713 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 8423925932686803201 0 0 17 5 0 0 0
/proc/20222/statm: 72332 5713 371 364 0 8247 0
[pid=20224] ppid=20222 vsize=6324 CPUtime=207.55 cores=0-7
/proc/20224/stat : 20224 (march_hi) R 20222 20224 20224 0 -1 4202496 1561 0 0 0 20755 0 0 0 25 0 1 0 627883668 6475776 1300 33554432000 4194304 4955518 140736461013904 18446744073709551615 4209628 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/20224/statm: 1581 1300 104 186 0 1394 0
[pid=20225] ppid=20213 vsize=299584 CPUtime=0.86 cores=0-7
/proc/20225/stat : 20225 (python) S 20213 20213 18999 0 -1 4202560 1001 0 0 0 45 41 0 0 15 0 1 0 627883671 306774016 5717 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 13298324861408477443 0 0 17 2 0 0 0
/proc/20225/statm: 74896 5717 371 364 0 10808 0
[pid=20227] ppid=20225 vsize=11060 CPUtime=283.08 cores=0-7
/proc/20227/stat : 20227 (march_hi) R 20225 20227 20227 0 -1 4202496 2341 0 0 0 28308 0 0 0 25 0 1 0 627883672 11325440 2079 33554432000 4194304 4955518 140737228553984 18446744073709551615 4282406 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/20227/statm: 2765 2079 104 186 0 2578 0
[pid=20228] ppid=20213 vsize=309840 CPUtime=0.84 cores=0-7
/proc/20228/stat : 20228 (python) S 20213 20213 18999 0 -1 4202560 1154 0 0 0 45 39 0 0 15 0 2 0 627883675 317276160 5751 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/20228/statm: 77460 5751 400 364 0 13369 0
[pid=20230] ppid=20228 vsize=8048 CPUtime=229.44 cores=0-7
/proc/20230/stat : 20230 (march_hi) R 20228 20230 20230 0 -1 4202496 1749 0 0 0 22943 1 0 0 25 0 1 0 627883676 8241152 1486 33554432000 4194304 4955518 140734863126448 18446744073709551615 4209688 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/20230/statm: 2012 1486 104 186 0 1825 0
[pid=20228/tid=20249] ppid=20213 vsize=309840 CPUtime=0 cores=0-7
/proc/20228/task/20249/stat : 20249 (python) S 20213 20213 18999 0 -1 4202560 48 0 0 0 0 0 0 0 15 0 2 0 627893654 317276160 5751 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 514 18446604440305132544 0 0 -1 2 0 0 0
[pid=20231] ppid=20213 vsize=385632 CPUtime=0.23 cores=0-7
/proc/20231/stat : 20231 (python) S 20213 20213 18999 0 -1 4202560 1131 0 0 0 11 12 0 0 18 0 2 0 627883679 394887168 5765 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427218379 0 0 16781312 514 18446744073709551615 0 0 17 2 0 0 0
/proc/20231/statm: 96408 5765 400 364 0 32314 0
[pid=20233] ppid=20231 vsize=1017800 CPUtime=99.12 cores=0-7
/proc/20233/stat : 20233 (gnovelty+2) T 20231 20233 20233 0 -1 4202496 434 0 0 0 9912 0 0 0 25 0 1 0 627883680 1042227200 173 33554432000 134512640 135663704 4292836944 18446744073709551615 134525286 0 0 16781312 0 18446744071562691360 0 0 17 1 0 0 0
/proc/20233/statm: 254450 173 113 282 0 254166 0
[pid=20231/tid=20245] ppid=20213 vsize=385632 CPUtime=0 cores=0-7
/proc/20231/task/20245/stat : 20245 (python) S 20213 20213 18999 0 -1 4202560 49 0 0 0 0 0 0 0 19 0 2 0 627893604 394887168 5765 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 514 18446604447216698688 0 0 -1 1 0 0 0
[pid=20234] ppid=20213 vsize=330524 CPUtime=0.84 cores=0-7
/proc/20234/stat : 20234 (python) S 20213 20213 18999 0 -1 4202560 1031 0 0 0 46 38 0 0 15 0 1 0 627883683 338456576 5756 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 14930083675616805123 0 0 17 7 0 0 0
/proc/20234/statm: 82631 5756 372 364 0 18534 0
[pid=20236] ppid=20234 vsize=8472 CPUtime=258.97 cores=0-7
/proc/20236/stat : 20236 (march_hi) R 20234 20236 20236 0 -1 4202496 2032 0 0 0 25897 0 0 0 25 0 1 0 627883684 8675328 1768 33554432000 4194304 4955518 140735863778944 18446744073709551615 4209675 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/20236/statm: 2118 1768 104 186 0 1931 0
[pid=20237] ppid=20213 vsize=340780 CPUtime=1.45 cores=0-7
/proc/20237/stat : 20237 (python) R 20213 20213 18999 0 -1 4202560 1042 0 0 0 96 49 0 0 16 0 1 0 627883687 348958720 5759 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427221823 0 0 16781312 514 0 0 0 17 4 0 0 0
/proc/20237/statm: 85195 5759 371 364 0 21095 0
[pid=20239] ppid=20237 vsize=0 CPUtime=303.61 cores=0-7
/proc/20239/stat : 20239 (march_hi) R 20237 20239 20239 0 -1 4202500 2688 0 0 0 30360 1 0 0 25 0 1 0 627883688 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/20239/statm: 0 0 0 0 0 0 0
[pid=20240] ppid=20213 vsize=351036 CPUtime=0.83 cores=0-7
/proc/20240/stat : 20240 (python) S 20213 20213 18999 0 -1 4202560 1079 0 0 0 44 39 0 0 15 0 1 0 627883691 359460864 5778 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 7620178534736036097 0 0 17 0 0 0 0
/proc/20240/statm: 87759 5778 377 364 0 23656 0
[pid=20242] ppid=20240 vsize=11060 CPUtime=292.94 cores=0-7
/proc/20242/stat : 20242 (march_hi) R 20240 20242 20242 0 -1 4202496 2507 0 0 0 29290 4 0 0 25 0 1 0 627883692 11325440 2240 33554432000 4194304 4955518 140735390915856 18446744073709551615 4209780 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/20242/statm: 2765 2240 104 186 0 2578 0
[pid=20246] ppid=20213 vsize=361292 CPUtime=0.57 cores=0-7
/proc/20246/stat : 20246 (python) S 20213 20213 18999 0 -1 4202560 1048 0 0 0 30 27 0 0 15 0 1 0 627893608 369963008 5776 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 15389930228973273347 0 0 17 1 0 0 0
/proc/20246/statm: 90323 5776 371 364 0 26217 0
[pid=20248] ppid=20246 vsize=6112 CPUtime=190.44 cores=0-7
/proc/20248/stat : 20248 (march_hi) R 20246 20248 20248 0 -1 4202496 1451 0 0 0 19044 0 0 0 25 0 1 0 627893608 6258688 1183 33554432000 4194304 4955518 140734556999200 18446744073709551615 4209701 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/20248/statm: 1528 1183 104 186 0 1341 0
[pid=20251] ppid=20213 vsize=371680 CPUtime=0.51 cores=0-7
/proc/20251/stat : 20251 (python) S 20213 20213 18999 0 -1 4202560 1038 0 0 0 27 24 0 0 18 0 1 0 627893690 380600320 5804 33554432000 4194304 5683644 140737200111600 18446744073709551615 256415411782 0 0 16781312 514 14930083675616805121 0 0 17 1 0 0 0
/proc/20251/statm: 92920 5804 371 364 0 28811 0
[pid=20253] ppid=20251 vsize=182452 CPUtime=190.52 cores=0-7
/proc/20253/stat : 20253 (kcnfs-2006) R 20251 20253 20253 0 -1 4202496 571 0 0 0 19052 0 0 0 25 0 1 0 627893691 186830848 307 33554432000 134512640 135163436 4288748592 18446744073709551615 134675067 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/20253/statm: 45613 307 77 159 0 45453 0
[pid=20213/tid=20220] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20220/stat : 20220 (python) S 20211 20213 18999 0 -1 4202560 25 934 0 0 0 0 0 0 19 0 11 0 627883663 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20223] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20223/stat : 20223 (python) S 20211 20213 18999 0 -1 4202560 24 934 0 0 0 0 0 0 21 0 11 0 627883668 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20226] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20226/stat : 20226 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 18 0 11 0 627883671 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 4 0 0 0
[pid=20213/tid=20229] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20229/stat : 20229 (python) S 20211 20213 18999 0 -1 4202560 37 934 0 0 0 0 0 0 18 0 11 0 627883676 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20232] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20232/stat : 20232 (python) S 20211 20213 18999 0 -1 4202560 26 934 0 0 0 0 0 0 20 0 11 0 627883679 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20235] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20235/stat : 20235 (python) S 20211 20213 18999 0 -1 4202560 25 934 0 0 0 0 0 0 20 0 11 0 627883684 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20238] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20238/stat : 20238 (python) S 20211 20213 18999 0 -1 4202560 24 934 0 0 0 0 0 0 22 0 11 0 627883687 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 2 0 0 0
[pid=20213/tid=20241] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20241/stat : 20241 (python) S 20211 20213 18999 0 -1 4202560 30 934 0 0 0 0 0 0 20 0 11 0 627883692 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20247] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20247/stat : 20247 (python) S 20211 20213 18999 0 -1 4202560 28 934 0 0 0 0 0 0 19 0 11 0 627893608 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
[pid=20213/tid=20252] ppid=20211 vsize=381924 CPUtime=0 cores=0-7
/proc/20213/task/20252/stat : 20252 (python) S 20211 20213 18999 0 -1 4202560 29 934 0 0 0 0 0 0 19 0 11 0 627893691 391090176 7352 33554432000 4194304 5683644 140737200111600 18446744073709551615 256427215281 0 0 16781312 2 18446604441496220864 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2164.03
Current children cumulated vsize (KiB) 5979588

Child status: 0
Real time (s): 306.275
CPU time (s): 2164.56
CPU user time (s): 2161.05
CPU system time (s): 3.50547
CPU usage (%): 706.738
Max. virtual memory (cumulated for all children) (KiB): 5991496

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2161.05
system time used= 3.50547
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 50248
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6062
involuntary context switches= 17619

runsolver used 1.70374 second user time and 3.51147 second system time

The end

Launcher Data 

Begin job on node115 at 2011-05-03 00:48:49
IDJOB=3322238
IDBENCH=83016
IDSOLVER=1712
FILE ID=node115/3322238-1304376529
RUNJOBID= node115-1304367627-19013
PBS_JOBID= 13181769
Free space on /tmp= 73484 MiB

SOLVER NAME= borg-sat 11.03.02-random
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v450-c1917-S1879031150-062.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3322238-1304376529/watcher-3322238-1304376529 -o /tmp/evaluation-result-3322238-1304376529/solver-3322238-1304376529 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3322238-1304376529.cnf -seed 1576083386 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= dea41a8eae52b60f8e05be01aec1734d
RANDOM SEED=1576083386

node115.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.62
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.77
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.811
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      21488616 kB
Buffers:        906288 kB
Cached:        2636848 kB
SwapCached:          0 kB
Active:        7950300 kB
Inactive:      3037096 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      21488616 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:           31816 kB
Writeback:           0 kB
AnonPages:     7444600 kB
Mapped:          26228 kB
Slab:           373808 kB
PageTables:      34288 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 61795280 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73452 MiB
End job on node115 at 2011-05-03 00:53:56