Trace number 3385667

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 304.226 38.7067

General information on the benchmark

Namerandom/medium/
unif-k3-r4.26-v400-c1704-S549184435-037.UNKNOWN.cnf
MD5SUMab99d3f634575de78557b69bc7096528
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark37.1314
Satisfiable
(Un)Satisfiability was proved
Number of variables400
Number of clauses1704
Sum of the clauses size5112
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 31704
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data 

0.32/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.47	c INFO: solving HOME/instance-3385667-1305572027.cnf
0.32/0.47	c INFO: computing task features
0.32/0.47	c INFO: computed variable-clause graph statistics
0.32/0.47	c INFO: computed clause balance statistics
0.41/0.50	c INFO: computed variable balance statistics
0.41/0.53	c INFO: computed small-clause counts
0.41/0.53	c INFO: computed Horn-clause counts
0.41/0.53	c INFO: computed variable graph statistics
0.41/0.53	c INFO: collected features for HOME/instance-3385667-1305572027.cnf in 0.06 s
0.41/0.56	c INFO: running gnovelty+2@0 for 97 with 4999 remaining (b = 0.56)
0.41/0.59	c INFO: running march_hi@0 for 194 with 4999 remaining (b = 0.26)
0.51/0.62	c INFO: running march_hi@0 for 194 with 4999 remaining (b = 0.18)
0.51/0.65	c INFO: running TNM@0 for 97 with 4999 remaining (b = 0.07)
0.51/0.69	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.28)
0.61/0.72	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.25)
0.61/0.75	c INFO: running march_hi@0 for 388 with 4999 remaining (b = 0.22)
0.61/0.79	c INFO: running march_hi@0 for 485 with 4999 remaining (b = 0.22)
303.39/38.68	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3385667-1305572027/watcher-3385667-1305572027 -o /tmp/evaluation-result-3385667-1305572027/solver-3385667-1305572027 -C 40000 -W 5100 -M 31000 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385667-1305572027.cnf -seed 2084976151 -budget 40000 -cores 8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 4.24 6.72 7.59 1/370 4402
/proc/meminfo: memFree=12656212/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=15384 CPUtime=0 cores=0-7
/proc/4402/stat : 4402 (python) S 4400 4402 4082 0 -1 4202496 809 0 0 0 0 0 0 0 25 0 1 0 668801513 15753216 696 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594864640 0 2147483391 16781312 2 18446744071699215550 0 0 17 2 0 0 0
/proc/4402/statm: 3846 696 334 364 0 333 0

[startup+0.050263 s]
/proc/loadavg: 4.24 6.72 7.59 1/370 4402
/proc/meminfo: memFree=12656212/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=29792 CPUtime=0.03 cores=0-7
/proc/4402/stat : 4402 (python) S 4400 4402 4082 0 -1 4202496 1454 0 0 0 2 1 0 0 25 0 1 0 668801513 30507008 1296 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594864640 0 2147483391 16781312 2 0 0 0 17 2 0 0 0
/proc/4402/statm: 7449 1297 443 364 0 819 0
Current children cumulated CPU time (s) 0.03
Current children cumulated vsize (KiB) 29792

[startup+0.10027 s]
/proc/loadavg: 4.24 6.72 7.59 1/370 4402
/proc/meminfo: memFree=12656212/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=104228 CPUtime=0.07 cores=0-7
/proc/4402/stat : 4402 (python) R 4400 4402 4082 0 -1 4202496 3057 0 0 0 5 2 0 0 25 0 1 0 668801513 106729472 2741 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594864640 0 0 16781312 2 0 0 0 17 2 0 0 0
/proc/4402/statm: 26057 2741 847 364 0 1805 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104228

[startup+0.3003 s]
/proc/loadavg: 4.24 6.72 7.59 1/370 4402
/proc/meminfo: memFree=12656212/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=170756 CPUtime=0.23 cores=0-7
/proc/4402/stat : 4402 (python) S 4400 4402 4082 0 -1 4202496 6731 935 1 0 17 6 0 0 25 0 1 0 668801513 174854144 4805 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594864640 0 2147483391 16781312 2 18446744071699215550 0 0 17 2 0 0 0
/proc/4402/statm: 42689 4805 1336 364 0 3454 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 170756

[startup+0.700147 s]
/proc/loadavg: 4.24 6.72 7.59 1/370 4402
/proc/meminfo: memFree=12656212/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=330488 CPUtime=0.61 cores=0-7
/proc/4402/stat : 4402 (python) R 4400 4402 4082 0 -1 4202496 13553 935 1 0 51 10 0 0 18 0 6 0 668801513 338419712 7259 33554432000 4194304 5683644 140736744942512 18446744073709551615 47980875046518 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4402/statm: 82622 7259 1882 364 0 18528 0
Current children cumulated CPU time (s) 0.61
Current children cumulated vsize (KiB) 330488

[startup+1.50101 s]
/proc/loadavg: 4.24 6.72 7.59 9/395 4432
/proc/meminfo: memFree=12598520/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=361292 CPUtime=0.7 cores=0-7
/proc/4402/stat : 4402 (python) S 4400 4402 4082 0 -1 4202496 15754 935 1 0 59 11 0 0 24 0 9 0 668801513 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606693323 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4402/statm: 90323 7287 1889 364 0 26220 0
[pid=4408] ppid=4402 vsize=278620 CPUtime=0 cores=0-7
/proc/4408/stat : 4408 (python) S 4402 4402 4082 0 -1 4202560 740 0 0 0 0 0 0 0 25 0 1 0 668801569 285306880 5568 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886671 0 0 16781312 514 3874128675007889407 0 0 17 7 0 0 0
/proc/4408/statm: 69655 5568 354 364 0 5573 0
[pid=4410] ppid=4408 vsize=1017800 CPUtime=0.93 cores=0-7
/proc/4410/stat : 4410 (gnovelty+2) R 4408 4410 4410 0 -1 4202496 431 0 0 0 93 0 0 0 25 0 1 0 668801570 1042227200 168 33554432000 134512640 135663704 4293055392 18446744073709551615 134544218 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4410/statm: 254450 168 113 282 0 254166 0
[pid=4411] ppid=4402 vsize=289308 CPUtime=0 cores=0-7
/proc/4411/stat : 4411 (python) S 4402 4402 4082 0 -1 4202560 747 0 0 0 0 0 0 0 17 0 1 0 668801572 296251392 5684 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 6499028518125404419 0 0 17 4 0 0 0
/proc/4411/statm: 72327 5684 348 364 0 8242 0
[pid=4413] ppid=4411 vsize=1928 CPUtime=0.9 cores=0-7
/proc/4413/stat : 4413 (march_hi) R 4411 4413 4413 0 -1 4202496 570 0 0 0 90 0 0 0 25 0 1 0 668801573 1974272 308 33554432000 4194304 4955518 140735657674016 18446744073709551615 4238727 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/4413/statm: 482 308 104 186 0 295 0
[pid=4414] ppid=4402 vsize=299192 CPUtime=0 cores=0-7
/proc/4414/stat : 4414 (python) S 4402 4402 4082 0 -1 4202560 757 0 0 0 0 0 0 0 17 0 1 0 668801575 306372608 5592 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 1565769708311904515 0 0 17 6 0 0 0
/proc/4414/statm: 74798 5592 348 364 0 10710 0
[pid=4416] ppid=4414 vsize=1928 CPUtime=0.87 cores=0-7
/proc/4416/stat : 4416 (march_hi) R 4414 4416 4416 0 -1 4202496 566 0 0 0 87 0 0 0 25 0 1 0 668801576 1974272 306 33554432000 4194304 4955518 140736152763360 18446744073709551615 4224704 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4416/statm: 482 306 104 186 0 295 0
[pid=4417] ppid=4402 vsize=309988 CPUtime=0 cores=0-7
/proc/4417/stat : 4417 (python) S 4402 4402 4082 0 -1 4202560 729 0 0 0 0 0 0 0 19 0 1 0 668801579 317427712 5713 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 47437883719548927 0 0 17 0 0 0 0
/proc/4417/statm: 77497 5713 344 364 0 13406 0
[pid=4419] ppid=4417 vsize=454908 CPUtime=0.82 cores=0-7
/proc/4419/stat : 4419 (TNM) R 4417 4419 4419 0 -1 4202496 426 0 0 0 82 0 0 0 23 0 1 0 668801579 465825792 160 33554432000 134512640 135008748 4286984496 18446744073709551615 134528639 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/4419/statm: 113727 160 46 122 0 112627 0
[pid=4420] ppid=4402 vsize=320244 CPUtime=0 cores=0-7
/proc/4420/stat : 4420 (python) S 4402 4402 4082 0 -1 4202560 764 0 0 0 0 0 0 0 15 0 1 0 668801582 327929856 5721 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 711388425512517890 0 0 17 2 0 0 0
/proc/4420/statm: 80061 5721 348 364 0 15967 0
[pid=4422] ppid=4420 vsize=1928 CPUtime=0.8 cores=0-7
/proc/4422/stat : 4422 (march_hi) R 4420 4422 4422 0 -1 4202496 574 0 0 0 80 0 0 0 25 0 1 0 668801582 1974272 307 33554432000 4194304 4955518 140734441682080 18446744073709551615 4227654 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4422/statm: 482 307 104 186 0 295 0
[pid=4423] ppid=4402 vsize=330500 CPUtime=0 cores=0-7
/proc/4423/stat : 4423 (python) S 4402 4402 4082 0 -1 4202560 764 0 0 0 0 0 0 0 15 0 1 0 668801585 338432000 5725 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 11614383221050933505 0 0 17 1 0 0 0
/proc/4423/statm: 82625 5725 348 364 0 18528 0
[pid=4425] ppid=4423 vsize=1928 CPUtime=0.77 cores=0-7
/proc/4425/stat : 4425 (march_hi) R 4423 4425 4425 0 -1 4202496 570 0 0 0 77 0 0 0 22 0 1 0 668801586 1974272 306 33554432000 4194304 4955518 140737140088656 18446744073709551615 4282800 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/4425/statm: 482 306 104 186 0 295 0
[pid=4426] ppid=4402 vsize=340184 CPUtime=0 cores=0-7
/proc/4426/stat : 4426 (python) S 4402 4402 4082 0 -1 4202560 779 0 0 0 0 0 0 0 15 0 1 0 668801589 348348416 5618 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 1541436141599686915 0 0 17 2 0 0 0
/proc/4426/statm: 85046 5618 348 364 0 20946 0
[pid=4429] ppid=4426 vsize=1928 CPUtime=0.66 cores=0-7
/proc/4429/stat : 4429 (march_hi) R 4426 4429 4429 0 -1 4202496 569 0 0 0 66 0 0 0 25 0 1 0 668801589 1974272 302 33554432000 4194304 4955518 140734754312848 18446744073709551615 4224679 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4429/statm: 482 302 104 186 0 295 0
[pid=4430] ppid=4402 vsize=351048 CPUtime=0 cores=0-7
/proc/4430/stat : 4430 (python) S 4402 4402 4082 0 -1 4202560 811 0 0 0 0 0 0 0 18 0 1 0 668801592 359473152 5748 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 4358354145636221185 0 0 17 2 0 0 0
/proc/4430/statm: 87762 5748 354 364 0 23659 0
[pid=4432] ppid=4430 vsize=1928 CPUtime=0.7 cores=0-7
/proc/4432/stat : 4432 (march_hi) R 4430 4432 4432 0 -1 4202496 577 0 0 0 70 0 0 0 25 0 1 0 668801592 1974272 306 33554432000 4194304 4955518 140736106439776 18446744073709551615 4224658 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4432/statm: 482 306 104 186 0 295 0
[pid=4402/tid=4409] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4409/stat : 4409 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 9 0 668801569 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4412] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4412/stat : 4412 (python) S 4400 4402 4082 0 -1 4202560 21 935 0 0 0 0 0 0 25 0 9 0 668801572 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 2 0 0 0
[pid=4402/tid=4415] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4415/stat : 4415 (python) S 4400 4402 4082 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 9 0 668801576 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 6 0 0 0
[pid=4402/tid=4418] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4418/stat : 4418 (python) S 4400 4402 4082 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 9 0 668801579 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4421] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4421/stat : 4421 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 22 0 9 0 668801582 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4424] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4424/stat : 4424 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 9 0 668801586 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4427] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4427/stat : 4427 (python) S 4400 4402 4082 0 -1 4202560 24 935 0 0 0 0 0 0 22 0 9 0 668801589 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4431] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4431/stat : 4431 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 24 0 9 0 668801592 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 7.15
Current children cumulated vsize (KiB) 4364652

[startup+3.10172 s]
/proc/loadavg: 4.62 6.76 7.60 9/395 4432
/proc/meminfo: memFree=12597156/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=361292 CPUtime=0.7 cores=0-7
/proc/4402/stat : 4402 (python) S 4400 4402 4082 0 -1 4202496 15754 935 1 0 59 11 0 0 24 0 9 0 668801513 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606693323 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4402/statm: 90323 7287 1889 364 0 26220 0
[pid=4408] ppid=4402 vsize=278620 CPUtime=0 cores=0-7
/proc/4408/stat : 4408 (python) S 4402 4402 4082 0 -1 4202560 740 0 0 0 0 0 0 0 18 0 1 0 668801569 285306880 5568 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886671 0 0 16781312 514 3874128675007856900 0 0 17 7 0 0 0
/proc/4408/statm: 69655 5568 354 364 0 5573 0
[pid=4410] ppid=4408 vsize=1017800 CPUtime=2.53 cores=0-7
/proc/4410/stat : 4410 (gnovelty+2) R 4408 4410 4410 0 -1 4202496 431 0 0 0 253 0 0 0 25 0 1 0 668801570 1042227200 168 33554432000 134512640 135663704 4293055392 18446744073709551615 134525255 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4410/statm: 254450 168 113 282 0 254166 0
[pid=4411] ppid=4402 vsize=289308 CPUtime=0 cores=0-7
/proc/4411/stat : 4411 (python) S 4402 4402 4082 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 668801572 296251392 5684 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 6499028518125404419 0 0 17 4 0 0 0
/proc/4411/statm: 72327 5684 348 364 0 8242 0
[pid=4413] ppid=4411 vsize=2060 CPUtime=2.5 cores=0-7
/proc/4413/stat : 4413 (march_hi) R 4411 4413 4413 0 -1 4202496 579 0 0 0 250 0 0 0 25 0 1 0 668801573 2109440 317 33554432000 4194304 4955518 140735657674016 18446744073709551615 4209717 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/4413/statm: 515 317 104 186 0 328 0
[pid=4414] ppid=4402 vsize=299192 CPUtime=0 cores=0-7
/proc/4414/stat : 4414 (python) S 4402 4402 4082 0 -1 4202560 761 0 0 0 0 0 0 0 15 0 1 0 668801575 306372608 5592 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 16050538805173649667 0 0 17 6 0 0 0
/proc/4414/statm: 74798 5592 348 364 0 10710 0
[pid=4416] ppid=4414 vsize=2060 CPUtime=2.47 cores=0-7
/proc/4416/stat : 4416 (march_hi) R 4414 4416 4416 0 -1 4202496 576 0 0 0 247 0 0 0 25 0 1 0 668801576 2109440 316 33554432000 4194304 4955518 140736152763360 18446744073709551615 4209717 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4416/statm: 515 316 104 186 0 328 0
[pid=4417] ppid=4402 vsize=309988 CPUtime=0 cores=0-7
/proc/4417/stat : 4417 (python) S 4402 4402 4082 0 -1 4202560 747 0 0 0 0 0 0 0 18 0 1 0 668801579 317427712 5717 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 47437883719516419 0 0 17 0 0 0 0
/proc/4417/statm: 77497 5717 348 364 0 13406 0
[pid=4419] ppid=4417 vsize=454908 CPUtime=2.4 cores=0-7
/proc/4419/stat : 4419 (TNM) R 4417 4419 4419 0 -1 4202496 426 0 0 0 240 0 0 0 25 0 1 0 668801579 465825792 160 33554432000 134512640 135008748 4286984496 18446744073709551615 134527111 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/4419/statm: 113727 160 46 122 0 112627 0
[pid=4420] ppid=4402 vsize=320244 CPUtime=0 cores=0-7
/proc/4420/stat : 4420 (python) S 4402 4402 4082 0 -1 4202560 767 0 0 0 0 0 0 0 15 0 1 0 668801582 327929856 5721 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 711388425512517890 0 0 17 2 0 0 0
/proc/4420/statm: 80061 5721 348 364 0 15967 0
[pid=4422] ppid=4420 vsize=2060 CPUtime=2.4 cores=0-7
/proc/4422/stat : 4422 (march_hi) R 4420 4422 4422 0 -1 4202496 584 0 0 0 240 0 0 0 25 0 1 0 668801582 2109440 317 33554432000 4194304 4955518 140734441682080 18446744073709551615 4346490 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4422/statm: 515 317 104 186 0 328 0
[pid=4423] ppid=4402 vsize=330500 CPUtime=0 cores=0-7
/proc/4423/stat : 4423 (python) S 4402 4402 4082 0 -1 4202560 767 0 0 0 0 0 0 0 15 0 1 0 668801585 338432000 5725 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 11614383221050933505 0 0 17 1 0 0 0
/proc/4423/statm: 82625 5725 348 364 0 18528 0
[pid=4425] ppid=4423 vsize=1928 CPUtime=2.37 cores=0-7
/proc/4425/stat : 4425 (march_hi) R 4423 4425 4425 0 -1 4202496 580 0 0 0 237 0 0 0 25 0 1 0 668801586 1974272 316 33554432000 4194304 4955518 140737140088656 18446744073709551615 4244760 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/4425/statm: 482 316 104 186 0 295 0
[pid=4426] ppid=4402 vsize=340184 CPUtime=0 cores=0-7
/proc/4426/stat : 4426 (python) S 4402 4402 4082 0 -1 4202560 780 0 0 0 0 0 0 0 15 0 1 0 668801589 348348416 5618 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 1541436141599686915 0 0 17 2 0 0 0
/proc/4426/statm: 85046 5618 348 364 0 20946 0
[pid=4429] ppid=4426 vsize=1928 CPUtime=2.26 cores=0-7
/proc/4429/stat : 4429 (march_hi) R 4426 4429 4429 0 -1 4202496 582 0 0 0 226 0 0 0 25 0 1 0 668801589 1974272 315 33554432000 4194304 4955518 140734754312848 18446744073709551615 4239009 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4429/statm: 482 315 104 186 0 295 0
[pid=4430] ppid=4402 vsize=351048 CPUtime=0 cores=0-7
/proc/4430/stat : 4430 (python) S 4402 4402 4082 0 -1 4202560 815 0 0 0 0 0 0 0 15 0 1 0 668801592 359473152 5748 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 5607389180012232961 0 0 17 2 0 0 0
/proc/4430/statm: 87762 5748 354 364 0 23659 0
[pid=4432] ppid=4430 vsize=1928 CPUtime=2.3 cores=0-7
/proc/4432/stat : 4432 (march_hi) R 4430 4432 4432 0 -1 4202496 587 0 0 0 230 0 0 0 25 0 1 0 668801592 1974272 316 33554432000 4194304 4955518 140736106439776 18446744073709551615 4227502 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4432/statm: 482 316 104 186 0 295 0
[pid=4402/tid=4409] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4409/stat : 4409 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 9 0 668801569 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4412] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4412/stat : 4412 (python) S 4400 4402 4082 0 -1 4202560 21 935 0 0 0 0 0 0 25 0 9 0 668801572 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 2 0 0 0
[pid=4402/tid=4415] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4415/stat : 4415 (python) S 4400 4402 4082 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 9 0 668801576 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 6 0 0 0
[pid=4402/tid=4418] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4418/stat : 4418 (python) S 4400 4402 4082 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 9 0 668801579 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4421] ppid=4400 vsize=361292 CPUtime=0 cores=0-7

################
# More data... #
################

/proc/4402/task/4415/stat : 4415 (python) S 4400 4402 4082 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 9 0 668801576 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 6 0 0 0
[pid=4402/tid=4418] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4418/stat : 4418 (python) S 4400 4402 4082 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 9 0 668801579 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4421] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4421/stat : 4421 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 22 0 9 0 668801582 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4424] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4424/stat : 4424 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 9 0 668801586 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4427] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4427/stat : 4427 (python) S 4400 4402 4082 0 -1 4202560 24 935 0 0 0 0 0 0 22 0 9 0 668801589 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4431] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4431/stat : 4431 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 24 0 9 0 668801592 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 288.22
Current children cumulated vsize (KiB) 4368108

[startup+37.5017 s]
/proc/loadavg: 6.12 6.90 7.62 9/395 4433
/proc/meminfo: memFree=12594900/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=361292 CPUtime=0.7 cores=0-7
/proc/4402/stat : 4402 (python) S 4400 4402 4082 0 -1 4202496 15754 935 1 0 59 11 0 0 24 0 9 0 668801513 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606693323 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4402/statm: 90323 7287 1889 364 0 26220 0
[pid=4408] ppid=4402 vsize=278620 CPUtime=0.09 cores=0-7
/proc/4408/stat : 4408 (python) S 4402 4402 4082 0 -1 4202560 916 0 0 0 4 5 0 0 18 0 1 0 668801569 285306880 5591 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886671 0 0 16781312 514 8452763919365144836 0 0 17 7 0 0 0
/proc/4408/statm: 69655 5591 377 364 0 5573 0
[pid=4410] ppid=4408 vsize=1017800 CPUtime=36.93 cores=0-7
/proc/4410/stat : 4410 (gnovelty+2) R 4408 4410 4410 0 -1 4202496 431 0 0 0 3693 0 0 0 25 0 1 0 668801570 1042227200 168 33554432000 134512640 135663704 4293055392 18446744073709551615 134524436 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4410/statm: 254450 168 113 282 0 254166 0
[pid=4411] ppid=4402 vsize=289308 CPUtime=0.12 cores=0-7
/proc/4411/stat : 4411 (python) S 4402 4402 4082 0 -1 4202560 939 0 0 0 7 5 0 0 15 0 1 0 668801572 296251392 5707 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 6499028518125404419 0 0 17 5 0 0 0
/proc/4411/statm: 72327 5707 371 364 0 8242 0
[pid=4413] ppid=4411 vsize=2656 CPUtime=36.84 cores=0-7
/proc/4413/stat : 4413 (march_hi) R 4411 4413 4413 0 -1 4202496 708 0 0 0 3684 0 0 0 25 0 1 0 668801573 2719744 446 33554432000 4194304 4955518 140735657674016 18446744073709551615 4227824 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/4413/statm: 664 446 104 186 0 477 0
[pid=4414] ppid=4402 vsize=299192 CPUtime=0.11 cores=0-7
/proc/4414/stat : 4414 (python) S 4402 4402 4082 0 -1 4202560 925 0 0 0 6 5 0 0 15 0 1 0 668801575 306372608 5615 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 16050538805173649667 0 0 17 6 0 0 0
/proc/4414/statm: 74798 5615 371 364 0 10710 0
[pid=4416] ppid=4414 vsize=2656 CPUtime=36.74 cores=0-7
/proc/4416/stat : 4416 (march_hi) R 4414 4416 4416 0 -1 4202496 704 0 0 0 3674 0 0 0 25 0 1 0 668801576 2719744 444 33554432000 4194304 4955518 140736152763360 18446744073709551615 4227679 0 0 16781312 0 0 0 0 17 6 0 0 0
/proc/4416/statm: 664 444 104 186 0 477 0
[pid=4417] ppid=4402 vsize=309988 CPUtime=0.09 cores=0-7
/proc/4417/stat : 4417 (python) S 4402 4402 4082 0 -1 4202560 919 0 0 0 5 4 0 0 18 0 1 0 668801579 317427712 5740 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 8452763919365144835 0 0 17 0 0 0 0
/proc/4417/statm: 77497 5740 371 364 0 13406 0
[pid=4419] ppid=4417 vsize=454908 CPUtime=36.29 cores=0-7
/proc/4419/stat : 4419 (TNM) R 4417 4419 4419 0 -1 4202496 426 0 0 0 3629 0 0 0 25 0 1 0 668801579 465825792 160 33554432000 134512640 135008748 4286984496 18446744073709551615 134528628 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/4419/statm: 113727 160 46 122 0 112627 0
[pid=4420] ppid=4402 vsize=320244 CPUtime=0.11 cores=0-7
/proc/4420/stat : 4420 (python) S 4402 4402 4082 0 -1 4202560 972 0 0 0 6 5 0 0 15 0 1 0 668801582 327929856 5744 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 5607389326041121026 0 0 17 0 0 0 0
/proc/4420/statm: 80061 5744 371 364 0 15967 0
[pid=4422] ppid=4420 vsize=2656 CPUtime=36.64 cores=0-7
/proc/4422/stat : 4422 (march_hi) R 4420 4422 4422 0 -1 4202496 712 0 0 0 3664 0 0 0 25 0 1 0 668801582 2719744 445 33554432000 4194304 4955518 140734441682080 18446744073709551615 4227556 0 0 16781312 0 0 0 0 17 0 0 0 0
/proc/4422/statm: 664 445 104 186 0 477 0
[pid=4423] ppid=4402 vsize=330500 CPUtime=0.12 cores=0-7
/proc/4423/stat : 4423 (python) S 4402 4402 4082 0 -1 4202560 960 0 0 0 7 5 0 0 15 0 1 0 668801585 338432000 5749 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 11614383221050933505 0 0 17 1 0 0 0
/proc/4423/statm: 82625 5749 372 364 0 18528 0
[pid=4425] ppid=4423 vsize=2656 CPUtime=36.64 cores=0-7
/proc/4425/stat : 4425 (march_hi) R 4423 4425 4425 0 -1 4202496 709 0 0 0 3664 0 0 0 25 0 1 0 668801586 2719744 445 33554432000 4194304 4955518 140737140088656 18446744073709551615 4209667 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/4425/statm: 664 445 104 186 0 477 0
[pid=4426] ppid=4402 vsize=340184 CPUtime=0.12 cores=0-7
/proc/4426/stat : 4426 (python) S 4402 4402 4082 0 -1 4202560 977 0 0 0 6 6 0 0 15 0 1 0 668801589 348348416 5641 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 1541436141599686915 0 0 17 4 0 0 0
/proc/4426/statm: 85046 5641 371 364 0 20946 0
[pid=4429] ppid=4426 vsize=2656 CPUtime=36.52 cores=0-7
/proc/4429/stat : 4429 (march_hi) R 4426 4429 4429 0 -1 4202496 710 0 0 0 3652 0 0 0 25 0 1 0 668801589 2719744 443 33554432000 4194304 4955518 140734754312848 18446744073709551615 4299850 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4429/statm: 664 443 104 186 0 477 0
[pid=4430] ppid=4402 vsize=351048 CPUtime=0.12 cores=0-7
/proc/4430/stat : 4430 (python) S 4402 4402 4082 0 -1 4202560 999 0 0 0 6 6 0 0 15 0 1 0 668801592 359473152 5771 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 5607389326041121025 0 0 17 2 0 0 0
/proc/4430/statm: 87762 5771 377 364 0 23659 0
[pid=4432] ppid=4430 vsize=2656 CPUtime=36.43 cores=0-7
/proc/4432/stat : 4432 (march_hi) R 4430 4432 4432 0 -1 4202496 713 0 0 0 3643 0 0 0 25 0 1 0 668801592 2719744 442 33554432000 4194304 4955518 140736106439776 18446744073709551615 4209679 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4432/statm: 664 442 104 186 0 477 0
[pid=4402/tid=4409] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4409/stat : 4409 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 9 0 668801569 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4412] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4412/stat : 4412 (python) S 4400 4402 4082 0 -1 4202560 21 935 0 0 0 0 0 0 25 0 9 0 668801572 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 2 0 0 0
[pid=4402/tid=4415] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4415/stat : 4415 (python) S 4400 4402 4082 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 9 0 668801576 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 6 0 0 0
[pid=4402/tid=4418] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4418/stat : 4418 (python) S 4400 4402 4082 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 9 0 668801579 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4421] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4421/stat : 4421 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 22 0 9 0 668801582 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4424] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4424/stat : 4424 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 9 0 668801586 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4427] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4427/stat : 4427 (python) S 4400 4402 4082 0 -1 4202560 24 935 0 0 0 0 0 0 22 0 9 0 668801589 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4431] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4431/stat : 4431 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 24 0 9 0 668801592 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 294.61
Current children cumulated vsize (KiB) 4369020

[startup+38.3015 s]
/proc/loadavg: 6.12 6.90 7.62 9/395 4433
/proc/meminfo: memFree=12594900/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=361292 CPUtime=0.7 cores=0-7
/proc/4402/stat : 4402 (python) S 4400 4402 4082 0 -1 4202496 15754 935 1 0 59 11 0 0 24 0 9 0 668801513 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606693323 0 0 16781312 2 18446744073709551615 0 0 17 4 0 0 0
/proc/4402/statm: 90323 7287 1889 364 0 26220 0
[pid=4408] ppid=4402 vsize=278620 CPUtime=0.09 cores=0-7
/proc/4408/stat : 4408 (python) S 4402 4402 4082 0 -1 4202560 916 0 0 0 4 5 0 0 18 0 1 0 668801569 285306880 5591 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886671 0 0 16781312 514 8452763919365144836 0 0 17 7 0 0 0
/proc/4408/statm: 69655 5591 377 364 0 5573 0
[pid=4410] ppid=4408 vsize=1017800 CPUtime=37.73 cores=0-7
/proc/4410/stat : 4410 (gnovelty+2) R 4408 4410 4410 0 -1 4202496 431 0 0 0 3773 0 0 0 25 0 1 0 668801570 1042227200 168 33554432000 134512640 135663704 4293055392 18446744073709551615 134545635 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/4410/statm: 254450 168 113 282 0 254166 0
[pid=4411] ppid=4402 vsize=289308 CPUtime=0.43 cores=0-7
/proc/4411/stat : 4411 (python) R 4402 4402 4082 0 -1 4202560 944 0 0 0 34 9 0 0 16 0 1 0 668801572 296251392 5707 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606696767 0 0 16781312 514 0 0 0 17 5 0 0 0
/proc/4411/statm: 72327 5707 371 364 0 8242 0
[pid=4413] ppid=4411 vsize=0 CPUtime=37.34 cores=0-7
/proc/4413/stat : 4413 (march_hi) R 4411 4413 4413 0 -1 4202500 712 0 0 0 3734 0 0 0 25 0 1 0 668801573 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 5 0 0 0
/proc/4413/statm: 0 0 0 0 0 0 0
[pid=4414] ppid=4402 vsize=299192 CPUtime=0.36 cores=0-7
/proc/4414/stat : 4414 (python) R 4402 4402 4082 0 -1 4202560 925 0 0 0 28 8 0 0 16 0 1 0 668801575 306372608 5615 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606696767 0 0 16781312 514 0 0 0 17 6 0 0 0
/proc/4414/statm: 74798 5615 371 364 0 10710 0
[pid=4416] ppid=4414 vsize=0 CPUtime=37.3 cores=0-7
/proc/4416/stat : 4416 (march_hi) R 4414 4416 4416 0 -1 4202500 709 0 0 0 3730 0 0 0 25 0 1 0 668801576 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 6 0 0 0
/proc/4416/statm: 0 0 0 0 0 0 0
[pid=4417] ppid=4402 vsize=309988 CPUtime=0.09 cores=0-7
/proc/4417/stat : 4417 (python) S 4402 4402 4082 0 -1 4202560 919 0 0 0 5 4 0 0 18 0 1 0 668801579 317427712 5740 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 8452763919365144835 0 0 17 0 0 0 0
/proc/4417/statm: 77497 5740 371 364 0 13406 0
[pid=4419] ppid=4417 vsize=454908 CPUtime=37.09 cores=0-7
/proc/4419/stat : 4419 (TNM) R 4417 4419 4419 0 -1 4202496 426 0 0 0 3709 0 0 0 25 0 1 0 668801579 465825792 160 33554432000 134512640 135008748 4286984496 18446744073709551615 134527475 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/4419/statm: 113727 160 46 122 0 112627 0
[pid=4420] ppid=4402 vsize=320244 CPUtime=0.25 cores=0-7
/proc/4420/stat : 4420 (python) R 4402 4402 4082 0 -1 4202560 976 0 0 0 18 7 0 0 15 0 1 0 668801582 327929856 5744 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606696767 0 0 16781312 514 0 0 0 17 0 0 0 0
/proc/4420/statm: 80061 5744 371 364 0 15967 0
[pid=4422] ppid=4420 vsize=0 CPUtime=37.31 cores=0-7
/proc/4422/stat : 4422 (march_hi) R 4420 4422 4422 0 -1 4202500 717 0 0 0 3731 0 0 0 25 0 1 0 668801582 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 0 0 0 0
/proc/4422/statm: 0 0 0 0 0 0 0
[pid=4423] ppid=4402 vsize=330500 CPUtime=0.21 cores=0-7
/proc/4423/stat : 4423 (python) R 4402 4402 4082 0 -1 4202560 961 0 0 0 15 6 0 0 15 0 1 0 668801585 338432000 5749 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606696767 0 0 16781312 514 0 0 0 17 1 0 0 0
/proc/4423/statm: 82625 5749 372 364 0 18528 0
[pid=4425] ppid=4423 vsize=0 CPUtime=37.34 cores=0-7
/proc/4425/stat : 4425 (march_hi) R 4423 4425 4425 0 -1 4202500 714 0 0 0 3734 0 0 0 25 0 1 0 668801586 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/4425/statm: 0 0 0 0 0 0 0
[pid=4426] ppid=4402 vsize=340184 CPUtime=0.12 cores=0-7
/proc/4426/stat : 4426 (python) S 4402 4402 4082 0 -1 4202560 977 0 0 0 6 6 0 0 15 0 1 0 668801589 348348416 5641 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 5607389330336088323 0 0 17 4 0 0 0
/proc/4426/statm: 85046 5641 371 364 0 20946 0
[pid=4429] ppid=4426 vsize=2656 CPUtime=37.31 cores=0-7
/proc/4429/stat : 4429 (march_hi) R 4426 4429 4429 0 -1 4202496 715 0 0 0 3731 0 0 0 25 0 1 0 668801589 2719744 448 33554432000 4194304 4955518 140734754312848 18446744073709551615 4209717 0 0 16781312 0 0 0 0 17 4 0 0 0
/proc/4429/statm: 664 448 104 186 0 477 0
[pid=4430] ppid=4402 vsize=351048 CPUtime=0.12 cores=0-7
/proc/4430/stat : 4430 (python) S 4402 4402 4082 0 -1 4202560 1003 0 0 0 6 6 0 0 15 0 1 0 668801592 359473152 5772 33554432000 4194304 5683644 140736744942512 18446744073709551615 235594886726 0 0 16781312 514 5607389330336088321 0 0 17 2 0 0 0
/proc/4430/statm: 87762 5772 378 364 0 23659 0
[pid=4432] ppid=4430 vsize=2656 CPUtime=37.23 cores=0-7
/proc/4432/stat : 4432 (march_hi) R 4430 4432 4432 0 -1 4202496 718 0 0 0 3723 0 0 0 25 0 1 0 668801592 2719744 447 33554432000 4194304 4955518 140736106439776 18446744073709551615 4227109 0 0 16781312 0 0 0 0 17 2 0 0 0
/proc/4432/statm: 664 447 104 186 0 477 0
[pid=4402/tid=4409] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4409/stat : 4409 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 25 0 9 0 668801569 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4412] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4412/stat : 4412 (python) S 4400 4402 4082 0 -1 4202560 21 935 0 0 0 0 0 0 25 0 9 0 668801572 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 2 0 0 0
[pid=4402/tid=4415] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4415/stat : 4415 (python) S 4400 4402 4082 0 -1 4202560 23 935 0 0 0 0 0 0 25 0 9 0 668801576 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 6 0 0 0
[pid=4402/tid=4418] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4418/stat : 4418 (python) S 4400 4402 4082 0 -1 4202560 22 935 0 0 0 0 0 0 18 0 9 0 668801579 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4421] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4421/stat : 4421 (python) S 4400 4402 4082 0 -1 4202560 26 935 0 0 0 0 0 0 22 0 9 0 668801582 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 0 0 0 0
[pid=4402/tid=4424] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4424/stat : 4424 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 20 0 9 0 668801586 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4427] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4427/stat : 4427 (python) S 4400 4402 4082 0 -1 4202560 24 935 0 0 0 0 0 0 22 0 9 0 668801589 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
[pid=4402/tid=4431] ppid=4400 vsize=361292 CPUtime=0 cores=0-7
/proc/4402/task/4431/stat : 4431 (python) S 4400 4402 4082 0 -1 4202560 27 935 0 0 0 0 0 0 24 0 9 0 668801592 369963008 7287 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 2 18446604440392349632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 301.02
Current children cumulated vsize (KiB) 4358396

[startup+38.7015 s]
/proc/loadavg: 6.12 6.90 7.62 14/395 4433
/proc/meminfo: memFree=12598992/32950928 swapFree=54713420/67111528
[pid=4402] ppid=4400 vsize=320212 CPUtime=304.2 cores=0-7
/proc/4402/stat : 4402 (python) R 4400 4402 4082 0 -1 4202496 19832 15102 1 0 60 11 30267 82 17 0 2 0 668801513 327897088 7257 33554432000 4194304 5683644 140736744942512 18446744073709551615 5293305 0 0 16781312 0 18446744073709551615 0 0 17 0 0 0 0
/proc/4402/statm: 80053 7257 1875 364 0 15976 0
[pid=4402/tid=4421] ppid=4400 vsize=320212 CPUtime=303.49 cores=0-7
/proc/4402/task/4421/stat : 4421 (python) S 4400 4402 4082 0 -1 4202560 27 15102 0 0 0 0 30267 82 18 0 2 0 668801582 327897088 7257 33554432000 4194304 5683644 140736744942512 18446744073709551615 235606690225 0 0 16781312 0 18446604440392349632 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 304.2
Current children cumulated vsize (KiB) 320212

Child status: 0
Real time (s): 38.7067
CPU time (s): 304.226
CPU user time (s): 303.284
CPU system time (s): 0.941856
CPU usage (%): 785.977
Max. virtual memory (cumulated for all children) (KiB): 4369020

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 303.284
system time used= 0.941856
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 35261
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4661
involuntary context switches= 4668

runsolver used 0.154976 second user time and 0.319951 second system time

The end

Launcher Data 

Begin job on node120 at 2011-05-16 20:53:47
IDJOB=3385667
IDBENCH=82932
IDSOLVER=1870
FILE ID=node120/3385667-1305572027
RUNJOBID= node120-1305571909-4096
PBS_JOBID= 13350152
Free space on /tmp= 73500 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k3-r4.26-v400-c1704-S549184435-037.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3385667-1305572027/watcher-3385667-1305572027 -o /tmp/evaluation-result-3385667-1305572027/solver-3385667-1305572027 -C 40000 -W 5100 -M 31000  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3385667-1305572027.cnf -seed 2084976151 -budget 40000 -cores 8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ab99d3f634575de78557b69bc7096528
RANDOM SEED=2084976151

node120.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      12656492 kB
Buffers:        128852 kB
Cached:        1492552 kB
SwapCached:      32576 kB
Active:        6898836 kB
Inactive:     13147532 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      12656492 kB
SwapTotal:    67111528 kB
SwapFree:     54713420 kB
Dirty:           31808 kB
Writeback:           0 kB
AnonPages:    18399372 kB
Mapped:          21400 kB
Slab:            95556 kB
PageTables:      80772 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 78397812 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73468 MiB
End job on node120 at 2011-05-16 20:54:26