Trace number 3378384

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
borg-sat 11.03.02-par-random (fixed)UNSAT 1298.65 326.175

General information on the benchmark

Namerandom/medium/
unif-k5-r21.3-v100-c2130-S1760258375-046.UNKNOWN.cnf
MD5SUMf913d39a64a2142ed34486c38d5636c0
Bench CategoryRANDOM (random instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark319.762
Satisfiable
(Un)Satisfiability was proved
Number of variables100
Number of clauses2130
Sum of the clauses size10650
Maximum clause length5
Minimum clause length5
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 52130
Number of clauses of size over 50

Solver Data 

0.32/0.47	c INFO: loaded portfolio from HOME/etc/borg-mix+class.random.pickle
0.32/0.47	c INFO: solving HOME/instance-3378384-1305465199.cnf
0.32/0.47	c INFO: computing task features
0.32/0.48	c INFO: computed variable-clause graph statistics
0.32/0.48	c INFO: computed clause balance statistics
0.40/0.53	c INFO: computed variable balance statistics
0.40/0.56	c INFO: computed small-clause counts
0.40/0.56	c INFO: computed Horn-clause counts
0.40/0.56	c INFO: computed variable graph statistics
0.40/0.56	c INFO: collected features for HOME/instance-3378384-1305465199.cnf in 0.10 s
0.40/0.57	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.62)
0.40/0.58	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.32)
0.40/0.59	c INFO: running gnovelty+2@0 for 97 with 1199 remaining (b = 0.07)
0.50/0.60	c INFO: running march_hi@0 for 388 with 1199 remaining (b = 0.30)
400.01/100.89	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.37)
400.40/100.90	c INFO: running march_hi@0 for 679 with 1099 remaining (b = 0.33)
1298.30/326.15	s UNSATISFIABLE

Verifier Data 

No possible verification on an UNSAT instance

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3378384-1305465199/watcher-3378384-1305465199 -o /tmp/evaluation-result-3378384-1305465199/solver-3378384-1305465199 -C 4800 -W 1300 -M 15500 python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378384-1305465199.cnf -seed 173778446 -budget 4800 -cores 4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 8.21 7.77 5/444 22797
/proc/meminfo: memFree=18816844/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=16236 CPUtime=0 cores=1,3,5,7
/proc/22797/stat : 22797 (python) R 22795 22797 21694 0 -1 4202496 1044 0 0 0 0 0 0 0 25 0 1 0 658122296 16625664 916 33554432000 4194304 5683644 140734409580320 18446744073709551615 212435710932 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/22797/statm: 4059 916 381 364 0 546 0

[startup+0.0795459 s]
/proc/loadavg: 8.09 8.21 7.77 5/444 22797
/proc/meminfo: memFree=18816844/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=85616 CPUtime=0.06 cores=1,3,5,7
/proc/22797/stat : 22797 (python) R 22795 22797 21694 0 -1 4202496 2627 0 0 0 4 2 0 0 18 0 1 0 658122296 87670784 2347 33554432000 4194304 5683644 140734409580320 18446744073709551615 5212738 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/22797/statm: 21404 2347 713 364 0 1621 0
Current children cumulated CPU time (s) 0.06
Current children cumulated vsize (KiB) 85616

[startup+0.100537 s]
/proc/loadavg: 8.09 8.21 7.77 5/444 22797
/proc/meminfo: memFree=18816844/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=104228 CPUtime=0.07 cores=1,3,5,7
/proc/22797/stat : 22797 (python) R 22795 22797 21694 0 -1 4202496 3066 0 0 0 5 2 0 0 17 0 1 0 658122296 106729472 2748 33554432000 4194304 5683644 140734409580320 18446744073709551615 4620076 0 0 16781312 2 0 0 0 17 1 0 0 0
/proc/22797/statm: 26056 2749 847 364 0 1804 0
Current children cumulated CPU time (s) 0.07
Current children cumulated vsize (KiB) 104228

[startup+0.30103 s]
/proc/loadavg: 8.09 8.21 7.77 5/444 22797
/proc/meminfo: memFree=18816844/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=187688 CPUtime=0.23 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 6824 938 1 0 16 7 0 0 18 0 1 0 658122296 192192512 4882 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436015104 0 2147483391 16781312 2 18446744071699207358 0 0 17 7 0 0 0
/proc/22797/statm: 46922 4882 1399 364 0 3723 0
Current children cumulated CPU time (s) 0.23
Current children cumulated vsize (KiB) 187688

[startup+0.700379 s]
/proc/loadavg: 8.09 8.21 7.77 5/444 22797
/proc/meminfo: memFree=18816844/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=320032 CPUtime=0.5 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 12149 938 1 0 40 10 0 0 24 0 5 0 658122296 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/22797/statm: 80008 7213 1873 364 0 15917 0
Current children cumulated CPU time (s) 0.5
Current children cumulated vsize (KiB) 320032

[startup+1.50123 s]
/proc/loadavg: 8.09 8.21 7.77 9/457 22814
/proc/meminfo: memFree=18778416/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=320032 CPUtime=0.5 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 12149 938 1 0 40 10 0 0 24 0 5 0 658122296 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/22797/statm: 80008 7213 1873 364 0 15917 0
[pid=22803] ppid=22797 vsize=278452 CPUtime=0 cores=1,3,5,7
/proc/22803/stat : 22803 (python) S 22797 22797 21694 0 -1 4202560 732 0 0 0 0 0 0 0 19 0 1 0 658122354 285134848 5528 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037135 0 0 16781312 514 2964946908046426111 0 0 17 1 0 0 0
/proc/22803/statm: 69613 5528 354 364 0 5531 0
[pid=22805] ppid=22803 vsize=1017800 CPUtime=0.92 cores=1,3,5,7
/proc/22805/stat : 22805 (gnovelty+2) R 22803 22805 22805 0 -1 4202496 433 0 0 0 92 0 0 0 23 0 1 0 658122354 1042227200 172 33554432000 134512640 135663704 4289597280 18446744073709551615 134524551 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22805/statm: 254450 172 113 282 0 254166 0
[pid=22806] ppid=22797 vsize=289276 CPUtime=0 cores=1,3,5,7
/proc/22806/stat : 22806 (python) S 22797 22797 21694 0 -1 4202560 748 0 0 0 0 0 0 0 15 0 1 0 658122355 296218624 5676 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 3374471558683656455 0 0 17 1 0 0 0
/proc/22806/statm: 72319 5676 348 364 0 8234 0
[pid=22808] ppid=22806 vsize=1560 CPUtime=0.91 cores=1,3,5,7
/proc/22808/stat : 22808 (march_hi) R 22806 22808 22808 0 -1 4202496 503 0 0 0 91 0 0 0 23 0 1 0 658122355 1597440 240 33554432000 4194304 4955518 140735294948496 18446744073709551615 4263536 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22808/statm: 390 240 106 186 0 203 0
[pid=22809] ppid=22797 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/22809/stat : 22809 (python) S 22797 22797 21694 0 -1 4202560 743 0 0 0 0 0 0 0 21 0 1 0 658122356 306720768 5680 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 15800273191139540991 0 0 17 5 0 0 0
/proc/22809/statm: 74883 5680 348 364 0 10795 0
[pid=22811] ppid=22809 vsize=1017800 CPUtime=0.88 cores=1,3,5,7
/proc/22811/stat : 22811 (gnovelty+2) R 22809 22811 22811 0 -1 4202496 432 0 0 0 88 0 0 0 25 0 1 0 658122356 1042227200 171 33554432000 134512640 135663704 4288016944 18446744073709551615 134544272 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22811/statm: 254450 171 113 282 0 254166 0
[pid=22812] ppid=22797 vsize=309788 CPUtime=0 cores=1,3,5,7
/proc/22812/stat : 22812 (python) S 22797 22797 21694 0 -1 4202560 756 0 0 0 0 0 0 0 17 0 1 0 658122357 317222912 5684 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 10904045932949504263 0 0 17 5 0 0 0
/proc/22812/statm: 77447 5684 348 364 0 13356 0
[pid=22814] ppid=22812 vsize=1560 CPUtime=0.88 cores=1,3,5,7
/proc/22814/stat : 22814 (march_hi) R 22812 22814 22814 0 -1 4202496 501 0 0 0 88 0 0 0 25 0 1 0 658122357 1597440 239 33554432000 4194304 4955518 140736276024512 18446744073709551615 4237750 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22814/statm: 390 239 106 186 0 203 0
[pid=22797/tid=22804] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22804/stat : 22804 (python) S 22795 22797 21694 0 -1 4202560 26 938 0 0 0 0 0 0 20 0 5 0 658122354 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22807] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22807/stat : 22807 (python) S 22795 22797 21694 0 -1 4202560 24 938 0 0 0 0 0 0 20 0 5 0 658122355 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 1 0 0 0
[pid=22797/tid=22810] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22810/stat : 22810 (python) S 22795 22797 21694 0 -1 4202560 35 938 0 0 0 0 0 0 22 0 5 0 658122356 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22813] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22813/stat : 22813 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 24 0 5 0 658122357 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 4.09
Current children cumulated vsize (KiB) 3535800

[startup+3.10092 s]
/proc/loadavg: 8.09 8.21 7.77 9/457 22814
/proc/meminfo: memFree=18777796/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=320032 CPUtime=0.5 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 12149 938 1 0 40 10 0 0 24 0 5 0 658122296 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/22797/statm: 80008 7213 1873 364 0 15917 0
[pid=22803] ppid=22797 vsize=278452 CPUtime=0 cores=1,3,5,7
/proc/22803/stat : 22803 (python) S 22797 22797 21694 0 -1 4202560 732 0 0 0 0 0 0 0 18 0 1 0 658122354 285134848 5528 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037135 0 0 16781312 514 2964946908046393608 0 0 17 1 0 0 0
/proc/22803/statm: 69613 5528 354 364 0 5531 0
[pid=22805] ppid=22803 vsize=1017800 CPUtime=2.52 cores=1,3,5,7
/proc/22805/stat : 22805 (gnovelty+2) R 22803 22805 22805 0 -1 4202496 433 0 0 0 252 0 0 0 25 0 1 0 658122354 1042227200 172 33554432000 134512640 135663704 4289597280 18446744073709551615 134525218 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22805/statm: 254450 172 113 282 0 254166 0
[pid=22806] ppid=22797 vsize=289276 CPUtime=0 cores=1,3,5,7
/proc/22806/stat : 22806 (python) S 22797 22797 21694 0 -1 4202560 755 0 0 0 0 0 0 0 15 0 1 0 658122355 296218624 5676 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 3374471558683656455 0 0 17 1 0 0 0
/proc/22806/statm: 72319 5676 348 364 0 8234 0
[pid=22808] ppid=22806 vsize=1560 CPUtime=2.51 cores=1,3,5,7
/proc/22808/stat : 22808 (march_hi) R 22806 22808 22808 0 -1 4202496 506 0 0 0 251 0 0 0 25 0 1 0 658122355 1597440 243 33554432000 4194304 4955518 140735294948496 18446744073709551615 4224473 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22808/statm: 390 243 106 186 0 203 0
[pid=22809] ppid=22797 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/22809/stat : 22809 (python) S 22797 22797 21694 0 -1 4202560 743 0 0 0 0 0 0 0 18 0 1 0 658122356 306720768 5680 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 15800273191139508485 0 0 17 5 0 0 0
/proc/22809/statm: 74883 5680 348 364 0 10795 0
[pid=22811] ppid=22809 vsize=1017800 CPUtime=2.47 cores=1,3,5,7
/proc/22811/stat : 22811 (gnovelty+2) R 22809 22811 22811 0 -1 4202496 432 0 0 0 247 0 0 0 25 0 1 0 658122356 1042227200 171 33554432000 134512640 135663704 4288016944 18446744073709551615 134545619 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22811/statm: 254450 171 113 282 0 254166 0
[pid=22812] ppid=22797 vsize=309788 CPUtime=0 cores=1,3,5,7
/proc/22812/stat : 22812 (python) S 22797 22797 21694 0 -1 4202560 756 0 0 0 0 0 0 0 15 0 1 0 658122357 317222912 5684 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 10904045932949504263 0 0 17 5 0 0 0
/proc/22812/statm: 77447 5684 348 364 0 13356 0
[pid=22814] ppid=22812 vsize=1560 CPUtime=2.48 cores=1,3,5,7
/proc/22814/stat : 22814 (march_hi) R 22812 22814 22814 0 -1 4202496 504 0 0 0 248 0 0 0 25 0 1 0 658122357 1597440 242 33554432000 4194304 4955518 140736276024512 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22814/statm: 390 242 106 186 0 203 0
[pid=22797/tid=22804] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22804/stat : 22804 (python) S 22795 22797 21694 0 -1 4202560 26 938 0 0 0 0 0 0 20 0 5 0 658122354 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22807] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22807/stat : 22807 (python) S 22795 22797 21694 0 -1 4202560 24 938 0 0 0 0 0 0 20 0 5 0 658122355 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 1 0 0 0
[pid=22797/tid=22810] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22810/stat : 22810 (python) S 22795 22797 21694 0 -1 4202560 35 938 0 0 0 0 0 0 22 0 5 0 658122356 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22813] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22813/stat : 22813 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 24 0 5 0 658122357 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 10.48
Current children cumulated vsize (KiB) 3535800

[startup+6.30132 s]
/proc/loadavg: 8.08 8.20 7.77 9/457 22814
/proc/meminfo: memFree=18775656/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=320032 CPUtime=0.5 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 12149 938 1 0 40 10 0 0 24 0 5 0 658122296 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/22797/statm: 80008 7213 1873 364 0 15917 0
[pid=22803] ppid=22797 vsize=278452 CPUtime=0.01 cores=1,3,5,7
/proc/22803/stat : 22803 (python) S 22797 22797 21694 0 -1 4202560 891 0 0 0 0 1 0 0 18 0 1 0 658122354 285134848 5551 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037135 0 0 16781312 514 2964946908046393608 0 0 17 1 0 0 0
/proc/22803/statm: 69613 5551 377 364 0 5531 0
[pid=22805] ppid=22803 vsize=1017800 CPUtime=5.72 cores=1,3,5,7
/proc/22805/stat : 22805 (gnovelty+2) R 22803 22805 22805 0 -1 4202496 433 0 0 0 572 0 0 0 25 0 1 0 658122354 1042227200 172 33554432000 134512640 135663704 4289597280 18446744073709551615 134524918 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22805/statm: 254450 172 113 282 0 254166 0
[pid=22806] ppid=22797 vsize=289276 CPUtime=0.01 cores=1,3,5,7
/proc/22806/stat : 22806 (python) S 22797 22797 21694 0 -1 4202560 919 0 0 0 1 0 0 0 15 0 1 0 658122355 296218624 5698 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 3374471558683656455 0 0 17 1 0 0 0
/proc/22806/statm: 72319 5698 370 364 0 8234 0
[pid=22808] ppid=22806 vsize=1776 CPUtime=5.67 cores=1,3,5,7
/proc/22808/stat : 22808 (march_hi) R 22806 22808 22808 0 -1 4202496 532 0 0 0 567 0 0 0 25 0 1 0 658122355 1818624 269 33554432000 4194304 4955518 140735294948496 18446744073709551615 4208624 0 0 16781312 0 0 0 0 17 1 0 0 0
/proc/22808/statm: 444 269 106 186 0 257 0
[pid=22809] ppid=22797 vsize=299532 CPUtime=0.01 cores=1,3,5,7
/proc/22809/stat : 22809 (python) S 22797 22797 21694 0 -1 4202560 903 0 0 0 0 1 0 0 18 0 1 0 658122356 306720768 5703 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 15800273191139508485 0 0 17 5 0 0 0
/proc/22809/statm: 74883 5703 371 364 0 10795 0
[pid=22811] ppid=22809 vsize=1017800 CPUtime=5.63 cores=1,3,5,7
/proc/22811/stat : 22811 (gnovelty+2) R 22809 22811 22811 0 -1 4202496 432 0 0 0 563 0 0 0 25 0 1 0 658122356 1042227200 171 33554432000 134512640 135663704 4288016944 18446744073709551615 134524888 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22811/statm: 254450 171 113 282 0 254166 0
[pid=22812] ppid=22797 vsize=309788 CPUtime=0.01 cores=1,3,5,7
/proc/22812/stat : 22812 (python) S 22797 22797 21694 0 -1 4202560 908 0 0 0 1 0 0 0 15 0 1 0 658122357 317222912 5706 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 10904045932949504263 0 0 17 5 0 0 0
/proc/22812/statm: 77447 5706 370 364 0 13356 0
[pid=22814] ppid=22812 vsize=1776 CPUtime=5.65 cores=1,3,5,7
/proc/22814/stat : 22814 (march_hi) R 22812 22814 22814 0 -1 4202496 530 0 0 0 565 0 0 0 25 0 1 0 658122357 1818624 268 33554432000 4194304 4955518 140736276024512 18446744073709551615 4237810 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22814/statm: 444 268 106 186 0 257 0
[pid=22797/tid=22804] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22804/stat : 22804 (python) S 22795 22797 21694 0 -1 4202560 26 938 0 0 0 0 0 0 20 0 5 0 658122354 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22807] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22807/stat : 22807 (python) S 22795 22797 21694 0 -1 4202560 24 938 0 0 0 0 0 0 20 0 5 0 658122355 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 1 0 0 0
[pid=22797/tid=22810] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22810/stat : 22810 (python) S 22795 22797 21694 0 -1 4202560 35 938 0 0 0 0 0 0 22 0 5 0 658122356 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22813] ppid=22795 vsize=320032 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22813/stat : 22813 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 24 0 5 0 658122357 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 23.21
Current children cumulated vsize (KiB) 3536232

[startup+12.7011 s]
/proc/loadavg: 8.07 8.20 7.78 9/457 22815
/proc/meminfo: memFree=18774928/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=320032 CPUtime=0.5 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 12149 938 1 0 40 10 0 0 24 0 5 0 658122296 327712768 7213 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/22797/statm: 80008 7213 1873 364 0 15917 0
[pid=22803] ppid=22797 vsize=278452 CPUtime=0.03 cores=1,3,5,7
/proc/22803/stat : 22803 (python) S 22797 22797 21694 0 -1 4202560 894 0 0 0 1 2 0 0 18 0 1 0 658122354 285134848 5551 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037135 0 0 16781312 514 18316526712608751880 0 0 17 1 0 0 0
/proc/22803/statm: 69613 5551 377 364 0 5531 0
[pid=22805] ppid=22803 vsize=1017800 CPUtime=12.12 cores=1,3,5,7
/proc/22805/stat : 22805 (gnovelty+2) R 22803 22805 22805 0 -1 4202496 433 0 0 0 1212 0 0 0 25 0 1 0 658122354 1042227200 172 33554432000 134512640 135663704 4289597280 18446744073709551615 134525361 0 0 16781312 0 0 0 0 17 3 0 0 0
/proc/22805/statm: 254450 172 113 282 0 254166 0
[pid=22806] ppid=22797 vsize=289276 CPUtime=0.03 cores=1,3,5,7

################
# More data... #
################

[pid=22803/tid=22819] ppid=22797 vsize=288696 CPUtime=0 cores=1,3,5,7
/proc/22803/task/22819/stat : 22819 (python) S 22797 22797 21694 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658132383 295624704 5595 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 514 18446604466487925632 0 0 -1 1 0 0 0
[pid=22806] ppid=22797 vsize=289276 CPUtime=2.66 cores=1,3,5,7
/proc/22806/stat : 22806 (python) R 22797 22797 21694 0 -1 4202560 1019 0 0 0 190 76 0 0 17 0 1 0 658122355 296218624 5699 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 0 0 0 17 3 0 0 0
/proc/22806/statm: 72319 5699 371 364 0 8234 0
[pid=22808] ppid=22806 vsize=0 CPUtime=322.62 cores=1,3,5,7
/proc/22808/stat : 22808 (march_hi) R 22806 22808 22808 0 -1 4202500 3159 0 0 0 32261 1 0 0 25 0 1 0 658122355 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/22808/statm: 0 0 0 0 0 0 0
[pid=22809] ppid=22797 vsize=299532 CPUtime=0.3 cores=1,3,5,7
/proc/22809/stat : 22809 (python) S 22797 22797 21694 0 -1 4202560 1098 0 0 0 15 15 0 0 19 0 2 0 658122356 306720768 5733 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22809/statm: 74883 5733 400 364 0 10795 0
[pid=22811] ppid=22809 vsize=1017800 CPUtime=99.02 cores=1,3,5,7
/proc/22811/stat : 22811 (gnovelty+2) T 22809 22811 22811 0 -1 4202496 432 0 0 0 9902 0 0 0 25 0 1 0 658122356 1042227200 171 33554432000 134512640 135663704 4288016944 18446744073709551615 134525247 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/22811/statm: 254450 171 113 282 0 254166 0
[pid=22809/tid=22822] ppid=22797 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/22809/task/22822/stat : 22822 (python) S 22797 22797 21694 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658132386 306720768 5733 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 514 18446604456591481024 0 0 -1 5 0 0 0
[pid=22812] ppid=22797 vsize=309788 CPUtime=1.72 cores=1,3,5,7
/proc/22812/stat : 22812 (python) R 22797 22797 21694 0 -1 4202560 1014 0 0 0 112 60 0 0 16 0 1 0 658122357 317222912 5708 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447847231 0 0 16781312 514 0 0 0 17 1 0 0 0
/proc/22812/statm: 77447 5708 372 364 0 13356 0
[pid=22814] ppid=22812 vsize=0 CPUtime=322.66 cores=1,3,5,7
/proc/22814/stat : 22814 (march_hi) R 22812 22814 22814 0 -1 4202500 3157 0 0 0 32265 1 0 0 25 0 1 0 658122357 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/22814/statm: 0 0 0 0 0 0 0
[pid=22820] ppid=22797 vsize=320044 CPUtime=0.77 cores=1,3,5,7
/proc/22820/stat : 22820 (python) S 22797 22797 21694 0 -1 4202560 994 0 0 0 42 35 0 0 15 0 1 0 658132384 327725056 5713 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 1917060099973546247 0 0 17 7 0 0 0
/proc/22820/statm: 80011 5713 371 364 0 15917 0
[pid=22823] ppid=22820 vsize=9452 CPUtime=220.95 cores=1,3,5,7
/proc/22823/stat : 22823 (march_hi) R 22820 22823 22823 0 -1 4202496 1865 0 0 0 22095 0 0 0 25 0 1 0 658132387 9678848 1588 33554432000 4194304 4955518 140737382730544 18446744073709551615 4224471 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22823/statm: 2363 1588 106 186 0 2176 0
[pid=22824] ppid=22797 vsize=330072 CPUtime=0.76 cores=1,3,5,7
/proc/22824/stat : 22824 (python) S 22797 22797 21694 0 -1 4202560 991 0 0 0 41 35 0 0 15 0 1 0 658132387 337993728 5642 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 3374475956730167558 0 0 17 5 0 0 0
/proc/22824/statm: 82518 5642 371 364 0 18421 0
[pid=22826] ppid=22824 vsize=9452 CPUtime=223.39 cores=1,3,5,7
/proc/22826/stat : 22826 (march_hi) R 22824 22826 22826 0 -1 4202496 1872 0 0 0 22339 0 0 0 25 0 1 0 658132387 9678848 1594 33554432000 4194304 4955518 140737016217408 18446744073709551615 4297477 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22826/statm: 2363 1594 106 186 0 2176 0
[pid=22797/tid=22804] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22804/stat : 22804 (python) S 22795 22797 21694 0 -1 4202560 26 938 0 0 0 0 0 0 20 0 7 0 658122354 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22807] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22807/stat : 22807 (python) S 22795 22797 21694 0 -1 4202560 24 938 0 0 0 0 0 0 20 0 7 0 658122355 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 1 0 0 0
[pid=22797/tid=22810] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22810/stat : 22810 (python) S 22795 22797 21694 0 -1 4202560 35 938 0 0 0 0 0 0 22 0 7 0 658122356 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22813] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22813/stat : 22813 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 24 0 7 0 658122357 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22821] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22821/stat : 22821 (python) S 22795 22797 21694 0 -1 4202560 22 938 0 0 0 0 0 0 20 0 7 0 658132384 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22825] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22825/stat : 22825 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 21 0 7 0 658132387 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1295.95
Current children cumulated vsize (KiB) 4232228

[startup+325.901 s]
/proc/loadavg: 8.30 8.24 7.91 11/472 22838
/proc/meminfo: memFree=18741788/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=340316 CPUtime=0.51 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 13761 938 1 0 41 10 0 0 21 0 7 0 658122296 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/22797/statm: 85079 7153 1878 364 0 20982 0
[pid=22803] ppid=22797 vsize=288696 CPUtime=0.3 cores=1,3,5,7
/proc/22803/stat : 22803 (python) S 22797 22797 21694 0 -1 4202560 1166 0 0 0 14 16 0 0 19 0 2 0 658122354 295624704 5595 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22803/statm: 72174 5595 416 364 0 8092 0
[pid=22805] ppid=22803 vsize=1017800 CPUtime=100.29 cores=1,3,5,7
/proc/22805/stat : 22805 (gnovelty+2) T 22803 22805 22805 0 -1 4202496 433 0 0 0 10029 0 0 0 25 0 1 0 658122354 1042227200 172 33554432000 134512640 135663704 4289597280 18446744073709551615 134545772 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22805/statm: 254450 172 113 282 0 254166 0
[pid=22803/tid=22819] ppid=22797 vsize=288696 CPUtime=0 cores=1,3,5,7
/proc/22803/task/22819/stat : 22819 (python) S 22797 22797 21694 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658132383 295624704 5595 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 514 18446604466487925632 0 0 -1 1 0 0 0
[pid=22806] ppid=22797 vsize=289276 CPUtime=3.05 cores=1,3,5,7
/proc/22806/stat : 22806 (python) R 22797 22797 21694 0 -1 4202560 1019 0 0 0 223 82 0 0 18 0 1 0 658122355 296218624 5699 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447847231 0 0 16781312 514 0 0 0 17 3 0 0 0
/proc/22806/statm: 72319 5699 371 364 0 8234 0
[pid=22808] ppid=22806 vsize=0 CPUtime=322.62 cores=1,3,5,7
/proc/22808/stat : 22808 (march_hi) R 22806 22808 22808 0 -1 4202500 3159 0 0 0 32261 1 0 0 25 0 1 0 658122355 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/22808/statm: 0 0 0 0 0 0 0
[pid=22809] ppid=22797 vsize=299532 CPUtime=0.3 cores=1,3,5,7
/proc/22809/stat : 22809 (python) S 22797 22797 21694 0 -1 4202560 1098 0 0 0 15 15 0 0 19 0 2 0 658122356 306720768 5733 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22809/statm: 74883 5733 400 364 0 10795 0
[pid=22811] ppid=22809 vsize=1017800 CPUtime=99.02 cores=1,3,5,7
/proc/22811/stat : 22811 (gnovelty+2) T 22809 22811 22811 0 -1 4202496 432 0 0 0 9902 0 0 0 25 0 1 0 658122356 1042227200 171 33554432000 134512640 135663704 4288016944 18446744073709551615 134525247 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/22811/statm: 254450 171 113 282 0 254166 0
[pid=22809/tid=22822] ppid=22797 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/22809/task/22822/stat : 22822 (python) S 22797 22797 21694 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658132386 306720768 5733 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 514 18446604456591481024 0 0 -1 5 0 0 0
[pid=22812] ppid=22797 vsize=309788 CPUtime=2.11 cores=1,3,5,7
/proc/22812/stat : 22812 (python) R 22797 22797 21694 0 -1 4202560 1014 0 0 0 146 65 0 0 17 0 1 0 658122357 317222912 5708 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447847231 0 0 16781312 514 0 0 0 17 1 0 0 0
/proc/22812/statm: 77447 5708 372 364 0 13356 0
[pid=22814] ppid=22812 vsize=0 CPUtime=322.66 cores=1,3,5,7
/proc/22814/stat : 22814 (march_hi) R 22812 22814 22814 0 -1 4202500 3157 0 0 0 32265 1 0 0 25 0 1 0 658122357 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/22814/statm: 0 0 0 0 0 0 0
[pid=22820] ppid=22797 vsize=320044 CPUtime=0.77 cores=1,3,5,7
/proc/22820/stat : 22820 (python) S 22797 22797 21694 0 -1 4202560 994 0 0 0 42 35 0 0 15 0 1 0 658132384 327725056 5713 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 1917060099973546247 0 0 17 7 0 0 0
/proc/22820/statm: 80011 5713 371 364 0 15917 0
[pid=22823] ppid=22820 vsize=9452 CPUtime=221.34 cores=1,3,5,7
/proc/22823/stat : 22823 (march_hi) R 22820 22823 22823 0 -1 4202496 1869 0 0 0 22134 0 0 0 25 0 1 0 658132387 9678848 1592 33554432000 4194304 4955518 140737382730544 18446744073709551615 4208624 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22823/statm: 2363 1592 106 186 0 2176 0
[pid=22824] ppid=22797 vsize=330072 CPUtime=0.76 cores=1,3,5,7
/proc/22824/stat : 22824 (python) S 22797 22797 21694 0 -1 4202560 991 0 0 0 41 35 0 0 15 0 1 0 658132387 337993728 5642 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 3374475956730167558 0 0 17 5 0 0 0
/proc/22824/statm: 82518 5642 371 364 0 18421 0
[pid=22826] ppid=22824 vsize=9452 CPUtime=223.78 cores=1,3,5,7
/proc/22826/stat : 22826 (march_hi) R 22824 22826 22826 0 -1 4202496 1874 0 0 0 22378 0 0 0 25 0 1 0 658132387 9678848 1596 33554432000 4194304 4955518 140737016217408 18446744073709551615 4208656 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22826/statm: 2363 1596 106 186 0 2176 0
[pid=22797/tid=22804] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22804/stat : 22804 (python) S 22795 22797 21694 0 -1 4202560 26 938 0 0 0 0 0 0 20 0 7 0 658122354 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22807] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22807/stat : 22807 (python) S 22795 22797 21694 0 -1 4202560 24 938 0 0 0 0 0 0 20 0 7 0 658122355 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 1 0 0 0
[pid=22797/tid=22810] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22810/stat : 22810 (python) S 22795 22797 21694 0 -1 4202560 35 938 0 0 0 0 0 0 22 0 7 0 658122356 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22813] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22813/stat : 22813 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 24 0 7 0 658122357 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22821] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22821/stat : 22821 (python) S 22795 22797 21694 0 -1 4202560 22 938 0 0 0 0 0 0 20 0 7 0 658132384 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22825] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22825/stat : 22825 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 21 0 7 0 658132387 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1297.51
Current children cumulated vsize (KiB) 4232228

[startup+326.101 s]
/proc/loadavg: 8.30 8.24 7.91 11/472 22838
/proc/meminfo: memFree=18741788/32951124 swapFree=57331800/67111528
[pid=22797] ppid=22795 vsize=340316 CPUtime=0.51 cores=1,3,5,7
/proc/22797/stat : 22797 (python) S 22795 22797 21694 0 -1 4202496 13761 938 1 0 41 10 0 0 21 0 7 0 658122296 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 2 18446744073709551615 0 0 17 5 0 0 0
/proc/22797/statm: 85079 7153 1878 364 0 20982 0
[pid=22803] ppid=22797 vsize=288696 CPUtime=0.3 cores=1,3,5,7
/proc/22803/stat : 22803 (python) S 22797 22797 21694 0 -1 4202560 1166 0 0 0 14 16 0 0 19 0 2 0 658122354 295624704 5595 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 1 0 0 0
/proc/22803/statm: 72174 5595 416 364 0 8092 0
[pid=22805] ppid=22803 vsize=1017800 CPUtime=100.29 cores=1,3,5,7
/proc/22805/stat : 22805 (gnovelty+2) T 22803 22805 22805 0 -1 4202496 433 0 0 0 10029 0 0 0 25 0 1 0 658122354 1042227200 172 33554432000 134512640 135663704 4289597280 18446744073709551615 134545772 0 0 16781312 0 18446744071562691360 0 0 17 3 0 0 0
/proc/22805/statm: 254450 172 113 282 0 254166 0
[pid=22803/tid=22819] ppid=22797 vsize=288696 CPUtime=0 cores=1,3,5,7
/proc/22803/task/22819/stat : 22819 (python) S 22797 22797 21694 0 -1 4202560 46 0 0 0 0 0 0 0 19 0 2 0 658132383 295624704 5595 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 514 18446604466487925632 0 0 -1 1 0 0 0
[pid=22806] ppid=22797 vsize=289276 CPUtime=3.24 cores=1,3,5,7
/proc/22806/stat : 22806 (python) R 22797 22797 21694 0 -1 4202560 1019 0 0 0 240 84 0 0 18 0 1 0 658122355 296218624 5699 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447847231 0 0 16781312 514 0 0 0 17 3 0 0 0
/proc/22806/statm: 72319 5699 371 364 0 8234 0
[pid=22808] ppid=22806 vsize=0 CPUtime=322.62 cores=1,3,5,7
/proc/22808/stat : 22808 (march_hi) R 22806 22808 22808 0 -1 4202500 3159 0 0 0 32261 1 0 0 25 0 1 0 658122355 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 3 0 0 0
/proc/22808/statm: 0 0 0 0 0 0 0
[pid=22809] ppid=22797 vsize=299532 CPUtime=0.3 cores=1,3,5,7
/proc/22809/stat : 22809 (python) S 22797 22797 21694 0 -1 4202560 1098 0 0 0 15 15 0 0 19 0 2 0 658122356 306720768 5733 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447843787 0 0 16781312 514 18446744073709551615 0 0 17 5 0 0 0
/proc/22809/statm: 74883 5733 400 364 0 10795 0
[pid=22811] ppid=22809 vsize=1017800 CPUtime=99.02 cores=1,3,5,7
/proc/22811/stat : 22811 (gnovelty+2) T 22809 22811 22811 0 -1 4202496 432 0 0 0 9902 0 0 0 25 0 1 0 658122356 1042227200 171 33554432000 134512640 135663704 4288016944 18446744073709551615 134525247 0 0 16781312 0 18446744071562691360 0 0 17 7 0 0 0
/proc/22811/statm: 254450 171 113 282 0 254166 0
[pid=22809/tid=22822] ppid=22797 vsize=299532 CPUtime=0 cores=1,3,5,7
/proc/22809/task/22822/stat : 22822 (python) S 22797 22797 21694 0 -1 4202560 47 0 0 0 0 0 0 0 19 0 2 0 658132386 306720768 5733 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 514 18446604456591481024 0 0 -1 5 0 0 0
[pid=22812] ppid=22797 vsize=309788 CPUtime=2.31 cores=1,3,5,7
/proc/22812/stat : 22812 (python) R 22797 22797 21694 0 -1 4202560 1014 0 0 0 163 68 0 0 17 0 1 0 658122357 317222912 5708 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447847231 0 0 16781312 514 0 0 0 17 1 0 0 0
/proc/22812/statm: 77447 5708 372 364 0 13356 0
[pid=22814] ppid=22812 vsize=0 CPUtime=322.66 cores=1,3,5,7
/proc/22814/stat : 22814 (march_hi) R 22812 22814 22814 0 -1 4202500 3157 0 0 0 32265 1 0 0 25 0 1 0 658122357 0 0 33554432000 0 0 0 0 0 0 0 16781312 0 18446744073709551615 0 0 17 1 0 0 0
/proc/22814/statm: 0 0 0 0 0 0 0
[pid=22820] ppid=22797 vsize=320044 CPUtime=0.77 cores=1,3,5,7
/proc/22820/stat : 22820 (python) S 22797 22797 21694 0 -1 4202560 994 0 0 0 42 35 0 0 15 0 1 0 658132384 327725056 5713 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 1917060099973546247 0 0 17 7 0 0 0
/proc/22820/statm: 80011 5713 371 364 0 15917 0
[pid=22823] ppid=22820 vsize=9452 CPUtime=221.54 cores=1,3,5,7
/proc/22823/stat : 22823 (march_hi) R 22820 22823 22823 0 -1 4202496 1871 0 0 0 22154 0 0 0 25 0 1 0 658132387 9678848 1594 33554432000 4194304 4955518 140737382730544 18446744073709551615 4297472 0 0 16781312 0 0 0 0 17 7 0 0 0
/proc/22823/statm: 2363 1594 106 186 0 2176 0
[pid=22824] ppid=22797 vsize=330072 CPUtime=0.78 cores=1,3,5,7
/proc/22824/stat : 22824 (python) S 22797 22797 21694 0 -1 4202560 992 0 0 0 42 36 0 0 16 0 1 0 658132387 337993728 5642 33554432000 4194304 5683644 140734409580320 18446744073709551615 212436037190 0 0 16781312 514 18316526712608751878 0 0 17 5 0 0 0
/proc/22824/statm: 82518 5642 371 364 0 18421 0
[pid=22826] ppid=22824 vsize=9452 CPUtime=223.96 cores=1,3,5,7
/proc/22826/stat : 22826 (march_hi) R 22824 22826 22826 0 -1 4202496 1876 0 0 0 22396 0 0 0 25 0 1 0 658132387 9678848 1598 33554432000 4194304 4955518 140737016217408 18446744073709551615 4208598 0 0 16781312 0 0 0 0 17 5 0 0 0
/proc/22826/statm: 2363 1598 106 186 0 2176 0
[pid=22797/tid=22804] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22804/stat : 22804 (python) S 22795 22797 21694 0 -1 4202560 26 938 0 0 0 0 0 0 20 0 7 0 658122354 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22807] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22807/stat : 22807 (python) S 22795 22797 21694 0 -1 4202560 24 938 0 0 0 0 0 0 20 0 7 0 658122355 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 1 0 0 0
[pid=22797/tid=22810] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22810/stat : 22810 (python) S 22795 22797 21694 0 -1 4202560 35 938 0 0 0 0 0 0 22 0 7 0 658122356 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22813] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22813/stat : 22813 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 24 0 7 0 658122357 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
[pid=22797/tid=22821] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22821/stat : 22821 (python) S 22795 22797 21694 0 -1 4202560 22 938 0 0 0 0 0 0 20 0 7 0 658132384 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 7 0 0 0
[pid=22797/tid=22825] ppid=22795 vsize=340316 CPUtime=0 cores=1,3,5,7
/proc/22797/task/22825/stat : 22825 (python) S 22795 22797 21694 0 -1 4202560 25 938 0 0 0 0 0 0 21 0 7 0 658132387 348483584 7153 33554432000 4194304 5683644 140734409580320 18446744073709551615 212447840689 0 0 16781312 2 18446604471010591040 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 1298.3
Current children cumulated vsize (KiB) 4232228

Child status: 0
Real time (s): 326.175
CPU time (s): 1298.65
CPU user time (s): 1295.91
CPU system time (s): 2.73858
CPU usage (%): 398.145
Max. virtual memory (cumulated for all children) (KiB): 4256668

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1295.91
system time used= 2.73858
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 36876
page faults= 1
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4707
involuntary context switches= 11824

runsolver used 1.26781 second user time and 2.91556 second system time

The end

Launcher Data 

Begin job on node102 at 2011-05-15 15:13:19
IDJOB=3378384
IDBENCH=82793
IDSOLVER=1859
FILE ID=node102/3378384-1305465199
RUNJOBID= node102-1305462764-21713
PBS_JOBID= 13324973
Free space on /tmp= 73580 MiB

SOLVER NAME= borg-sat 11.03.02-par-random (fixed)
BENCH NAME= SAT11/random/medium/unif-k5-r21.3-v100-c2130-S1760258375-046.UNKNOWN.cnf
COMMAND LINE= python DIR/solve DIR/etc/borg-mix+class.random.pickle BENCHNAME -seed RANDOMSEED -budget TIMEOUT -cores NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3378384-1305465199/watcher-3378384-1305465199 -o /tmp/evaluation-result-3378384-1305465199/solver-3378384-1305465199 -C 4800 -W 1300 -M 15500  python HOME/solve HOME/etc/borg-mix+class.random.pickle HOME/instance-3378384-1305465199.cnf -seed 173778446 -budget 4800 -cores 4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f913d39a64a2142ed34486c38d5636c0
RANDOM SEED=173778446

node102.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.828
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18817248 kB
Buffers:         41780 kB
Cached:         767868 kB
SwapCached:      89980 kB
Active:        4632048 kB
Inactive:      9288400 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18817248 kB
SwapTotal:    67111528 kB
SwapFree:     57331800 kB
Dirty:           32180 kB
Writeback:           0 kB
AnonPages:    13031560 kB
Mapped:          26376 kB
Slab:            66128 kB
PageTables:      80040 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 101155024 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73552 MiB
End job on node102 at 2011-05-15 15:18:46