Trace number 3263764

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-03-24? (TO) 4800.29 1201.76

General information on the benchmark

NameSAT03/handmade/stanion/hwb/
hwb-n28-02-S818962541.shuffled-as.sat03-1628.cnf
MD5SUMd705043763df769aab1e385bbbc2b11e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables184
Number of clauses896
Sum of the clauses size3042
Maximum clause length4
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 214
Number of clauses of size 3514
Number of clauses of size 4368
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3263764-1304101836/watcher-3263764-1304101836 -o /tmp/evaluation-result-3263764-1304101836/solver-3263764-1304101836 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3263764-1304101836.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.18 8.11 8.02 3/173 3663
/proc/meminfo: memFree=27649976/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=49360 CPUtime=0 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 486 0 0 0 0 0 0 0 25 0 4 0 521781303 50544640 422 33554432000 4194304 4623224 140733986632272 18446744073709551615 47293341870000 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 12340 422 301 105 0 7844 0
[pid=3663/tid=3665] ppid=3661 vsize=49360 CPUtime=0 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 521781303 50544640 422 33554432000 4194304 4623224 140733986632272 18446744073709551615 218905787377 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=49360 CPUtime=0 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 521781303 50544640 422 33554432000 4194304 4623224 140733986632272 18446744073709551615 218905787377 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=49360 CPUtime=0 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 1 0 0 0 0 0 0 0 25 0 4 0 521781303 50544640 422 33554432000 4194304 4623224 140733986632272 18446744073709551615 218900762856 0 0 4096 2 0 0 0 -1 0 0 0 0

[startup+0.0261291 s]
/proc/loadavg: 8.18 8.11 8.02 3/173 3663
/proc/meminfo: memFree=27649976/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=246712 CPUtime=0.08 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 933 0 0 0 8 0 0 0 25 0 4 0 521781303 252633088 862 33554432000 4194304 4623224 140733986632272 18446744073709551615 4360048 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 61678 862 343 105 0 57182 0
[pid=3663/tid=3665] ppid=3661 vsize=246712 CPUtime=0.01 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 95 0 0 0 1 0 0 0 25 0 4 0 521781303 252633088 862 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=246712 CPUtime=0.01 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 84 0 0 0 1 0 0 0 25 0 4 0 521781303 252633088 862 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=246712 CPUtime=0.01 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 139 0 0 0 1 0 0 0 25 0 4 0 521781303 252633088 862 33554432000 4194304 4623224 140733986632272 18446744073709551615 4276086 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 246712

[startup+0.101116 s]
/proc/loadavg: 8.18 8.11 8.02 3/173 3663
/proc/meminfo: memFree=27649976/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=247436 CPUtime=0.38 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 1603 0 0 0 38 0 0 0 25 0 4 0 521781303 253374464 1203 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510199 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 61859 1203 343 105 0 57363 0
[pid=3663/tid=3665] ppid=3661 vsize=247436 CPUtime=0.09 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 263 0 0 0 9 0 0 0 25 0 4 0 521781303 253374464 1203 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=247436 CPUtime=0.09 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 237 0 0 0 9 0 0 0 25 0 4 0 521781303 253374464 1203 33554432000 4194304 4623224 140733986632272 18446744073709551615 4377495 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=247436 CPUtime=0.09 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 401 0 0 0 9 0 0 0 25 0 4 0 521781303 253374464 1203 33554432000 4194304 4623224 140733986632272 18446744073709551615 4366364 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.38
Current children cumulated vsize (KiB) 247436

[startup+0.30108 s]
/proc/loadavg: 8.18 8.11 8.02 3/173 3663
/proc/meminfo: memFree=27649976/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=249244 CPUtime=1.17 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 3149 0 0 0 117 0 0 0 25 0 4 0 521781303 255225856 2102 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510472 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 62311 2102 344 105 0 57815 0
[pid=3663/tid=3665] ppid=3661 vsize=249244 CPUtime=0.29 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 724 0 0 0 29 0 0 0 25 0 4 0 521781303 255225856 2102 33554432000 4194304 4623224 140733986632272 18446744073709551615 4464402 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=249244 CPUtime=0.29 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 685 0 0 0 29 0 0 0 25 0 4 0 521781303 255225856 2102 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=249244 CPUtime=0.29 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 872 0 0 0 29 0 0 0 25 0 4 0 521781303 255225856 2102 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376858 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1.17
Current children cumulated vsize (KiB) 249244

[startup+0.701009 s]
/proc/loadavg: 8.18 8.11 8.02 3/173 3663
/proc/meminfo: memFree=27649976/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=251048 CPUtime=2.77 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 6440 0 0 0 277 0 0 0 25 0 4 0 521781303 257073152 3576 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510086 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 62762 3576 344 105 0 58266 0
[pid=3663/tid=3665] ppid=3661 vsize=251048 CPUtime=0.69 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 1578 0 0 0 69 0 0 0 25 0 4 0 521781303 257073152 3576 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=251048 CPUtime=0.69 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 1470 0 0 0 69 0 0 0 25 0 4 0 521781303 257073152 3577 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376646 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=251048 CPUtime=0.69 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 2149 0 0 0 69 0 0 0 25 0 4 0 521781303 257073152 3577 33554432000 4194304 4623224 140733986632272 18446744073709551615 218905392161 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.77
Current children cumulated vsize (KiB) 251048

[startup+1.50086 s]
/proc/loadavg: 8.16 8.11 8.02 9/184 3675
/proc/meminfo: memFree=27599940/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=254908 CPUtime=5.97 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 12526 0 0 0 596 1 0 0 25 0 4 0 521781303 261025792 5538 33554432000 4194304 4623224 140733986632272 18446744073709551615 4384231 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 63727 5538 347 105 0 59231 0
[pid=3663/tid=3665] ppid=3661 vsize=254908 CPUtime=1.48 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 2405 0 0 0 148 0 0 0 25 0 4 0 521781303 261025792 5538 33554432000 4194304 4623224 140733986632272 18446744073709551615 4366381 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=254908 CPUtime=1.49 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 2177 0 0 0 149 0 0 0 25 0 4 0 521781303 261025792 5538 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=254908 CPUtime=1.48 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 6199 0 0 0 148 0 0 0 25 0 4 0 521781303 261025792 5538 33554432000 4194304 4623224 140733986632272 18446744073709551615 4544799 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.97
Current children cumulated vsize (KiB) 254908

[startup+3.10058 s]
/proc/loadavg: 8.16 8.11 8.02 9/184 3675
/proc/meminfo: memFree=27572536/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=260492 CPUtime=12.36 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 17304 0 0 0 1234 2 0 0 25 0 4 0 521781303 266743808 8401 33554432000 4194304 4623224 140733986632272 18446744073709551615 4416149 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 65123 8401 355 105 0 60627 0
[pid=3663/tid=3665] ppid=3661 vsize=260492 CPUtime=3.07 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3710 0 0 0 307 0 0 0 25 0 4 0 521781303 266743808 8401 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=260492 CPUtime=3.08 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3700 0 0 0 308 0 0 0 25 0 4 0 521781303 266743808 8401 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=260492 CPUtime=3.09 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 7492 0 0 0 308 1 0 0 25 0 4 0 521781303 266743808 8401 33554432000 4194304 4623224 140733986632272 18446744073709551615 4545983 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.36
Current children cumulated vsize (KiB) 260492

[startup+6.301 s]
/proc/loadavg: 8.16 8.11 8.02 9/184 3675
/proc/meminfo: memFree=27514408/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=271776 CPUtime=25.15 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 32906 0 0 0 2509 6 0 0 25 0 4 0 521781303 278298624 13024 33554432000 4194304 4623224 140733986632272 18446744073709551615 4517494 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 67944 13024 385 105 0 63448 0
[pid=3663/tid=3665] ppid=3661 vsize=271776 CPUtime=6.26 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 5958 0 0 0 625 1 0 0 25 0 4 0 521781303 278298624 13024 33554432000 4194304 4623224 140733986632272 18446744073709551615 4330719 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=271776 CPUtime=6.29 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 5834 0 0 0 628 1 0 0 25 0 4 0 521781303 278298624 13024 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376742 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=271776 CPUtime=6.29 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 17364 0 0 0 626 3 0 0 25 0 4 0 521781303 278298624 13024 33554432000 4194304 4623224 140733986632272 18446744073709551615 4377613 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.15
Current children cumulated vsize (KiB) 271776

[startup+12.7008 s]
/proc/loadavg: 8.14 8.11 8.02 9/184 3675
/proc/meminfo: memFree=27440892/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=277176 CPUtime=50.71 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 58749 0 0 0 5062 9 0 0 25 0 4 0 521781303 283828224 18061 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510978 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 69294 18061 393 105 0 64798 0
[pid=3663/tid=3665] ppid=3661 vsize=277176 CPUtime=12.62 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 9486 0 0 0 1260 2 0 0 25 0 4 0 521781303 283828224 18061 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376708 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=277176 CPUtime=12.69 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 8120 0 0 0 1268 1 0 0 25 0 4 0 521781303 283828224 18061 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=277176 CPUtime=12.69 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 36339 0 0 0 1264 5 0 0 25 0 4 0 521781303 283828224 18061 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.71
Current children cumulated vsize (KiB) 277176

[startup+25.5005 s]
/proc/loadavg: 8.11 8.10 8.02 9/184 3676
/proc/meminfo: memFree=27329652/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=299044 CPUtime=101.83 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 109688 0 0 0 10166 17 0 0 25 0 4 0 521781303 306221056 27956 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510017 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 74761 27956 469 105 0 70265 0
[pid=3663/tid=3665] ppid=3661 vsize=299044 CPUtime=25.35 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 13451 0 0 0 2532 3 0 0 25 0 4 0 521781303 306221056 27956 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376666 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=299044 CPUtime=25.49 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 16774 0 0 0 2546 3 0 0 25 0 4 0 521781303 306221056 27956 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376892 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=299044 CPUtime=25.49 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 72406 0 0 0 2539 10 0 0 25 0 4 0 521781303 306221056 27956 33554432000 4194304 4623224 140733986632272 18446744073709551615 4577456 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.83
Current children cumulated vsize (KiB) 299044

[startup+51.1009 s]
/proc/loadavg: 8.22 8.13 8.03 9/184 3676
/proc/meminfo: memFree=27108172/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=321064 CPUtime=204.1 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 194058 0 0 0 20377 33 0 0 25 0 4 0 521781303 328769536 42611 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 80266 42611 637 105 0 75770 0
[pid=3663/tid=3665] ppid=3661 vsize=321064 CPUtime=50.81 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 19018 0 0 0 5077 4 0 0 25 0 4 0 521781303 328769536 42611 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=321064 CPUtime=51.09 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 25133 0 0 0 5104 5 0 0 25 0 4 0 521781303 328769536 42611 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=321064 CPUtime=51.08 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 137675 0 0 0 5088 20 0 0 25 0 4 0 521781303 328769536 42611 33554432000 4194304 4623224 140733986632272 18446744073709551615 4545317 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 204.1
Current children cumulated vsize (KiB) 321064

[startup+102.307 s]
/proc/loadavg: 8.23 8.14 8.03 9/184 3678
/proc/meminfo: memFree=26659012/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=384784 CPUtime=408.63 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 426267 0 0 0 40792 71 0 0 25 0 4 0 521781303 394018816 62378 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 96196 62378 640 105 0 91700 0
[pid=3663/tid=3665] ppid=3661 vsize=384784 CPUtime=101.73 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 27526 0 0 0 10167 6 0 0 25 0 4 0 521781303 394018816 62378 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=384784 CPUtime=102.29 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 40154 0 0 0 10221 8 0 0 25 0 4 0 521781303 394018816 62378 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=384784 CPUtime=102.29 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 342565 0 0 0 10177 52 0 0 25 0 4 0 521781303 394018816 62378 33554432000 4194304 4623224 140733986632272 18446744073709551615 4577512 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 408.63
Current children cumulated vsize (KiB) 384784

[startup+162.301 s]

################
# More data... #
################

[pid=3663] ppid=3661 vsize=948520 CPUtime=2805.33 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 2598565 0 0 0 280086 447 0 0 25 0 4 0 521781303 971284480 178328 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 237130 178328 2185 105 0 232634 0
[pid=3663/tid=3665] ppid=3661 vsize=948520 CPUtime=698.47 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 79325 0 0 0 69811 36 0 0 25 0 4 0 521781303 971284480 178328 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376662 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=948520 CPUtime=702.3 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 93527 0 0 0 70210 20 0 0 25 0 4 0 521781303 971284480 178328 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=948520 CPUtime=702.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 2336841 0 0 0 69855 373 0 0 25 0 4 0 521781303 971284480 178328 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2805.33
Current children cumulated vsize (KiB) 948520

[startup+762.307 s]
/proc/loadavg: 8.09 8.07 8.02 9/184 3695
/proc/meminfo: memFree=24750768/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=985168 CPUtime=3045.03 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 2912692 0 0 0 304002 501 0 0 25 0 4 0 521781303 1008812032 181529 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 246292 181529 2139 105 0 241796 0
[pid=3663/tid=3665] ppid=3661 vsize=985168 CPUtime=758.16 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 99950 0 0 0 75776 40 0 0 25 0 4 0 521781303 1008812032 181529 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=985168 CPUtime=762.31 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 93836 0 0 0 76210 21 0 0 25 0 4 0 521781303 1008812032 181529 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=985168 CPUtime=762.29 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 2629859 0 0 0 75806 423 0 0 25 0 4 0 521781303 1008812032 181529 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376749 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3045.03
Current children cumulated vsize (KiB) 985168

[startup+822.301 s]
/proc/loadavg: 8.06 8.07 8.02 9/184 3697
/proc/meminfo: memFree=24649920/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=985168 CPUtime=3284.67 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 3220454 0 0 0 327914 553 0 0 25 0 4 0 521781303 1008812032 187246 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510116 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 246292 187267 2139 105 0 241796 0
[pid=3663/tid=3665] ppid=3661 vsize=985168 CPUtime=817.82 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 103271 0 0 0 81740 42 0 0 25 0 4 0 521781303 1008812032 187275 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=985168 CPUtime=822.3 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 95549 0 0 0 82209 21 0 0 25 0 4 0 521781303 1008812032 187278 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376675 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=985168 CPUtime=822.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 2932499 0 0 0 81756 472 0 0 25 0 4 0 521781303 1008812032 187278 33554432000 4194304 4623224 140733986632272 18446744073709551615 4577678 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3284.67
Current children cumulated vsize (KiB) 985168

[startup+882.3 s]
/proc/loadavg: 8.02 8.06 8.01 9/184 3698
/proc/meminfo: memFree=24591484/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=985168 CPUtime=3524.34 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 3532414 0 0 0 351828 606 0 0 25 0 4 0 521781303 1008812032 191997 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510978 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 246292 191997 1845 105 0 241796 0
[pid=3663/tid=3665] ppid=3661 vsize=985168 CPUtime=877.5 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 104742 0 0 0 87705 45 0 0 25 0 4 0 521781303 1008812032 191997 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=985168 CPUtime=882.31 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 96201 0 0 0 88209 22 0 0 25 0 4 0 521781303 1008812032 191997 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=985168 CPUtime=882.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3241450 0 0 0 87706 522 0 0 25 0 4 0 521781303 1008812032 191997 33554432000 4194304 4623224 140733986632272 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3524.34
Current children cumulated vsize (KiB) 985168

[startup+942.3 s]
/proc/loadavg: 8.01 8.04 8.00 9/184 3700
/proc/meminfo: memFree=24505400/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=1027696 CPUtime=3764.01 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 3840728 0 0 0 375744 657 0 0 25 0 4 0 521781303 1052360704 196906 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 256924 196906 1774 105 0 252428 0
[pid=3663/tid=3665] ppid=3661 vsize=1027696 CPUtime=937.18 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 106193 0 0 0 93671 47 0 0 25 0 4 0 521781303 1052360704 196906 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=1027696 CPUtime=942.31 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 96565 0 0 0 94209 22 0 0 25 0 4 0 521781303 1052360704 196906 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376640 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=1027696 CPUtime=942.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3524633 0 0 0 93659 569 0 0 25 0 4 0 521781303 1052360704 196906 33554432000 4194304 4623224 140733986632272 18446744073709551615 4550436 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3764.01
Current children cumulated vsize (KiB) 1027696

[startup+1002.3 s]
/proc/loadavg: 8.07 8.05 8.00 9/184 3702
/proc/meminfo: memFree=24308080/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=1095936 CPUtime=4003.68 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 4034657 0 0 0 399676 692 0 0 25 0 4 0 521781303 1122238464 205008 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 273984 205008 1749 105 0 269488 0
[pid=3663/tid=3665] ppid=3661 vsize=1095936 CPUtime=996.85 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 113602 0 0 0 99635 50 0 0 25 0 4 0 521781303 1122238464 205008 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=1095936 CPUtime=1002.31 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 96927 0 0 0 100209 22 0 0 25 0 4 0 521781303 1122238464 205008 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376659 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=1095936 CPUtime=1002.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3710047 0 0 0 99628 600 0 0 25 0 4 0 521781303 1122238464 205008 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4003.68
Current children cumulated vsize (KiB) 1095936

[startup+1062.3 s]
/proc/loadavg: 8.09 8.05 8.00 9/184 3703
/proc/meminfo: memFree=24231404/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=1095936 CPUtime=4243.36 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 4039686 0 0 0 423641 695 0 0 25 0 4 0 521781303 1122238464 209405 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 273984 209405 1142 105 0 269488 0
[pid=3663/tid=3665] ppid=3661 vsize=1095936 CPUtime=1056.53 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 115689 0 0 0 105601 52 0 0 25 0 4 0 521781303 1122238464 209405 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=1095936 CPUtime=1062.31 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 99357 0 0 0 106208 23 0 0 25 0 4 0 521781303 1122238464 209405 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=1095936 CPUtime=1062.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3710234 0 0 0 105628 600 0 0 25 0 4 0 521781303 1122238464 209405 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4243.36
Current children cumulated vsize (KiB) 1095936

[startup+1122.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/184 3705
/proc/meminfo: memFree=24131424/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=1163016 CPUtime=4483.03 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 4078612 0 0 0 447600 703 0 0 25 0 4 0 521781303 1190928384 220296 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 290754 220296 831 105 0 286258 0
[pid=3663/tid=3665] ppid=3661 vsize=1163016 CPUtime=1116.2 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 117823 0 0 0 111566 54 0 0 25 0 4 0 521781303 1190928384 220296 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=1163016 CPUtime=1122.31 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 134574 0 0 0 112203 28 0 0 25 0 4 0 521781303 1190928384 220296 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=1163016 CPUtime=1122.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3710584 0 0 0 111628 600 0 0 25 0 4 0 521781303 1190928384 220296 33554432000 4194304 4623224 140733986632272 18446744073709551615 4550411 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4483.03
Current children cumulated vsize (KiB) 1163016

[startup+1182.3 s]
/proc/loadavg: 8.09 8.06 8.00 9/184 3706
/proc/meminfo: memFree=24014960/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=1228552 CPUtime=4722.7 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 4082073 0 0 0 471564 706 0 0 25 0 4 0 521781303 1258037248 223717 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 307138 223717 831 105 0 302642 0
[pid=3663/tid=3665] ppid=3661 vsize=1228552 CPUtime=1175.87 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 119218 0 0 0 117531 56 0 0 25 0 4 0 521781303 1258037248 223717 33554432000 4194304 4623224 140733986632272 18446744073709551615 4377587 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=1228552 CPUtime=1182.32 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 135991 0 0 0 118203 29 0 0 25 0 4 0 521781303 1258037248 223717 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=1228552 CPUtime=1182.28 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3710711 0 0 0 117628 600 0 0 25 0 4 0 521781303 1258037248 223717 33554432000 4194304 4623224 140733986632272 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4722.7
Current children cumulated vsize (KiB) 1228552



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.06 8.06 8.00 9/184 3707
/proc/meminfo: memFree=23671960/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=1228552 CPUtime=4800.21 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 4083572 0 0 0 479314 707 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 307138 225205 831 105 0 302642 0
[pid=3663/tid=3665] ppid=3661 vsize=1228552 CPUtime=1195.17 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 119803 0 0 0 119460 57 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4377576 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=1228552 CPUtime=1201.72 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 136298 0 0 0 120143 29 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=1228552 CPUtime=1201.69 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3711275 0 0 0 119568 601 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 1228552

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.06 8.06 8.00 9/184 3707
/proc/meminfo: memFree=23671960/32951124 swapFree=67111356/67111528
[pid=3663] ppid=3661 vsize=1228552 CPUtime=4800.21 cores=0,2,4,6
/proc/3663/stat : 3663 (SArTagnan) R 3661 3663 3527 0 -1 4202496 4083572 0 0 0 479314 707 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/3663/statm: 307138 225205 831 105 0 302642 0
[pid=3663/tid=3665] ppid=3661 vsize=1228552 CPUtime=1195.17 cores=0,2,4,6
/proc/3663/task/3665/stat : 3665 (SArTagnan) R 3661 3663 3527 0 -1 4202560 119803 0 0 0 119460 57 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4377576 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=3663/tid=3666] ppid=3661 vsize=1228552 CPUtime=1201.72 cores=0,2,4,6
/proc/3663/task/3666/stat : 3666 (SArTagnan) R 3661 3663 3527 0 -1 4202560 136298 0 0 0 120143 29 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=3663/tid=3667] ppid=3661 vsize=1228552 CPUtime=1201.69 cores=0,2,4,6
/proc/3663/task/3667/stat : 3667 (SArTagnan) R 3661 3663 3527 0 -1 4202560 3711275 0 0 0 119568 601 0 0 25 0 4 0 521781303 1258037248 225205 33554432000 4194304 4623224 140733986632272 18446744073709551615 4550465 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.21
Current children cumulated vsize (KiB) 1228552

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.76
CPU time (s): 4800.29
CPU user time (s): 4793.17
CPU system time (s): 7.12292
CPU usage (%): 399.439
Max. virtual memory (cumulated for all children) (KiB): 1274448

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4793.17
system time used= 7.12292
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4083572
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 369
involuntary context switches= 23063

runsolver used 1.9657 second user time and 4.5973 second system time

The end

Launcher Data 

Begin job on node137 at 2011-04-29 20:30:36
IDJOB=3263764
IDBENCH=84018
IDSOLVER=1656
FILE ID=node137/3263764-1304101836
RUNJOBID= node137-1304100632-3551
PBS_JOBID= 13170625
Free space on /tmp= 71620 MiB

SOLVER NAME= SArTagnan 2011-03-24
BENCH NAME= SAT03/handmade/stanion/hwb/hwb-n28-02-S818962541.shuffled-as.sat03-1628.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3263764-1304101836/watcher-3263764-1304101836 -o /tmp/evaluation-result-3263764-1304101836/solver-3263764-1304101836 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3263764-1304101836.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d705043763df769aab1e385bbbc2b11e
RANDOM SEED=1991247410

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      27650124 kB
Buffers:        922644 kB
Cached:        3850080 kB
SwapCached:         56 kB
Active:         336284 kB
Inactive:      4499584 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      27650124 kB
SwapTotal:    67111528 kB
SwapFree:     67111356 kB
Dirty:             820 kB
Writeback:           0 kB
AnonPages:       62500 kB
Mapped:          14680 kB
Slab:           401700 kB
PageTables:       4240 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   185788 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71620 MiB
End job on node137 at 2011-04-29 20:50:40