Trace number 3266316

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-03-24? (TO) 4800.42 1201.77

General information on the benchmark

NameSAT02/handmade/pehoushek/plainoldcnf/
5cnf_3900_3900_220.shuffled.cnf
MD5SUMf0b7e0675915478b15cab3695fd0fd62
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark0.035994
Satisfiable
(Un)Satisfiability was proved
Number of variables220
Number of clauses3432
Sum of the clauses size17000
Maximum clause length5
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 32
Number of clauses of size 4156
Number of clauses of size 53274
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3266316-1304111560/watcher-3266316-1304111560 -o /tmp/evaluation-result-3266316-1304111560/solver-3266316-1304111560 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3266316-1304111560.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 6.93 7.96 8.09 2/177 8969
/proc/meminfo: memFree=29713656/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=39980 CPUtime=0 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 668 0 0 0 0 0 0 0 19 0 3 0 522753913 40939520 593 33554432000 4194304 4623224 140734034790768 18446744073709551615 255377357809 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 9995 593 302 105 0 5499 0
[pid=8969/tid=8972] ppid=8966 vsize=39980 CPUtime=0 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 1 0 0 0 0 0 0 0 20 0 3 0 522753913 40939520 593 33554432000 4194304 4623224 140734034790768 18446744073709551615 47791484802984 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=39980 CPUtime=0 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 0 0 0 0 0 0 0 0 20 0 3 0 522753913 40939520 593 33554432000 4194304 4623224 140734034790768 18446744073709551615 255389099062 0 0 4096 2 0 0 0 -1 1 0 0 0

[startup+0.286615 s]
/proc/loadavg: 6.93 7.96 8.09 2/177 8969
/proc/meminfo: memFree=29713656/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=255416 CPUtime=3.03 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 10487 0 0 0 302 1 0 0 18 0 4 0 522753913 261545984 4687 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510316 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 63854 4687 341 105 0 59358 0
[pid=8969/tid=8972] ppid=8966 vsize=255416 CPUtime=0.75 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 1695 0 0 0 75 0 0 0 19 0 4 0 522753913 261545984 4687 33554432000 4194304 4623224 140734034790768 18446744073709551615 255377344695 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=255416 CPUtime=0.75 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2022 0 0 0 75 0 0 0 18 0 4 0 522753913 261545984 4687 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.03
Current children cumulated vsize (KiB) 255416

[startup+0.781601 s]
/proc/loadavg: 6.93 7.96 8.09 2/177 8969
/proc/meminfo: memFree=29713656/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=255416 CPUtime=3.09 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 10657 0 0 0 308 1 0 0 18 0 4 0 522753913 261545984 4672 33554432000 4194304 4623224 140734034790768 18446744073709551615 4360030 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 63854 4672 341 105 0 59358 0
[pid=8969/tid=8972] ppid=8966 vsize=255416 CPUtime=0.77 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 1706 0 0 0 77 0 0 0 19 0 4 0 522753913 261545984 4672 33554432000 4194304 4623224 140734034790768 18446744073709551615 255377344695 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=255416 CPUtime=0.76 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) D 8966 8969 8042 0 -1 4202560 2041 0 0 0 76 0 0 0 19 0 4 0 522753913 261545984 4672 33554432000 4194304 4623224 140734034790768 18446744073709551615 255377344695 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 255416

[startup+0.807006 s]
/proc/loadavg: 6.93 7.96 8.09 2/177 8969
/proc/meminfo: memFree=29713656/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=255416 CPUtime=3.19 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 10984 0 0 0 318 1 0 0 18 0 4 0 522753913 261545984 4804 33554432000 4194304 4623224 140734034790768 18446744073709551615 4384653 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 63854 4804 341 105 0 59358 0
[pid=8969/tid=8972] ppid=8966 vsize=255416 CPUtime=0.79 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 1733 0 0 0 79 0 0 0 19 0 4 0 522753913 261545984 4804 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376846 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=255416 CPUtime=0.79 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2078 0 0 0 79 0 0 0 19 0 4 0 522753913 261545984 4804 33554432000 4194304 4623224 140734034790768 18446744073709551615 255376962593 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.19
Current children cumulated vsize (KiB) 255416

[startup+0.900991 s]
/proc/loadavg: 6.93 7.96 8.09 2/177 8969
/proc/meminfo: memFree=29713656/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=255416 CPUtime=3.56 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 11972 0 0 0 355 1 0 0 19 0 4 0 522753913 261545984 5178 33554432000 4194304 4623224 140734034790768 18446744073709551615 4415995 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 63854 5178 341 105 0 59358 0
[pid=8969/tid=8972] ppid=8966 vsize=255416 CPUtime=0.89 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 1821 0 0 0 89 0 0 0 19 0 4 0 522753913 261545984 5178 33554432000 4194304 4623224 140734034790768 18446744073709551615 4460986 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=255416 CPUtime=0.88 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2189 0 0 0 88 0 0 0 19 0 4 0 522753913 261545984 5178 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366349 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 3.56
Current children cumulated vsize (KiB) 255416

[startup+1.50088 s]
/proc/loadavg: 6.93 7.96 8.09 2/177 8969
/proc/meminfo: memFree=29713656/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=260376 CPUtime=5.97 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 16544 0 0 0 595 2 0 0 20 0 4 0 522753913 266625024 7200 33554432000 4194304 4623224 140734034790768 18446744073709551615 4414978 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 65094 7200 341 105 0 60598 0
[pid=8969/tid=8972] ppid=8966 vsize=260376 CPUtime=1.49 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2778 0 0 0 149 0 0 0 20 0 4 0 522753913 266625024 7200 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=260376 CPUtime=1.48 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 3277 0 0 0 148 0 0 0 20 0 4 0 522753913 266625024 7200 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 5.97
Current children cumulated vsize (KiB) 260376

[startup+3.10059 s]
/proc/loadavg: 6.93 7.96 8.09 9/188 8983
/proc/meminfo: memFree=29641412/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=266812 CPUtime=12.36 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 22885 0 0 0 1232 4 0 0 25 0 4 0 522753913 273215488 11278 33554432000 4194304 4623224 140734034790768 18446744073709551615 4425628 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 66703 11278 341 105 0 62207 0
[pid=8969/tid=8972] ppid=8966 vsize=266812 CPUtime=3.08 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 4660 0 0 0 307 1 0 0 25 0 4 0 522753913 273215488 11278 33554432000 4194304 4623224 140734034790768 18446744073709551615 4461046 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=266812 CPUtime=3.08 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 5222 0 0 0 307 1 0 0 25 0 4 0 522753913 273215488 11278 33554432000 4194304 4623224 140734034790768 18446744073709551615 255376962593 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=266812 CPUtime=3.08 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 8957 0 0 0 308 0 0 0 25 0 4 0 522753913 273215488 11278 33554432000 4194304 4623224 140734034790768 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 12.36
Current children cumulated vsize (KiB) 266812

[startup+6.30099 s]
/proc/loadavg: 7.10 7.97 8.10 10/188 8983
/proc/meminfo: memFree=29600656/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=277652 CPUtime=25.14 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 31962 0 0 0 2507 7 0 0 25 0 4 0 522753913 284315648 16884 33554432000 4194304 4623224 140734034790768 18446744073709551615 4360117 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 69413 16884 342 105 0 64917 0
[pid=8969/tid=8972] ppid=8966 vsize=277652 CPUtime=6.26 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 7124 0 0 0 624 2 0 0 25 0 4 0 522753913 284315648 16884 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376625 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=277652 CPUtime=6.28 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 8016 0 0 0 626 2 0 0 25 0 4 0 522753913 284315648 16884 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376652 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=277652 CPUtime=6.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 11053 0 0 0 627 1 0 0 25 0 4 0 522753913 284315648 16884 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366247 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 25.14
Current children cumulated vsize (KiB) 277652

[startup+12.7068 s]
/proc/loadavg: 7.17 7.97 8.09 9/188 8983
/proc/meminfo: memFree=29540040/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=295272 CPUtime=50.73 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 77672 0 0 0 5058 15 0 0 25 0 4 0 522753913 302358528 24801 33554432000 4194304 4623224 140734034790768 18446744073709551615 4511659 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 73818 24801 345 105 0 69322 0
[pid=8969/tid=8972] ppid=8966 vsize=295272 CPUtime=12.62 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 10778 0 0 0 1259 3 0 0 25 0 4 0 522753913 302358528 24801 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=295272 CPUtime=12.69 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 14592 0 0 0 1266 3 0 0 25 0 4 0 522753913 302358528 24802 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376832 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=295272 CPUtime=12.69 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 44104 0 0 0 1263 6 0 0 25 0 4 0 522753913 302358528 24802 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366247 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 50.73
Current children cumulated vsize (KiB) 295272

[startup+25.5004 s]
/proc/loadavg: 7.35 7.97 8.09 9/188 8983
/proc/meminfo: memFree=29468856/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=319184 CPUtime=101.84 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 184595 0 0 0 10147 37 0 0 25 0 4 0 522753913 326844416 39123 33554432000 4194304 4623224 140734034790768 18446744073709551615 4355951 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 79796 39123 375 105 0 75300 0
[pid=8969/tid=8972] ppid=8966 vsize=319184 CPUtime=25.35 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 15678 0 0 0 2529 6 0 0 25 0 4 0 522753913 326844416 39123 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=319184 CPUtime=25.48 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 22238 0 0 0 2543 5 0 0 25 0 4 0 522753913 326844416 39123 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376678 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=319184 CPUtime=25.48 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 134551 0 0 0 2525 23 0 0 25 0 4 0 522753913 326844416 39123 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366336 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 101.84
Current children cumulated vsize (KiB) 319184

[startup+51.1007 s]
/proc/loadavg: 7.57 7.97 8.09 9/188 8984
/proc/meminfo: memFree=29353264/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=351048 CPUtime=204.12 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 336794 0 0 0 20347 65 0 0 25 0 4 0 522753913 359473152 52319 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 87762 52319 346 105 0 83266 0
[pid=8969/tid=8972] ppid=8966 vsize=351048 CPUtime=50.81 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 21923 0 0 0 5074 7 0 0 25 0 4 0 522753913 359473152 52321 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376895 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=351048 CPUtime=51.09 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 30524 0 0 0 5101 8 0 0 25 0 4 0 522753913 359473152 52321 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=351048 CPUtime=51.08 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 265514 0 0 0 5062 46 0 0 25 0 4 0 522753913 359473152 52324 33554432000 4194304 4623224 140734034790768 18446744073709551615 4577476 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 204.12
Current children cumulated vsize (KiB) 351048

[startup+102.301 s]
/proc/loadavg: 7.95 8.02 8.10 9/188 8985
/proc/meminfo: memFree=29166996/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=485272 CPUtime=408.62 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 454150 0 0 0 40765 97 0 0 25 0 4 0 522753913 496918528 79310 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510316 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 121318 79310 439 105 0 116822 0
[pid=8969/tid=8972] ppid=8966 vsize=485272 CPUtime=101.74 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 33436 0 0 0 10157 17 0 0 25 0 4 0 522753913 496918528 79310 33554432000 4194304 4623224 140734034790768 18446744073709551615 4357641 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=485272 CPUtime=102.28 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 47670 0 0 0 10215 13 0 0 25 0 4 0 522753913 496918528 79310 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376666 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=485272 CPUtime=102.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 345142 0 0 0 10167 61 0 0 25 0 4 0 522753913 496918528 79310 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 408.62
Current children cumulated vsize (KiB) 485272

[startup+162.307 s]
/proc/loadavg: 8.23 8.09 8.11 9/188 8987
/proc/meminfo: memFree=28992252/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=667980 CPUtime=648.32 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 562619 0 0 0 64703 129 0 0 25 0 4 0 522753913 684011520 107983 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 166995 107983 462 105 0 162499 0
[pid=8969/tid=8972] ppid=8966 vsize=667980 CPUtime=161.42 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 49429 0 0 0 16115 27 0 0 25 0 4 0 522753913 684011520 107983 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376794 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=667980 CPUtime=162.29 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 56222 0 0 0 16210 19 0 0 25 0 4 0 522753913 684011520 107983 33554432000 4194304 4623224 140734034790768 18446744073709551615 255376962245 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=667980 CPUtime=162.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 416665 0 0 0 16152 76 0 0 25 0 4 0 522753913 684011520 107983 33554432000 4194304 4623224 140734034790768 18446744073709551615 255376998748 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 648.32

################
# More data... #
################

[pid=8969] ppid=8966 vsize=1138012 CPUtime=2805.41 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 3848738 0 0 0 279774 767 0 0 25 0 4 0 522753913 1165324288 220596 33554432000 4194304 4623224 140734034790768 18446744073709551615 4414972 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 284503 220596 409 105 0 280007 0
[pid=8969/tid=8972] ppid=8966 vsize=1138012 CPUtime=698.48 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 109370 0 0 0 69793 55 0 0 25 0 4 0 522753913 1165324288 220596 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376745 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1138012 CPUtime=702.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 118615 0 0 0 70173 58 0 0 25 0 4 0 522753913 1165324288 220596 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376652 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1138012 CPUtime=702.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2229081 0 0 0 69842 386 0 0 25 0 4 0 522753913 1165324288 220596 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 2805.41
Current children cumulated vsize (KiB) 1138012

[startup+762.301 s]
/proc/loadavg: 8.38 8.26 8.18 9/188 9003
/proc/meminfo: memFree=28147572/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1145932 CPUtime=3045.07 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 4234832 0 0 0 303665 842 0 0 25 0 4 0 522753913 1173434368 227188 33554432000 4194304 4623224 140734034790768 18446744073709551615 4517479 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 286483 227188 409 105 0 281987 0
[pid=8969/tid=8972] ppid=8966 vsize=1145932 CPUtime=758.16 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 111878 0 0 0 75759 57 0 0 25 0 4 0 522753913 1173434368 227188 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1145932 CPUtime=762.3 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 121977 0 0 0 76168 62 0 0 25 0 4 0 522753913 1173434368 227188 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1145932 CPUtime=762.27 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2229341 0 0 0 75841 386 0 0 25 0 4 0 522753913 1173434368 227188 33554432000 4194304 4623224 140734034790768 18446744073709551615 4364914 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3045.07
Current children cumulated vsize (KiB) 1145932

[startup+822.307 s]
/proc/loadavg: 8.20 8.23 8.17 9/188 9005
/proc/meminfo: memFree=28106268/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1173748 CPUtime=3284.77 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 4634210 0 0 0 327559 918 0 0 25 0 4 0 522753913 1201917952 235332 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 293437 235332 341 105 0 288941 0
[pid=8969/tid=8972] ppid=8966 vsize=1173748 CPUtime=817.84 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 116342 0 0 0 81724 60 0 0 25 0 4 0 522753913 1201917952 235332 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376625 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1173748 CPUtime=822.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 124556 0 0 0 82165 66 0 0 25 0 4 0 522753913 1201917952 235332 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1173748 CPUtime=822.29 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2239639 0 0 0 81841 388 0 0 25 0 4 0 522753913 1201917952 235332 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376732 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3284.77
Current children cumulated vsize (KiB) 1173748

[startup+882.301 s]
/proc/loadavg: 8.34 8.25 8.18 9/188 9006
/proc/meminfo: memFree=28049328/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1181932 CPUtime=3524.42 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 5031922 0 0 0 351456 986 0 0 25 0 4 0 522753913 1210298368 241465 33554432000 4194304 4623224 140734034790768 18446744073709551615 4355951 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 295483 241465 341 105 0 290987 0
[pid=8969/tid=8972] ppid=8966 vsize=1181932 CPUtime=877.51 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 119003 0 0 0 87689 62 0 0 25 0 4 0 522753913 1210298368 241465 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1181932 CPUtime=882.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 126880 0 0 0 88161 70 0 0 25 0 4 0 522753913 1210298368 241465 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1181932 CPUtime=882.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2261502 0 0 0 87837 391 0 0 25 0 4 0 522753913 1210298368 241465 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3524.42
Current children cumulated vsize (KiB) 1181932

[startup+942.301 s]
/proc/loadavg: 8.19 8.21 8.18 9/188 9008
/proc/meminfo: memFree=28002200/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1235496 CPUtime=3764.1 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 5532605 0 0 0 375338 1072 0 0 25 0 4 0 522753913 1265147904 250515 33554432000 4194304 4623224 140734034790768 18446744073709551615 4517494 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 308874 250515 341 105 0 304378 0
[pid=8969/tid=8972] ppid=8966 vsize=1235496 CPUtime=937.18 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 121512 0 0 0 93654 64 0 0 25 0 4 0 522753913 1265147904 250515 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1235496 CPUtime=942.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 154683 0 0 0 94153 78 0 0 25 0 4 0 522753913 1265147904 250515 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376640 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1235496 CPUtime=942.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2373283 0 0 0 93818 410 0 0 25 0 4 0 522753913 1265147904 250515 33554432000 4194304 4623224 140734034790768 18446744073709551615 4545317 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 3764.1
Current children cumulated vsize (KiB) 1235496

[startup+1002.3 s]
/proc/loadavg: 8.31 8.23 8.18 9/188 9009
/proc/meminfo: memFree=27961260/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1241964 CPUtime=4003.78 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 6116499 0 0 0 399201 1177 0 0 25 0 4 0 522753913 1271771136 256748 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 310491 256748 341 105 0 305995 0
[pid=8969/tid=8972] ppid=8966 vsize=1241964 CPUtime=996.86 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 123780 0 0 0 99620 66 0 0 25 0 4 0 522753913 1271771136 256748 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1241964 CPUtime=1002.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 157068 0 0 0 100150 81 0 0 25 0 4 0 522753913 1271771136 256748 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1241964 CPUtime=1002.27 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2597887 0 0 0 99781 446 0 0 25 0 4 0 522753913 1271771136 256748 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4003.78
Current children cumulated vsize (KiB) 1241964

[startup+1062.3 s]
/proc/loadavg: 8.15 8.20 8.18 9/188 9011
/proc/meminfo: memFree=27911008/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1380032 CPUtime=4243.45 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 6760686 0 0 0 423056 1289 0 0 25 0 4 0 522753913 1413152768 265081 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510316 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 345008 265081 389 105 0 340512 0
[pid=8969/tid=8972] ppid=8966 vsize=1380032 CPUtime=1056.53 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 126868 0 0 0 105586 67 0 0 25 0 4 0 522753913 1413152768 265081 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1380032 CPUtime=1062.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 159485 0 0 0 106146 85 0 0 25 0 4 0 522753913 1413152768 265081 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1380032 CPUtime=1062.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 2886619 0 0 0 105735 493 0 0 25 0 4 0 522753913 1413152768 265081 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4243.45
Current children cumulated vsize (KiB) 1380032

[startup+1122.3 s]
/proc/loadavg: 8.42 8.26 8.19 9/188 9013
/proc/meminfo: memFree=27868440/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1322356 CPUtime=4483.14 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 7446548 0 0 0 446904 1410 0 0 25 0 4 0 522753913 1354092544 269280 33554432000 4194304 4623224 140734034790768 18446744073709551615 4414885 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 330589 269281 341 105 0 326093 0
[pid=8969/tid=8972] ppid=8966 vsize=1322356 CPUtime=1116.2 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 127820 0 0 0 111553 67 0 0 25 0 4 0 522753913 1354092544 269281 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1322356 CPUtime=1122.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 161902 0 0 0 112141 90 0 0 25 0 4 0 522753913 1354092544 269281 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1322356 CPUtime=1122.28 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 3226553 0 0 0 111677 551 0 0 25 0 4 0 522753913 1354092544 269283 33554432000 4194304 4623224 140734034790768 18446744073709551615 4577678 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4483.14
Current children cumulated vsize (KiB) 1322356

[startup+1182.3 s]
/proc/loadavg: 8.26 8.24 8.19 9/188 9014
/proc/meminfo: memFree=27773936/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1413368 CPUtime=4722.8 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 8007274 0 0 0 470769 1511 0 0 25 0 4 0 522753913 1447288832 289023 33554432000 4194304 4623224 140734034790768 18446744073709551615 4414880 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 353342 289023 342 105 0 348846 0
[pid=8969/tid=8972] ppid=8966 vsize=1413368 CPUtime=1175.87 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 128861 0 0 0 117520 67 0 0 25 0 4 0 522753913 1447288832 289023 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1413368 CPUtime=1182.31 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 175220 0 0 0 118135 96 0 0 25 0 4 0 522753913 1447288832 289023 33554432000 4194304 4623224 140734034790768 18446744073709551615 255376962593 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1413368 CPUtime=1182.27 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 3425397 0 0 0 117643 584 0 0 25 0 4 0 522753913 1447288832 289023 33554432000 4194304 4623224 140734034790768 18446744073709551615 4364232 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4722.8
Current children cumulated vsize (KiB) 1413368



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.7 s]
/proc/loadavg: 8.18 8.22 8.18 10/188 9015
/proc/meminfo: memFree=27758672/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1413368 CPUtime=4800.33 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 8124895 0 0 0 478498 1535 0 0 25 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 353342 291046 341 105 0 348846 0
[pid=8969/tid=8972] ppid=8966 vsize=1413368 CPUtime=1195.17 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 129223 0 0 0 119450 67 0 0 25 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1413368 CPUtime=1201.72 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 176582 0 0 0 120074 98 0 0 25 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1413368 CPUtime=1201.69 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 3426251 0 0 0 119584 585 0 0 19 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 1413368

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+1201.7 s]
/proc/loadavg: 8.18 8.22 8.18 10/188 9015
/proc/meminfo: memFree=27758672/32951124 swapFree=67111352/67111528
[pid=8969] ppid=8966 vsize=1413368 CPUtime=4800.33 cores=1,3,5,7
/proc/8969/stat : 8969 (SArTagnan) R 8966 8969 8042 0 -1 4202496 8124895 0 0 0 478498 1535 0 0 25 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 1 0 0 0
/proc/8969/statm: 353342 291046 341 105 0 348846 0
[pid=8969/tid=8972] ppid=8966 vsize=1413368 CPUtime=1195.17 cores=1,3,5,7
/proc/8969/task/8972/stat : 8972 (SArTagnan) R 8966 8969 8042 0 -1 4202560 129223 0 0 0 119450 67 0 0 25 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=8969/tid=8973] ppid=8966 vsize=1413368 CPUtime=1201.72 cores=1,3,5,7
/proc/8969/task/8973/stat : 8973 (SArTagnan) R 8966 8969 8042 0 -1 4202560 176582 0 0 0 120074 98 0 0 25 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=8969/tid=8975] ppid=8966 vsize=1413368 CPUtime=1201.69 cores=1,3,5,7
/proc/8969/task/8975/stat : 8975 (SArTagnan) R 8966 8969 8042 0 -1 4202560 3426251 0 0 0 119584 585 0 0 19 0 4 0 522753913 1447288832 291046 33554432000 4194304 4623224 140734034790768 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 3 0 0 0
Current children cumulated CPU time (s) 4800.33
Current children cumulated vsize (KiB) 1413368

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.77
CPU time (s): 4800.42
CPU user time (s): 4785.01
CPU system time (s): 15.4127
CPU usage (%): 399.445
Max. virtual memory (cumulated for all children) (KiB): 1417728

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4785.01
system time used= 15.4127
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 8124895
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1905
involuntary context switches= 31683

runsolver used 1.69974 second user time and 4.86626 second system time

The end

Launcher Data 

Begin job on node135 at 2011-04-29 23:12:40
IDJOB=3266316
IDBENCH=87453
IDSOLVER=1656
FILE ID=node135/3266316-1304111560
RUNJOBID= node135-1304111560-8957
PBS_JOBID= 13170648
Free space on /tmp= 71120 MiB

SOLVER NAME= SArTagnan 2011-03-24
BENCH NAME= SAT02/handmade/pehoushek/plainoldcnf/5cnf_3900_3900_220.shuffled.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3266316-1304111560/watcher-3266316-1304111560 -o /tmp/evaluation-result-3266316-1304111560/solver-3266316-1304111560 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3266316-1304111560.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= f0b7e0675915478b15cab3695fd0fd62
RANDOM SEED=1502952425

node135.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.70
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 6640.14
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.851
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      29713944 kB
Buffers:        918060 kB
Cached:        1735560 kB
SwapCached:        172 kB
Active:         120496 kB
Inactive:      2598896 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      29713944 kB
SwapTotal:    67111528 kB
SwapFree:     67111352 kB
Dirty:             560 kB
Writeback:           0 kB
AnonPages:       65536 kB
Mapped:          14496 kB
Slab:           453992 kB
PageTables:       4720 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:   191056 kB
VmallocTotal: 34359738367 kB
VmallocUsed:     33868 kB
VmallocChunk: 34359703231 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71120 MiB
End job on node135 at 2011-04-29 23:32:44