Trace number 3386197

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4800.54 1205.29

General information on the benchmark

NameAPPLICATIONS/satComp09_BioInstances/
ndhf_xits_19_UNKNOWN.cnf
MD5SUM4cf2a693df1b6e5a084b88e84ec5f2af
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark14.5608
Satisfiable
(Un)Satisfiability was proved
Number of variables4020
Number of clauses466486
Sum of the clauses size1393798
Maximum clause length20
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 26187
Number of clauses of size 3460268
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 531

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386197-1305577907/watcher-3386197-1305577907 -o /tmp/evaluation-result-3386197-1305577907/solver-3386197-1305577907 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386197-1305577907.cnf -threads=4 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.87 8.05 8.15 6/481 10626
/proc/meminfo: memFree=13814912/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=31116 CPUtime=0 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 1049 0 0 0 0 0 0 0 21 0 1 0 669387424 31862784 860 33554432000 4194304 4625164 140736384967616 18446744073709551615 4339447 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/10626/statm: 7779 862 296 106 0 3282 0

[startup+0.0322089 s]
/proc/loadavg: 7.87 8.05 8.15 6/481 10626
/proc/meminfo: memFree=13814912/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=34736 CPUtime=0.02 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 3206 0 0 0 2 0 0 0 21 0 1 0 669387424 35569664 1909 33554432000 4194304 4625164 140736384967616 18446744073709551615 254500352709 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/10626/statm: 8684 1916 296 106 0 4187 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 34736

[startup+0.100204 s]
/proc/loadavg: 7.87 8.05 8.15 6/481 10626
/proc/meminfo: memFree=13814912/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=46640 CPUtime=0.09 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 10073 0 0 0 8 1 0 0 21 0 1 0 669387424 47759360 5275 33554432000 4194304 4625164 140736384967616 18446744073709551615 4361099 0 0 4096 2 0 0 0 17 3 0 0 0
/proc/10626/statm: 11660 5276 296 106 0 7163 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 46640

[startup+0.301088 s]
/proc/loadavg: 7.87 8.05 8.15 6/481 10626
/proc/meminfo: memFree=13814912/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=221832 CPUtime=0.35 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 38104 0 0 0 31 4 0 0 22 0 4 0 669387424 227155968 26836 33554432000 4194304 4625164 140736384967616 18446744073709551615 254500347873 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 55458 26836 458 106 0 50961 0
Current children cumulated CPU time (s) 0.35
Current children cumulated vsize (KiB) 221832

[startup+0.701006 s]
/proc/loadavg: 7.87 8.05 8.15 6/481 10626
/proc/meminfo: memFree=13814912/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=418920 CPUtime=1.92 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 60208 0 0 0 186 6 0 0 25 0 4 0 669387424 428974080 33464 33554432000 4194304 4625164 140736384967616 18446744073709551615 4518211 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 104730 33465 341 106 0 100233 0
Current children cumulated CPU time (s) 1.92
Current children cumulated vsize (KiB) 418920

[startup+1.50085 s]
/proc/loadavg: 7.87 8.05 8.15 9/485 10630
/proc/meminfo: memFree=13671048/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=418920 CPUtime=5.1 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 75633 0 0 0 502 8 0 0 25 0 4 0 669387424 428974080 36236 33554432000 4194304 4625164 140736384967616 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 104730 36236 341 106 0 100233 0
[pid=10626/tid=10628] ppid=10624 vsize=418920 CPUtime=1.21 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 6111 0 0 0 120 1 0 0 25 0 4 0 669387451 428974080 36236 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=418920 CPUtime=1.17 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 5294 0 0 0 117 0 0 0 25 0 4 0 669387451 428974080 36236 33554432000 4194304 4625164 140736384967616 18446744073709551615 4375320 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=418920 CPUtime=1.21 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 23524 0 0 0 119 2 0 0 25 0 4 0 669387451 428974080 36236 33554432000 4194304 4625164 140736384967616 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 5.1
Current children cumulated vsize (KiB) 418920

[startup+3.10053 s]
/proc/loadavg: 7.87 8.05 8.15 9/485 10630
/proc/meminfo: memFree=13659464/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=418920 CPUtime=11.48 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 100920 0 0 0 1135 13 0 0 25 0 4 0 669387424 428974080 38790 33554432000 4194304 4625164 140736384967616 18446744073709551615 4509199 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 104730 38790 341 106 0 100233 0
[pid=10626/tid=10628] ppid=10624 vsize=418920 CPUtime=2.81 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 8129 0 0 0 279 2 0 0 25 0 4 0 669387451 428974080 38790 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=418920 CPUtime=2.77 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 5707 0 0 0 277 0 0 0 25 0 4 0 669387451 428974080 38790 33554432000 4194304 4625164 140736384967616 18446744073709551615 4467485 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=418920 CPUtime=2.78 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 46248 0 0 0 273 5 0 0 25 0 4 0 669387451 428974080 38790 33554432000 4194304 4625164 140736384967616 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 11.48
Current children cumulated vsize (KiB) 418920

[startup+6.3009 s]
/proc/loadavg: 7.88 8.05 8.15 9/485 10630
/proc/meminfo: memFree=13635656/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=423312 CPUtime=24.24 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 112161 0 0 0 2406 18 0 0 25 0 4 0 669387424 433471488 45105 33554432000 4194304 4625164 140736384967616 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 105828 45106 342 106 0 101331 0
[pid=10626/tid=10628] ppid=10624 vsize=423312 CPUtime=6.01 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 10659 0 0 0 597 4 0 0 25 0 4 0 669387451 433471488 45106 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=423312 CPUtime=5.97 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 8519 0 0 0 596 1 0 0 25 0 4 0 669387451 433471488 45106 33554432000 4194304 4625164 140736384967616 18446744073709551615 254500353057 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=423312 CPUtime=5.94 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 52134 0 0 0 587 7 0 0 25 0 4 0 669387451 433471488 45106 33554432000 4194304 4625164 140736384967616 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 24.24
Current children cumulated vsize (KiB) 423312

[startup+12.7006 s]
/proc/loadavg: 7.89 8.05 8.15 9/485 10630
/proc/meminfo: memFree=13585436/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=446252 CPUtime=49.72 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 131700 0 0 0 4947 25 0 0 25 0 4 0 669387424 456962048 56097 33554432000 4194304 4625164 140736384967616 18446744073709551615 4512046 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 111563 56097 342 106 0 107066 0
[pid=10626/tid=10628] ppid=10624 vsize=446252 CPUtime=12.41 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 18687 0 0 0 1234 7 0 0 25 0 4 0 669387451 456962048 56097 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=446252 CPUtime=12.37 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 18310 0 0 0 1232 5 0 0 25 0 4 0 669387451 456962048 56097 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=446252 CPUtime=12.23 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 53800 0 0 0 1216 7 0 0 25 0 4 0 669387451 456962048 56097 33554432000 4194304 4625164 140736384967616 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 49.72
Current children cumulated vsize (KiB) 446252

[startup+25.5011 s]
/proc/loadavg: 7.91 8.05 8.15 9/485 10631
/proc/meminfo: memFree=13488952/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=528080 CPUtime=100.71 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 155960 0 0 0 10034 37 0 0 25 0 4 0 669387424 540753920 76077 33554432000 4194304 4625164 140736384967616 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 132020 76077 342 106 0 127523 0
[pid=10626/tid=10628] ppid=10624 vsize=528080 CPUtime=25.21 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 25918 0 0 0 2509 12 0 0 25 0 4 0 669387451 540753920 76077 33554432000 4194304 4625164 140736384967616 18446744073709551615 4352296 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=528080 CPUtime=25.17 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 28303 0 0 0 2505 12 0 0 25 0 4 0 669387451 540753920 76077 33554432000 4194304 4625164 140736384967616 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=528080 CPUtime=24.82 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 60558 0 0 0 2474 8 0 0 25 0 4 0 669387451 540753920 76077 33554432000 4194304 4625164 140736384967616 18446744073709551615 4579399 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 100.71
Current children cumulated vsize (KiB) 528080

[startup+51.101 s]
/proc/loadavg: 8.08 8.07 8.15 9/485 10631
/proc/meminfo: memFree=13305848/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=701308 CPUtime=202.71 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 203277 0 0 0 20207 64 0 0 25 0 4 0 669387424 718139392 110544 33554432000 4194304 4625164 140736384967616 18446744073709551615 4519467 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 175327 110544 343 106 0 170830 0
[pid=10626/tid=10628] ppid=10624 vsize=701308 CPUtime=50.81 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 43520 0 0 0 5058 23 0 0 25 0 4 0 669387451 718139392 110544 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=701308 CPUtime=50.77 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 52681 0 0 0 5051 26 0 0 25 0 4 0 669387451 718139392 110544 33554432000 4194304 4625164 140736384967616 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=701308 CPUtime=50.01 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 65068 0 0 0 4992 9 0 0 25 0 4 0 669387451 718139392 110544 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351871 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 202.71
Current children cumulated vsize (KiB) 701308

[startup+102.319 s]
/proc/loadavg: 8.18 8.09 8.16 9/485 10633
/proc/meminfo: memFree=13078524/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=885308 CPUtime=406.73 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 283710 0 0 0 40567 106 0 0 25 0 4 0 669387424 906555392 156892 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393051 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 221327 156892 347 106 0 216830 0
[pid=10626/tid=10628] ppid=10624 vsize=885308 CPUtime=102.03 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 66573 0 0 0 10167 36 0 0 25 0 4 0 669387451 906555392 156892 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=885308 CPUtime=101.98 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 85469 0 0 0 10148 50 0 0 25 0 4 0 669387451 906555392 156892 33554432000 4194304 4625164 140736384967616 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=885308 CPUtime=100.38 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 83932 0 0 0 10026 12 0 0 25 0 4 0 669387451 906555392 156892 33554432000 4194304 4625164 140736384967616 18446744073709551615 4551147 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 406.73
Current children cumulated vsize (KiB) 885308

[startup+162.301 s]
/proc/loadavg: 8.21 8.11 8.15 9/485 10634
/proc/meminfo: memFree=12838296/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1081544 CPUtime=645.71 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 352594 0 0 0 64423 148 0 0 25 0 4 0 669387424 1107501056 206430 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393112 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 270386 206430 1068 106 0 265889 0
[pid=10626/tid=10628] ppid=10624 vsize=1081544 CPUtime=162.02 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 107023 0 0 0 16129 73 0 0 25 0 4 0 669387451 1107501056 206430 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1081544 CPUtime=161.97 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 88603 0 0 0 16146 51 0 0 25 0 4 0 669387451 1107501056 206430 33554432000 4194304 4625164 140736384967616 18446744073709551615 4352014 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1081544 CPUtime=159.4 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 88560 0 0 0 15927 13 0 0 25 0 4 0 669387451 1107501056 206430 33554432000 4194304 4625164 140736384967616 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 645.71
Current children cumulated vsize (KiB) 1081544

[startup+222.301 s]
/proc/loadavg: 8.14 8.10 8.15 9/485 10636
/proc/meminfo: memFree=12567812/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1309340 CPUtime=884.73 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 650457 0 0 0 88256 217 0 0 25 0 4 0 669387424 1340764160 256373 33554432000 4194304 4625164 140736384967616 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 327335 256373 348 106 0 322838 0
[pid=10626/tid=10628] ppid=10624 vsize=1309340 CPUtime=222.02 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 129107 0 0 0 22106 96 0 0 25 0 4 0 669387451 1340764160 256373 33554432000 4194304 4625164 140736384967616 18446744073709551615 4462955 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1309340 CPUtime=221.98 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 113383 0 0 0 22139 59 0 0 25 0 4 0 669387451 1340764160 256373 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351996 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1309340 CPUtime=218.41 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 315752 0 0 0 21795 46 0 0 25 0 4 0 669387451 1340764160 256373 33554432000 4194304 4625164 140736384967616 18446744073709551615 4547554 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 884.73
Current children cumulated vsize (KiB) 1309340

[startup+282.301 s]

################
# More data... #
################

[pid=10626] ppid=10624 vsize=1760232 CPUtime=2557.95 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 2621281 0 0 0 255230 565 0 0 25 0 4 0 669387424 1802477568 360861 33554432000 4194304 4625164 140736384967616 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 440058 360861 347 106 0 435561 0
[pid=10626/tid=10628] ppid=10624 vsize=1760232 CPUtime=642.05 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 162344 0 0 0 64101 104 0 0 25 0 4 0 669387451 1802477568 360861 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1760232 CPUtime=642.01 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 130010 0 0 0 64121 80 0 0 25 0 4 0 669387451 1802477568 360861 33554432000 4194304 4625164 140736384967616 18446744073709551615 4330776 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1760232 CPUtime=631.57 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 2102840 0 0 0 62814 343 0 0 25 0 4 0 669387451 1802477568 360861 33554432000 4194304 4625164 140736384967616 18446744073709551615 4547221 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2557.95
Current children cumulated vsize (KiB) 1760232

[startup+702.301 s]
/proc/loadavg: 8.12 8.09 8.10 9/485 10680
/proc/meminfo: memFree=12407112/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1789216 CPUtime=2796.94 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 3283731 0 0 0 279017 677 0 0 25 0 4 0 669387424 1832157184 366649 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393089 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 447304 366649 350 106 0 442807 0
[pid=10626/tid=10628] ppid=10624 vsize=1789216 CPUtime=702.04 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 162471 0 0 0 70100 104 0 0 25 0 4 0 669387451 1832157184 366649 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1789216 CPUtime=701.99 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 130037 0 0 0 70119 80 0 0 25 0 4 0 669387451 1832157184 366649 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1789216 CPUtime=690.59 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 2744200 0 0 0 68609 450 0 0 25 0 4 0 669387451 1832157184 366649 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 2796.94
Current children cumulated vsize (KiB) 1789216

[startup+762.319 s]
/proc/loadavg: 8.12 8.10 8.10 9/485 10681
/proc/meminfo: memFree=12012632/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1789216 CPUtime=3036.04 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 3887469 0 0 0 302822 782 0 0 25 0 4 0 669387424 1832157184 376696 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 447304 376696 353 106 0 442807 0
[pid=10626/tid=10628] ppid=10624 vsize=1789216 CPUtime=762.06 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 162628 0 0 0 76102 104 0 0 25 0 4 0 669387451 1832157184 376696 33554432000 4194304 4625164 140736384967616 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1789216 CPUtime=762.01 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 130055 0 0 0 76121 80 0 0 25 0 4 0 669387451 1832157184 376696 33554432000 4194304 4625164 140736384967616 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1789216 CPUtime=749.61 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3337052 0 0 0 74410 551 0 0 25 0 4 0 669387451 1832157184 376696 33554432000 4194304 4625164 140736384967616 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3036.04
Current children cumulated vsize (KiB) 1789216

[startup+822.301 s]
/proc/loadavg: 8.16 8.11 8.10 9/485 10683
/proc/meminfo: memFree=11678284/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1791656 CPUtime=3275 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4156320 0 0 0 326672 828 0 0 25 0 4 0 669387424 1834655744 379557 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 447914 379557 347 106 0 443417 0
[pid=10626/tid=10628] ppid=10624 vsize=1791656 CPUtime=822.04 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 162680 0 0 0 82100 104 0 0 25 0 4 0 669387451 1834655744 379557 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1791656 CPUtime=822 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 130067 0 0 0 82120 80 0 0 25 0 4 0 669387451 1834655744 379557 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1791656 CPUtime=808.63 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3600614 0 0 0 80267 596 0 0 25 0 4 0 669387451 1834655744 379557 33554432000 4194304 4625164 140736384967616 18446744073709551615 254500389198 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3275
Current children cumulated vsize (KiB) 1791656

[startup+882.301 s]
/proc/loadavg: 8.31 8.15 8.11 9/485 10685
/proc/meminfo: memFree=11573716/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1799564 CPUtime=3514.02 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4159684 0 0 0 350573 829 0 0 25 0 4 0 669387424 1842753536 382759 33554432000 4194304 4625164 140736384967616 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 449891 382759 347 106 0 445394 0
[pid=10626/tid=10628] ppid=10624 vsize=1799564 CPUtime=882.05 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 162735 0 0 0 88101 104 0 0 25 0 4 0 669387451 1842753536 382759 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1799564 CPUtime=882 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 130078 0 0 0 88120 80 0 0 25 0 4 0 669387451 1842753536 382759 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1799564 CPUtime=867.65 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3603366 0 0 0 86168 597 0 0 25 0 4 0 669387451 1842753536 382759 33554432000 4194304 4625164 140736384967616 18446744073709551615 4547314 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3514.02
Current children cumulated vsize (KiB) 1799564

[startup+942.301 s]
/proc/loadavg: 8.25 8.15 8.11 9/485 10686
/proc/meminfo: memFree=11451532/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1810336 CPUtime=3753.06 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4164948 0 0 0 374475 831 0 0 25 0 4 0 669387424 1853784064 387613 33554432000 4194304 4625164 140736384967616 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 452584 387613 347 106 0 448087 0
[pid=10626/tid=10628] ppid=10624 vsize=1810336 CPUtime=942.05 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 162788 0 0 0 94101 104 0 0 25 0 4 0 669387451 1853784064 387613 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1810336 CPUtime=942 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 130089 0 0 0 94120 80 0 0 25 0 4 0 669387451 1853784064 387613 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1810336 CPUtime=926.67 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3606373 0 0 0 92070 597 0 0 25 0 4 0 669387451 1853784064 387613 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3753.06
Current children cumulated vsize (KiB) 1810336

[startup+1002.3 s]
/proc/loadavg: 8.23 8.16 8.11 9/485 10688
/proc/meminfo: memFree=11253092/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1957924 CPUtime=3992.07 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4250345 0 0 0 398362 845 0 0 25 0 4 0 669387424 2004914176 400466 33554432000 4194304 4625164 140736384967616 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 489481 400466 347 106 0 484984 0
[pid=10626/tid=10628] ppid=10624 vsize=1957924 CPUtime=1002.05 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 163062 0 0 0 100101 104 0 0 25 0 4 0 669387451 2004914176 400466 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1957924 CPUtime=1002.01 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 133705 0 0 0 100120 81 0 0 25 0 4 0 669387451 2004914176 400466 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1957924 CPUtime=985.69 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3623703 0 0 0 97969 600 0 0 25 0 4 0 669387451 2004914176 400466 33554432000 4194304 4625164 140736384967616 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 3992.07
Current children cumulated vsize (KiB) 1957924

[startup+1062.3 s]
/proc/loadavg: 8.21 8.16 8.11 9/485 10689
/proc/meminfo: memFree=11149512/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1966960 CPUtime=4231.1 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4264591 0 0 0 422261 849 0 0 25 0 4 0 669387424 2014167040 409184 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393110 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 491740 409184 347 106 0 487243 0
[pid=10626/tid=10628] ppid=10624 vsize=1966960 CPUtime=1062.05 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 163119 0 0 0 106101 104 0 0 25 0 4 0 669387451 2014167040 409184 33554432000 4194304 4625164 140736384967616 18446744073709551615 4330855 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1966960 CPUtime=1062.01 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 133745 0 0 0 106120 81 0 0 25 0 4 0 669387451 2014167040 409184 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1966960 CPUtime=1044.72 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3626335 0 0 0 103871 601 0 0 25 0 4 0 669387451 2014167040 409184 33554432000 4194304 4625164 140736384967616 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4231.1
Current children cumulated vsize (KiB) 1966960

[startup+1122.3 s]
/proc/loadavg: 8.20 8.16 8.11 9/485 10691
/proc/meminfo: memFree=10919208/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1967780 CPUtime=4470.12 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4285306 0 0 0 446159 853 0 0 25 0 4 0 669387424 2015006720 410662 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393127 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 491945 410662 347 106 0 487448 0
[pid=10626/tid=10628] ppid=10624 vsize=1967780 CPUtime=1122.05 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 163172 0 0 0 112101 104 0 0 25 0 4 0 669387451 2015006720 410662 33554432000 4194304 4625164 140736384967616 18446744073709551615 4330810 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1967780 CPUtime=1122.01 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 133789 0 0 0 112120 81 0 0 25 0 4 0 669387451 2015006720 410662 33554432000 4194304 4625164 140736384967616 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1967780 CPUtime=1103.74 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3636146 0 0 0 109771 603 0 0 25 0 4 0 669387451 2015006720 410662 33554432000 4194304 4625164 140736384967616 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4470.12
Current children cumulated vsize (KiB) 1967780

[startup+1182.3 s]
/proc/loadavg: 8.13 8.14 8.10 9/485 10693
/proc/meminfo: memFree=10831128/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1975544 CPUtime=4709.16 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4299798 0 0 0 470058 858 0 0 25 0 4 0 669387424 2022957056 414154 33554432000 4194304 4625164 140736384967616 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 493886 414154 347 106 0 489389 0
[pid=10626/tid=10628] ppid=10624 vsize=1975544 CPUtime=1182.06 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 163282 0 0 0 118102 104 0 0 25 0 4 0 669387451 2022957056 414154 33554432000 4194304 4625164 140736384967616 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1975544 CPUtime=1182.01 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 133885 0 0 0 118120 81 0 0 25 0 4 0 669387451 2022957056 414154 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1975544 CPUtime=1162.76 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3638529 0 0 0 115673 603 0 0 25 0 4 0 669387451 2022957056 414154 33554432000 4194304 4625164 140736384967616 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4709.16
Current children cumulated vsize (KiB) 1975544



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.2 s]
/proc/loadavg: 8.08 8.13 8.10 9/485 10693
/proc/meminfo: memFree=10782500/32950928 swapFree=45358800/67111528
[pid=10626] ppid=10624 vsize=1981852 CPUtime=4800.43 cores=1,3,5,7
/proc/10626/stat : 10626 (SArTagnan) R 10624 10626 10402 0 -1 4202496 4306660 0 0 0 479183 860 0 0 25 0 4 0 669387424 2029416448 416580 33554432000 4194304 4625164 140736384967616 18446744073709551615 4393080 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/10626/statm: 495463 416580 347 106 0 490966 0
[pid=10626/tid=10628] ppid=10624 vsize=1981852 CPUtime=1204.97 cores=1,3,5,7
/proc/10626/task/10628/stat : 10628 (SArTagnan) R 10624 10626 10402 0 -1 4202560 163284 0 0 0 120393 104 0 0 25 0 4 0 669387451 2029416448 416580 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=10626/tid=10629] ppid=10624 vsize=1981852 CPUtime=1204.93 cores=1,3,5,7
/proc/10626/task/10629/stat : 10629 (SArTagnan) R 10624 10626 10402 0 -1 4202560 133921 0 0 0 120412 81 0 0 25 0 4 0 669387451 2029416448 416580 33554432000 4194304 4625164 140736384967616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=10626/tid=10630] ppid=10624 vsize=1981852 CPUtime=1185.29 cores=1,3,5,7
/proc/10626/task/10630/stat : 10630 (SArTagnan) R 10624 10626 10402 0 -1 4202560 3640212 0 0 0 117925 604 0 0 25 0 4 0 669387451 2029416448 416580 33554432000 4194304 4625164 140736384967616 18446744073709551615 4335146 0 0 4096 2 0 0 0 -1 7 0 0 0
Current children cumulated CPU time (s) 4800.43
Current children cumulated vsize (KiB) 1981852

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.29
CPU time (s): 4800.54
CPU user time (s): 4791.87
CPU system time (s): 8.66868
CPU usage (%): 398.288
Max. virtual memory (cumulated for all children) (KiB): 2124164

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4791.87
system time used= 8.66868
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4306660
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 1682
involuntary context switches= 32325

runsolver used 5.17121 second user time and 14.5488 second system time

The end

Launcher Data 

Begin job on node149 at 2011-05-16 22:31:47
IDJOB=3386197
IDBENCH=71095
IDSOLVER=1871
FILE ID=node149/3386197-1305577907
RUNJOBID= node149-1305576699-10421
PBS_JOBID= 13350208
Free space on /tmp= 73524 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT09/APPLICATIONS/satComp09_BioInstances/ndhf_xits_19_UNKNOWN.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386197-1305577907/watcher-3386197-1305577907 -o /tmp/evaluation-result-3386197-1305577907/solver-3386197-1305577907 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386197-1305577907.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 4cf2a693df1b6e5a084b88e84ec5f2af
RANDOM SEED=1852870939

node149.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.806
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      13815068 kB
Buffers:         84672 kB
Cached:         543320 kB
SwapCached:      36208 kB
Active:        9723184 kB
Inactive:      9132904 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      13815068 kB
SwapTotal:    67111528 kB
SwapFree:     45358800 kB
Dirty:            8164 kB
Writeback:           0 kB
AnonPages:    18204368 kB
Mapped:          27472 kB
Slab:            97784 kB
PageTables:     114104 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 157371520 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73520 MiB
End job on node149 at 2011-05-16 22:51:55