Trace number 3386127

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 4800.18 1205.17

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n220-s24359701-sat.cnf
MD5SUMceff918324677440f95867755c4e7133
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark5.76412
Satisfiable
(Un)Satisfiability was proved
Number of variables220
Number of clauses528
Sum of the clauses size1320
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2440
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 588
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3386127-1305580476/watcher-3386127-1305580476 -o /tmp/evaluation-result-3386127-1305580476/solver-3386127-1305580476 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3386127-1305580476.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 3.68 4.97 6.59 3/485 2008
/proc/meminfo: memFree=16071200/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=28724 CPUtime=0 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 446 0 0 0 0 0 0 0 24 0 2 0 669645336 29413376 383 33554432000 4194304 4625164 140733714175072 18446744073709551615 218905787377 0 0 4096 2 18446744073709551615 0 0 17 2 0 0 0
/proc/2008/statm: 7181 383 300 106 0 2684 0
[pid=2008/tid=2014] ppid=2005 vsize=49212 CPUtime=0.01 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1 0 0 0 1 0 0 0 25 0 4 0 669645337 50393088 385 33554432000 4194304 4625164 140733714175072 18446744073709551615 47185750185906 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=49212 CPUtime=0 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 0 0 0 0 0 0 0 0 25 0 4 0 669645337 50393088 385 33554432000 4194304 4625164 140733714175072 18446744073709551615 218905787377 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=49212 CPUtime=0.01 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 0 0 0 0 1 0 0 0 25 0 4 0 669645337 50393088 385 33554432000 4194304 4625164 140733714175072 18446744073709551615 47185750185910 0 0 4096 2 0 0 0 -1 0 0 0 0

[startup+0.170064 s]
/proc/loadavg: 3.68 4.97 6.59 3/485 2008
/proc/meminfo: memFree=16071200/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=248628 CPUtime=0.64 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 2362 0 0 0 64 0 0 0 18 0 4 0 669645336 254595072 1614 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512167 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 62157 1614 345 106 0 57660 0
[pid=2008/tid=2014] ppid=2005 vsize=248628 CPUtime=0.16 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 483 0 0 0 16 0 0 0 25 0 4 0 669645337 254595072 1614 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352005 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=248628 CPUtime=0.14 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 525 0 0 0 14 0 0 0 25 0 4 0 669645337 254595072 1614 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=248628 CPUtime=0.16 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 537 0 0 0 16 0 0 0 25 0 4 0 669645337 254595072 1614 33554432000 4194304 4625164 140733714175072 18446744073709551615 4339655 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.64
Current children cumulated vsize (KiB) 248628

[startup+0.201058 s]
/proc/loadavg: 3.68 4.97 6.59 3/485 2008
/proc/meminfo: memFree=16071200/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=249012 CPUtime=0.77 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 2691 0 0 0 77 0 0 0 18 0 4 0 669645336 254988288 1812 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512057 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 62253 1812 345 106 0 57756 0
[pid=2008/tid=2014] ppid=2005 vsize=249012 CPUtime=0.19 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 538 0 0 0 19 0 0 0 25 0 4 0 669645337 254988288 1812 33554432000 4194304 4625164 140733714175072 18446744073709551615 4288861 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=249012 CPUtime=0.17 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 710 0 0 0 17 0 0 0 25 0 4 0 669645337 254988288 1812 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=249012 CPUtime=0.19 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 619 0 0 0 19 0 0 0 25 0 4 0 669645337 254988288 1812 33554432000 4194304 4625164 140733714175072 18446744073709551615 4288867 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.77
Current children cumulated vsize (KiB) 249012

[startup+0.301041 s]
/proc/loadavg: 3.68 4.97 6.59 3/485 2008
/proc/meminfo: memFree=16071200/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=250012 CPUtime=1.16 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 3792 0 0 0 116 0 0 0 18 0 4 0 669645336 256012288 2273 33554432000 4194304 4625164 140733714175072 18446744073709551615 4356856 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 62503 2273 345 106 0 58006 0
[pid=2008/tid=2014] ppid=2005 vsize=250012 CPUtime=0.29 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 838 0 0 0 29 0 0 0 25 0 4 0 669645337 256012288 2273 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=250012 CPUtime=0.27 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1074 0 0 0 27 0 0 0 25 0 4 0 669645337 256012288 2273 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=250012 CPUtime=0.29 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 917 0 0 0 29 0 0 0 25 0 4 0 669645337 256012288 2273 33554432000 4194304 4625164 140733714175072 18446744073709551615 4579312 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 250012

[startup+0.700969 s]
/proc/loadavg: 3.68 4.97 6.59 3/485 2008
/proc/meminfo: memFree=16071200/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=253824 CPUtime=2.76 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 7427 0 0 0 276 0 0 0 18 0 4 0 669645336 259915776 3901 33554432000 4194304 4625164 140733714175072 18446744073709551615 4511986 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 63456 3901 345 106 0 58959 0
[pid=2008/tid=2014] ppid=2005 vsize=253824 CPUtime=0.69 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1861 0 0 0 69 0 0 0 25 0 4 0 669645337 259915776 3901 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=253824 CPUtime=0.67 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1912 0 0 0 67 0 0 0 25 0 4 0 669645337 259915776 3901 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=253824 CPUtime=0.69 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 2494 0 0 0 69 0 0 0 25 0 4 0 669645337 259915776 3902 33554432000 4194304 4625164 140733714175072 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.76
Current children cumulated vsize (KiB) 253824

[startup+1.50083 s]
/proc/loadavg: 3.68 4.97 6.59 10/495 2018
/proc/meminfo: memFree=16028728/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=258340 CPUtime=5.95 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 17779 0 0 0 594 1 0 0 20 0 4 0 669645336 264540160 6476 33554432000 4194304 4625164 140733714175072 18446744073709551615 4513220 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 64585 6476 351 106 0 60088 0
[pid=2008/tid=2014] ppid=2005 vsize=258340 CPUtime=1.49 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 3117 0 0 0 149 0 0 0 25 0 4 0 669645337 264540160 6477 33554432000 4194304 4625164 140733714175072 18446744073709551615 218905391813 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=258340 CPUtime=1.45 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 3505 0 0 0 145 0 0 0 25 0 4 0 669645337 264540160 6477 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=258340 CPUtime=1.49 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 9294 0 0 0 149 0 0 0 25 0 4 0 669645337 264540160 6477 33554432000 4194304 4625164 140733714175072 18446744073709551615 4579376 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.95
Current children cumulated vsize (KiB) 258340

[startup+3.10054 s]
/proc/loadavg: 3.68 4.97 6.59 10/495 2018
/proc/meminfo: memFree=16004176/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=267896 CPUtime=12.33 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 26809 0 0 0 1230 3 0 0 25 0 4 0 669645336 274325504 9782 33554432000 4194304 4625164 140733714175072 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 66974 9782 357 106 0 62477 0
[pid=2008/tid=2014] ppid=2005 vsize=267896 CPUtime=3.09 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 5096 0 0 0 309 0 0 0 25 0 4 0 669645337 274325504 9782 33554432000 4194304 4625164 140733714175072 18446744073709551615 218905391813 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=267896 CPUtime=3.03 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 4218 0 0 0 303 0 0 0 25 0 4 0 669645337 274325504 9782 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=267896 CPUtime=3.08 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 15260 0 0 0 307 1 0 0 25 0 4 0 669645337 274325504 9782 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 12.33
Current children cumulated vsize (KiB) 267896

[startup+6.30095 s]
/proc/loadavg: 4.03 5.02 6.59 10/495 2018
/proc/meminfo: memFree=15962652/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=277688 CPUtime=25.08 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 40973 0 0 0 2502 6 0 0 25 0 4 0 669645336 284352512 14666 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512066 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 69422 14666 356 106 0 64925 0
[pid=2008/tid=2014] ppid=2005 vsize=277688 CPUtime=6.28 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 8094 0 0 0 628 0 0 0 25 0 4 0 669645337 284352512 14666 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=277688 CPUtime=6.18 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 8324 0 0 0 616 2 0 0 25 0 4 0 669645337 284352512 14666 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=277688 CPUtime=6.29 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 21334 0 0 0 627 2 0 0 25 0 4 0 669645337 284352512 14666 33554432000 4194304 4625164 140733714175072 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 25.08
Current children cumulated vsize (KiB) 277688

[startup+12.7008 s]
/proc/loadavg: 4.35 5.07 6.60 10/495 2019
/proc/meminfo: memFree=15906616/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=292592 CPUtime=50.55 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 94197 0 0 0 5041 14 0 0 25 0 4 0 669645336 299614208 21533 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 73148 21534 380 106 0 68651 0
[pid=2008/tid=2014] ppid=2005 vsize=292592 CPUtime=12.69 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 12782 0 0 0 1268 1 0 0 25 0 4 0 669645337 299614208 21534 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=292592 CPUtime=12.47 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 12263 0 0 0 1244 3 0 0 25 0 4 0 669645337 299614208 21534 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=292592 CPUtime=12.68 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 64653 0 0 0 1260 8 0 0 25 0 4 0 669645337 299614208 21534 33554432000 4194304 4625164 140733714175072 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.55
Current children cumulated vsize (KiB) 292592

[startup+25.5005 s]
/proc/loadavg: 5.15 5.21 6.62 10/495 2019
/proc/meminfo: memFree=15811504/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=319872 CPUtime=101.54 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 117044 0 0 0 10134 20 0 0 25 0 4 0 669645336 327548928 32938 33554432000 4194304 4625164 140733714175072 18446744073709551615 4519365 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 79968 32938 375 106 0 75471 0
[pid=2008/tid=2014] ppid=2005 vsize=319872 CPUtime=25.49 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 19506 0 0 0 2546 3 0 0 25 0 4 0 669645337 327548928 32938 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=319872 CPUtime=25.06 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 18617 0 0 0 2501 5 0 0 25 0 4 0 669645337 327548928 32938 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=319872 CPUtime=25.48 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 71699 0 0 0 2538 10 0 0 25 0 4 0 669645337 327548928 32938 33554432000 4194304 4625164 140733714175072 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.54
Current children cumulated vsize (KiB) 319872

[startup+51.1008 s]
/proc/loadavg: 6.12 5.43 6.66 10/495 2020
/proc/meminfo: memFree=15694424/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=362768 CPUtime=203.52 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 309047 0 0 0 20299 53 0 0 25 0 4 0 669645336 371474432 47895 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 90692 47895 357 106 0 86195 0
[pid=2008/tid=2014] ppid=2005 vsize=362768 CPUtime=51.09 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 28415 0 0 0 5102 7 0 0 25 0 4 0 669645337 371474432 47895 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=362768 CPUtime=50.24 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 26796 0 0 0 5018 6 0 0 25 0 4 0 669645337 371474432 47895 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=362768 CPUtime=51.08 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 239712 0 0 0 5071 37 0 0 25 0 4 0 669645337 371474432 47895 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351808 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.52
Current children cumulated vsize (KiB) 362768

[startup+102.301 s]
/proc/loadavg: 7.24 5.84 6.73 9/495 2021
/proc/meminfo: memFree=15513468/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=442756 CPUtime=407.48 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 703997 0 0 0 40630 118 0 0 25 0 4 0 669645336 453382144 71376 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 110689 71376 814 106 0 106192 0
[pid=2008/tid=2014] ppid=2005 vsize=442756 CPUtime=102.31 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 40448 0 0 0 10221 10 0 0 25 0 4 0 669645337 453382144 71385 33554432000 4194304 4625164 140733714175072 18446744073709551615 4339639 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=442756 CPUtime=100.6 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 30639 0 0 0 10053 7 0 0 25 0 4 0 669645337 453382144 71385 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=442756 CPUtime=102.3 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 610560 0 0 0 10134 96 0 0 25 0 4 0 669645337 453382144 71385 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.48
Current children cumulated vsize (KiB) 442756

[startup+162.301 s]

################
# More data... #
################

[pid=2008] ppid=2005 vsize=963528 CPUtime=2558.37 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1566946 0 0 0 255573 264 0 0 25 0 4 0 669645336 986652672 187609 33554432000 4194304 4625164 140733714175072 18446744073709551615 4511982 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 240882 187609 1283 106 0 236385 0
[pid=2008/tid=2014] ppid=2005 vsize=963528 CPUtime=642.3 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 112983 0 0 0 64208 22 0 0 25 0 4 0 669645337 986652672 187609 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=963528 CPUtime=631.62 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 92044 0 0 0 63137 25 0 0 25 0 4 0 669645337 986652672 187609 33554432000 4194304 4625164 140733714175072 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=963528 CPUtime=642.3 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1193851 0 0 0 64046 184 0 0 25 0 4 0 669645337 986652672 187609 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2558.37
Current children cumulated vsize (KiB) 963528

[startup+702.301 s]
/proc/loadavg: 8.21 7.83 7.39 10/495 2037
/proc/meminfo: memFree=14560552/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1029064 CPUtime=2797.36 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1670714 0 0 0 279448 288 0 0 25 0 4 0 669645336 1053761536 193411 33554432000 4194304 4625164 140733714175072 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 257266 193411 583 106 0 252769 0
[pid=2008/tid=2014] ppid=2005 vsize=1029064 CPUtime=702.31 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 115027 0 0 0 70207 24 0 0 25 0 4 0 669645337 1053761536 193411 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1029064 CPUtime=690.61 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 95175 0 0 0 69033 28 0 0 25 0 4 0 669645337 1053761536 193411 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1029064 CPUtime=702.31 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1194436 0 0 0 70046 185 0 0 25 0 4 0 669645337 1053761536 193411 33554432000 4194304 4625164 140733714175072 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2797.36
Current children cumulated vsize (KiB) 1029064

[startup+762.301 s]
/proc/loadavg: 8.21 7.89 7.43 10/495 2039
/proc/meminfo: memFree=14505072/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1051044 CPUtime=3036.37 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1747986 0 0 0 303331 306 0 0 25 0 4 0 669645336 1076269056 199936 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 262761 199936 392 106 0 258264 0
[pid=2008/tid=2014] ppid=2005 vsize=1051044 CPUtime=762.33 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 117449 0 0 0 76207 26 0 0 25 0 4 0 669645337 1076269056 199936 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1051044 CPUtime=749.61 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 98002 0 0 0 74930 31 0 0 25 0 4 0 669645337 1076269056 199936 33554432000 4194304 4625164 140733714175072 18446744073709551615 4467459 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1051044 CPUtime=762.32 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1206262 0 0 0 76046 186 0 0 25 0 4 0 669645337 1076269056 199936 33554432000 4194304 4625164 140733714175072 18446744073709551615 4579376 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3036.37
Current children cumulated vsize (KiB) 1051044

[startup+822.301 s]
/proc/loadavg: 8.20 7.94 7.48 10/495 2040
/proc/meminfo: memFree=14435948/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1051044 CPUtime=3275.31 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1754709 0 0 0 327220 311 0 0 25 0 4 0 669645336 1076269056 205794 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 262761 205794 346 106 0 258264 0
[pid=2008/tid=2014] ppid=2005 vsize=1051044 CPUtime=822.31 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 119982 0 0 0 82203 28 0 0 25 0 4 0 669645337 1076269056 205794 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1051044 CPUtime=808.59 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 100205 0 0 0 80826 33 0 0 25 0 4 0 669645337 1076269056 205794 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1051044 CPUtime=822.3 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1206462 0 0 0 82044 186 0 0 25 0 4 0 669645337 1076269056 205794 33554432000 4194304 4625164 140733714175072 18446744073709551615 4551660 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3275.31
Current children cumulated vsize (KiB) 1051044

[startup+882.32 s]
/proc/loadavg: 8.07 7.95 7.51 10/495 2042
/proc/meminfo: memFree=14376004/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1164584 CPUtime=3514.38 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1796414 0 0 0 351116 322 0 0 25 0 4 0 669645336 1192534016 217186 33554432000 4194304 4625164 140733714175072 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 291146 217186 346 106 0 286649 0
[pid=2008/tid=2014] ppid=2005 vsize=1164584 CPUtime=882.34 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 123722 0 0 0 88203 31 0 0 25 0 4 0 669645337 1192534016 217186 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1164584 CPUtime=867.61 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 106640 0 0 0 86726 35 0 0 25 0 4 0 669645337 1192534016 217186 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1164584 CPUtime=882.33 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1206873 0 0 0 88046 187 0 0 25 0 4 0 669645337 1192534016 217186 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3514.38
Current children cumulated vsize (KiB) 1164584

[startup+942.301 s]
/proc/loadavg: 8.13 7.98 7.55 9/495 2043
/proc/meminfo: memFree=14277384/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1243184 CPUtime=3753.3 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1887786 0 0 0 374988 342 0 0 25 0 4 0 669645336 1273020416 226561 33554432000 4194304 4625164 140733714175072 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 310796 226561 346 106 0 306299 0
[pid=2008/tid=2014] ppid=2005 vsize=1243184 CPUtime=942.32 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 130489 0 0 0 94199 33 0 0 25 0 4 0 669645337 1273020416 226561 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1243184 CPUtime=926.58 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 107011 0 0 0 92623 35 0 0 25 0 4 0 669645337 1273020416 226561 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1243184 CPUtime=942.31 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1207998 0 0 0 94044 187 0 0 25 0 4 0 669645337 1273020416 226561 33554432000 4194304 4625164 140733714175072 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3753.3
Current children cumulated vsize (KiB) 1243184

[startup+1002.3 s]
/proc/loadavg: 8.26 8.05 7.59 9/495 2045
/proc/meminfo: memFree=14204180/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1334992 CPUtime=3992.27 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1930322 0 0 0 398879 348 0 0 25 0 4 0 669645336 1367031808 235369 33554432000 4194304 4625164 140733714175072 18446744073709551615 4519319 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 333748 235369 346 106 0 329251 0
[pid=2008/tid=2014] ppid=2005 vsize=1334992 CPUtime=1002.32 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 168850 0 0 0 100195 37 0 0 25 0 4 0 669645337 1367031808 235369 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1334992 CPUtime=985.57 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 107940 0 0 0 98521 36 0 0 25 0 4 0 669645337 1367031808 235369 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1334992 CPUtime=1002.31 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1209082 0 0 0 100044 187 0 0 25 0 4 0 669645337 1367031808 235369 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 3992.27
Current children cumulated vsize (KiB) 1334992

[startup+1062.3 s]
/proc/loadavg: 8.18 8.06 7.63 9/495 2047
/proc/meminfo: memFree=14157752/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1338724 CPUtime=4231.27 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1933170 0 0 0 422776 351 0 0 25 0 4 0 669645336 1370853376 238141 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512066 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 334681 238141 346 106 0 330184 0
[pid=2008/tid=2014] ppid=2005 vsize=1338724 CPUtime=1062.31 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 169095 0 0 0 106194 37 0 0 25 0 4 0 669645337 1370853376 238141 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1338724 CPUtime=1044.58 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 108242 0 0 0 104422 36 0 0 25 0 4 0 669645337 1370853376 238141 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1338724 CPUtime=1062.32 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1210369 0 0 0 106043 189 0 0 25 0 4 0 669645337 1370853376 238141 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4231.27
Current children cumulated vsize (KiB) 1338724

[startup+1122.3 s]
/proc/loadavg: 8.07 8.05 7.65 10/495 2048
/proc/meminfo: memFree=14109224/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1395208 CPUtime=4470.24 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 1967926 0 0 0 446666 358 0 0 25 0 4 0 669645336 1428692992 245991 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 348802 245991 346 106 0 344305 0
[pid=2008/tid=2014] ppid=2005 vsize=1395208 CPUtime=1122.32 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 170199 0 0 0 112193 39 0 0 25 0 4 0 669645337 1428692992 245991 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1395208 CPUtime=1103.56 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 140417 0 0 0 110314 42 0 0 25 0 4 0 669645337 1428692992 245991 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1395208 CPUtime=1122.32 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1210774 0 0 0 112043 189 0 0 25 0 4 0 669645337 1428692992 245991 33554432000 4194304 4625164 140733714175072 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4470.24
Current children cumulated vsize (KiB) 1395208

[startup+1182.3 s]
/proc/loadavg: 8.02 8.04 7.66 10/495 2050
/proc/meminfo: memFree=14054120/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1427056 CPUtime=4709.23 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 2190576 0 0 0 470533 390 0 0 25 0 4 0 669645336 1461305344 254480 33554432000 4194304 4625164 140733714175072 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 356764 254480 409 106 0 352267 0
[pid=2008/tid=2014] ppid=2005 vsize=1427056 CPUtime=1182.32 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 171328 0 0 0 118192 40 0 0 25 0 4 0 669645337 1461305344 254480 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1427056 CPUtime=1162.58 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 141863 0 0 0 116214 44 0 0 25 0 4 0 669645337 1461305344 254480 33554432000 4194304 4625164 140733714175072 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1427056 CPUtime=1182.32 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1412946 0 0 0 118015 217 0 0 25 0 4 0 669645337 1461305344 254480 33554432000 4194304 4625164 140733714175072 18446744073709551615 4579396 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4709.23
Current children cumulated vsize (KiB) 1427056



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1205.1 s]
/proc/loadavg: 8.01 8.03 7.67 5/491 2051
/proc/meminfo: memFree=15047748/32951124 swapFree=62248504/67111528
[pid=2008] ppid=2005 vsize=1427056 CPUtime=4800.06 cores=0,2,4,6
/proc/2008/stat : 2008 (SArTagnan) R 2005 2008 1492 0 -1 4202496 2355575 0 0 0 479592 414 0 0 25 0 4 0 669645336 1461305344 254201 33554432000 4194304 4625164 140733714175072 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/2008/statm: 356764 254201 364 106 0 352267 0
[pid=2008/tid=2014] ppid=2005 vsize=1427056 CPUtime=1205.14 cores=0,2,4,6
/proc/2008/task/2014/stat : 2014 (SArTagnan) R 2005 2008 1492 0 -1 4202560 171409 0 0 0 120474 40 0 0 25 0 4 0 669645337 1461305344 254208 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=2008/tid=2015] ppid=2005 vsize=1427056 CPUtime=1184.99 cores=0,2,4,6
/proc/2008/task/2015/stat : 2015 (SArTagnan) R 2005 2008 1492 0 -1 4202560 141979 0 0 0 118455 44 0 0 25 0 4 0 669645337 1461305344 254208 33554432000 4194304 4625164 140733714175072 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=2008/tid=2016] ppid=2005 vsize=1427056 CPUtime=1205.13 cores=0,2,4,6
/proc/2008/task/2016/stat : 2016 (SArTagnan) R 2005 2008 1492 0 -1 4202560 1577711 0 0 0 120273 240 0 0 25 0 4 0 669645337 1461305344 254208 33554432000 4194304 4625164 140733714175072 18446744073709551615 4552315 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 4800.06
Current children cumulated vsize (KiB) 1427056

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1205.17
CPU time (s): 4800.18
CPU user time (s): 4795.99
CPU system time (s): 4.19236
CPU usage (%): 398.299
Max. virtual memory (cumulated for all children) (KiB): 1502564

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4795.99
system time used= 4.19236
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2355593
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 296
involuntary context switches= 41273

runsolver used 5.53616 second user time and 14.5798 second system time

The end

Launcher Data 

Begin job on node137 at 2011-05-16 23:14:36
IDJOB=3386127
IDBENCH=83146
IDSOLVER=1871
FILE ID=node137/3386127-1305580476
RUNJOBID= node137-1305580476-1991
PBS_JOBID= 13350207
Free space on /tmp= 73664 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n220-s24359701-sat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3386127-1305580476/watcher-3386127-1305580476 -o /tmp/evaluation-result-3386127-1305580476/solver-3386127-1305580476 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3386127-1305580476.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= ceff918324677440f95867755c4e7133
RANDOM SEED=833074947

node137.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.68
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.844
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      16071752 kB
Buffers:        330336 kB
Cached:        4367152 kB
SwapCached:      25020 kB
Active:        6901128 kB
Inactive:      9703900 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      16071752 kB
SwapTotal:    67111528 kB
SwapFree:     62248504 kB
Dirty:             884 kB
Writeback:           0 kB
AnonPages:    11892168 kB
Mapped:          30784 kB
Slab:           137720 kB
PageTables:      68520 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 146924760 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73664 MiB
End job on node137 at 2011-05-16 23:34:43