Trace number 3438706

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.8 5011.52

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n190-s29985700-unsat.cnf
MD5SUMf2cf61fc47972fb7e89be65d1a83332e
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables193
Number of clauses396
Sum of the clauses size1188
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3396
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438706-1306922101/watcher-3438706-1306922101 -o /tmp/evaluation-result-3438706-1306922101/solver-3438706-1306922101 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438706-1306922101.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 8.08 8.04 2/507 22957
/proc/meminfo: memFree=19812540/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=90456 CPUtime=0.06 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 500 0 0 0 6 0 0 0 25 0 8 0 882440876 92626944 437 33554432000 4194304 4625164 140735580001488 18446744073709551615 47961627944882 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22957/statm: 22614 437 300 106 0 18117 0
[pid=22957/tid=22958] ppid=22954 vsize=287212 CPUtime=0.01 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 15 0 0 0 1 0 0 0 25 0 8 0 882440877 294105088 624 33554432000 4194304 4625164 140735580001488 18446744073709551615 4487022 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=287212 CPUtime=0.01 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 2 0 0 0 1 0 0 0 25 0 8 0 882440877 294105088 624 33554432000 4194304 4625164 140735580001488 18446744073709551615 4353587 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=287212 CPUtime=0.01 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 25 0 0 0 1 0 0 0 25 0 8 0 882440877 294105088 624 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=287212 CPUtime=0.01 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) S 22954 22957 22892 0 -1 4202560 1 0 0 0 1 0 0 0 25 0 8 0 882440877 294105088 624 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988697582 0 0 4096 2 18446604466393592896 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=287212 CPUtime=0 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 8 0 0 0 0 0 0 0 25 0 8 0 882440877 294105088 629 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988697582 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=287212 CPUtime=0.01 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 33 0 0 0 1 0 0 0 25 0 8 0 882440877 294105088 629 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988259842 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=287212 CPUtime=0.01 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 5 0 0 0 1 0 0 0 25 0 8 0 882440877 294105088 630 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351909 0 0 4096 2 0 0 0 -1 0 0 0 0

[startup+0.0771581 s]
/proc/loadavg: 8.09 8.08 8.04 2/507 22957
/proc/meminfo: memFree=19812540/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=354760 CPUtime=0.53 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 2212 0 0 0 53 0 0 0 25 0 8 0 882440876 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988697582 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 88690 1872 351 106 0 84193 0
[pid=22957/tid=22958] ppid=22954 vsize=354760 CPUtime=0.07 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 230 0 0 0 7 0 0 0 25 0 8 0 882440877 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351925 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=354760 CPUtime=0.06 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 185 0 0 0 6 0 0 0 25 0 8 0 882440877 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=354760 CPUtime=0.07 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 295 0 0 0 7 0 0 0 25 0 8 0 882440877 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988697631 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=354760 CPUtime=0.06 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 202 0 0 0 6 0 0 0 25 0 8 0 882440877 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352818 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=354760 CPUtime=0.05 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 184 0 0 0 5 0 0 0 25 0 8 0 882440877 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 4350518 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=354760 CPUtime=0.06 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 216 0 0 0 6 0 0 0 25 0 8 0 882440877 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 216004576283 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=354760 CPUtime=0.06 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 151 0 0 0 6 0 0 0 25 0 8 0 882440877 363274240 1872 33554432000 4194304 4625164 140735580001488 18446744073709551615 4462600 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.53
Current children cumulated vsize (KiB) 354760

[startup+0.103091 s]
/proc/loadavg: 8.09 8.08 8.04 2/507 22957
/proc/meminfo: memFree=19812540/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=355660 CPUtime=0.74 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 2957 0 0 0 74 0 0 0 25 0 8 0 882440876 364195840 2311 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988697582 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 88915 2311 351 106 0 84418 0
[pid=22957/tid=22958] ppid=22954 vsize=355660 CPUtime=0.09 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 331 0 0 0 9 0 0 0 25 0 8 0 882440877 364195840 2311 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988253381 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=355660 CPUtime=0.09 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 282 0 0 0 9 0 0 0 25 0 8 0 882440877 364195840 2311 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=355660 CPUtime=0.09 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 517 0 0 0 9 0 0 0 25 0 8 0 882440877 364195840 2311 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=355660 CPUtime=0.09 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 297 0 0 0 9 0 0 0 25 0 8 0 882440877 364195840 2311 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=355660 CPUtime=0.08 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 276 0 0 0 8 0 0 0 25 0 8 0 882440877 364195840 2311 33554432000 4194304 4625164 140735580001488 18446744073709551615 4349768 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=355660 CPUtime=0.09 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 247 0 0 0 9 0 0 0 25 0 8 0 882440877 364195840 2311 33554432000 4194304 4625164 140735580001488 18446744073709551615 216004576283 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=355660 CPUtime=0.08 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 176 0 0 0 8 0 0 0 25 0 8 0 882440877 364195840 2312 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988697631 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 0.74
Current children cumulated vsize (KiB) 355660

[startup+0.300672 s]
/proc/loadavg: 8.09 8.08 8.04 2/507 22957
/proc/meminfo: memFree=19812540/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=361924 CPUtime=2.32 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 7707 0 0 0 232 0 0 0 25 0 8 0 882440876 370610176 5296 33554432000 4194304 4625164 140735580001488 18446744073709551615 4396709 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 90481 5297 351 106 0 85984 0
[pid=22957/tid=22958] ppid=22954 vsize=361924 CPUtime=0.29 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 849 0 0 0 29 0 0 0 25 0 8 0 882440877 370610176 5297 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=361924 CPUtime=0.29 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 806 0 0 0 29 0 0 0 25 0 8 0 882440877 370610176 5297 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988253381 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=361924 CPUtime=0.29 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1653 0 0 0 29 0 0 0 25 0 8 0 882440877 370610176 5297 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=361924 CPUtime=0.29 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 794 0 0 0 29 0 0 0 25 0 8 0 882440877 370610176 5297 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351981 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=361924 CPUtime=0.27 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 844 0 0 0 27 0 0 0 25 0 8 0 882440877 370610176 5297 33554432000 4194304 4625164 140735580001488 18446744073709551615 215988252044 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=361924 CPUtime=0.29 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 816 0 0 0 29 0 0 0 25 0 8 0 882440877 370610176 5298 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=361924 CPUtime=0.28 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 668 0 0 0 28 0 0 0 25 0 8 0 882440877 370610176 5298 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352208 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 2.32
Current children cumulated vsize (KiB) 361924

[startup+0.700606 s]
/proc/loadavg: 8.09 8.08 8.04 2/507 22957
/proc/meminfo: memFree=19812540/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=371052 CPUtime=5.52 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 15839 0 0 0 550 2 0 0 25 0 8 0 882440876 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 4356581 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 92763 9777 351 106 0 88266 0
[pid=22957/tid=22958] ppid=22954 vsize=371052 CPUtime=0.69 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1727 0 0 0 69 0 0 0 25 0 8 0 882440877 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=371052 CPUtime=0.68 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1792 0 0 0 68 0 0 0 25 0 8 0 882440877 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=371052 CPUtime=0.68 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 4131 0 0 0 68 0 0 0 25 0 8 0 882440877 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 4579390 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=371052 CPUtime=0.69 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1618 0 0 0 69 0 0 0 25 0 8 0 882440877 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=371052 CPUtime=0.67 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1780 0 0 0 67 0 0 0 25 0 8 0 882440877 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 4348867 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=371052 CPUtime=0.69 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1749 0 0 0 69 0 0 0 25 0 8 0 882440877 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 216004576283 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=371052 CPUtime=0.68 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1175 0 0 0 68 0 0 0 25 0 8 0 882440877 379957248 9777 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 5.52
Current children cumulated vsize (KiB) 371052

[startup+1.50047 s]
/proc/loadavg: 8.08 8.08 8.04 9/515 22967
/proc/meminfo: memFree=19762140/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=384368 CPUtime=11.89 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 27540 0 0 0 1183 6 0 0 25 0 8 0 882440876 393592832 16035 33554432000 4194304 4625164 140735580001488 18446744073709551615 4519388 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 96092 16035 351 106 0 91595 0
[pid=22957/tid=22958] ppid=22954 vsize=384368 CPUtime=1.49 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 2636 0 0 0 149 0 0 0 25 0 8 0 882440877 393592832 16035 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=384368 CPUtime=1.48 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 2724 0 0 0 148 0 0 0 18 0 8 0 882440877 393592832 16035 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=384368 CPUtime=1.48 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 8750 0 0 0 147 1 0 0 25 0 8 0 882440877 393592832 16035 33554432000 4194304 4625164 140735580001488 18446744073709551615 4552315 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=384368 CPUtime=1.48 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 3474 0 0 0 148 0 0 0 25 0 8 0 882440877 393592832 16035 33554432000 4194304 4625164 140735580001488 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=384368 CPUtime=1.47 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 2745 0 0 0 147 0 0 0 25 0 8 0 882440877 393592832 16035 33554432000 4194304 4625164 140735580001488 18446744073709551615 4349047 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=384368 CPUtime=1.48 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 2668 0 0 0 148 0 0 0 18 0 8 0 882440877 393592832 16036 33554432000 4194304 4625164 140735580001488 18446744073709551615 4468692 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=384368 CPUtime=1.47 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 1919 0 0 0 147 0 0 0 25 0 8 0 882440877 393592832 16036 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 11.89
Current children cumulated vsize (KiB) 384368

[startup+3.10119 s]
/proc/loadavg: 8.08 8.08 8.04 9/515 22967
/proc/meminfo: memFree=19735124/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=398604 CPUtime=24.67 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 46837 0 0 0 2459 8 0 0 25 0 8 0 882440876 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4338016 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 99651 25118 363 106 0 95154 0
[pid=22957/tid=22958] ppid=22954 vsize=398604 CPUtime=3.09 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 4254 0 0 0 309 0 0 0 25 0 8 0 882440877 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=398604 CPUtime=3.09 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 4047 0 0 0 308 1 0 0 21 0 8 0 882440877 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=398604 CPUtime=3.08 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 16679 0 0 0 306 2 0 0 25 0 8 0 882440877 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=398604 CPUtime=3.08 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 5457 0 0 0 308 0 0 0 25 0 8 0 882440877 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=398604 CPUtime=3.07 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 5596 0 0 0 306 1 0 0 25 0 8 0 882440877 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4349779 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=398604 CPUtime=3.08 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 4111 0 0 0 308 0 0 0 20 0 8 0 882440877 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=398604 CPUtime=3.07 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 3088 0 0 0 307 0 0 0 25 0 8 0 882440877 408170496 25118 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.67
Current children cumulated vsize (KiB) 398604

[startup+6.30063 s]
/proc/loadavg: 8.08 8.08 8.03 9/515 22967
/proc/meminfo: memFree=19668864/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=442584 CPUtime=50.21 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 78004 0 0 0 5007 14 0 0 25 0 8 0 882440876 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4396555 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 110646 39144 351 106 0 106149 0
[pid=22957/tid=22958] ppid=22954 vsize=442584 CPUtime=6.29 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 6349 0 0 0 629 0 0 0 25 0 8 0 882440877 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=442584 CPUtime=6.28 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 6198 0 0 0 627 1 0 0 25 0 8 0 882440877 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=442584 CPUtime=6.28 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 28507 0 0 0 623 5 0 0 25 0 8 0 882440877 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4548692 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=442584 CPUtime=6.29 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 8423 0 0 0 628 1 0 0 25 0 8 0 882440877 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=442584 CPUtime=6.27 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 8662 0 0 0 626 1 0 0 25 0 8 0 882440877 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4348851 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=442584 CPUtime=6.28 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 8440 0 0 0 627 1 0 0 25 0 8 0 882440877 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352285 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=442584 CPUtime=6.28 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 6392 0 0 0 627 1 0 0 25 0 8 0 882440877 453206016 39144 33554432000 4194304 4625164 140735580001488 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 0 0 0 0

################
# More data... #
################

/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 319140 0 0 0 466035 74 0 0 25 0 8 0 882440877 5058691072 1041652 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=4940128 CPUtime=4662.31 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 190240 0 0 0 466184 47 0 0 25 0 8 0 882440877 5058691072 1041652 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37214.8
Current children cumulated vsize (KiB) 4940128

[startup+4722.32 s]
/proc/loadavg: 8.05 8.05 8.02 10/515 23125
/proc/meminfo: memFree=15634552/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=4940128 CPUtime=37693.8 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 14398802 0 0 0 3766683 2698 0 0 25 0 8 0 882440876 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 1235032 1042746 550 106 0 1230535 0
[pid=22957/tid=22958] ppid=22954 vsize=4940128 CPUtime=4722.41 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 179785 0 0 0 472200 41 0 0 25 0 8 0 882440877 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=4940128 CPUtime=4722.4 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 225610 0 0 0 472190 50 0 0 25 0 8 0 882440877 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=4940128 CPUtime=4722.17 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 10180395 0 0 0 470406 1811 0 0 25 0 8 0 882440877 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=4940128 CPUtime=4722.39 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 230539 0 0 0 472187 52 0 0 25 0 8 0 882440877 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=4940128 CPUtime=4722.39 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 164985 0 0 0 472202 37 0 0 25 0 8 0 882440877 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=4940128 CPUtime=4721.09 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 319319 0 0 0 472035 74 0 0 25 0 8 0 882440877 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=4940128 CPUtime=4722.33 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 190252 0 0 0 472186 47 0 0 25 0 8 0 882440877 5058691072 1042746 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37693.8
Current children cumulated vsize (KiB) 4940128

[startup+4782.3 s]
/proc/loadavg: 8.05 8.05 8.02 9/515 23126
/proc/meminfo: memFree=15612800/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=5013140 CPUtime=38172.6 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 14612048 0 0 0 3814522 2737 0 0 25 0 8 0 882440876 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 1253285 1047558 455 106 0 1248788 0
[pid=22957/tid=22958] ppid=22954 vsize=5013140 CPUtime=4782.4 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 179794 0 0 0 478199 41 0 0 25 0 8 0 882440877 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=5013140 CPUtime=4782.38 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 225613 0 0 0 478188 50 0 0 25 0 8 0 882440877 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=5013140 CPUtime=4782.15 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 10344249 0 0 0 476373 1842 0 0 25 0 8 0 882440877 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4579396 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=5013140 CPUtime=4782.38 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 230548 0 0 0 478186 52 0 0 25 0 8 0 882440877 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=5013140 CPUtime=4782.37 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 164995 0 0 0 478200 37 0 0 25 0 8 0 882440877 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4460422 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=5013140 CPUtime=4781.06 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 319367 0 0 0 478032 74 0 0 25 0 8 0 882440877 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=5013140 CPUtime=4782.32 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 190262 0 0 0 478185 47 0 0 25 0 8 0 882440877 5133455360 1047558 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38172.6
Current children cumulated vsize (KiB) 5013140

[startup+4842.3 s]
/proc/loadavg: 8.05 8.05 8.02 9/515 23128
/proc/meminfo: memFree=15610380/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=5016308 CPUtime=38651.5 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 14787282 0 0 0 3862377 2771 0 0 25 0 8 0 882440876 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 1254077 1048858 473 106 0 1249580 0
[pid=22957/tid=22958] ppid=22954 vsize=5016308 CPUtime=4842.4 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 179804 0 0 0 484199 41 0 0 25 0 8 0 882440877 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=5016308 CPUtime=4842.38 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 225616 0 0 0 484188 50 0 0 25 0 8 0 882440877 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=5016308 CPUtime=4842.14 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 10519006 0 0 0 482338 1876 0 0 25 0 8 0 882440877 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=5016308 CPUtime=4842.38 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 230557 0 0 0 484186 52 0 0 25 0 8 0 882440877 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=5016308 CPUtime=4842.37 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 165004 0 0 0 484200 37 0 0 25 0 8 0 882440877 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=5016308 CPUtime=4841.04 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 319496 0 0 0 484030 74 0 0 25 0 8 0 882440877 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=5016308 CPUtime=4842.32 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 190271 0 0 0 484185 47 0 0 25 0 8 0 882440877 5136699392 1048858 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38651.5
Current children cumulated vsize (KiB) 5016308

[startup+4902.3 s]
/proc/loadavg: 8.20 8.10 8.03 9/515 23129
/proc/meminfo: memFree=15598924/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=5081844 CPUtime=39130.4 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 14972212 0 0 0 3910236 2807 0 0 25 0 8 0 882440876 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 1270461 1052018 612 106 0 1265964 0
[pid=22957/tid=22958] ppid=22954 vsize=5081844 CPUtime=4902.4 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 179820 0 0 0 490199 41 0 0 25 0 8 0 882440877 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=5081844 CPUtime=4902.38 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 225624 0 0 0 490188 50 0 0 25 0 8 0 882440877 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=5081844 CPUtime=4902.15 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 10703024 0 0 0 488303 1912 0 0 25 0 8 0 882440877 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=5081844 CPUtime=4902.38 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 230572 0 0 0 490186 52 0 0 25 0 8 0 882440877 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=5081844 CPUtime=4902.38 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 165125 0 0 0 490201 37 0 0 25 0 8 0 882440877 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4460436 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=5081844 CPUtime=4901.03 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 319917 0 0 0 490029 74 0 0 25 0 8 0 882440877 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=5081844 CPUtime=4902.32 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 190285 0 0 0 490185 47 0 0 25 0 8 0 882440877 5203808256 1052018 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39130.4
Current children cumulated vsize (KiB) 5081844

[startup+4962.3 s]
/proc/loadavg: 8.07 8.08 8.03 10/515 23131
/proc/meminfo: memFree=15611272/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=5016308 CPUtime=39609.3 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 15131648 0 0 0 3958096 2837 0 0 25 0 8 0 882440876 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4512012 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 1254077 1048474 359 106 0 1249580 0
[pid=22957/tid=22958] ppid=22954 vsize=5016308 CPUtime=4962.4 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 179829 0 0 0 496199 41 0 0 25 0 8 0 882440877 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=5016308 CPUtime=4962.39 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 225628 0 0 0 496189 50 0 0 25 0 8 0 882440877 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=5016308 CPUtime=4962.14 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 10861987 0 0 0 494273 1941 0 0 25 0 8 0 882440877 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4552399 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=5016308 CPUtime=4962.38 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 230591 0 0 0 496186 52 0 0 25 0 8 0 882440877 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=5016308 CPUtime=4962.38 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 165134 0 0 0 496201 37 0 0 25 0 8 0 882440877 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=5016308 CPUtime=4961.01 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 320302 0 0 0 496027 74 0 0 25 0 8 0 882440877 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=5016308 CPUtime=4962.32 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 190297 0 0 0 496185 47 0 0 25 0 8 0 882440877 5136699392 1048474 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39609.3
Current children cumulated vsize (KiB) 5016308



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5011.3 s]
/proc/loadavg: 8.03 8.06 8.02 9/515 23132
/proc/meminfo: memFree=15599948/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=5016308 CPUtime=40000.6 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4202496 15267389 0 0 0 3997193 2863 0 0 25 0 8 0 882440876 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/22957/statm: 1254077 1051393 352 106 0 1249580 0
[pid=22957/tid=22958] ppid=22954 vsize=5016308 CPUtime=5011.42 cores=0-7
/proc/22957/task/22958/stat : 22958 (SArTagnan) R 22954 22957 22892 0 -1 4202560 179836 0 0 0 501101 41 0 0 25 0 8 0 882440877 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22957/tid=22959] ppid=22954 vsize=5016308 CPUtime=5011.41 cores=0-7
/proc/22957/task/22959/stat : 22959 (SArTagnan) R 22954 22957 22892 0 -1 4202560 225632 0 0 0 501091 50 0 0 25 0 8 0 882440877 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=22957/tid=22960] ppid=22954 vsize=5016308 CPUtime=5011.16 cores=0-7
/proc/22957/task/22960/stat : 22960 (SArTagnan) R 22954 22957 22892 0 -1 4202560 10997429 0 0 0 499149 1967 0 0 25 0 8 0 882440877 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=22957/tid=22961] ppid=22954 vsize=5016308 CPUtime=5011.4 cores=0-7
/proc/22957/task/22961/stat : 22961 (SArTagnan) R 22954 22957 22892 0 -1 4202560 230602 0 0 0 501088 52 0 0 25 0 8 0 882440877 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=22957/tid=22962] ppid=22954 vsize=5016308 CPUtime=5011.4 cores=0-7
/proc/22957/task/22962/stat : 22962 (SArTagnan) R 22954 22957 22892 0 -1 4202560 165143 0 0 0 501103 37 0 0 25 0 8 0 882440877 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=22957/tid=22963] ppid=22954 vsize=5016308 CPUtime=5010.02 cores=0-7
/proc/22957/task/22963/stat : 22963 (SArTagnan) R 22954 22957 22892 0 -1 4202560 320522 0 0 0 500928 74 0 0 25 0 8 0 882440877 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=22957/tid=22964] ppid=22954 vsize=5016308 CPUtime=5011.34 cores=0-7
/proc/22957/task/22964/stat : 22964 (SArTagnan) R 22954 22957 22892 0 -1 4202560 190311 0 0 0 501087 47 0 0 25 0 8 0 882440877 5136699392 1051393 33554432000 4194304 4625164 140735580001488 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 5016308

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5011.43 s]
/proc/loadavg: 8.03 8.06 8.02 9/515 23132
/proc/meminfo: memFree=15599948/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=0 CPUtime=40000.7 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4203524 15267389 0 0 0 3997201 2873 0 0 25 0 1 0 882440876 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/22957/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

[startup+5011.5 s]
/proc/loadavg: 8.03 8.06 8.02 2/509 23133
/proc/meminfo: memFree=19311440/32950940 swapFree=67111364/67111528
[pid=22957] ppid=22954 vsize=0 CPUtime=40000.8 cores=0-7
/proc/22957/stat : 22957 (SArTagnan) R 22954 22957 22892 0 -1 4203524 15267389 0 0 0 3997201 2882 0 0 25 0 1 0 882440876 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 3 0 0 0
/proc/22957/statm: 0 0 0 0 0 0 0
Current children cumulated CPU time (s) 40000.8
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5011.52
CPU time (s): 40000.8
CPU user time (s): 39972
CPU system time (s): 28.8336
CPU usage (%): 798.177
Max. virtual memory (cumulated for all children) (KiB): 5219144

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39972
system time used= 28.8336
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 15267389
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 921
involuntary context switches= 176098

runsolver used 23.1865 second user time and 64.9831 second system time

The end

Launcher Data 

Begin job on node117 at 2011-06-01 11:55:01
IDJOB=3438706
IDBENCH=83156
IDSOLVER=1936
FILE ID=node117/3438706-1306922101
RUNJOBID= node117-1306922035-22906
PBS_JOBID= 13488552
Free space on /tmp= 73628 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n190-s29985700-unsat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438706-1306922101/watcher-3438706-1306922101 -o /tmp/evaluation-result-3438706-1306922101/solver-3438706-1306922101 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438706-1306922101.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f2cf61fc47972fb7e89be65d1a83332e
RANDOM SEED=1088935728

node117.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.849
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5267.44
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950940 kB
MemFree:      19812820 kB
Buffers:        313836 kB
Cached:        2658532 kB
SwapCached:          0 kB
Active:        9972108 kB
Inactive:      2785356 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950940 kB
LowFree:      19812820 kB
SwapTotal:    67111528 kB
SwapFree:     67111364 kB
Dirty:             392 kB
Writeback:           0 kB
AnonPages:     9784808 kB
Mapped:          41568 kB
Slab:           250848 kB
PageTables:      57836 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586996 kB
Committed_AS: 149469568 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73628 MiB
End job on node117 at 2011-06-01 13:18:34