Trace number 3438661

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.7 5009.53

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n120-s55656844-unsat.cnf
MD5SUM4877ca9f3fe317c84d3087c8296a2a0b
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark716.736
Satisfiable
(Un)Satisfiability was proved
Number of variables121
Number of clauses252
Sum of the clauses size756
Maximum clause length3
Minimum clause length3
Number of clauses of size 10
Number of clauses of size 20
Number of clauses of size 3252
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438661-1306921268/watcher-3438661-1306921268 -o /tmp/evaluation-result-3438661-1306921268/solver-3438661-1306921268 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438661-1306921268.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.08 8.11 8.00 2/436 1629
/proc/meminfo: memFree=30255536/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 474 0 0 0 0 0 0 0 23 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895546865 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 22581 410 300 106 0 18084 0
[pid=1629/tid=1630] ppid=1627 vsize=90324 CPUtime=0.01 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1 0 0 0 1 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387954 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895546865 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895546865 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895546865 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387948 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=90324 CPUtime=0.01 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 1 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387954 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=90324 CPUtime=0.01 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 1 0 0 0 25 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387958 0 0 4096 2 0 0 0 -1 1 0 0 0

[startup+0.0202921 s]
/proc/loadavg: 8.08 8.11 8.00 2/436 1629
/proc/meminfo: memFree=30255536/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=90324 CPUtime=0.08 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 474 0 0 0 8 0 0 0 23 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387948 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 22581 410 300 106 0 18084 0
[pid=1629/tid=1630] ppid=1627 vsize=90324 CPUtime=0.01 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1 0 0 0 1 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387954 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895546865 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895546865 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=90324 CPUtime=0 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 0 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 231907288118 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=90324 CPUtime=0.01 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 1 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387954 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=90324 CPUtime=0.01 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 1 0 0 0 24 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387944 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=90324 CPUtime=0.01 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 0 0 0 0 1 0 0 0 25 0 8 0 882357487 92491776 410 33554432000 4194304 4625164 140735220666832 18446744073709551615 46954217387954 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.08
Current children cumulated vsize (KiB) 90324

[startup+0.10122 s]
/proc/loadavg: 8.08 8.11 8.00 2/436 1629
/proc/meminfo: memFree=30255536/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=354736 CPUtime=0.64 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 2481 0 0 0 64 0 0 0 25 0 8 0 882357487 363249664 1868 33554432000 4194304 4625164 140735220666832 18446744073709551615 4519268 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 88684 1870 351 106 0 84187 0
[pid=1629/tid=1630] ppid=1627 vsize=354736 CPUtime=0.08 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 157 0 0 0 8 0 0 0 25 0 8 0 882357487 363249664 1870 33554432000 4194304 4625164 140735220666832 18446744073709551615 4340391 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=354736 CPUtime=0.02 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 55 0 0 0 2 0 0 0 24 0 8 0 882357487 363249664 1870 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352539 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=354736 CPUtime=0.07 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 421 0 0 0 7 0 0 0 18 0 8 0 882357487 363249664 1870 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=354736 CPUtime=0.07 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 271 0 0 0 7 0 0 0 18 0 8 0 882357487 363249664 1870 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352665 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=354736 CPUtime=0.09 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 248 0 0 0 9 0 0 0 25 0 8 0 882357487 363249664 1870 33554432000 4194304 4625164 140735220666832 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=354736 CPUtime=0.09 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 236 0 0 0 9 0 0 0 25 0 8 0 882357487 363249664 1870 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=354736 CPUtime=0.09 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 352 0 0 0 9 0 0 0 18 0 8 0 882357487 363249664 1870 33554432000 4194304 4625164 140735220666832 18446744073709551615 4291937 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 0.64
Current children cumulated vsize (KiB) 354736

[startup+0.301184 s]
/proc/loadavg: 8.08 8.11 8.00 2/436 1629
/proc/meminfo: memFree=30255536/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=360508 CPUtime=2.24 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 6903 0 0 0 223 1 0 0 25 0 8 0 882357487 369160192 4744 33554432000 4194304 4625164 140735220666832 18446744073709551615 4513077 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 90127 4744 351 106 0 85630 0
[pid=1629/tid=1630] ppid=1627 vsize=360508 CPUtime=0.28 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 666 0 0 0 28 0 0 0 25 0 8 0 882357487 369160192 4744 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352765 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=360508 CPUtime=0.22 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 587 0 0 0 22 0 0 0 25 0 8 0 882357487 369160192 4744 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=360508 CPUtime=0.27 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1322 0 0 0 27 0 0 0 18 0 8 0 882357487 369160192 4746 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895145468 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=360508 CPUtime=0.27 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 765 0 0 0 27 0 0 0 18 0 8 0 882357487 369160192 4746 33554432000 4194304 4625164 140735220666832 18446744073709551615 4462569 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=360508 CPUtime=0.29 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 896 0 0 0 29 0 0 0 25 0 8 0 882357487 369160192 4746 33554432000 4194304 4625164 140735220666832 18446744073709551615 4349067 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=360508 CPUtime=0.29 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 923 0 0 0 29 0 0 0 25 0 8 0 882357487 369160192 4746 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=360508 CPUtime=0.29 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 651 0 0 0 29 0 0 0 18 0 8 0 882357487 369160192 4746 33554432000 4194304 4625164 140735220666832 18446744073709551615 4339865 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 2.24
Current children cumulated vsize (KiB) 360508

[startup+0.701111 s]
/proc/loadavg: 8.08 8.11 8.00 2/436 1629
/proc/meminfo: memFree=30255536/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=368664 CPUtime=5.44 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 13898 0 0 0 542 2 0 0 25 0 8 0 882357487 377511936 8481 33554432000 4194304 4625164 140735220666832 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 92166 8481 351 106 0 87669 0
[pid=1629/tid=1630] ppid=1627 vsize=368664 CPUtime=0.68 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1490 0 0 0 68 0 0 0 25 0 8 0 882357487 377511936 8481 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=368664 CPUtime=0.62 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1401 0 0 0 62 0 0 0 25 0 8 0 882357487 377511936 8481 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351827 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=368664 CPUtime=0.66 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 3549 0 0 0 66 0 0 0 18 0 8 0 882357487 377511936 8481 33554432000 4194304 4625164 140735220666832 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=368664 CPUtime=0.67 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1626 0 0 0 67 0 0 0 18 0 8 0 882357487 377511936 8481 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351909 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=368664 CPUtime=0.69 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1470 0 0 0 69 0 0 0 25 0 8 0 882357487 377511936 8483 33554432000 4194304 4625164 140735220666832 18446744073709551615 4348870 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=368664 CPUtime=0.69 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1525 0 0 0 69 0 0 0 25 0 8 0 882357487 377511936 8483 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351989 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=368664 CPUtime=0.69 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 1416 0 0 0 69 0 0 0 18 0 8 0 882357487 377511936 8483 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351804 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 5.44
Current children cumulated vsize (KiB) 368664

[startup+1.50096 s]
/proc/loadavg: 8.07 8.11 8.00 9/444 1637
/proc/meminfo: memFree=30213008/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=380156 CPUtime=11.83 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 24859 0 0 0 1179 4 0 0 25 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 4511982 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 95039 13307 351 106 0 90542 0
[pid=1629/tid=1630] ppid=1627 vsize=380156 CPUtime=1.47 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 2295 0 0 0 147 0 0 0 25 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352296 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=380156 CPUtime=1.4 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 2181 0 0 0 140 0 0 0 25 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=380156 CPUtime=1.47 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 7688 0 0 0 146 1 0 0 19 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 231895157722 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=380156 CPUtime=1.47 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 2530 0 0 0 147 0 0 0 20 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=380156 CPUtime=1.48 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 2957 0 0 0 148 0 0 0 25 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 4436924 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=380156 CPUtime=1.49 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 3054 0 0 0 149 0 0 0 25 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=380156 CPUtime=1.49 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 2189 0 0 0 149 0 0 0 20 0 8 0 882357487 389279744 13307 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 11.83
Current children cumulated vsize (KiB) 380156

[startup+3.10067 s]
/proc/loadavg: 8.07 8.11 8.00 9/444 1637
/proc/meminfo: memFree=30192016/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=396720 CPUtime=24.61 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 43753 0 0 0 2454 7 0 0 25 0 8 0 882357487 406241280 20874 33554432000 4194304 4625164 140735220666832 18446744073709551615 4512156 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 99180 20875 354 106 0 94683 0
[pid=1629/tid=1630] ppid=1627 vsize=396720 CPUtime=3.07 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 3700 0 0 0 307 0 0 0 25 0 8 0 882357487 406241280 20876 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352310 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=396720 CPUtime=2.98 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 4751 0 0 0 298 0 0 0 25 0 8 0 882357487 406241280 20877 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=396720 CPUtime=3.07 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 14387 0 0 0 305 2 0 0 25 0 8 0 882357487 406241280 20877 33554432000 4194304 4625164 140735220666832 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=396720 CPUtime=3.07 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 4058 0 0 0 307 0 0 0 25 0 8 0 882357487 406241280 20877 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351989 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=396720 CPUtime=3.08 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 4746 0 0 0 308 0 0 0 25 0 8 0 882357487 406241280 20878 33554432000 4194304 4625164 140735220666832 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=396720 CPUtime=3.08 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 4731 0 0 0 308 0 0 0 25 0 8 0 882357487 406241280 20880 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=396720 CPUtime=3.09 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 4658 0 0 0 308 1 0 0 25 0 8 0 882357487 406241280 20880 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 24.61
Current children cumulated vsize (KiB) 396720

[startup+6.30107 s]
/proc/loadavg: 8.07 8.10 8.00 9/444 1637
/proc/meminfo: memFree=30140448/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=422808 CPUtime=50.16 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 71722 0 0 0 5005 11 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4396755 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 105702 31739 385 106 0 101205 0
[pid=1629/tid=1630] ppid=1627 vsize=422808 CPUtime=6.27 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 5817 0 0 0 627 0 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=422808 CPUtime=6.13 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 7439 0 0 0 613 0 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351792 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=422808 CPUtime=6.27 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 28673 0 0 0 623 4 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=422808 CPUtime=6.27 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 6550 0 0 0 627 0 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=422808 CPUtime=6.29 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 7297 0 0 0 628 1 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4348804 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=422808 CPUtime=6.28 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 5164 0 0 0 628 0 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=422808 CPUtime=6.29 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 6918 0 0 0 628 1 0 0 25 0 8 0 882357487 432955392 31739 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0

################
# More data... #
################

Current children cumulated vsize (KiB) 4918036

[startup+4722.32 s]
/proc/loadavg: 8.08 8.08 8.03 9/444 1777
/proc/meminfo: memFree=26303600/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=4922672 CPUtime=37708.6 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 15938742 0 0 0 3768068 2795 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 1230668 985694 357 106 0 1226171 0
[pid=1629/tid=1630] ppid=1627 vsize=4922672 CPUtime=4721.24 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 230247 0 0 0 472078 46 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=4922672 CPUtime=4653.67 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 314991 0 0 0 465305 62 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=4922672 CPUtime=4722.4 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 11195498 0 0 0 470269 1971 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=4922672 CPUtime=4722.44 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 278068 0 0 0 472185 59 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=4922672 CPUtime=4722.23 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 128273 0 0 0 472192 31 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=4922672 CPUtime=4721.85 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 82151 0 0 0 472151 34 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=4922672 CPUtime=4722.29 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 305748 0 0 0 472166 63 0 0 25 0 8 0 882357487 5040816128 985694 33554432000 4194304 4625164 140735220666832 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 37708.6
Current children cumulated vsize (KiB) 4922672

[startup+4782.3 s]
/proc/loadavg: 8.06 8.07 8.02 9/444 1778
/proc/meminfo: memFree=26246644/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=5070220 CPUtime=38187.6 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 16242132 0 0 0 3815919 2843 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 1267555 1001042 460 106 0 1263058 0
[pid=1629/tid=1630] ppid=1627 vsize=5070220 CPUtime=4781.2 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 305964 0 0 0 478066 54 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=5070220 CPUtime=4712.79 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 316060 0 0 0 471216 63 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=5070220 CPUtime=4782.38 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 11355160 0 0 0 476240 1998 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=5070220 CPUtime=4782.43 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 278111 0 0 0 478184 59 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=5070220 CPUtime=4782.21 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 128589 0 0 0 478190 31 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=5070220 CPUtime=4781.83 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 82439 0 0 0 478149 34 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4339628 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=5070220 CPUtime=4782.27 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 305784 0 0 0 478164 63 0 0 25 0 8 0 882357487 5191905280 1001042 33554432000 4194304 4625164 140735220666832 18446744073709551615 4330787 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38187.6
Current children cumulated vsize (KiB) 5070220

[startup+4842.3 s]
/proc/loadavg: 8.06 8.07 8.02 9/444 1789
/proc/meminfo: memFree=26236712/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=5073380 CPUtime=38666.7 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 16453212 0 0 0 3863788 2881 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 1268345 1002382 351 106 0 1263848 0
[pid=1629/tid=1630] ppid=1627 vsize=5073380 CPUtime=4841.19 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 306231 0 0 0 484064 55 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=5073380 CPUtime=4771.9 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 316201 0 0 0 477127 63 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=5073380 CPUtime=4842.38 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 11518033 0 0 0 482210 2028 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=5073380 CPUtime=4842.43 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 278149 0 0 0 484184 59 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=5073380 CPUtime=4842.21 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 128629 0 0 0 484190 31 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4348826 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=5073380 CPUtime=4841.83 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 82460 0 0 0 484148 35 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=5073380 CPUtime=4842.26 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 305821 0 0 0 484163 63 0 0 25 0 8 0 882357487 5195141120 1002382 33554432000 4194304 4625164 140735220666832 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 38666.7
Current children cumulated vsize (KiB) 5073380

[startup+4902.3 s]
/proc/loadavg: 8.13 8.09 8.03 9/444 1790
/proc/meminfo: memFree=26217032/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=5073380 CPUtime=39145.8 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 16619888 0 0 0 3911670 2913 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 1268345 1007083 393 106 0 1263848 0
[pid=1629/tid=1630] ppid=1627 vsize=5073380 CPUtime=4901.18 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 306347 0 0 0 490063 55 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=5073380 CPUtime=4831.03 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 316299 0 0 0 483040 63 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=5073380 CPUtime=4902.38 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 11645081 0 0 0 488187 2051 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=5073380 CPUtime=4902.43 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 278797 0 0 0 490184 59 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=5073380 CPUtime=4902.21 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 128721 0 0 0 490190 31 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4348867 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=5073380 CPUtime=4901.83 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 82460 0 0 0 490148 35 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=5073380 CPUtime=4902.27 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 305868 0 0 0 490163 64 0 0 25 0 8 0 882357487 5195141120 1007083 33554432000 4194304 4625164 140735220666832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39145.8
Current children cumulated vsize (KiB) 5073380

[startup+4962.3 s]
/proc/loadavg: 8.12 8.09 8.03 9/444 1792
/proc/meminfo: memFree=26203536/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=5073380 CPUtime=39624.9 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 16847320 0 0 0 3959541 2954 0 0 25 0 8 0 882357487 5195141120 1010285 33554432000 4194304 4625164 140735220666832 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 1268345 1010286 372 106 0 1263848 0
[pid=1629/tid=1630] ppid=1627 vsize=5073380 CPUtime=4961.16 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 306493 0 0 0 496061 55 0 0 25 0 8 0 882357487 5195141120 1010286 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=5073380 CPUtime=4890.15 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 316371 0 0 0 488952 63 0 0 25 0 8 0 882357487 5195141120 1010286 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=5073380 CPUtime=4962.39 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 11792830 0 0 0 494160 2079 0 0 25 0 8 0 882357487 5195141120 1010286 33554432000 4194304 4625164 140735220666832 18446744073709551615 4579396 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=5073380 CPUtime=4962.43 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 278839 0 0 0 496184 59 0 0 25 0 8 0 882357487 5195141120 1010286 33554432000 4194304 4625164 140735220666832 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=5073380 CPUtime=4962.21 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 128961 0 0 0 496190 31 0 0 25 0 8 0 882357487 5195141120 1010286 33554432000 4194304 4625164 140735220666832 18446744073709551615 4348864 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=5073380 CPUtime=4961.83 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 82520 0 0 0 496148 35 0 0 25 0 8 0 882357487 5195141120 1010286 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=5073380 CPUtime=4962.27 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 305910 0 0 0 496163 64 0 0 25 0 8 0 882357487 5195141120 1010286 33554432000 4194304 4625164 140735220666832 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 39624.9
Current children cumulated vsize (KiB) 5073380



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.3 s]
/proc/loadavg: 8.06 8.07 8.02 10/444 1793
/proc/meminfo: memFree=26177044/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=5105640 CPUtime=40000.4 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) R 1627 1629 1412 0 -1 4202496 17022055 0 0 0 3997056 2984 0 0 25 0 8 0 882357487 5228175360 1017212 33554432000 4194304 4625164 140735220666832 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 1276410 1017212 364 106 0 1271913 0
[pid=1629/tid=1630] ppid=1627 vsize=5105640 CPUtime=5008.18 cores=0-7
/proc/1629/task/1630/stat : 1630 (SArTagnan) R 1627 1629 1412 0 -1 4202560 306585 0 0 0 500763 55 0 0 25 0 8 0 882357487 5228175360 1017217 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=5105640 CPUtime=4936.48 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4202560 316629 0 0 0 493585 63 0 0 25 0 8 0 882357487 5228175360 1017217 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=1629/tid=1632] ppid=1627 vsize=5105640 CPUtime=5009.42 cores=0-7
/proc/1629/task/1632/stat : 1632 (SArTagnan) R 1627 1629 1412 0 -1 4202560 11906107 0 0 0 498843 2099 0 0 25 0 8 0 882357487 5228175360 1017217 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=1629/tid=1633] ppid=1627 vsize=5105640 CPUtime=5009.47 cores=0-7
/proc/1629/task/1633/stat : 1633 (SArTagnan) R 1627 1629 1412 0 -1 4202560 278873 0 0 0 500888 59 0 0 25 0 8 0 882357487 5228175360 1017217 33554432000 4194304 4625164 140735220666832 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=1629/tid=1634] ppid=1627 vsize=5105640 CPUtime=5009.25 cores=0-7
/proc/1629/task/1634/stat : 1634 (SArTagnan) R 1627 1629 1412 0 -1 4202560 129043 0 0 0 500894 31 0 0 25 0 8 0 882357487 5228175360 1017217 33554432000 4194304 4625164 140735220666832 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=1629/tid=1635] ppid=1627 vsize=5105640 CPUtime=5008.87 cores=0-7
/proc/1629/task/1635/stat : 1635 (SArTagnan) R 1627 1629 1412 0 -1 4202560 82762 0 0 0 500852 35 0 0 25 0 8 0 882357487 5228175360 1017217 33554432000 4194304 4625164 140735220666832 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=1629/tid=1636] ppid=1627 vsize=5105640 CPUtime=5009.31 cores=0-7
/proc/1629/task/1636/stat : 1636 (SArTagnan) R 1627 1629 1412 0 -1 4202560 305944 0 0 0 500867 64 0 0 25 0 8 0 882357487 5228175360 1017217 33554432000 4194304 4625164 140735220666832 18446744073709551615 4330847 0 0 4096 2 0 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 5105640

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.44 s]
/proc/loadavg: 8.06 8.07 8.02 10/444 1793
/proc/meminfo: memFree=26177044/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=0 CPUtime=40000.6 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) Z 1627 1629 1412 0 -1 4203548 17022060 0 0 0 3997069 2994 0 0 25 0 2 0 882357487 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 0 0 0 0 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=0 CPUtime=4936.58 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4203588 316629 0 0 0 493585 73 0 0 25 0 2 0 882357487 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

[startup+5009.5 s]
/proc/loadavg: 8.06 8.07 8.02 10/444 1793
/proc/meminfo: memFree=26177044/32950928 swapFree=52246700/67111528
[pid=1629] ppid=1627 vsize=0 CPUtime=40000.7 cores=0-7
/proc/1629/stat : 1629 (SArTagnan) Z 1627 1629 1412 0 -1 4203548 17022060 0 0 0 3997069 3001 0 0 25 0 2 0 882357487 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 0 0 0 0
/proc/1629/statm: 0 0 0 0 0 0 0
[pid=1629/tid=1631] ppid=1627 vsize=0 CPUtime=4936.64 cores=0-7
/proc/1629/task/1631/stat : 1631 (SArTagnan) R 1627 1629 1412 0 -1 4203588 316629 0 0 0 493585 79 0 0 25 0 2 0 882357487 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 1 0 0 0
Current children cumulated CPU time (s) 40000.7
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5009.53
CPU time (s): 40000.7
CPU user time (s): 39970.7
CPU system time (s): 30.0414
CPU usage (%): 798.493
Max. virtual memory (cumulated for all children) (KiB): 5341380

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39970.7
system time used= 30.0414
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 17022060
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4415
involuntary context switches= 169031

runsolver used 19.897 second user time and 53.0549 second system time

The end

Launcher Data 

Begin job on node114 at 2011-06-01 11:41:08
IDJOB=3438661
IDBENCH=83147
IDSOLVER=1936
FILE ID=node114/3438661-1306921268
RUNJOBID= node114-1306916256-1426
PBS_JOBID= 13488515
Free space on /tmp= 73652 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n120-s55656844-unsat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438661-1306921268/watcher-3438661-1306921268 -o /tmp/evaluation-result-3438661-1306921268/solver-3438661-1306921268 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438661-1306921268.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 4877ca9f3fe317c84d3087c8296a2a0b
RANDOM SEED=739206748

node114.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.15
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.848
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30255932 kB
Buffers:        167400 kB
Cached:         224500 kB
SwapCached:      10660 kB
Active:        2160376 kB
Inactive:       267048 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30255932 kB
SwapTotal:    67111528 kB
SwapFree:     52246700 kB
Dirty:             408 kB
Writeback:           4 kB
AnonPages:     2032996 kB
Mapped:          20644 kB
Slab:           135356 kB
PageTables:      61980 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 108158404 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73648 MiB
End job on node114 at 2011-06-01 13:04:39