Trace number 3438656

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.5 5009.81

General information on the benchmark

Namecrafted/spence/sgen/
sgen3-n220-s24359701-sat.cnf
MD5SUMceff918324677440f95867755c4e7133
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark6.70998
Satisfiable
(Un)Satisfiability was proved
Number of variables220
Number of clauses528
Sum of the clauses size1320
Maximum clause length5
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2440
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 588
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3438656-1306921252/watcher-3438656-1306921252 -o /tmp/evaluation-result-3438656-1306921252/solver-3438656-1306921252 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3438656-1306921252.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.44 7.95 7.96 2/439 28359
/proc/meminfo: memFree=32293308/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=90456 CPUtime=0.04 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 486 0 0 0 4 0 0 0 25 0 8 0 803723962 92626944 423 33554432000 4194304 4625164 140737344502752 18446744073709551615 46951506331568 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 22614 423 300 106 0 18117 0
[pid=28359/tid=28360] ppid=28357 vsize=155992 CPUtime=0 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) S 28357 28359 28333 0 -1 4202560 9 0 0 0 0 0 0 0 25 0 8 0 803723962 159735808 482 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594971630 0 0 4096 2 18446604445874481856 0 0 -1 4 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=155992 CPUtime=0 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 2 0 0 0 0 0 0 0 25 0 8 0 803723962 159735808 482 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594971630 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=155992 CPUtime=0 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 18 0 0 0 0 0 0 0 25 0 8 0 803723962 159735808 482 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594527429 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=155992 CPUtime=0.01 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) S 28357 28359 28333 0 -1 4202560 2 0 0 0 1 0 0 0 25 0 8 0 803723962 159735808 482 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594971630 0 0 4096 2 18446604445874481856 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=155992 CPUtime=0.01 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) S 28357 28359 28333 0 -1 4202560 8 0 0 0 1 0 0 0 25 0 8 0 803723962 159735808 486 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594971630 0 0 4096 2 18446604445874481856 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=155992 CPUtime=0 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 14 0 0 0 0 0 0 0 25 0 8 0 803723962 159735808 486 33554432000 4194304 4625164 140737344502752 18446744073709551615 235610850331 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=155992 CPUtime=0.01 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 16 0 0 0 1 0 0 0 25 0 8 0 803723962 159735808 486 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594563339 0 0 4096 2 0 0 0 -1 4 0 0 0

[startup+0.0443689 s]
/proc/loadavg: 7.44 7.95 7.96 2/439 28359
/proc/meminfo: memFree=32293308/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=418992 CPUtime=0.21 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 1310 0 0 0 21 0 0 0 25 0 8 0 803723962 429047808 1224 33554432000 4194304 4625164 140737344502752 18446744073709551615 4512315 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 104748 1224 352 106 0 100251 0
[pid=28359/tid=28360] ppid=28357 vsize=418992 CPUtime=0.02 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 75 0 0 0 2 0 0 0 25 0 8 0 803723962 429047808 1224 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594971630 0 0 4096 2 0 0 0 -1 6 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=418992 CPUtime=0.02 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 92 0 0 0 2 0 0 0 25 0 8 0 803723962 429047808 1224 33554432000 4194304 4625164 140737344502752 18446744073709551615 4288861 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=418992 CPUtime=0.02 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 115 0 0 0 2 0 0 0 25 0 8 0 803723962 429047808 1224 33554432000 4194304 4625164 140737344502752 18446744073709551615 235610850331 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=418992 CPUtime=0.03 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 108 0 0 0 3 0 0 0 25 0 8 0 803723962 429047808 1224 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352056 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=418992 CPUtime=0.03 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 81 0 0 0 3 0 0 0 25 0 8 0 803723962 429047808 1224 33554432000 4194304 4625164 140737344502752 18446744073709551615 4459376 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=418992 CPUtime=0.01 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 98 0 0 0 1 0 0 0 25 0 8 0 803723962 429047808 1224 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594971630 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=418992 CPUtime=0.02 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 66 0 0 0 2 0 0 0 25 0 8 0 803723962 429047808 1225 33554432000 4194304 4625164 140737344502752 18446744073709551615 4465165 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 0.21
Current children cumulated vsize (KiB) 418992

[startup+0.168348 s]
/proc/loadavg: 7.44 7.95 7.96 2/439 28359
/proc/meminfo: memFree=32293308/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=422500 CPUtime=1.16 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 3983 0 0 0 116 0 0 0 25 0 8 0 803723962 432640000 2774 33554432000 4194304 4625164 140737344502752 18446744073709551615 4508828 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 105625 2774 352 106 0 101128 0
[pid=28359/tid=28360] ppid=28357 vsize=422500 CPUtime=0.12 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 309 0 0 0 12 0 0 0 25 0 8 0 803723962 432640000 2775 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352816 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=422500 CPUtime=0.15 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 448 0 0 0 15 0 0 0 25 0 8 0 803723962 432640000 2776 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352827 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=422500 CPUtime=0.14 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 743 0 0 0 14 0 0 0 25 0 8 0 803723962 432640000 2776 33554432000 4194304 4625164 140737344502752 18446744073709551615 4559735 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=422500 CPUtime=0.15 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 459 0 0 0 15 0 0 0 25 0 8 0 803723962 432640000 2776 33554432000 4194304 4625164 140737344502752 18446744073709551615 235594527429 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=422500 CPUtime=0.16 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 406 0 0 0 16 0 0 0 25 0 8 0 803723962 432640000 2776 33554432000 4194304 4625164 140737344502752 18446744073709551615 4348771 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=422500 CPUtime=0.12 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 376 0 0 0 12 0 0 0 25 0 8 0 803723962 432640000 2776 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=422500 CPUtime=0.14 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 434 0 0 0 14 0 0 0 25 0 8 0 803723962 432640000 2776 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 1.16
Current children cumulated vsize (KiB) 422500

[startup+0.300328 s]
/proc/loadavg: 7.44 7.95 7.96 2/439 28359
/proc/meminfo: memFree=32293308/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=425040 CPUtime=2.22 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 6786 0 0 0 222 0 0 0 25 0 8 0 803723962 435240960 4392 33554432000 4194304 4625164 140737344502752 18446744073709551615 4511997 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 106260 4392 352 106 0 101763 0
[pid=28359/tid=28360] ppid=28357 vsize=425040 CPUtime=0.25 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 632 0 0 0 25 0 0 0 25 0 8 0 803723962 435240960 4394 33554432000 4194304 4625164 140737344502752 18446744073709551615 4465309 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=425040 CPUtime=0.28 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 726 0 0 0 28 0 0 0 25 0 8 0 803723962 435240960 4394 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339865 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=425040 CPUtime=0.28 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 1547 0 0 0 28 0 0 0 25 0 8 0 803723962 435240960 4394 33554432000 4194304 4625164 140737344502752 18446744073709551615 4562846 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=425040 CPUtime=0.29 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 694 0 0 0 29 0 0 0 25 0 8 0 803723962 435240960 4394 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=425040 CPUtime=0.29 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 921 0 0 0 29 0 0 0 25 0 8 0 803723962 435240960 4396 33554432000 4194304 4625164 140737344502752 18446744073709551615 4328133 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=425040 CPUtime=0.25 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 616 0 0 0 25 0 0 0 25 0 8 0 803723962 435240960 4396 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352728 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=425040 CPUtime=0.27 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 668 0 0 0 27 0 0 0 25 0 8 0 803723962 435240960 4397 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 2.22
Current children cumulated vsize (KiB) 425040

[startup+0.701258 s]
/proc/loadavg: 7.44 7.95 7.96 2/439 28359
/proc/meminfo: memFree=32293308/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=433012 CPUtime=5.43 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 14274 0 0 0 541 2 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4512747 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 108253 8229 352 106 0 103756 0
[pid=28359/tid=28360] ppid=28357 vsize=433012 CPUtime=0.65 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 1432 0 0 0 65 0 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351972 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=433012 CPUtime=0.68 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 1474 0 0 0 68 0 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=433012 CPUtime=0.67 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 4084 0 0 0 67 0 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=433012 CPUtime=0.68 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 1495 0 0 0 68 0 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4466961 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=433012 CPUtime=0.69 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 1545 0 0 0 69 0 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4349705 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=433012 CPUtime=0.65 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 1438 0 0 0 65 0 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4335264 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=433012 CPUtime=0.67 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 1449 0 0 0 67 0 0 0 25 0 8 0 803723962 443404288 8229 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339536 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 5.43
Current children cumulated vsize (KiB) 433012

[startup+1.50112 s]
/proc/loadavg: 7.44 7.95 7.96 9/447 28367
/proc/meminfo: memFree=32253116/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=442728 CPUtime=11.81 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 25974 0 0 0 1177 4 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4512315 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 110682 13419 352 106 0 106185 0
[pid=28359/tid=28360] ppid=28357 vsize=442728 CPUtime=1.45 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 2323 0 0 0 145 0 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352072 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=442728 CPUtime=1.48 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 2315 0 0 0 148 0 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=442728 CPUtime=1.47 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 9417 0 0 0 146 1 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=442728 CPUtime=1.48 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 2389 0 0 0 148 0 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4462583 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=442728 CPUtime=1.49 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 2368 0 0 0 149 0 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4348817 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=442728 CPUtime=1.43 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 2982 0 0 0 143 0 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351996 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=442728 CPUtime=1.47 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 2336 0 0 0 147 0 0 0 25 0 8 0 803723962 453353472 13419 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 11.81
Current children cumulated vsize (KiB) 442728

[startup+3.10084 s]
/proc/loadavg: 7.44 7.95 7.96 9/447 28367
/proc/meminfo: memFree=32225092/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=458940 CPUtime=24.59 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 46832 0 0 0 2453 6 0 0 25 0 8 0 803723962 469954560 21124 33554432000 4194304 4625164 140737344502752 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 114735 21124 352 106 0 110238 0
[pid=28359/tid=28360] ppid=28357 vsize=458940 CPUtime=3.05 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 3772 0 0 0 305 0 0 0 25 0 8 0 803723962 469954560 21124 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=458940 CPUtime=3.07 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 3676 0 0 0 307 0 0 0 25 0 8 0 803723962 469954560 21124 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351814 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=458940 CPUtime=3.07 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 19845 0 0 0 305 2 0 0 25 0 8 0 803723962 469954560 21124 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=458940 CPUtime=3.08 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 3924 0 0 0 308 0 0 0 25 0 8 0 803723962 469954560 21124 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352739 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=458940 CPUtime=3.08 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 4755 0 0 0 308 0 0 0 25 0 8 0 803723962 469954560 21124 33554432000 4194304 4625164 140737344502752 18446744073709551615 4460211 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=458940 CPUtime=3.01 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 4801 0 0 0 301 0 0 0 25 0 8 0 803723962 469954560 21124 33554432000 4194304 4625164 140737344502752 18446744073709551615 4353239 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=458940 CPUtime=3.07 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 3730 0 0 0 307 0 0 0 25 0 8 0 803723962 469954560 21125 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 4 0 0 0
Current children cumulated CPU time (s) 24.59
Current children cumulated vsize (KiB) 458940

[startup+6.30128 s]
/proc/loadavg: 7.49 7.95 7.96 9/447 28367
/proc/meminfo: memFree=32175120/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=493612 CPUtime=50.14 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 93830 0 0 0 5001 13 0 0 25 0 8 0 803723962 505458688 33816 33554432000 4194304 4625164 140737344502752 18446744073709551615 4511967 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 123403 33816 352 106 0 118906 0
[pid=28359/tid=28360] ppid=28357 vsize=493612 CPUtime=6.25 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 7653 0 0 0 625 0 0 0 25 0 8 0 803723962 505458688 33816 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=493612 CPUtime=6.27 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 5746 0 0 0 627 0 0 0 25 0 8 0 803723962 505458688 33817 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=493612 CPUtime=6.28 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 44206 0 0 0 622 6 0 0 25 0 8 0 803723962 505458688 33823 33554432000 4194304 4625164 140737344502752 18446744073709551615 4579598 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=493612 CPUtime=6.28 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 8163 0 0 0 627 1 0 0 25 0 8 0 803723962 505458688 33823 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351777 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=493612 CPUtime=6.29 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 7305 0 0 0 628 1 0 0 25 0 8 0 803723962 505458688 33823 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339536 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=493612 CPUtime=6.17 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 9547 0 0 0 616 1 0 0 25 0 8 0 803723962 505458688 33823 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=493612 CPUtime=6.27 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 7694 0 0 0 626 1 0 0 25 0 8 0 803723962 505458688 33824 33554432000 4194304 4625164 140737344502752 18446744073709551615 4467522 0 0 4096 2 0 0 0 -1 4 0 0 0

################
# More data... #
################

[pid=28359/tid=28362] ppid=28357 vsize=5070084 CPUtime=4661.08 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 20797966 0 0 0 463028 3080 0 0 25 0 8 0 803723962 5191766016 994741 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=5070084 CPUtime=4662.39 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 247134 0 0 0 466185 54 0 0 25 0 8 0 803723962 5191766016 994741 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=5070084 CPUtime=4662.33 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 152738 0 0 0 466183 50 0 0 25 0 8 0 803723962 5191766016 994741 33554432000 4194304 4625164 140737344502752 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=5070084 CPUtime=4592.71 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 265442 0 0 0 459203 68 0 0 25 0 8 0 803723962 5191766016 994741 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=5070084 CPUtime=4662.34 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 244868 0 0 0 466182 52 0 0 25 0 8 0 803723962 5191766016 994741 33554432000 4194304 4625164 140737344502752 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37227.2
Current children cumulated vsize (KiB) 5070084

[startup+4722.32 s]
/proc/loadavg: 8.10 8.05 8.00 9/447 28505
/proc/meminfo: memFree=28295800/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=5070084 CPUtime=37706.4 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 25701947 0 0 0 3766587 4050 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 1267521 997443 500 106 0 1263024 0
[pid=28359/tid=28360] ppid=28357 vsize=5070084 CPUtime=4722.35 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 224065 0 0 0 472183 52 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=5070084 CPUtime=4722.4 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 285112 0 0 0 472179 61 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4352007 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=5070084 CPUtime=4721.09 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 21140334 0 0 0 468975 3134 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4552369 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=5070084 CPUtime=4722.41 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 247153 0 0 0 472187 54 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4330855 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=5070084 CPUtime=4722.34 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 152994 0 0 0 472184 50 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4349054 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=5070084 CPUtime=4651.79 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 265903 0 0 0 465111 68 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=5070084 CPUtime=4722.36 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 244888 0 0 0 472184 52 0 0 25 0 8 0 803723962 5191766016 997443 33554432000 4194304 4625164 140737344502752 18446744073709551615 4330829 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 37706.4
Current children cumulated vsize (KiB) 5070084

[startup+4782.3 s]
/proc/loadavg: 8.10 8.05 8.00 9/447 28506
/proc/meminfo: memFree=28300232/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=5070084 CPUtime=38185.3 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 26167925 0 0 0 3814403 4130 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4512857 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 1267521 995946 352 106 0 1263024 0
[pid=28359/tid=28360] ppid=28357 vsize=5070084 CPUtime=4782.34 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 224083 0 0 0 478182 52 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=5070084 CPUtime=4782.39 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 285236 0 0 0 478178 61 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=5070084 CPUtime=4781.06 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 21482367 0 0 0 474913 3193 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4579394 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=5070084 CPUtime=4782.39 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 247171 0 0 0 478185 54 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=5070084 CPUtime=4782.34 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 153855 0 0 0 478183 51 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=5070084 CPUtime=4710.87 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 267374 0 0 0 471018 69 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=5070084 CPUtime=4782.35 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 244909 0 0 0 478183 52 0 0 25 0 8 0 803723962 5191766016 995946 33554432000 4194304 4625164 140737344502752 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38185.3
Current children cumulated vsize (KiB) 5070084

[startup+4842.3 s]
/proc/loadavg: 8.15 8.07 8.01 9/447 28517
/proc/meminfo: memFree=28283732/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=5070084 CPUtime=38664.3 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 26614615 0 0 0 3862223 4211 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 1267521 1001087 355 106 0 1263024 0
[pid=28359/tid=28360] ppid=28357 vsize=5070084 CPUtime=4842.31 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 224100 0 0 0 484179 52 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=5070084 CPUtime=4842.39 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 285324 0 0 0 484178 61 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351764 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=5070084 CPUtime=4841.02 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 21815667 0 0 0 480854 3248 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=5070084 CPUtime=4842.39 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 247190 0 0 0 484185 54 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=5070084 CPUtime=4842.32 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 154796 0 0 0 484181 51 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=5070084 CPUtime=4769.94 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 267971 0 0 0 476925 69 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351824 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=5070084 CPUtime=4842.34 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 244927 0 0 0 484182 52 0 0 25 0 8 0 803723962 5191766016 1001087 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 38664.3
Current children cumulated vsize (KiB) 5070084

[startup+4902.3 s]
/proc/loadavg: 8.10 8.07 8.01 9/447 28518
/proc/meminfo: memFree=28257920/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=5070084 CPUtime=39143.4 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 26957609 0 0 0 3910070 4272 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4519334 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 1267521 1005433 432 106 0 1263024 0
[pid=28359/tid=28360] ppid=28357 vsize=5070084 CPUtime=4902.31 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 224118 0 0 0 490179 52 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=5070084 CPUtime=4902.39 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 285838 0 0 0 490177 62 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351830 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=5070084 CPUtime=4901.01 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 22062717 0 0 0 486812 3289 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351866 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=5070084 CPUtime=4902.39 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 247208 0 0 0 490185 54 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=5070084 CPUtime=4902.33 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 155915 0 0 0 490181 52 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=5070084 CPUtime=4829.03 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 268365 0 0 0 482833 70 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=5070084 CPUtime=4902.34 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 244948 0 0 0 490182 52 0 0 25 0 8 0 803723962 5191766016 1005433 33554432000 4194304 4625164 140737344502752 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39143.4
Current children cumulated vsize (KiB) 5070084

[startup+4962.3 s]
/proc/loadavg: 8.09 8.07 8.01 9/447 28520
/proc/meminfo: memFree=28257384/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=5070084 CPUtime=39622.5 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 27282414 0 0 0 3957923 4326 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 1267521 1009595 575 106 0 1263024 0
[pid=28359/tid=28360] ppid=28357 vsize=5070084 CPUtime=4962.31 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 224138 0 0 0 496179 52 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=5070084 CPUtime=4962.39 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 286101 0 0 0 496177 62 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=5070084 CPUtime=4960.99 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 22386782 0 0 0 492757 3342 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=5070084 CPUtime=4962.39 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 247226 0 0 0 496185 54 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=5070084 CPUtime=4962.33 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 156037 0 0 0 496181 52 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=5070084 CPUtime=4888.1 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 268429 0 0 0 488740 70 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=5070084 CPUtime=4962.35 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 244975 0 0 0 496183 52 0 0 25 0 8 0 803723962 5191766016 1009595 33554432000 4194304 4625164 140737344502752 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 39622.5
Current children cumulated vsize (KiB) 5070084



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5009.6 s]
/proc/loadavg: 8.04 8.06 8.00 9/447 28523
/proc/meminfo: memFree=28237512/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=5070084 CPUtime=40000.2 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) R 28357 28359 28333 0 -1 4202496 27505602 0 0 0 3995660 4363 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 1267521 1011868 574 106 0 1263024 0
[pid=28359/tid=28360] ppid=28357 vsize=5070084 CPUtime=5009.63 cores=0-7
/proc/28359/task/28360/stat : 28360 (SArTagnan) R 28357 28359 28333 0 -1 4202560 224151 0 0 0 500911 52 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=28359/tid=28361] ppid=28357 vsize=5070084 CPUtime=5009.7 cores=0-7
/proc/28359/task/28361/stat : 28361 (SArTagnan) R 28357 28359 28333 0 -1 4202560 286174 0 0 0 500908 62 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=28359/tid=28362] ppid=28357 vsize=5070084 CPUtime=5008.29 cores=0-7
/proc/28359/task/28362/stat : 28362 (SArTagnan) R 28357 28359 28333 0 -1 4202560 22609218 0 0 0 497450 3379 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4552371 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=28359/tid=28363] ppid=28357 vsize=5070084 CPUtime=5009.71 cores=0-7
/proc/28359/task/28363/stat : 28363 (SArTagnan) R 28357 28359 28333 0 -1 4202560 247241 0 0 0 500917 54 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=28359/tid=28364] ppid=28357 vsize=5070084 CPUtime=5009.65 cores=0-7
/proc/28359/task/28364/stat : 28364 (SArTagnan) R 28357 28359 28333 0 -1 4202560 156469 0 0 0 500912 53 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4348870 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=5070084 CPUtime=4934.67 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4202560 268509 0 0 0 493397 70 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=28359/tid=28366] ppid=28357 vsize=5070084 CPUtime=5009.66 cores=0-7
/proc/28359/task/28366/stat : 28366 (SArTagnan) R 28357 28359 28333 0 -1 4202560 244990 0 0 0 500914 52 0 0 25 0 8 0 803723962 5191766016 1011868 33554432000 4194304 4625164 140737344502752 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 5070084

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5009.8 s]
/proc/loadavg: 8.04 8.06 8.00 9/447 28523
/proc/meminfo: memFree=28237512/32950928 swapFree=34365992/67111528
[pid=28359] ppid=28357 vsize=0 CPUtime=40000.5 cores=0-7
/proc/28359/stat : 28359 (SArTagnan) Z 28357 28359 28333 0 -1 4203548 27505602 0 0 0 3995668 4381 0 0 25 0 2 0 803723962 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/28359/statm: 0 0 0 0 0 0 0
[pid=28359/tid=28365] ppid=28357 vsize=0 CPUtime=4934.86 cores=0-7
/proc/28359/task/28365/stat : 28365 (SArTagnan) R 28357 28359 28333 0 -1 4203588 268509 0 0 0 493398 88 0 0 25 0 2 0 803723962 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.5
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5009.81
CPU time (s): 40000.5
CPU user time (s): 39956.7
CPU system time (s): 43.8273
CPU usage (%): 798.443
Max. virtual memory (cumulated for all children) (KiB): 5306696

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39956.7
system time used= 43.8273
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 27505602
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 2149
involuntary context switches= 169145

runsolver used 19.926 second user time and 55.0916 second system time

The end

Launcher Data 

Begin job on node120 at 2011-06-01 11:40:52
IDJOB=3438656
IDBENCH=83146
IDSOLVER=1936
FILE ID=node120/3438656-1306921252
RUNJOBID= node120-1306921252-28347
PBS_JOBID= 13488550
Free space on /tmp= 73936 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT11/crafted/spence/sgen/sgen3-n220-s24359701-sat.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3438656-1306921252/watcher-3438656-1306921252 -o /tmp/evaluation-result-3438656-1306921252/solver-3438656-1306921252 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3438656-1306921252.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= ceff918324677440f95867755c4e7133
RANDOM SEED=649647487

node120.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.61
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5226.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.78
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.809
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      32293712 kB
Buffers:        149904 kB
Cached:         196380 kB
SwapCached:      12852 kB
Active:         129564 kB
Inactive:       235804 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      32293712 kB
SwapTotal:    67111528 kB
SwapFree:     34365992 kB
Dirty:             396 kB
Writeback:           0 kB
AnonPages:       17428 kB
Mapped:           8832 kB
Slab:           129620 kB
PageTables:      91660 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 105368108 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73932 MiB
End job on node120 at 2011-06-01 13:04:24