Trace number 3439231

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.5 5012.71

General information on the benchmark

NameSAT03/handmade/gomes/bqwh/
bqwh.60.1152.shuffled-as.sat03-1650.cnf
MD5SUM98d6317b259165bb4fb7a39e783668e0
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark373.939
Satisfiable
(Un)Satisfiability was proved
Number of variables7534
Number of clauses72812
Sum of the clauses size161368
Maximum clause length14
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 269406
Number of clauses of size 3128
Number of clauses of size 4291
Number of clauses of size 5538
Number of clauses of size over 52449

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3439231-1306960407/watcher-3439231-1306960407 -o /tmp/evaluation-result-3439231-1306960407/solver-3439231-1306960407 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3439231-1306960407.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.00 0.00 0.00 2/554 17690
/proc/meminfo: memFree=30392516/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=22044 CPUtime=0.01 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 1096 0 0 0 1 0 0 0 25 0 1 0 807639426 22573056 778 33554432000 4194304 4625164 140733863142880 18446744073709551615 251814240000 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/17690/statm: 5511 778 296 106 0 1014 0

[startup+0.0224571 s]
/proc/loadavg: 0.00 0.00 0.00 2/554 17690
/proc/meminfo: memFree=30392516/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=22364 CPUtime=0.02 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 1313 0 0 0 2 0 0 0 25 0 1 0 807639426 22900736 834 33554432000 4194304 4625164 140733863142880 18446744073709551615 4340576 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/17690/statm: 5591 835 296 106 0 1094 0
Current children cumulated CPU time (s) 0.02
Current children cumulated vsize (KiB) 22364

[startup+0.100443 s]
/proc/loadavg: 0.00 0.00 0.00 2/554 17690
/proc/meminfo: memFree=30392516/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=501672 CPUtime=0.41 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 7263 0 0 0 41 0 0 0 25 0 8 0 807639426 513712128 6651 33554432000 4194304 4625164 140733863142880 18446744073709551615 4398379 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17690/statm: 125418 6691 349 106 0 120921 0
Current children cumulated CPU time (s) 0.41
Current children cumulated vsize (KiB) 501672

[startup+0.300414 s]
/proc/loadavg: 0.00 0.00 0.00 2/554 17690
/proc/meminfo: memFree=30392516/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=523868 CPUtime=1.82 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 21947 0 0 0 179 3 0 0 25 0 8 0 807639426 536440832 14315 33554432000 4194304 4625164 140733863142880 18446744073709551615 4519391 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17690/statm: 130967 14316 351 106 0 126470 0
Current children cumulated CPU time (s) 1.82
Current children cumulated vsize (KiB) 523868

[startup+0.700345 s]
/proc/loadavg: 0.00 0.00 0.00 2/554 17690
/proc/meminfo: memFree=30392516/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=527124 CPUtime=4.87 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 27366 0 0 0 483 4 0 0 25 0 8 0 807639426 539774976 15796 33554432000 4194304 4625164 140733863142880 18446744073709551615 251813899699 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17690/statm: 131781 15797 352 106 0 127284 0
Current children cumulated CPU time (s) 4.87
Current children cumulated vsize (KiB) 527124

[startup+1.50121 s]
/proc/loadavg: 0.00 0.00 0.00 9/562 17698
/proc/meminfo: memFree=30326616/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=531756 CPUtime=11.25 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 33079 0 0 0 1120 5 0 0 25 0 8 0 807639426 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4396555 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17690/statm: 132939 18836 352 106 0 128442 0
[pid=17690/tid=17692] ppid=17688 vsize=531756 CPUtime=1.45 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 2848 0 0 0 145 0 0 0 19 0 8 0 807639429 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352749 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=531756 CPUtime=1.46 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 2636 0 0 0 146 0 0 0 19 0 8 0 807639429 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339680 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=531756 CPUtime=1.44 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 8734 0 0 0 143 1 0 0 19 0 8 0 807639429 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4557107 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=531756 CPUtime=1.44 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 2810 0 0 0 144 0 0 0 20 0 8 0 807639429 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352170 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=531756 CPUtime=1.44 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 2863 0 0 0 144 0 0 0 19 0 8 0 807639429 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=531756 CPUtime=1.06 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 1796 0 0 0 106 0 0 0 25 0 8 0 807639429 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4288883 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=531756 CPUtime=1.44 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 2874 0 0 0 144 0 0 0 25 0 8 0 807639429 544518144 18836 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.25
Current children cumulated vsize (KiB) 531756

[startup+3.10094 s]
/proc/loadavg: 0.64 0.13 0.04 9/562 17698
/proc/meminfo: memFree=30305416/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=540924 CPUtime=24.03 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 50611 0 0 0 2392 11 0 0 25 0 8 0 807639426 553906176 27143 33554432000 4194304 4625164 140733863142880 18446744073709551615 4396555 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17690/statm: 135231 27143 353 106 0 130734 0
[pid=17690/tid=17692] ppid=17688 vsize=540924 CPUtime=3.04 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 4343 0 0 0 304 0 0 0 25 0 8 0 807639429 553906176 27143 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=540924 CPUtime=3.06 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 3190 0 0 0 306 0 0 0 25 0 8 0 807639429 553906176 27144 33554432000 4194304 4625164 140733863142880 18446744073709551615 4466334 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=540924 CPUtime=3.04 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 19603 0 0 0 300 4 0 0 25 0 8 0 807639429 553906176 27144 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352280 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=540924 CPUtime=3.04 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 3428 0 0 0 304 0 0 0 25 0 8 0 807639429 553906176 27146 33554432000 4194304 4625164 140733863142880 18446744073709551615 251813901345 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=540924 CPUtime=3.04 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 3323 0 0 0 304 0 0 0 25 0 8 0 807639429 553906176 27147 33554432000 4194304 4625164 140733863142880 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=540924 CPUtime=2.66 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 2271 0 0 0 266 0 0 0 25 0 8 0 807639429 553906176 27148 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=540924 CPUtime=3.04 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 4125 0 0 0 304 0 0 0 25 0 8 0 807639429 553906176 27148 33554432000 4194304 4625164 140733863142880 18446744073709551615 251813901345 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.03
Current children cumulated vsize (KiB) 540924

[startup+6.3004 s]
/proc/loadavg: 0.64 0.13 0.04 9/562 17698
/proc/meminfo: memFree=30232380/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=559440 CPUtime=49.56 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 88967 0 0 0 4937 19 0 0 25 0 8 0 807639426 572866560 45045 33554432000 4194304 4625164 140733863142880 18446744073709551615 4393101 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17690/statm: 139860 45045 353 106 0 135363 0
[pid=17690/tid=17692] ppid=17688 vsize=559440 CPUtime=6.25 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 6132 0 0 0 624 1 0 0 25 0 8 0 807639429 572866560 45046 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351818 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=559440 CPUtime=6.26 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 5832 0 0 0 625 1 0 0 25 0 8 0 807639429 572866560 45046 33554432000 4194304 4625164 140733863142880 18446744073709551615 4462556 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=559440 CPUtime=6.24 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 42455 0 0 0 616 8 0 0 25 0 8 0 807639429 572866560 45046 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=559440 CPUtime=6.24 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 5895 0 0 0 623 1 0 0 25 0 8 0 807639429 572866560 45046 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=559440 CPUtime=6.23 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 5397 0 0 0 623 0 0 0 25 0 8 0 807639429 572866560 45047 33554432000 4194304 4625164 140733863142880 18446744073709551615 251814345247 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=559440 CPUtime=5.86 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 3904 0 0 0 585 1 0 0 25 0 8 0 807639429 572866560 45047 33554432000 4194304 4625164 140733863142880 18446744073709551615 251830223899 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=559440 CPUtime=6.24 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 6911 0 0 0 623 1 0 0 25 0 8 0 807639429 572866560 45048 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 49.56
Current children cumulated vsize (KiB) 559440

[startup+12.7003 s]
/proc/loadavg: 1.77 0.39 0.13 9/562 17698
/proc/meminfo: memFree=30092708/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=594984 CPUtime=100.63 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 161649 0 0 0 10027 36 0 0 25 0 8 0 807639426 609263616 78741 33554432000 4194304 4625164 140733863142880 18446744073709551615 4338085 0 0 4096 2 18446744073709551615 0 0 17 7 0 0 0
/proc/17690/statm: 148746 78741 353 106 0 144249 0
[pid=17690/tid=17692] ppid=17688 vsize=594984 CPUtime=12.64 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 11797 0 0 0 1261 3 0 0 25 0 8 0 807639429 609263616 78741 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=594984 CPUtime=12.66 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 10673 0 0 0 1263 3 0 0 25 0 8 0 807639429 609263616 78741 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351839 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=594984 CPUtime=12.64 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 85368 0 0 0 1249 15 0 0 25 0 8 0 807639429 609263616 78741 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=594984 CPUtime=12.64 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 10109 0 0 0 1262 2 0 0 25 0 8 0 807639429 609263616 78742 33554432000 4194304 4625164 140733863142880 18446744073709551615 4288743 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=594984 CPUtime=12.64 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 9401 0 0 0 1263 1 0 0 25 0 8 0 807639429 609263616 78742 33554432000 4194304 4625164 140733863142880 18446744073709551615 251813901345 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=594984 CPUtime=12.26 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 6730 0 0 0 1224 2 0 0 25 0 8 0 807639429 609263616 78742 33554432000 4194304 4625164 140733863142880 18446744073709551615 4288758 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=594984 CPUtime=12.64 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 11787 0 0 0 1261 3 0 0 25 0 8 0 807639429 609263616 78743 33554432000 4194304 4625164 140733863142880 18446744073709551615 4486772 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 100.63
Current children cumulated vsize (KiB) 594984

[startup+25.5012 s]
/proc/loadavg: 2.80 0.66 0.22 9/562 17699
/proc/meminfo: memFree=29840608/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=1035048 CPUtime=202.76 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 299687 0 0 0 20208 68 0 0 25 0 8 0 807639426 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 4499272 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 258762 138924 353 106 0 254265 0
[pid=17690/tid=17692] ppid=17688 vsize=1035048 CPUtime=25.44 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 19626 0 0 0 2538 6 0 0 25 0 8 0 807639429 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352044 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=1035048 CPUtime=25.45 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 17674 0 0 0 2539 6 0 0 25 0 8 0 807639429 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=1035048 CPUtime=25.39 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 174860 0 0 0 2508 31 0 0 25 0 8 0 807639429 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=1035048 CPUtime=25.43 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 17901 0 0 0 2538 5 0 0 25 0 8 0 807639429 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=1035048 CPUtime=25.44 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 15889 0 0 0 2541 3 0 0 25 0 8 0 807639429 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339591 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=1035048 CPUtime=25.05 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 12206 0 0 0 2501 4 0 0 25 0 8 0 807639429 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 251813907418 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=1035048 CPUtime=25.44 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 19614 0 0 0 2540 4 0 0 25 0 8 0 807639429 1059889152 138924 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 202.76
Current children cumulated vsize (KiB) 1035048

[startup+51.1009 s]
/proc/loadavg: 4.57 1.25 0.42 9/562 17700
/proc/meminfo: memFree=29439904/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=1214208 CPUtime=407.07 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 538233 0 0 0 40581 126 0 0 25 0 8 0 807639426 1243348992 240005 33554432000 4194304 4625164 140733863142880 18446744073709551615 4396709 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 303552 240005 353 106 0 299055 0
[pid=17690/tid=17692] ppid=17688 vsize=1214208 CPUtime=51.04 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 33720 0 0 0 5091 13 0 0 25 0 8 0 807639429 1243348992 240005 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352005 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=1214208 CPUtime=51.04 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 31639 0 0 0 5093 11 0 0 25 0 8 0 807639429 1243348992 240005 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352002 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=1214208 CPUtime=50.51 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 323673 0 0 0 4993 58 0 0 25 0 8 0 807639429 1243348992 240005 33554432000 4194304 4625164 140733863142880 18446744073709551615 4579356 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=1214208 CPUtime=51.03 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 31185 0 0 0 5094 9 0 0 25 0 8 0 807639429 1243348992 240006 33554432000 4194304 4625164 140733863142880 18446744073709551615 251813901345 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=1214208 CPUtime=51.03 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 27382 0 0 0 5097 6 0 0 25 0 8 0 807639429 1243348992 240006 33554432000 4194304 4625164 140733863142880 18446744073709551615 4349019 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=1214208 CPUtime=50.65 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 21872 0 0 0 5057 8 0 0 25 0 8 0 807639429 1243348992 240006 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339612 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=1214208 CPUtime=51.04 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 38267 0 0 0 5095 9 0 0 25 0 8 0 807639429 1243348992 240006 33554432000 4194304 4625164 140733863142880 18446744073709551615 4352522 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 407.07

################
# More data... #
################

[pid=17690/tid=17694] ppid=17688 vsize=6361448 CPUtime=4573.17 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 17625194 0 0 0 453682 3635 0 0 25 0 8 0 807639429 6514122752 1424598 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=6361448 CPUtime=4662.3 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 131899 0 0 0 466178 52 0 0 25 0 8 0 807639429 6514122752 1424598 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=6361448 CPUtime=4662.31 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 135441 0 0 0 466172 59 0 0 25 0 8 0 807639429 6514122752 1424598 33554432000 4194304 4625164 140733863142880 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=6361448 CPUtime=4661.93 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 167107 0 0 0 466100 93 0 0 25 0 8 0 807639429 6514122752 1424598 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=6361448 CPUtime=4662.15 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 132067 0 0 0 466174 41 0 0 25 0 8 0 807639429 6514122752 1424598 33554432000 4194304 4625164 140733863142880 18446744073709551615 4330976 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37206.1
Current children cumulated vsize (KiB) 6361448

[startup+4722.32 s]
/proc/loadavg: 8.11 8.06 7.98 9/562 17836
/proc/meminfo: memFree=24648404/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=6468892 CPUtime=37685.1 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 19448381 0 0 0 3764300 4208 0 0 25 0 8 0 807639426 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4392981 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 1617223 1433376 353 106 0 1612726 0
[pid=17690/tid=17692] ppid=17688 vsize=6468892 CPUtime=4722.15 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 155540 0 0 0 472159 56 0 0 25 0 8 0 807639429 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=6468892 CPUtime=4720.16 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 130535 0 0 0 471971 45 0 0 25 0 8 0 807639429 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=6468892 CPUtime=4632.02 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 17888521 0 0 0 459515 3687 0 0 25 0 8 0 807639429 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4552340 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=6468892 CPUtime=4722.33 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 153748 0 0 0 472178 55 0 0 25 0 8 0 807639429 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=6468892 CPUtime=4722.33 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 135560 0 0 0 472174 59 0 0 25 0 8 0 807639429 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=6468892 CPUtime=4721.95 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 167384 0 0 0 472102 93 0 0 25 0 8 0 807639429 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=6468892 CPUtime=4722.17 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 132238 0 0 0 472176 41 0 0 25 0 8 0 807639429 6624145408 1433376 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 37685.1
Current children cumulated vsize (KiB) 6468892

[startup+4782.3 s]
/proc/loadavg: 8.11 8.07 7.98 9/562 17838
/proc/meminfo: memFree=24614652/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=6468892 CPUtime=38163.8 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 19705599 0 0 0 3812114 4262 0 0 25 0 8 0 807639426 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 1617223 1442216 353 106 0 1612726 0
[pid=17690/tid=17692] ppid=17688 vsize=6468892 CPUtime=4782.13 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 155672 0 0 0 478157 56 0 0 25 0 8 0 807639429 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=6468892 CPUtime=4780.12 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 130728 0 0 0 477967 45 0 0 25 0 8 0 807639429 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=6468892 CPUtime=4690.87 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 18140306 0 0 0 465347 3740 0 0 25 0 8 0 807639429 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4335165 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=6468892 CPUtime=4782.31 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 153869 0 0 0 478176 55 0 0 25 0 8 0 807639429 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=6468892 CPUtime=4782.31 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 135840 0 0 0 478172 59 0 0 25 0 8 0 807639429 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4460412 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=6468892 CPUtime=4781.93 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 167558 0 0 0 478100 93 0 0 25 0 8 0 807639429 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=6468892 CPUtime=4782.15 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 132426 0 0 0 478174 41 0 0 25 0 8 0 807639429 6624145408 1442216 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38163.8
Current children cumulated vsize (KiB) 6468892

[startup+4842.3 s]
/proc/loadavg: 8.10 8.07 7.99 9/562 17839
/proc/meminfo: memFree=24580280/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=6468892 CPUtime=38642.6 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 19967867 0 0 0 3859941 4316 0 0 25 0 8 0 807639426 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 1617223 1449432 353 106 0 1612726 0
[pid=17690/tid=17692] ppid=17688 vsize=6468892 CPUtime=4842.13 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 155760 0 0 0 484157 56 0 0 25 0 8 0 807639429 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=6468892 CPUtime=4840.09 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 130942 0 0 0 483964 45 0 0 25 0 8 0 807639429 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=6468892 CPUtime=4749.71 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 18397527 0 0 0 471179 3792 0 0 25 0 8 0 807639429 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4579598 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=6468892 CPUtime=4842.31 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 153965 0 0 0 484176 55 0 0 25 0 8 0 807639429 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=6468892 CPUtime=4842.31 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 136001 0 0 0 484172 59 0 0 25 0 8 0 807639429 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4460403 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=6468892 CPUtime=4841.93 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 167713 0 0 0 484100 93 0 0 25 0 8 0 807639429 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=6468892 CPUtime=4842.15 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 132580 0 0 0 484174 41 0 0 25 0 8 0 807639429 6624145408 1449432 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 38642.6
Current children cumulated vsize (KiB) 6468892

[startup+4902.3 s]
/proc/loadavg: 8.15 8.08 8.00 9/562 17841
/proc/meminfo: memFree=24473496/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=6850896 CPUtime=39121.4 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 20404257 0 0 0 3907746 4394 0 0 25 0 8 0 807639426 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4512084 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 1712724 1476253 353 106 0 1708227 0
[pid=17690/tid=17692] ppid=17688 vsize=6850896 CPUtime=4902.13 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 155980 0 0 0 490157 56 0 0 25 0 8 0 807639429 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=6850896 CPUtime=4900.08 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 131073 0 0 0 489963 45 0 0 25 0 8 0 807639429 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=6850896 CPUtime=4808.54 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 18651983 0 0 0 477009 3845 0 0 25 0 8 0 807639429 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4547445 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=6850896 CPUtime=4902.31 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 154193 0 0 0 490176 55 0 0 25 0 8 0 807639429 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=6850896 CPUtime=4902.31 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 136176 0 0 0 490172 59 0 0 25 0 8 0 807639429 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=6850896 CPUtime=4901.93 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 167857 0 0 0 490100 93 0 0 25 0 8 0 807639429 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=6850896 CPUtime=4902.15 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 132742 0 0 0 490174 41 0 0 25 0 8 0 807639429 7015317504 1476253 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39121.4
Current children cumulated vsize (KiB) 6850896

[startup+4962.3 s]
/proc/loadavg: 8.16 8.10 8.01 9/562 17842
/proc/meminfo: memFree=24440608/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=6850896 CPUtime=39600.2 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 20677228 0 0 0 3955567 4455 0 0 25 0 8 0 807639426 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 1712724 1484901 353 106 0 1708227 0
[pid=17690/tid=17692] ppid=17688 vsize=6850896 CPUtime=4962.12 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 156155 0 0 0 496156 56 0 0 25 0 8 0 807639429 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=6850896 CPUtime=4960.07 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 131197 0 0 0 495962 45 0 0 25 0 8 0 807639429 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=6850896 CPUtime=4867.37 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 18920543 0 0 0 482833 3904 0 0 25 0 8 0 807639429 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4334485 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=6850896 CPUtime=4962.31 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 154381 0 0 0 496176 55 0 0 25 0 8 0 807639429 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4488037 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=6850896 CPUtime=4962.31 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 136350 0 0 0 496172 59 0 0 25 0 8 0 807639429 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=6850896 CPUtime=4961.93 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 168043 0 0 0 496100 93 0 0 25 0 8 0 807639429 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=6850896 CPUtime=4962.15 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 132896 0 0 0 496174 41 0 0 25 0 8 0 807639429 7015317504 1484901 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 39600.2
Current children cumulated vsize (KiB) 6850896



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5012.4 s]
/proc/loadavg: 8.07 8.08 8.00 9/562 17844
/proc/meminfo: memFree=24414300/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=6850896 CPUtime=40000.2 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) R 17688 17690 17664 0 -1 4202496 20902588 0 0 0 3995513 4505 0 0 25 0 8 0 807639426 7015317504 1491465 33554432000 4194304 4625164 140733863142880 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 1712724 1491465 353 106 0 1708227 0
[pid=17690/tid=17692] ppid=17688 vsize=6850896 CPUtime=5012.25 cores=0-7
/proc/17690/task/17692/stat : 17692 (SArTagnan) R 17688 17690 17664 0 -1 4202560 156278 0 0 0 501168 57 0 0 25 0 8 0 807639429 7015317504 1491466 33554432000 4194304 4625164 140733863142880 18446744073709551615 4339639 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=17690/tid=17693] ppid=17688 vsize=6850896 CPUtime=5010.17 cores=0-7
/proc/17690/task/17693/stat : 17693 (SArTagnan) R 17688 17690 17664 0 -1 4202560 131326 0 0 0 500972 45 0 0 25 0 8 0 807639429 7015317504 1491466 33554432000 4194304 4625164 140733863142880 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=6850896 CPUtime=4916.52 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4202560 19142381 0 0 0 487699 3953 0 0 25 0 8 0 807639429 7015317504 1491466 33554432000 4194304 4625164 140733863142880 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=17690/tid=17695] ppid=17688 vsize=6850896 CPUtime=5012.43 cores=0-7
/proc/17690/task/17695/stat : 17695 (SArTagnan) R 17688 17690 17664 0 -1 4202560 154477 0 0 0 501188 55 0 0 25 0 8 0 807639429 7015317504 1491466 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=17690/tid=17696] ppid=17688 vsize=6850896 CPUtime=5012.43 cores=0-7
/proc/17690/task/17696/stat : 17696 (SArTagnan) R 17688 17690 17664 0 -1 4202560 136452 0 0 0 501184 59 0 0 25 0 8 0 807639429 7015317504 1491466 33554432000 4194304 4625164 140733863142880 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=17690/tid=17697] ppid=17688 vsize=6850896 CPUtime=5012.05 cores=0-7
/proc/17690/task/17697/stat : 17697 (SArTagnan) R 17688 17690 17664 0 -1 4202560 168141 0 0 0 501112 93 0 0 25 0 8 0 807639429 7015317504 1491466 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=17690/tid=17698] ppid=17688 vsize=6850896 CPUtime=5012.28 cores=0-7
/proc/17690/task/17698/stat : 17698 (SArTagnan) R 17688 17690 17664 0 -1 4202560 133029 0 0 0 501186 42 0 0 25 0 8 0 807639429 7015317504 1491466 33554432000 4194304 4625164 140733863142880 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 40000.2
Current children cumulated vsize (KiB) 6850896

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5012.6 s]
/proc/loadavg: 8.07 8.08 8.00 2/557 17845
/proc/meminfo: memFree=28150540/32950928 swapFree=41528364/67111528
[pid=17690] ppid=17688 vsize=0 CPUtime=40000.4 cores=0-7
/proc/17690/stat : 17690 (SArTagnan) Z 17688 17690 17664 0 -1 4203548 20902590 0 0 0 3995519 4524 0 0 25 0 2 0 807639426 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 5 0 0 0
/proc/17690/statm: 0 0 0 0 0 0 0
[pid=17690/tid=17694] ppid=17688 vsize=0 CPUtime=4916.71 cores=0-7
/proc/17690/task/17694/stat : 17694 (SArTagnan) R 17688 17690 17664 0 -1 4203588 19142382 0 0 0 487699 3972 0 0 25 0 2 0 807639429 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5012.71
CPU time (s): 40000.5
CPU user time (s): 39955.2
CPU system time (s): 45.3391
CPU usage (%): 797.982
Max. virtual memory (cumulated for all children) (KiB): 7483484

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39955.2
system time used= 45.3391
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 20902590
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 6079
involuntary context switches= 165368

runsolver used 25.1022 second user time and 71.2072 second system time

The end

Launcher Data 

Begin job on node126 at 2011-06-01 22:33:27
IDJOB=3439231
IDBENCH=83797
IDSOLVER=1936
FILE ID=node126/3439231-1306960407
RUNJOBID= node126-1306960407-17678
PBS_JOBID= 13497625
Free space on /tmp= 73664 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT03/handmade/gomes/bqwh/bqwh.60.1152.shuffled-as.sat03-1650.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3439231-1306960407/watcher-3439231-1306960407 -o /tmp/evaluation-result-3439231-1306960407/solver-3439231-1306960407 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3439231-1306960407.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 98d6317b259165bb4fb7a39e783668e0
RANDOM SEED=1427911303

node126.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.56
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.80
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.82
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.783
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      30392804 kB
Buffers:        200724 kB
Cached:         284144 kB
SwapCached:      15328 kB
Active:        1924756 kB
Inactive:       314972 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      30392804 kB
SwapTotal:    67111528 kB
SwapFree:     41528364 kB
Dirty:            1588 kB
Writeback:           0 kB
AnonPages:     1752556 kB
Mapped:          23188 kB
Slab:           145148 kB
PageTables:      99924 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 176375700 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73664 MiB
End job on node126 at 2011-06-01 23:57:02