Trace number 3439396

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-05-15 (fixed)? (TO) 40000.6 5010.81

General information on the benchmark

NameSAT04/handmade/kexu/frb-0.8/
frb59-26-2.used-as.sat04-892.cnf
MD5SUMfd8d5783ed404e5c6590d627ffdcca50
Bench CategoryCRAFTED (crafted instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1534
Number of clauses132295
Sum of the clauses size266006
Maximum clause length26
Minimum clause length2
Number of clauses of size 10
Number of clauses of size 2132236
Number of clauses of size 30
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 559

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 977)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3439396-1306961974/watcher-3439396-1306961974 -o /tmp/evaluation-result-3439396-1306961974/solver-3439396-1306961974 -C 40000 -W 5100 -M 31000 SArTagnan HOME/instance-3439396-1306961974.cnf -threads=8 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 40000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 40030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 8.09 7.96 6.15 2/491 31936
/proc/meminfo: memFree=27172072/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=23036 CPUtime=0.01 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 1229 0 0 0 1 0 0 0 24 0 1 0 886428106 23588864 633 33554432000 4194304 4625164 140734955805616 18446744073709551615 251367590023 0 0 4096 2 0 0 0 17 6 0 0 0
/proc/31936/statm: 5759 633 296 106 0 1262 0

[startup+0.099857 s]
/proc/loadavg: 8.09 7.96 6.15 2/491 31936
/proc/meminfo: memFree=27172072/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=440828 CPUtime=0.67 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 10034 0 0 0 66 1 0 0 19 0 8 0 886428106 451407872 7496 33554432000 4194304 4625164 140734955805616 18446744073709551615 251367590071 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 111531 8867 341 106 0 107034 0
Current children cumulated CPU time (s) 0.67
Current children cumulated vsize (KiB) 440828

[startup+0.17043 s]
/proc/loadavg: 8.09 7.96 6.15 2/491 31936
/proc/meminfo: memFree=27172072/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=509520 CPUtime=1.3 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 14196 0 0 0 128 2 0 0 19 0 8 0 886428106 521748480 10424 33554432000 4194304 4625164 140734955805616 18446744073709551615 251367202823 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 127380 10482 349 106 0 122883 0
Current children cumulated CPU time (s) 1.3
Current children cumulated vsize (KiB) 509520

[startup+0.528282 s]
/proc/loadavg: 8.09 7.96 6.15 2/491 31936
/proc/meminfo: memFree=27172072/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=517240 CPUtime=4.9 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 25063 0 0 0 486 4 0 0 19 0 8 0 886428106 529653760 17198 33554432000 4194304 4625164 140734955805616 18446744073709551615 4513489 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 130179 22100 353 106 0 125682 0
Current children cumulated CPU time (s) 4.9
Current children cumulated vsize (KiB) 517240

[startup+1.14817 s]
/proc/loadavg: 8.09 7.96 6.15 2/491 31936
/proc/meminfo: memFree=27172072/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=531876 CPUtime=8.85 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 33667 0 0 0 880 5 0 0 20 0 8 0 886428106 544641024 25257 33554432000 4194304 4625164 140734955805616 18446744073709551615 251367244245 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 132969 25279 364 106 0 128472 0
Current children cumulated CPU time (s) 8.85
Current children cumulated vsize (KiB) 531876

[startup+1.54809 s]
/proc/loadavg: 8.09 7.96 6.15 2/491 31936
/proc/meminfo: memFree=27172072/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=524440 CPUtime=12.04 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 37454 0 0 0 1199 5 0 0 21 0 8 0 886428106 537026560 26138 33554432000 4194304 4625164 140734955805616 18446744073709551615 4512205 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 131110 26138 355 106 0 126613 0
Current children cumulated CPU time (s) 12.04
Current children cumulated vsize (KiB) 524440

[startup+3.10081 s]
/proc/loadavg: 8.08 7.96 6.16 9/499 31944
/proc/meminfo: memFree=27017688/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=530020 CPUtime=24.42 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 54617 0 0 0 2434 8 0 0 25 0 8 0 886428106 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4508988 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 132505 39236 363 106 0 128008 0
[pid=31936/tid=31938] ppid=31934 vsize=530020 CPUtime=3.06 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 4870 0 0 0 305 1 0 0 24 0 8 0 886428110 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352035 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=530020 CPUtime=3 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 4264 0 0 0 300 0 0 0 24 0 8 0 886428110 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=530020 CPUtime=3.05 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 6262 0 0 0 304 1 0 0 25 0 8 0 886428110 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4334248 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=530020 CPUtime=3.01 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 4436 0 0 0 301 0 0 0 24 0 8 0 886428110 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=530020 CPUtime=3.05 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 4267 0 0 0 305 0 0 0 24 0 8 0 886428110 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4339559 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=530020 CPUtime=3.05 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 5649 0 0 0 305 0 0 0 25 0 8 0 886428110 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351975 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=530020 CPUtime=3.05 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 3418 0 0 0 305 0 0 0 24 0 8 0 886428110 542740480 39236 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352089 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 24.42
Current children cumulated vsize (KiB) 530020

[startup+6.31921 s]
/proc/loadavg: 8.08 7.96 6.16 9/499 31944
/proc/meminfo: memFree=26991904/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=555460 CPUtime=50.12 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 77041 0 0 0 4999 13 0 0 25 0 8 0 886428106 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4511948 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 138865 44954 359 106 0 134368 0
[pid=31936/tid=31938] ppid=31934 vsize=555460 CPUtime=6.27 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 7115 0 0 0 626 1 0 0 25 0 8 0 886428110 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352047 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=555460 CPUtime=6.22 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 6723 0 0 0 622 0 0 0 25 0 8 0 886428110 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=555460 CPUtime=6.26 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 9541 0 0 0 625 1 0 0 25 0 8 0 886428110 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=555460 CPUtime=6.17 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 4825 0 0 0 617 0 0 0 25 0 8 0 886428110 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=555460 CPUtime=6.26 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 6275 0 0 0 626 0 0 0 25 0 8 0 886428110 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4349075 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=555460 CPUtime=6.27 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 8476 0 0 0 626 1 0 0 25 0 8 0 886428110 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=555460 CPUtime=6.27 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 5461 0 0 0 627 0 0 0 25 0 8 0 886428110 568791040 44954 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352002 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 50.12
Current children cumulated vsize (KiB) 555460

[startup+12.701 s]
/proc/loadavg: 8.07 7.96 6.17 9/499 31944
/proc/meminfo: memFree=26924300/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=586256 CPUtime=101.08 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 121209 0 0 0 10088 20 0 0 25 0 8 0 886428106 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4512763 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 146564 63115 464 106 0 142067 0
[pid=31936/tid=31938] ppid=31934 vsize=586256 CPUtime=12.65 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 11177 0 0 0 1263 2 0 0 25 0 8 0 886428110 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=586256 CPUtime=12.61 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 7454 0 0 0 1260 1 0 0 25 0 8 0 886428110 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4291963 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=586256 CPUtime=12.65 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 26450 0 0 0 1261 4 0 0 25 0 8 0 886428110 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4334728 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=586256 CPUtime=12.47 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 6813 0 0 0 1246 1 0 0 25 0 8 0 886428110 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4288686 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=586256 CPUtime=12.65 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 9714 0 0 0 1264 1 0 0 25 0 8 0 886428110 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4319706 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=586256 CPUtime=12.65 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 13382 0 0 0 1263 2 0 0 25 0 8 0 886428110 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=586256 CPUtime=12.65 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 6235 0 0 0 1264 1 0 0 25 0 8 0 886428110 600326144 63115 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 101.08
Current children cumulated vsize (KiB) 586256

[startup+25.5006 s]
/proc/loadavg: 8.06 7.96 6.19 9/499 31945
/proc/meminfo: memFree=26799768/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=732384 CPUtime=203.27 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 161387 0 0 0 20300 27 0 0 25 0 8 0 886428106 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4512009 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 183096 94167 774 106 0 178599 0
[pid=31936/tid=31938] ppid=31934 vsize=732384 CPUtime=25.46 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 16010 0 0 0 2543 3 0 0 25 0 8 0 886428110 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=732384 CPUtime=25.4 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 10422 0 0 0 2539 1 0 0 25 0 8 0 886428110 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=732384 CPUtime=25.44 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 35501 0 0 0 2539 5 0 0 25 0 8 0 886428110 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=732384 CPUtime=25.05 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 9619 0 0 0 2504 1 0 0 25 0 8 0 886428110 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=732384 CPUtime=25.45 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 14160 0 0 0 2543 2 0 0 25 0 8 0 886428110 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4348903 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=732384 CPUtime=25.45 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 19528 0 0 0 2542 3 0 0 25 0 8 0 886428110 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351901 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=732384 CPUtime=25.45 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 9177 0 0 0 2544 1 0 0 25 0 8 0 886428110 749961216 94167 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352760 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 203.27
Current children cumulated vsize (KiB) 732384

[startup+51.1009 s]
/proc/loadavg: 8.11 7.98 6.25 9/499 31945
/proc/meminfo: memFree=26679788/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=858144 CPUtime=407.64 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 267591 0 0 0 40720 44 0 0 25 0 8 0 886428106 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 214536 123975 1030 106 0 210039 0
[pid=31936/tid=31938] ppid=31934 vsize=858144 CPUtime=51.06 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 23226 0 0 0 5102 4 0 0 25 0 8 0 886428110 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=858144 CPUtime=51 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 15861 0 0 0 5098 2 0 0 25 0 8 0 886428110 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=858144 CPUtime=51.05 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 91028 0 0 0 5092 13 0 0 25 0 8 0 886428110 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=858144 CPUtime=50.23 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 10839 0 0 0 5022 1 0 0 25 0 8 0 886428110 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=858144 CPUtime=51.05 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 21921 0 0 0 5102 3 0 0 25 0 8 0 886428110 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4339546 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=858144 CPUtime=51.04 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 28794 0 0 0 5099 5 0 0 25 0 8 0 886428110 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351798 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=858144 CPUtime=51.05 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 13453 0 0 0 5103 2 0 0 25 0 8 0 886428110 878739456 123975 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 407.64
Current children cumulated vsize (KiB) 858144

[startup+102.3 s]
/proc/loadavg: 8.18 8.01 6.35 9/499 31956
/proc/meminfo: memFree=26571040/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=870424 CPUtime=816.35 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 455249 0 0 0 81557 78 0 0 25 0 8 0 886428106 891314176 151134 33554432000 4194304 4625164 140734955805616 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 217606 151134 875 106 0 213109 0
[pid=31936/tid=31938] ppid=31934 vsize=870424 CPUtime=102.23 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 25184 0 0 0 10218 5 0 0 25 0 8 0 886428110 891314176 151134 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351781 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=870424 CPUtime=102.21 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 18556 0 0 0 10217 4 0 0 25 0 8 0 886428110 891314176 151134 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352027 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=870424 CPUtime=102.24 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 228735 0 0 0 10189 35 0 0 25 0 8 0 886428110 891314176 151134 33554432000 4194304 4625164 140734955805616 18446744073709551615 4335141 0 0 4096 2 0 0 0 -1 3 0 0 0

################
# More data... #
################

[pid=31936/tid=31940] ppid=31934 vsize=4677704 CPUtime=4661.46 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 26300853 0 0 0 462060 4086 0 0 25 0 8 0 886428110 4789968896 945164 33554432000 4194304 4625164 140734955805616 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=4677704 CPUtime=4655.33 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 82883 0 0 0 465513 20 0 0 25 0 8 0 886428110 4789968896 945164 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=4677704 CPUtime=4662.41 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 175384 0 0 0 466206 35 0 0 25 0 8 0 886428110 4789968896 945164 33554432000 4194304 4625164 140734955805616 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=4677704 CPUtime=4661.11 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 166955 0 0 0 466073 38 0 0 25 0 8 0 886428110 4789968896 945164 33554432000 4194304 4625164 140734955805616 18446744073709551615 4330822 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=4677704 CPUtime=4662.32 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 84598 0 0 0 466185 47 0 0 25 0 8 0 886428110 4789968896 945165 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37219.7
Current children cumulated vsize (KiB) 4677704

[startup+4722.3 s]
/proc/loadavg: 8.03 8.05 8.00 9/499 32091
/proc/meminfo: memFree=23348272/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=4725704 CPUtime=37698.7 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 29302879 0 0 0 3765107 4765 0 0 25 0 8 0 886428106 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 1181426 953646 353 106 0 1176929 0
[pid=31936/tid=31938] ppid=31934 vsize=4725704 CPUtime=4722.37 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 132287 0 0 0 472207 30 0 0 25 0 8 0 886428110 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=4725704 CPUtime=4651.23 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 169770 0 0 0 464996 127 0 0 25 0 8 0 886428110 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352765 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=4725704 CPUtime=4721.46 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 26723971 0 0 0 467996 4150 0 0 25 0 8 0 886428110 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4552402 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=4725704 CPUtime=4715.33 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 82949 0 0 0 471513 20 0 0 25 0 8 0 886428110 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351844 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=4725704 CPUtime=4722.41 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 175521 0 0 0 472206 35 0 0 25 0 8 0 886428110 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4460393 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=4725704 CPUtime=4721.1 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 179322 0 0 0 472071 39 0 0 25 0 8 0 886428110 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=4725704 CPUtime=4722.32 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 84646 0 0 0 472185 47 0 0 25 0 8 0 886428110 4839120896 953646 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351894 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 37698.7
Current children cumulated vsize (KiB) 4725704

[startup+4782.3 s]
/proc/loadavg: 8.01 8.04 8.00 9/499 32093
/proc/meminfo: memFree=23191620/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=5213556 CPUtime=38177.7 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 30046676 0 0 0 3812886 4880 0 0 25 0 8 0 886428106 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 1303389 992881 352 106 0 1298892 0
[pid=31936/tid=31938] ppid=31934 vsize=5213556 CPUtime=4782.37 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 132555 0 0 0 478207 30 0 0 25 0 8 0 886428110 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=5213556 CPUtime=4710.24 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 173654 0 0 0 470889 135 0 0 25 0 8 0 886428110 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=5213556 CPUtime=4781.43 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 27180398 0 0 0 473924 4219 0 0 25 0 8 0 886428110 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=5213556 CPUtime=4775.33 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 82984 0 0 0 477513 20 0 0 25 0 8 0 886428110 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351863 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=5213556 CPUtime=4782.41 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 175678 0 0 0 478206 35 0 0 25 0 8 0 886428110 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=5213556 CPUtime=4781.09 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 179553 0 0 0 478070 39 0 0 25 0 8 0 886428110 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=5213556 CPUtime=4782.31 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 86631 0 0 0 478182 49 0 0 25 0 8 0 886428110 5338681344 992881 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38177.7
Current children cumulated vsize (KiB) 5213556

[startup+4842.3 s]
/proc/loadavg: 8.00 8.03 8.00 9/499 32094
/proc/meminfo: memFree=23174464/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=5213556 CPUtime=38656.6 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 30498017 0 0 0 3860710 4955 0 0 25 0 8 0 886428106 5338681344 997252 33554432000 4194304 4625164 140734955805616 18446744073709551615 4329151 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 1303389 997253 352 106 0 1298892 0
[pid=31936/tid=31938] ppid=31934 vsize=5213556 CPUtime=4842.38 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 132648 0 0 0 484208 30 0 0 25 0 8 0 886428110 5338681344 997254 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=5213556 CPUtime=4769.23 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 173776 0 0 0 476788 135 0 0 25 0 8 0 886428110 5338681344 997254 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=5213556 CPUtime=4841.43 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 27627569 0 0 0 479853 4290 0 0 25 0 8 0 886428110 5338681344 997255 33554432000 4194304 4625164 140734955805616 18446744073709551615 4339447 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=5213556 CPUtime=4835.33 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 83078 0 0 0 483513 20 0 0 25 0 8 0 886428110 5338681344 997255 33554432000 4194304 4625164 140734955805616 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=5213556 CPUtime=4842.41 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 175849 0 0 0 484206 35 0 0 25 0 8 0 886428110 5338681344 997256 33554432000 4194304 4625164 140734955805616 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=5213556 CPUtime=4841.07 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 179801 0 0 0 484068 39 0 0 25 0 8 0 886428110 5338681344 997256 33554432000 4194304 4625164 140734955805616 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=5213556 CPUtime=4842.31 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 88705 0 0 0 484179 52 0 0 25 0 8 0 886428110 5338681344 997256 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 38656.7
Current children cumulated vsize (KiB) 5213556

[startup+4902.3 s]
/proc/loadavg: 8.04 8.04 8.00 9/499 32096
/proc/meminfo: memFree=23152716/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=5261556 CPUtime=39135.8 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 30903050 0 0 0 3908556 5023 0 0 25 0 8 0 886428106 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4511904 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 1315389 1002553 352 106 0 1310892 0
[pid=31936/tid=31938] ppid=31934 vsize=5261556 CPUtime=4902.4 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 144799 0 0 0 490208 32 0 0 25 0 8 0 886428110 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4488046 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=5261556 CPUtime=4828.24 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 173928 0 0 0 482689 135 0 0 25 0 8 0 886428110 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352731 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=5261556 CPUtime=4901.45 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 28018391 0 0 0 485789 4356 0 0 25 0 8 0 886428110 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352021 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=5261556 CPUtime=4895.35 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 83177 0 0 0 489515 20 0 0 25 0 8 0 886428110 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=5261556 CPUtime=4902.43 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 175977 0 0 0 490208 35 0 0 25 0 8 0 886428110 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=5261556 CPUtime=4901.07 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 180048 0 0 0 490068 39 0 0 25 0 8 0 886428110 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=5261556 CPUtime=4902.33 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 88728 0 0 0 490181 52 0 0 25 0 8 0 886428110 5387833344 1002553 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39135.8
Current children cumulated vsize (KiB) 5261556

[startup+4962.3 s]
/proc/loadavg: 8.05 8.04 8.00 9/499 32097
/proc/meminfo: memFree=23136180/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=5327092 CPUtime=39614.7 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 31298420 0 0 0 3956379 5087 0 0 25 0 8 0 886428106 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4393097 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 1331773 1005048 352 106 0 1327276 0
[pid=31936/tid=31938] ppid=31934 vsize=5327092 CPUtime=4962.39 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 145090 0 0 0 496207 32 0 0 25 0 8 0 886428110 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4330841 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=5327092 CPUtime=4887.24 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 173944 0 0 0 488589 135 0 0 25 0 8 0 886428110 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4352032 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=5327092 CPUtime=4961.43 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 28411606 0 0 0 491724 4419 0 0 25 0 8 0 886428110 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4547280 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=5327092 CPUtime=4955.34 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 83256 0 0 0 495514 20 0 0 25 0 8 0 886428110 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351849 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=5327092 CPUtime=4962.41 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 176102 0 0 0 496206 35 0 0 25 0 8 0 886428110 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4329721 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=5327092 CPUtime=4961.05 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 180430 0 0 0 496066 39 0 0 25 0 8 0 886428110 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=5327092 CPUtime=4962.31 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 88851 0 0 0 496179 52 0 0 25 0 8 0 886428110 5454942208 1005048 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 39614.7
Current children cumulated vsize (KiB) 5327092



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+5010.6 s]
/proc/loadavg: 8.06 8.05 8.00 9/499 32099
/proc/meminfo: memFree=23122628/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=5327092 CPUtime=40000.4 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) R 31934 31936 31837 0 -1 4202496 31641710 0 0 0 3994902 5137 0 0 25 0 8 0 886428106 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4511930 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 1331773 1010137 352 106 0 1327276 0
[pid=31936/tid=31938] ppid=31934 vsize=5327092 CPUtime=5010.71 cores=0-7
/proc/31936/task/31938/stat : 31938 (SArTagnan) R 31934 31936 31837 0 -1 4202560 145226 0 0 0 501039 32 0 0 25 0 8 0 886428110 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 1 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=5327092 CPUtime=4934.74 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4202560 174207 0 0 0 493339 135 0 0 25 0 8 0 886428110 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4339643 0 0 4096 2 0 0 0 -1 7 0 0 0
[pid=31936/tid=31940] ppid=31934 vsize=5327092 CPUtime=5009.75 cores=0-7
/proc/31936/task/31940/stat : 31940 (SArTagnan) R 31934 31936 31837 0 -1 4202560 28752847 0 0 0 496506 4469 0 0 25 0 8 0 886428110 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 3 0 0 0
[pid=31936/tid=31941] ppid=31934 vsize=5327092 CPUtime=5003.66 cores=0-7
/proc/31936/task/31941/stat : 31941 (SArTagnan) R 31934 31936 31837 0 -1 4202560 83343 0 0 0 500346 20 0 0 25 0 8 0 886428110 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 5 0 0 0
[pid=31936/tid=31942] ppid=31934 vsize=5327092 CPUtime=5010.74 cores=0-7
/proc/31936/task/31942/stat : 31942 (SArTagnan) R 31934 31936 31837 0 -1 4202560 176261 0 0 0 501038 36 0 0 25 0 8 0 886428110 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4329981 0 0 4096 2 0 0 0 -1 4 0 0 0
[pid=31936/tid=31943] ppid=31934 vsize=5327092 CPUtime=5009.35 cores=0-7
/proc/31936/task/31943/stat : 31943 (SArTagnan) R 31934 31936 31837 0 -1 4202560 180680 0 0 0 500896 39 0 0 25 0 8 0 886428110 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4331101 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=31936/tid=31944] ppid=31934 vsize=5327092 CPUtime=5010.63 cores=0-7
/proc/31936/task/31944/stat : 31944 (SArTagnan) R 31934 31936 31837 0 -1 4202560 88873 0 0 0 501011 52 0 0 25 0 8 0 886428110 5454942208 1010137 33554432000 4194304 4625164 140734955805616 18446744073709551615 4351979 0 0 4096 2 0 0 0 -1 0 0 0 0
Current children cumulated CPU time (s) 40000.4
Current children cumulated vsize (KiB) 5327092

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

[startup+5010.8 s]
/proc/loadavg: 8.06 8.05 8.00 9/499 32099
/proc/meminfo: memFree=23122628/32950928 swapFree=34421304/67111528
[pid=31936] ppid=31934 vsize=0 CPUtime=40000.6 cores=0-7
/proc/31936/stat : 31936 (SArTagnan) Z 31934 31936 31837 0 -1 4203548 31641711 0 0 0 3994906 5155 0 0 25 0 2 0 886428106 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 17 6 0 0 0
/proc/31936/statm: 0 0 0 0 0 0 0
[pid=31936/tid=31939] ppid=31934 vsize=0 CPUtime=4934.92 cores=0-7
/proc/31936/task/31939/stat : 31939 (SArTagnan) R 31934 31936 31837 0 -1 4203588 174208 0 0 0 493339 153 0 0 25 0 2 0 886428110 0 0 33554432000 0 0 0 0 0 0 0 4096 2 18446744073709551615 0 0 -1 5 0 0 0
Current children cumulated CPU time (s) 40000.6
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5010.81
CPU time (s): 40000.6
CPU user time (s): 39949.1
CPU system time (s): 51.5602
CPU usage (%): 798.286
Max. virtual memory (cumulated for all children) (KiB): 6184940

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 39949.1
system time used= 51.5602
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 31641711
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 8405
involuntary context switches= 168914

runsolver used 21.9047 second user time and 60.9037 second system time

The end

Launcher Data 

Begin job on node113 at 2011-06-01 22:59:34
IDJOB=3439396
IDBENCH=84560
IDSOLVER=1936
FILE ID=node113/3439396-1306961974
RUNJOBID= node113-1306960629-31851
PBS_JOBID= 13497640
Free space on /tmp= 73532 MiB

SOLVER NAME= SArTagnan 2011-05-15 (fixed)
BENCH NAME= SAT04/handmade/kexu/frb-0.8/frb59-26-2.used-as.sat04-892.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3439396-1306961974/watcher-3439396-1306961974 -o /tmp/evaluation-result-3439396-1306961974/solver-3439396-1306961974 -C 40000 -W 5100 -M 31000  SArTagnan HOME/instance-3439396-1306961974.cnf -threads=8

TIME LIMIT= 40000 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= fd8d5783ed404e5c6590d627ffdcca50
RANDOM SEED=538612366

node113.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.846
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32950928 kB
MemFree:      27172352 kB
Buffers:        111216 kB
Cached:         360116 kB
SwapCached:      13476 kB
Active:        5159108 kB
Inactive:       338824 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32950928 kB
LowFree:      27172352 kB
SwapTotal:    67111528 kB
SwapFree:     34421304 kB
Dirty:            1960 kB
Writeback:           0 kB
AnonPages:     5024468 kB
Mapped:          20632 kB
Slab:            92772 kB
PageTables:     116968 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83586992 kB
Committed_AS: 154762216 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73532 MiB
End job on node113 at 2011-06-02 00:23:07