Trace number 3288071

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
SArTagnan 2011-03-24? (TO) 4800.22 1201.87

General information on the benchmark

Nameapplication/jarvisalo/
HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k45-pdtpmsgoodbakery-tseitin.cnf
MD5SUM3fad4722cb319d6fee4192f2fe2c31ed
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark166.168
Satisfiable
(Un)Satisfiability was proved
Number of variables98935
Number of clauses296405
Sum of the clauses size691609
Maximum clause length3
Minimum clause length1
Number of clauses of size 12
Number of clauses of size 2197602
Number of clauses of size 398801
Number of clauses of size 40
Number of clauses of size 50
Number of clauses of size over 50

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 935)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3288071-1304072801/watcher-3288071-1304072801 -o /tmp/evaluation-result-3288071-1304072801/solver-3288071-1304072801 -C 4800 -W 1300 -M 15500 SArTagnan HOME/instance-3288071-1304072801.cnf -threads=4 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 4800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 4830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 1300 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 7.75 7.99 7.97 5/180 22853
/proc/meminfo: memFree=11517752/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=25072 CPUtime=0 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 1487 0 0 0 0 0 0 0 23 0 1 0 597510819 25673728 1424 33554432000 4194304 4623224 140734457990960 18446744073709551615 4296444 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/22853/statm: 6268 1435 290 105 0 1772 0

[startup+0.0511641 s]
/proc/loadavg: 7.75 7.99 7.97 5/180 22853
/proc/meminfo: memFree=11517752/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=40388 CPUtime=0.04 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 5253 0 0 0 4 0 0 0 23 0 1 0 597510819 41357312 3644 33554432000 4194304 4623224 140734457990960 18446744073709551615 216734876117 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/22853/statm: 10097 3667 297 105 0 5601 0
Current children cumulated CPU time (s) 0.04
Current children cumulated vsize (KiB) 40388

[startup+0.101155 s]
/proc/loadavg: 7.75 7.99 7.97 5/180 22853
/proc/meminfo: memFree=11517752/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=43944 CPUtime=0.09 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 8226 0 0 0 8 1 0 0 23 0 1 0 597510819 44998656 4948 33554432000 4194304 4623224 140734457990960 18446744073709551615 4366247 0 0 4096 2 0 0 0 17 4 0 0 0
/proc/22853/statm: 10986 4949 297 105 0 6490 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 43944

[startup+0.301082 s]
/proc/loadavg: 7.75 7.99 7.97 5/180 22853
/proc/meminfo: memFree=11517752/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=329272 CPUtime=0.75 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 22662 0 0 0 73 2 0 0 25 0 4 0 597510819 337174528 18240 33554432000 4194304 4623224 140734457990960 18446744073709551615 4412468 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 82318 18240 335 105 0 77822 0
Current children cumulated CPU time (s) 0.75
Current children cumulated vsize (KiB) 329272

[startup+0.701012 s]
/proc/loadavg: 7.75 7.99 7.97 5/180 22853
/proc/meminfo: memFree=11517752/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=329272 CPUtime=2.36 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 24015 0 0 0 233 3 0 0 25 0 4 0 597510819 337174528 19464 33554432000 4194304 4623224 140734457990960 18446744073709551615 4412473 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 82318 19464 343 105 0 77822 0
Current children cumulated CPU time (s) 2.36
Current children cumulated vsize (KiB) 329272

[startup+1.50087 s]
/proc/loadavg: 7.75 7.99 7.97 9/184 22857
/proc/meminfo: memFree=11442708/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=335880 CPUtime=5.55 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 29859 0 0 0 551 4 0 0 25 0 4 0 597510819 343941120 20575 33554432000 4194304 4623224 140734457990960 18446744073709551615 4412464 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 83970 20575 343 105 0 79474 0
[pid=22853/tid=22855] ppid=22851 vsize=335880 CPUtime=1.35 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 2050 0 0 0 135 0 0 0 25 0 4 0 597510834 343941120 20575 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289678 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=335880 CPUtime=1.35 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 2043 0 0 0 135 0 0 0 25 0 4 0 597510834 343941120 20575 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289714 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=335880 CPUtime=1.35 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 5028 0 0 0 134 1 0 0 25 0 4 0 597510834 343941120 20575 33554432000 4194304 4623224 140734457990960 18446744073709551615 4364669 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 5.55
Current children cumulated vsize (KiB) 335880

[startup+3.10059 s]
/proc/loadavg: 7.75 7.99 7.97 9/184 22857
/proc/meminfo: memFree=11429936/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=603060 CPUtime=11.86 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 237445 0 0 0 1151 35 0 0 25 0 4 0 597510819 617533440 103505 33554432000 4194304 4623224 140734457990960 18446744073709551615 4306456 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 150765 103507 343 105 0 146269 0
[pid=22853/tid=22855] ppid=22851 vsize=603060 CPUtime=2.92 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 40418 0 0 0 288 4 0 0 18 0 4 0 597510834 617533440 103507 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376672 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=603060 CPUtime=2.92 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 39513 0 0 0 287 5 0 0 18 0 4 0 597510834 617533440 103510 33554432000 4194304 4623224 140734457990960 18446744073709551615 4366247 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=603060 CPUtime=2.91 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 44213 0 0 0 285 6 0 0 18 0 4 0 597510834 617533440 103512 33554432000 4194304 4623224 140734457990960 18446744073709551615 4488767 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 11.86
Current children cumulated vsize (KiB) 603060

[startup+6.30104 s]
/proc/loadavg: 7.85 8.01 7.97 9/184 22857
/proc/meminfo: memFree=11069096/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=670644 CPUtime=24.63 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 264999 0 0 0 2424 39 0 0 25 0 4 0 597510819 686739456 114534 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 167661 114534 344 105 0 163165 0
[pid=22853/tid=22855] ppid=22851 vsize=670644 CPUtime=6.12 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 41908 0 0 0 608 4 0 0 25 0 4 0 597510834 686739456 114534 33554432000 4194304 4623224 140734457990960 18446744073709551615 4377133 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=670644 CPUtime=6.12 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 42857 0 0 0 606 6 0 0 25 0 4 0 597510834 686739456 114534 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=670644 CPUtime=6.09 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 62510 0 0 0 600 9 0 0 25 0 4 0 597510834 686739456 114534 33554432000 4194304 4623224 140734457990960 18446744073709551615 4577479 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 24.63
Current children cumulated vsize (KiB) 670644

[startup+12.7009 s]
/proc/loadavg: 7.86 8.01 7.97 9/184 22857
/proc/meminfo: memFree=11048440/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=670644 CPUtime=50.2 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 316846 0 0 0 4974 46 0 0 25 0 4 0 597510819 686739456 117049 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510169 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 167661 117049 344 105 0 163165 0
[pid=22853/tid=22855] ppid=22851 vsize=670644 CPUtime=12.52 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 41954 0 0 0 1248 4 0 0 25 0 4 0 597510834 686739456 117049 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=670644 CPUtime=12.52 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 42945 0 0 0 1246 6 0 0 25 0 4 0 597510834 686739456 117049 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=670644 CPUtime=12.45 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 113691 0 0 0 1229 16 0 0 25 0 4 0 597510834 686739456 117049 33554432000 4194304 4623224 140734457990960 18446744073709551615 4545358 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 50.2
Current children cumulated vsize (KiB) 670644

[startup+25.5007 s]
/proc/loadavg: 7.96 8.02 7.98 9/184 22857
/proc/meminfo: memFree=11028100/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=670644 CPUtime=101.32 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 430090 0 0 0 10070 62 0 0 25 0 4 0 597510819 686739456 118174 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510285 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 167661 118174 344 105 0 163165 0
[pid=22853/tid=22855] ppid=22851 vsize=670644 CPUtime=25.32 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 42000 0 0 0 2528 4 0 0 25 0 4 0 597510834 686739456 118174 33554432000 4194304 4623224 140734457990960 18446744073709551615 4463773 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=670644 CPUtime=25.32 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 42999 0 0 0 2526 6 0 0 25 0 4 0 597510834 686739456 118174 33554432000 4194304 4623224 140734457990960 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=670644 CPUtime=25.18 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 225700 0 0 0 2487 31 0 0 25 0 4 0 597510834 686739456 118174 33554432000 4194304 4623224 140734457990960 18446744073709551615 4545358 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 101.32
Current children cumulated vsize (KiB) 670644

[startup+51.1002 s]
/proc/loadavg: 7.97 8.02 7.98 9/184 22858
/proc/meminfo: memFree=11000164/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=690840 CPUtime=203.59 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 641015 0 0 0 20261 98 0 0 25 0 4 0 597510819 707420160 122448 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510105 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 172710 122448 344 105 0 168214 0
[pid=22853/tid=22855] ppid=22851 vsize=690840 CPUtime=50.92 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 42037 0 0 0 5088 4 0 0 25 0 4 0 597510834 707420160 122448 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376625 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=690840 CPUtime=50.93 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 43028 0 0 0 5086 7 0 0 25 0 4 0 597510834 707420160 122448 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289788 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=690840 CPUtime=50.64 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 425151 0 0 0 5000 64 0 0 25 0 4 0 597510834 707420160 122448 33554432000 4194304 4623224 140734457990960 18446744073709551615 4364906 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 203.59
Current children cumulated vsize (KiB) 690840

[startup+102.306 s]
/proc/loadavg: 8.11 8.04 7.99 9/184 22859
/proc/meminfo: memFree=10937908/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=690840 CPUtime=408.13 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 1061960 0 0 0 40651 162 0 0 25 0 4 0 597510819 707420160 128293 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 172710 128293 344 105 0 168214 0
[pid=22853/tid=22855] ppid=22851 vsize=690840 CPUtime=102.13 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 42230 0 0 0 10209 4 0 0 25 0 4 0 597510834 707420160 128293 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=690840 CPUtime=102.13 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 43090 0 0 0 10206 7 0 0 25 0 4 0 597510834 707420160 128293 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376729 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=690840 CPUtime=101.56 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 843271 0 0 0 10031 125 0 0 25 0 4 0 597510834 707420160 128293 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289742 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 408.13
Current children cumulated vsize (KiB) 690840

[startup+162.301 s]
/proc/loadavg: 8.09 8.05 7.99 9/184 22861
/proc/meminfo: memFree=10859636/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=722752 CPUtime=647.78 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 1497705 0 0 0 64550 228 0 0 25 0 4 0 597510819 740098048 141849 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510090 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 180688 141849 344 105 0 176192 0
[pid=22853/tid=22855] ppid=22851 vsize=722752 CPUtime=162.13 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 54265 0 0 0 16206 7 0 0 25 0 4 0 597510834 740098048 141849 33554432000 4194304 4623224 140734457990960 18446744073709551615 4377133 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=722752 CPUtime=162.13 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 43886 0 0 0 16206 7 0 0 25 0 4 0 597510834 740098048 141849 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289853 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=722752 CPUtime=161.22 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 1259870 0 0 0 15936 186 0 0 25 0 4 0 597510834 740098048 141849 33554432000 4194304 4623224 140734457990960 18446744073709551615 4363989 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 647.78
Current children cumulated vsize (KiB) 722752

[startup+222.3 s]
/proc/loadavg: 8.15 8.07 8.00 9/184 22863
/proc/meminfo: memFree=10773920/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=871912 CPUtime=887.45 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 1895679 0 0 0 88452 293 0 0 25 0 4 0 597510819 892837888 155342 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510142 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 217978 155342 344 105 0 213482 0
[pid=22853/tid=22855] ppid=22851 vsize=871912 CPUtime=222.13 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 57629 0 0 0 22203 10 0 0 25 0 4 0 597510834 892837888 155342 33554432000 4194304 4623224 140734457990960 18446744073709551615 4354519 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=871912 CPUtime=222.12 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 54932 0 0 0 22204 8 0 0 25 0 4 0 597510834 892837888 155342 33554432000 4194304 4623224 140734457990960 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=871912 CPUtime=220.89 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 1616041 0 0 0 21846 243 0 0 25 0 4 0 597510834 892837888 155342 33554432000 4194304 4623224 140734457990960 18446744073709551615 4364424 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 887.45
Current children cumulated vsize (KiB) 871912

[startup+282.301 s]

################
# More data... #
################

[pid=22853] ppid=22851 vsize=1056544 CPUtime=2565.2 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 4492499 0 0 0 255820 700 0 0 25 0 4 0 597510819 1081901056 225718 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 264136 225718 344 105 0 259640 0
[pid=22853/tid=22855] ppid=22851 vsize=1056544 CPUtime=642.14 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 95670 0 0 0 64184 30 0 0 25 0 4 0 597510834 1081901056 225718 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1056544 CPUtime=642.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 60659 0 0 0 64203 11 0 0 25 0 4 0 597510834 1081901056 225718 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1056544 CPUtime=638.6 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 4157476 0 0 0 63239 621 0 0 25 0 4 0 597510834 1081901056 225718 33554432000 4194304 4623224 140734457990960 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2565.2
Current children cumulated vsize (KiB) 1056544

[startup+702.301 s]
/proc/loadavg: 8.10 8.06 8.01 9/184 22906
/proc/meminfo: memFree=10868120/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1145556 CPUtime=2804.85 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 4663490 0 0 0 279758 727 0 0 25 0 4 0 597510819 1173049344 236868 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 286389 236868 344 105 0 281893 0
[pid=22853/tid=22855] ppid=22851 vsize=1145556 CPUtime=702.13 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 97305 0 0 0 70182 31 0 0 25 0 4 0 597510834 1173049344 236868 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289678 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1145556 CPUtime=702.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 61400 0 0 0 70203 11 0 0 25 0 4 0 597510834 1173049344 236868 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1145556 CPUtime=698.28 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 4324542 0 0 0 69181 647 0 0 25 0 4 0 597510834 1173049344 236868 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376612 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 2804.85
Current children cumulated vsize (KiB) 1145556

[startup+762.306 s]
/proc/loadavg: 8.37 8.14 8.03 9/184 22911
/proc/meminfo: memFree=10800080/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1262268 CPUtime=3044.54 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 5101206 0 0 0 303659 795 0 0 25 0 4 0 597510819 1292562432 247605 33554432000 4194304 4623224 140734457990960 18446744073709551615 4509955 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 315567 247605 344 105 0 311071 0
[pid=22853/tid=22855] ppid=22851 vsize=1262268 CPUtime=762.14 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 121543 0 0 0 76180 34 0 0 25 0 4 0 597510834 1292562432 247605 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1262268 CPUtime=762.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 63039 0 0 0 76202 12 0 0 25 0 4 0 597510834 1292562432 247605 33554432000 4194304 4623224 140734457990960 18446744073709551615 4366391 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1262268 CPUtime=757.94 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 4734488 0 0 0 75084 710 0 0 25 0 4 0 597510834 1292562432 247605 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3044.54
Current children cumulated vsize (KiB) 1262268

[startup+822.301 s]
/proc/loadavg: 8.17 8.13 8.03 9/184 22913
/proc/meminfo: memFree=10724272/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1347188 CPUtime=3284.18 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 5427849 0 0 0 327574 844 0 0 25 0 4 0 597510819 1379520512 257968 33554432000 4194304 4623224 140734457990960 18446744073709551615 4507171 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 336797 257968 344 105 0 332301 0
[pid=22853/tid=22855] ppid=22851 vsize=1347188 CPUtime=822.14 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 123148 0 0 0 82179 35 0 0 25 0 4 0 597510834 1379520512 257968 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289853 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1347188 CPUtime=822.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 63353 0 0 0 82202 12 0 0 25 0 4 0 597510834 1379520512 257968 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1347188 CPUtime=817.62 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 5019530 0 0 0 81009 753 0 0 25 0 4 0 597510834 1379520512 257968 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3284.18
Current children cumulated vsize (KiB) 1347188

[startup+882.3 s]
/proc/loadavg: 8.10 8.11 8.03 9/184 22914
/proc/meminfo: memFree=10666096/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1347188 CPUtime=3523.85 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 5804126 0 0 0 351485 900 0 0 25 0 4 0 597510819 1379520512 264919 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510034 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 336797 264919 344 105 0 332301 0
[pid=22853/tid=22855] ppid=22851 vsize=1347188 CPUtime=882.14 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 125479 0 0 0 88178 36 0 0 25 0 4 0 597510834 1379520512 264919 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1347188 CPUtime=882.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 63940 0 0 0 88202 12 0 0 25 0 4 0 597510834 1379520512 264919 33554432000 4194304 4623224 140734457990960 18446744073709551615 4330746 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1347188 CPUtime=877.28 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 5391850 0 0 0 86921 807 0 0 25 0 4 0 597510834 1379520512 264919 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289678 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3523.85
Current children cumulated vsize (KiB) 1347188

[startup+942.301 s]
/proc/loadavg: 8.20 8.14 8.04 9/184 22916
/proc/meminfo: memFree=10603820/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1377500 CPUtime=3763.53 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 6230912 0 0 0 375391 962 0 0 25 0 4 0 597510819 1410560000 272062 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 344375 272062 344 105 0 339879 0
[pid=22853/tid=22855] ppid=22851 vsize=1377500 CPUtime=942.14 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 127575 0 0 0 94177 37 0 0 25 0 4 0 597510834 1410560000 272062 33554432000 4194304 4623224 140734457990960 18446744073709551615 4463552 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1377500 CPUtime=942.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 80399 0 0 0 94199 15 0 0 25 0 4 0 597510834 1410560000 272062 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376711 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1377500 CPUtime=936.96 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 5799166 0 0 0 92831 865 0 0 25 0 4 0 597510834 1410560000 272062 33554432000 4194304 4623224 140734457990960 18446744073709551615 4545317 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 3763.53
Current children cumulated vsize (KiB) 1377500

[startup+1002.3 s]
/proc/loadavg: 8.20 8.14 8.05 9/184 22917
/proc/meminfo: memFree=10548240/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1377500 CPUtime=4003.2 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 6672922 0 0 0 399288 1032 0 0 25 0 4 0 597510819 1410560000 279130 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510316 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 344375 279130 344 105 0 339879 0
[pid=22853/tid=22855] ppid=22851 vsize=1377500 CPUtime=1002.15 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 129525 0 0 0 100176 39 0 0 25 0 4 0 597510834 1410560000 279130 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376629 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1377500 CPUtime=1002.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 80938 0 0 0 100199 15 0 0 25 0 4 0 597510834 1410560000 279130 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289723 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1377500 CPUtime=996.63 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 6237668 0 0 0 98730 933 0 0 25 0 4 0 597510834 1410560000 279130 33554432000 4194304 4623224 140734457990960 18446744073709551615 216734876117 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4003.2
Current children cumulated vsize (KiB) 1377500

[startup+1062.3 s]
/proc/loadavg: 8.19 8.14 8.05 9/184 22919
/proc/meminfo: memFree=10482868/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1512980 CPUtime=4242.88 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 7007425 0 0 0 423200 1088 0 0 25 0 4 0 597510819 1549291520 289237 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510316 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 378245 289237 344 105 0 373749 0
[pid=22853/tid=22855] ppid=22851 vsize=1512980 CPUtime=1062.15 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 131448 0 0 0 106175 40 0 0 25 0 4 0 597510834 1549291520 289237 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289757 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1512980 CPUtime=1062.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 82573 0 0 0 106197 17 0 0 25 0 4 0 597510834 1549291520 289237 33554432000 4194304 4623224 140734457990960 18446744073709551615 4377576 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1512980 CPUtime=1056.32 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 6567606 0 0 0 104646 986 0 0 25 0 4 0 597510834 1549291520 289237 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4242.88
Current children cumulated vsize (KiB) 1512980

[startup+1122.3 s]
/proc/loadavg: 8.11 8.13 8.05 9/184 22921
/proc/meminfo: memFree=10421960/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1525756 CPUtime=4482.54 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 7213895 0 0 0 447137 1117 0 0 25 0 4 0 597510819 1562374144 296510 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510238 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 381439 296510 344 105 0 376943 0
[pid=22853/tid=22855] ppid=22851 vsize=1525756 CPUtime=1122.14 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 132553 0 0 0 112174 40 0 0 25 0 4 0 597510834 1562374144 296510 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1525756 CPUtime=1122.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 82751 0 0 0 112197 17 0 0 25 0 4 0 597510834 1562374144 296510 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376827 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1525756 CPUtime=1115.99 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 6770106 0 0 0 110585 1014 0 0 25 0 4 0 597510834 1562374144 296510 33554432000 4194304 4623224 140734457990960 18446744073709551615 4552693 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4482.54
Current children cumulated vsize (KiB) 1525756

[startup+1182.3 s]
/proc/loadavg: 8.08 8.12 8.05 9/184 22922
/proc/meminfo: memFree=10350016/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1529112 CPUtime=4722.23 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 7548680 0 0 0 471060 1163 0 0 25 0 4 0 597510819 1565810688 309224 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510064 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 382278 309224 345 105 0 377782 0
[pid=22853/tid=22855] ppid=22851 vsize=1529112 CPUtime=1182.15 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 133468 0 0 0 118174 41 0 0 25 0 4 0 597510834 1565810688 309224 33554432000 4194304 4623224 140734457990960 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1529112 CPUtime=1182.14 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 83432 0 0 0 118197 17 0 0 25 0 4 0 597510834 1565810688 309224 33554432000 4194304 4623224 140734457990960 18446744073709551615 4461212 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1529112 CPUtime=1175.67 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 7097411 0 0 0 116509 1058 0 0 25 0 4 0 597510834 1565810688 309224 33554432000 4194304 4623224 140734457990960 18446744073709551615 4364911 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4722.23
Current children cumulated vsize (KiB) 1529112



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+1201.8 s]
/proc/loadavg: 8.06 8.11 8.05 9/184 22923
/proc/meminfo: memFree=10316536/32951124 swapFree=67102140/67111528
[pid=22853] ppid=22851 vsize=1531628 CPUtime=4800.13 cores=0,2,4,6
/proc/22853/stat : 22853 (SArTagnan) R 22851 22853 22146 0 -1 4202496 7588586 0 0 0 478845 1168 0 0 25 0 4 0 597510819 1568387072 314093 33554432000 4194304 4623224 140734457990960 18446744073709551615 4510034 0 0 4096 2 18446744073709551615 0 0 17 4 0 0 0
/proc/22853/statm: 382907 314093 345 105 0 378411 0
[pid=22853/tid=22855] ppid=22851 vsize=1531628 CPUtime=1201.65 cores=0,2,4,6
/proc/22853/task/22855/stat : 22855 (SArTagnan) R 22851 22853 22146 0 -1 4202560 133506 0 0 0 120124 41 0 0 25 0 4 0 597510834 1568387072 314093 33554432000 4194304 4623224 140734457990960 18446744073709551615 4289691 0 0 4096 2 0 0 0 -1 2 0 0 0
[pid=22853/tid=22856] ppid=22851 vsize=1531628 CPUtime=1201.65 cores=0,2,4,6
/proc/22853/task/22856/stat : 22856 (SArTagnan) R 22851 22853 22146 0 -1 4202560 83559 0 0 0 120148 17 0 0 25 0 4 0 597510834 1568387072 314093 33554432000 4194304 4623224 140734457990960 18446744073709551615 4366412 0 0 4096 2 0 0 0 -1 0 0 0 0
[pid=22853/tid=22857] ppid=22851 vsize=1531628 CPUtime=1195.07 cores=0,2,4,6
/proc/22853/task/22857/stat : 22857 (SArTagnan) R 22851 22853 22146 0 -1 4202560 7134508 0 0 0 118444 1063 0 0 25 0 4 0 597510834 1568387072 314093 33554432000 4194304 4623224 140734457990960 18446744073709551615 4376579 0 0 4096 2 0 0 0 -1 6 0 0 0
Current children cumulated CPU time (s) 4800.13
Current children cumulated vsize (KiB) 1531628

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)
Real time (s): 1201.87
CPU time (s): 4800.22
CPU user time (s): 4788.48
CPU system time (s): 11.7442
CPU usage (%): 399.396
Max. virtual memory (cumulated for all children) (KiB): 1531628

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4788.48
system time used= 11.7442
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 7588586
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3788
involuntary context switches= 19167

runsolver used 1.86072 second user time and 4.66529 second system time

The end

Launcher Data 

Begin job on node119 at 2011-04-29 12:26:42
IDJOB=3288071
IDBENCH=82482
IDSOLVER=1656
FILE ID=node119/3288071-1304072801
RUNJOBID= node119-1304071597-22496
PBS_JOBID= 13170477
Free space on /tmp= 71800 MiB

SOLVER NAME= SArTagnan 2011-03-24
BENCH NAME= SAT11/application/jarvisalo/HWMCC10-combination-timeframe-expansion/hwmcc10-timeframe-expansion-k45-pdtpmsgoodbakery-tseitin.cnf
COMMAND LINE= SArTagnan BENCHNAME -threads=NBCORE
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3288071-1304072801/watcher-3288071-1304072801 -o /tmp/evaluation-result-3288071-1304072801/solver-3288071-1304072801 -C 4800 -W 1300 -M 15500  SArTagnan HOME/instance-3288071-1304072801.cnf -threads=4

TIME LIMIT= 4800 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 3fad4722cb319d6fee4192f2fe2c31ed
RANDOM SEED=1657355390

node119.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.63
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.67
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.815
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.83
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      11517916 kB
Buffers:       2233864 kB
Cached:       17316708 kB
SwapCached:       5504 kB
Active:        9202020 kB
Inactive:     11645212 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      11517916 kB
SwapTotal:    67111528 kB
SwapFree:     67102140 kB
Dirty:            5444 kB
Writeback:           8 kB
AnonPages:     1293388 kB
Mapped:          15316 kB
Slab:           521168 kB
PageTables:       7288 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS:  1689764 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 71804 MiB
End job on node119 at 2011-04-29 12:46:45