Trace number 3372471

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Plingeling 587f (fixed)? (TO) 157895 5102.11

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
MD5SUM6c2d25eda70423643946d76935ceb950
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables1512832
Number of clauses8233086
Sum of the clauses size19530286
Maximum clause length1178
Minimum clause length1
Number of clauses of size 1846
Number of clauses of size 28165680
Number of clauses of size 31280
Number of clauses of size 41600
Number of clauses of size 51600
Number of clauses of size over 562080

Solver Data


Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3372471-1305662191/watcher-3372471-1305662191 -o /tmp/evaluation-result-3372471-1305662191/solver-3372471-1305662191 -C 160000 -W 5100 -M 266338304 plingeling -t 32 HOME/instance-3372471-1305662191.cnf 

running on 32 cores: 0,4,8,12,16,20,24,28,1,5,9,13,17,21,25,29,2,6,10,14,18,22,26,30,3,7,11,15,19,23,27,31

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 160000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 160030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 272730423296 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 272730474496 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 0.15 0.03 0.01 1/413 26181
/proc/meminfo: memFree=255363124/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=47808 CPUtime=0.03 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 10010 1250 0 0 1 2 0 0 23 0 1 0 677706235 48955392 9940 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 11952 9955 142 55 0 9838 0

[startup+0.109022 s]
/proc/loadavg: 0.15 0.03 0.01 1/413 26181
/proc/meminfo: memFree=255363124/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=69792 CPUtime=0.09 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 15520 1250 0 0 6 3 0 0 23 0 1 0 677706235 71467008 15450 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650574314 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 17448 15450 142 55 0 15334 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 69792

[startup+0.201012 s]
/proc/loadavg: 0.15 0.03 0.01 1/413 26181
/proc/meminfo: memFree=255363124/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=80148 CPUtime=0.19 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 18109 1250 0 0 15 4 0 0 23 0 1 0 677706235 82071552 18039 268435456000 4194304 4417452 140735443751376 18446744073709551615 4264909 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 20037 18039 142 55 0 17923 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 80148

[startup+0.301017 s]
/proc/loadavg: 0.15 0.03 0.01 1/413 26181
/proc/meminfo: memFree=255363124/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=80148 CPUtime=0.29 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 18109 1250 0 0 25 4 0 0 24 0 1 0 677706235 82071552 18039 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232651 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 20037 18039 142 55 0 17923 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 80148

[startup+0.70103 s]
/proc/loadavg: 0.15 0.03 0.01 1/413 26181
/proc/meminfo: memFree=255363124/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=129320 CPUtime=0.68 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 30421 1250 0 0 62 6 0 0 25 0 1 0 677706235 132423680 30351 268435456000 4194304 4417452 140735443751376 18446744073709551615 4235997 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 32330 30351 145 55 0 30216 0
Current children cumulated CPU time (s) 0.68
Current children cumulated vsize (KiB) 129320

[startup+1.50108 s]
/proc/loadavg: 0.15 0.03 0.01 2/414 26187
/proc/meminfo: memFree=255229048/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=220036 CPUtime=1.48 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 52853 1250 0 0 137 11 0 0 25 0 1 0 677706235 225316864 52783 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 55009 52800 146 55 0 52895 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 220036

[startup+3.10111 s]
/proc/loadavg: 0.15 0.03 0.01 2/414 26187
/proc/meminfo: memFree=255109956/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=378224 CPUtime=3.09 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 92656 1250 0 0 289 20 0 0 25 0 1 0 677706235 387301376 92578 268435456000 4194304 4417452 140735443751376 18446744073709551615 4231477 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 94556 92578 146 55 0 92442 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 378224

[startup+6.30122 s]
/proc/loadavg: 0.22 0.05 0.02 2/414 26187
/proc/meminfo: memFree=254876448/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=743540 CPUtime=6.28 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 184001 1250 0 0 590 38 0 0 25 0 1 0 677706235 761384960 183907 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236017 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 185885 183907 146 55 0 183771 0
Current children cumulated CPU time (s) 6.28
Current children cumulated vsize (KiB) 743540

[startup+12.7004 s]
/proc/loadavg: 0.28 0.06 0.02 2/414 26187
/proc/meminfo: memFree=254390356/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=979840 CPUtime=12.68 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 243100 1250 0 0 1218 50 0 0 25 0 1 0 677706235 1003356160 242982 268435456000 4194304 4417452 140735443751376 18446744073709551615 4209679 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 244960 242982 146 55 0 242846 0
Current children cumulated CPU time (s) 12.68
Current children cumulated vsize (KiB) 979840

[startup+25.5009 s]
/proc/loadavg: 0.44 0.11 0.03 2/414 26187
/proc/meminfo: memFree=253470016/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=1899972 CPUtime=25.48 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 473133 1250 0 0 2456 92 0 0 25 0 1 0 677706235 1945571328 473015 268435456000 4194304 4417452 140735443751376 18446744073709551615 4231537 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 474993 473015 146 55 0 472879 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 1899972

[startup+51.1008 s]
/proc/loadavg: 0.70 0.20 0.06 2/414 26189
/proc/meminfo: memFree=251682168/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=3685328 CPUtime=51.08 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 919472 1250 0 0 4932 176 0 0 25 0 1 0 677706235 3773775872 919354 268435456000 4194304 4417452 140735443751376 18446744073709551615 4323376 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 921332 919354 146 55 0 919218 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 3685328

[startup+102.301 s]
/proc/loadavg: 0.94 0.33 0.12 2/414 26189
/proc/meminfo: memFree=248101716/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=7259100 CPUtime=102.3 cores=0-31
/proc/26181/stat : 26181 (plingeling) R 26179 26181 26155 0 -1 4202496 1812915 1250 0 0 9884 346 0 0 25 0 1 0 677706235 7433318400 1812797 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650955760 0 0 4096 17442 0 0 0 17 4 0 0 0
/proc/26181/statm: 1814775 1812797 146 55 0 1812661 0
Current children cumulated CPU time (s) 102.3
Current children cumulated vsize (KiB) 7259100

[startup+162.302 s]
/proc/loadavg: 7.97 2.00 0.68 12/446 26223
/proc/meminfo: memFree=246143180/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=10298740 CPUtime=576.44 cores=0-31
/proc/26181/stat : 26181 (plingeling) S 26179 26181 26155 0 -1 4202496 4110134 1250 0 0 53687 3957 0 0 18 0 33 0 677706235 10545909760 2339783 268435456000 4194304 4417452 140735443751376 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26181/statm: 2574685 2339863 172 55 0 2572571 0
[pid=26181/tid=26192] ppid=26179 vsize=10298740 CPUtime=13.24 cores=0-31
/proc/26181/task/26192/stat : 26192 (plingeling) R 26179 26181 26155 0 -1 4202560 47305 1250 0 0 1218 106 0 0 18 0 33 0 677720999 10545909760 2339896 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=26181/tid=26193] ppid=26179 vsize=10298740 CPUtime=13.84 cores=0-31
/proc/26181/task/26193/stat : 26193 (plingeling) R 26179 26181 26155 0 -1 4202560 44720 1250 0 0 1286 98 0 0 18 0 33 0 677720999 10545909760 2339922 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26181/tid=26194] ppid=26179 vsize=10298740 CPUtime=13.84 cores=0-31
/proc/26181/task/26194/stat : 26194 (plingeling) R 26179 26181 26155 0 -1 4202560 100595 1250 0 0 1260 124 0 0 25 0 33 0 677720999 10545909760 2339955 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26181/tid=26195] ppid=26179 vsize=10298740 CPUtime=14.29 cores=0-31
/proc/26181/task/26195/stat : 26195 (plingeling) R 26179 26181 26155 0 -1 4202560 32792 1250 0 0 1354 75 0 0 25 0 33 0 677721000 10545909760 2339989 268435456000 4194304 4417452 140735443751376 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=26181/tid=26196] ppid=26179 vsize=10298740 CPUtime=13.04 cores=0-31
/proc/26181/task/26196/stat : 26196 (plingeling) R 26179 26181 26155 0 -1 4202560 48493 1250 0 0 1217 87 0 0 18 0 33 0 677721000 10545909760 2340026 268435456000 4194304 4417452 140735443751376 18446744073709551615 4268576 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=26181/tid=26197] ppid=26179 vsize=10298740 CPUtime=13.43 cores=0-31
/proc/26181/task/26197/stat : 26197 (plingeling) R 26179 26181 26155 0 -1 4202560 40471 1250 0 0 1239 104 0 0 23 0 33 0 677721000 10545909760 2340077 268435456000 4194304 4417452 140735443751376 18446744073709551615 4302950 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=26181/tid=26198] ppid=26179 vsize=10298740 CPUtime=13.06 cores=0-31
/proc/26181/task/26198/stat : 26198 (plingeling) R 26179 26181 26155 0 -1 4202560 89712 1250 0 0 1180 126 0 0 16 0 33 0 677721001 10545909760 2340109 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=26181/tid=26199] ppid=26179 vsize=10298740 CPUtime=13.54 cores=0-31
/proc/26181/task/26199/stat : 26199 (plingeling) R 26179 26181 26155 0 -1 4202560 46084 1250 0 0 1251 103 0 0 21 0 33 0 677721002 10545909760 2340143 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649923 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=26181/tid=26200] ppid=26179 vsize=10298740 CPUtime=13.6 cores=0-31
/proc/26181/task/26200/stat : 26200 (plingeling) R 26179 26181 26155 0 -1 4202560 39033 1250 0 0 1272 88 0 0 25 0 33 0 677721002 10545909760 2340169 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26181/tid=26201] ppid=26179 vsize=10298740 CPUtime=13.62 cores=0-31
/proc/26181/task/26201/stat : 26201 (plingeling) R 26179 26181 26155 0 -1 4202560 38433 1250 0 0 1261 101 0 0 25 0 33 0 677721003 10545909760 2340207 268435456000 4194304 4417452 140735443751376 18446744073709551615 4260714 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=26181/tid=26202] ppid=26179 vsize=10298740 CPUtime=12.81 cores=0-31
/proc/26181/task/26202/stat : 26202 (plingeling) R 26179 26181 26155 0 -1 4202560 70912 1250 0 0 1149 132 0 0 16 0 33 0 677721003 10545909760 2340237 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=26181/tid=26203] ppid=26179 vsize=10298740 CPUtime=13.61 cores=0-31
/proc/26181/task/26203/stat : 26203 (plingeling) R 26179 26181 26155 0 -1 4202560 40533 1250 0 0 1267 94 0 0 25 0 33 0 677721004 10545909760 2340259 268435456000 4194304 4417452 140735443751376 18446744073709551615 4302950 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=26181/tid=26204] ppid=26179 vsize=10298740 CPUtime=13.49 cores=0-31
/proc/26181/task/26204/stat : 26204 (plingeling) R 26179 26181 26155 0 -1 4202560 44468 1250 0 0 1225 124 0 0 18 0 33 0 677721004 10545909760 2340282 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=26181/tid=26205] ppid=26179 vsize=10298740 CPUtime=13.07 cores=0-31
/proc/26181/task/26205/stat : 26205 (plingeling) R 26179 26181 26155 0 -1 4202560 39515 1250 0 0 1221 86 0 0 23 0 33 0 677721005 10545909760 2340302 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232508 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=26181/tid=26206] ppid=26179 vsize=10298740 CPUtime=13.5 cores=0-31
/proc/26181/task/26206/stat : 26206 (plingeling) R 26179 26181 26155 0 -1 4202560 92814 1250 0 0 1234 116 0 0 19 0 33 0 677721006 10545909760 2340326 268435456000 4194304 4417452 140735443751376 18446744073709551615 4265940 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=26181/tid=26207] ppid=26179 vsize=10298740 CPUtime=13.7 cores=0-31
/proc/26181/task/26207/stat : 26207 (plingeling) R 26179 26181 26155 0 -1 4202560 46565 1250 0 0 1253 117 0 0 24 0 33 0 677721006 10545909760 2340367 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649923 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=26181/tid=26208] ppid=26179 vsize=10298740 CPUtime=13.81 cores=0-31
/proc/26181/task/26208/stat : 26208 (plingeling) R 26179 26181 26155 0 -1 4202560 40483 1250 0 0 1284 97 0 0 25 0 33 0 677721007 10545909760 2340412 268435456000 4194304 4417452 140735443751376 18446744073709551615 4265937 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=26181/tid=26209] ppid=26179 vsize=10298740 CPUtime=13.6 cores=0-31
/proc/26181/task/26209/stat : 26209 (plingeling) R 26179 26181 26155 0 -1 4202560 46091 1250 0 0 1279 81 0 0 18 0 33 0 677721008 10545909760 2340451 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=26181/tid=26210] ppid=26179 vsize=10298740 CPUtime=13 cores=0-31
/proc/26181/task/26210/stat : 26210 (plingeling) R 26179 26181 26155 0 -1 4202560 74015 1250 0 0 1177 123 0 0 18 0 33 0 677721008 10545909760 2340492 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=26181/tid=26211] ppid=26179 vsize=10298740 CPUtime=12.88 cores=0-31
/proc/26181/task/26211/stat : 26211 (plingeling) R 26179 26181 26155 0 -1 4202560 53977 1250 0 0 1195 93 0 0 18 0 33 0 677721009 10545909760 2340531 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 16 0 0 0
[pid=26181/tid=26212] ppid=26179 vsize=10298740 CPUtime=13.65 cores=0-31
/proc/26181/task/26212/stat : 26212 (plingeling) R 26179 26181 26155 0 -1 4202560 38421 1250 0 0 1263 102 0 0 23 0 33 0 677721010 10545909760 2340564 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236598 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=26181/tid=26213] ppid=26179 vsize=10298740 CPUtime=13.45 cores=0-31
/proc/26181/task/26213/stat : 26213 (plingeling) R 26179 26181 26155 0 -1 4202560 45350 1250 0 0 1236 109 0 0 18 0 33 0 677721010 10545909760 2340612 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26181/tid=26214] ppid=26179 vsize=10298740 CPUtime=13.04 cores=0-31
/proc/26181/task/26214/stat : 26214 (plingeling) R 26179 26181 26155 0 -1 4202560 84824 1250 0 0 1166 138 0 0 18 0 33 0 677721011 10545909760 2340657 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=26181/tid=26215] ppid=26179 vsize=10298740 CPUtime=13.39 cores=0-31
/proc/26181/task/26215/stat : 26215 (plingeling) R 26179 26181 26155 0 -1 4202560 43382 1250 0 0 1231 108 0 0 24 0 33 0 677721012 10545909760 2340693 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649903 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=26181/tid=26216] ppid=26179 vsize=10298740 CPUtime=13.81 cores=0-31
/proc/26181/task/26216/stat : 26216 (plingeling) R 26179 26181 26155 0 -1 4202560 44267 1250 0 0 1264 117 0 0 25 0 33 0 677721013 10545909760 2340741 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26181/tid=26217] ppid=26179 vsize=10298740 CPUtime=13.53 cores=0-31
/proc/26181/task/26217/stat : 26217 (plingeling) R 26179 26181 26155 0 -1 4202560 44298 1250 0 0 1243 110 0 0 18 0 33 0 677721013 10545909760 2340782 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649903 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=26181/tid=26218] ppid=26179 vsize=10298740 CPUtime=13.08 cores=0-31
/proc/26181/task/26218/stat : 26218 (plingeling) R 26179 26181 26155 0 -1 4202560 81463 1250 0 0 1165 143 0 0 16 0 33 0 677721014 10545909760 2340823 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=26181/tid=26219] ppid=26179 vsize=10298740 CPUtime=13.2 cores=0-31
/proc/26181/task/26219/stat : 26219 (plingeling) R 26179 26181 26155 0 -1 4202560 49415 1250 0 0 1207 113 0 0 18 0 33 0 677721015 10545909760 2340868 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26181/tid=26220] ppid=26179 vsize=10298740 CPUtime=13.27 cores=0-31
/proc/26181/task/26220/stat : 26220 (plingeling) R 26179 26181 26155 0 -1 4202560 45583 1250 0 0 1204 123 0 0 19 0 33 0 677721017 10545909760 2340902 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649918 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26181/tid=26221] ppid=26179 vsize=10298740 CPUtime=13.06 cores=0-31
/proc/26181/task/26221/stat : 26221 (plingeling) R 26179 26181 26155 0 -1 4202560 41429 1250 0 0 1218 88 0 0 20 0 33 0 677721017 10545909760 2340932 268435456000 4194304 4417452 140735443751376 18446744073709551615 4268108 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26181/tid=26222] ppid=26179 vsize=10298740 CPUtime=12.85 cores=0-31
/proc/26181/task/26222/stat : 26222 (plingeling) R 26179 26181 26155 0 -1 4202560 64477 1250 0 0 1152 133 0 0 16 0 33 0 677721019 10545909760 2340969 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=26181/tid=26223] ppid=26179 vsize=10298740 CPUtime=13.26 cores=0-31
/proc/26181/task/26223/stat : 26223 (plingeling) R 26179 26181 26155 0 -1 4202560 48142 1250 0 0 1204 122 0 0 18 0 33 0 677721020 10545909760 2341016 268435456000 4194304 4417452 140735443751376 18446744073709551615 258650649933 0 0 4096 17442 0 0 0 -1 14 0 0 0
Current children cumulated CPU time (s) 576.44
Current children cumulated vsize (KiB) 10298740


################
# More data... #
################

[pid=26181/tid=26205] ppid=26179 vsize=33884524 CPUtime=4913.15 cores=0-31
/proc/26181/task/26205/stat : 26205 (plingeling) R 26179 26181 26155 0 -1 4202560 1138881 1250 0 0 489889 1426 0 0 25 0 33 0 677721005 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4265465 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=26181/tid=26206] ppid=26179 vsize=33884524 CPUtime=4911.57 cores=0-31
/proc/26181/task/26206/stat : 26206 (plingeling) R 26179 26181 26155 0 -1 4202560 1707212 1250 0 0 488442 2715 0 0 25 0 33 0 677721006 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=26181/tid=26207] ppid=26179 vsize=33884524 CPUtime=4906.02 cores=0-31
/proc/26181/task/26207/stat : 26207 (plingeling) R 26179 26181 26155 0 -1 4202560 1539268 1250 0 0 488497 2105 0 0 25 0 33 0 677721006 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=26181/tid=26208] ppid=26179 vsize=33884524 CPUtime=4913.06 cores=0-31
/proc/26181/task/26208/stat : 26208 (plingeling) R 26179 26181 26155 0 -1 4202560 1780083 1250 0 0 488616 2690 0 0 25 0 33 0 677721007 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4270130 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=26181/tid=26209] ppid=26179 vsize=33884524 CPUtime=4918.95 cores=0-31
/proc/26181/task/26209/stat : 26209 (plingeling) R 26179 26181 26155 0 -1 4202560 946807 1250 0 0 490974 921 0 0 25 0 33 0 677721008 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26181/tid=26210] ppid=26179 vsize=33884524 CPUtime=4900.26 cores=0-31
/proc/26181/task/26210/stat : 26210 (plingeling) R 26179 26181 26155 0 -1 4202560 1660152 1250 0 0 487522 2504 0 0 25 0 33 0 677721008 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26181/tid=26211] ppid=26179 vsize=33884524 CPUtime=4920.36 cores=0-31
/proc/26181/task/26211/stat : 26211 (plingeling) R 26179 26181 26155 0 -1 4202560 1056102 1250 0 0 490877 1159 0 0 25 0 33 0 677721009 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232651 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26181/tid=26212] ppid=26179 vsize=33884524 CPUtime=4911.94 cores=0-31
/proc/26181/task/26212/stat : 26212 (plingeling) R 26179 26181 26155 0 -1 4202560 1600663 1250 0 0 488754 2440 0 0 25 0 33 0 677721010 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4309888 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=26181/tid=26213] ppid=26179 vsize=33884524 CPUtime=4916.32 cores=0-31
/proc/26181/task/26213/stat : 26213 (plingeling) R 26179 26181 26155 0 -1 4202560 1017449 1250 0 0 490132 1500 0 0 25 0 33 0 677721010 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4250633 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=26181/tid=26214] ppid=26179 vsize=33884524 CPUtime=4909.58 cores=0-31
/proc/26181/task/26214/stat : 26214 (plingeling) R 26179 26181 26155 0 -1 4202560 1839711 1250 0 0 488116 2842 0 0 25 0 33 0 677721011 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26181/tid=26215] ppid=26179 vsize=33884524 CPUtime=4913.19 cores=0-31
/proc/26181/task/26215/stat : 26215 (plingeling) R 26179 26181 26155 0 -1 4202560 1584865 1250 0 0 488832 2487 0 0 25 0 33 0 677721012 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331729 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26181/tid=26216] ppid=26179 vsize=33884524 CPUtime=4914.99 cores=0-31
/proc/26181/task/26216/stat : 26216 (plingeling) R 26179 26181 26155 0 -1 4202560 1757467 1250 0 0 488992 2507 0 0 25 0 33 0 677721013 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4357283 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=26181/tid=26217] ppid=26179 vsize=33884524 CPUtime=4910.86 cores=0-31
/proc/26181/task/26217/stat : 26217 (plingeling) R 26179 26181 26155 0 -1 4202560 950427 1250 0 0 489804 1282 0 0 25 0 33 0 677721013 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4300504 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=26181/tid=26218] ppid=26179 vsize=33884524 CPUtime=4902.98 cores=0-31
/proc/26181/task/26218/stat : 26218 (plingeling) R 26179 26181 26155 0 -1 4202560 1692992 1250 0 0 487341 2957 0 0 25 0 33 0 677721014 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331715 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=26181/tid=26219] ppid=26179 vsize=33884524 CPUtime=4914.01 cores=0-31
/proc/26181/task/26219/stat : 26219 (plingeling) R 26179 26181 26155 0 -1 4202560 1262787 1250 0 0 489379 2022 0 0 23 0 33 0 677721015 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4281863 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=26181/tid=26220] ppid=26179 vsize=33884524 CPUtime=4920.78 cores=0-31
/proc/26181/task/26220/stat : 26220 (plingeling) R 26179 26181 26155 0 -1 4202560 1259920 1250 0 0 490466 1612 0 0 25 0 33 0 677721017 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236634 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=26181/tid=26221] ppid=26179 vsize=33884524 CPUtime=4914.01 cores=0-31
/proc/26181/task/26221/stat : 26221 (plingeling) R 26179 26181 26155 0 -1 4202560 1045713 1250 0 0 489881 1520 0 0 25 0 33 0 677721017 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232595 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26181/tid=26222] ppid=26179 vsize=33884524 CPUtime=4911.07 cores=0-31
/proc/26181/task/26222/stat : 26222 (plingeling) R 26179 26181 26155 0 -1 4202560 1554044 1250 0 0 488732 2375 0 0 25 0 33 0 677721019 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232698 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=26181/tid=26223] ppid=26179 vsize=33884524 CPUtime=4914.78 cores=0-31
/proc/26181/task/26223/stat : 26223 (plingeling) R 26179 26181 26155 0 -1 4202560 1996036 1250 0 0 488100 3378 0 0 25 0 33 0 677721020 34697752576 8202106 268435456000 4194304 4417452 140735443751376 18446744073709551615 4281668 0 0 4096 17442 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 157326
Current children cumulated vsize (KiB) 33884524



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+5100 s]
/proc/loadavg: 32.08 32.02 31.85 33/446 26376
/proc/meminfo: memFree=222131256/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=34250048 CPUtime=157892 cores=0-31
/proc/26181/stat : 26181 (plingeling) S 26179 26181 26155 0 -1 4202496 51026808 1250 0 0 15714936 74309 0 0 18 0 33 0 677706235 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 258662759221 0 0 4096 17442 18446744073709551615 0 0 17 4 0 0 0
/proc/26181/statm: 8562512 8293487 175 55 0 8560398 0
[pid=26181/tid=26192] ppid=26179 vsize=34250048 CPUtime=4933.01 cores=0-31
/proc/26181/task/26192/stat : 26192 (plingeling) R 26179 26181 26155 0 -1 4202560 1752682 1250 0 0 490308 2993 0 0 25 0 33 0 677720999 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 14 0 0 0
[pid=26181/tid=26193] ppid=26179 vsize=34250048 CPUtime=4932.24 cores=0-31
/proc/26181/task/26193/stat : 26193 (plingeling) R 26179 26181 26155 0 -1 4202560 1138906 1250 0 0 491270 1954 0 0 25 0 33 0 677720999 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4232801 0 0 4096 17442 0 0 0 -1 24 0 0 0
[pid=26181/tid=26194] ppid=26179 vsize=34250048 CPUtime=4926.69 cores=0-31
/proc/26181/task/26194/stat : 26194 (plingeling) R 26179 26181 26155 0 -1 4202560 1807679 1250 0 0 489936 2733 0 0 25 0 33 0 677720999 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 15 0 0 0
[pid=26181/tid=26195] ppid=26179 vsize=34250048 CPUtime=4916.26 cores=0-31
/proc/26181/task/26195/stat : 26195 (plingeling) R 26179 26181 26155 0 -1 4202560 2768742 1250 0 0 487593 4033 0 0 25 0 33 0 677721000 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4356852 0 0 4096 17442 0 0 0 -1 20 0 0 0
[pid=26181/tid=26196] ppid=26179 vsize=34250048 CPUtime=4933.19 cores=0-31
/proc/26181/task/26196/stat : 26196 (plingeling) R 26179 26181 26155 0 -1 4202560 1763705 1250 0 0 490477 2842 0 0 25 0 33 0 677721000 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 31 0 0 0
[pid=26181/tid=26197] ppid=26179 vsize=34250048 CPUtime=4917.48 cores=0-31
/proc/26181/task/26197/stat : 26197 (plingeling) R 26179 26181 26155 0 -1 4202560 1100307 1250 0 0 489933 1815 0 0 25 0 33 0 677721000 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4352536 0 0 4096 17442 0 0 0 -1 22 0 0 0
[pid=26181/tid=26198] ppid=26179 vsize=34250048 CPUtime=4931.68 cores=0-31
/proc/26181/task/26198/stat : 26198 (plingeling) R 26179 26181 26155 0 -1 4202560 1539439 1250 0 0 490714 2454 0 0 25 0 33 0 677721001 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 19 0 0 0
[pid=26181/tid=26199] ppid=26179 vsize=34250048 CPUtime=4928.32 cores=0-31
/proc/26181/task/26199/stat : 26199 (plingeling) R 26179 26181 26155 0 -1 4202560 1267270 1250 0 0 490864 1968 0 0 25 0 33 0 677721002 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 1 0 0 0
[pid=26181/tid=26200] ppid=26179 vsize=34250048 CPUtime=4926.33 cores=0-31
/proc/26181/task/26200/stat : 26200 (plingeling) R 26179 26181 26155 0 -1 4202560 2089079 1250 0 0 488985 3648 0 0 25 0 33 0 677721002 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 9 0 0 0
[pid=26181/tid=26201] ppid=26179 vsize=34250048 CPUtime=4934.62 cores=0-31
/proc/26181/task/26201/stat : 26201 (plingeling) R 26179 26181 26155 0 -1 4202560 1086341 1250 0 0 492107 1355 0 0 25 0 33 0 677721003 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 10 0 0 0
[pid=26181/tid=26202] ppid=26179 vsize=34250048 CPUtime=4929.96 cores=0-31
/proc/26181/task/26202/stat : 26202 (plingeling) R 26179 26181 26155 0 -1 4202560 1711076 1250 0 0 490230 2766 0 0 25 0 33 0 677721003 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4364907 0 0 4096 17442 0 0 0 -1 21 0 0 0
[pid=26181/tid=26203] ppid=26179 vsize=34250048 CPUtime=4927.88 cores=0-31
/proc/26181/task/26203/stat : 26203 (plingeling) R 26179 26181 26155 0 -1 4202560 1619756 1250 0 0 490209 2579 0 0 25 0 33 0 677721004 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4366065 0 0 4096 17442 0 0 0 -1 13 0 0 0
[pid=26181/tid=26204] ppid=26179 vsize=34250048 CPUtime=4932.52 cores=0-31
/proc/26181/task/26204/stat : 26204 (plingeling) R 26179 26181 26155 0 -1 4202560 1490958 1250 0 0 491081 2171 0 0 25 0 33 0 677721004 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4358116 0 0 4096 17442 0 0 0 -1 2 0 0 0
[pid=26181/tid=26205] ppid=26179 vsize=34250048 CPUtime=4930.87 cores=0-31
/proc/26181/task/26205/stat : 26205 (plingeling) R 26179 26181 26155 0 -1 4202560 1173107 1250 0 0 491633 1454 0 0 25 0 33 0 677721005 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4357260 0 0 4096 17442 0 0 0 -1 18 0 0 0
[pid=26181/tid=26206] ppid=26179 vsize=34250048 CPUtime=4929.29 cores=0-31
/proc/26181/task/26206/stat : 26206 (plingeling) R 26179 26181 26155 0 -1 4202560 1707212 1250 0 0 490214 2715 0 0 25 0 33 0 677721006 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 23 0 0 0
[pid=26181/tid=26207] ppid=26179 vsize=34250048 CPUtime=4923.74 cores=0-31
/proc/26181/task/26207/stat : 26207 (plingeling) R 26179 26181 26155 0 -1 4202560 1552923 1250 0 0 490231 2143 0 0 25 0 33 0 677721006 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 11 0 0 0
[pid=26181/tid=26208] ppid=26179 vsize=34250048 CPUtime=4930.78 cores=0-31
/proc/26181/task/26208/stat : 26208 (plingeling) R 26179 26181 26155 0 -1 4202560 1780083 1250 0 0 490387 2691 0 0 25 0 33 0 677721007 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4309884 0 0 4096 17442 0 0 0 -1 8 0 0 0
[pid=26181/tid=26209] ppid=26179 vsize=34250048 CPUtime=4936.67 cores=0-31
/proc/26181/task/26209/stat : 26209 (plingeling) R 26179 26181 26155 0 -1 4202560 955347 1250 0 0 492744 923 0 0 25 0 33 0 677721008 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4364653 0 0 4096 17442 0 0 0 -1 0 0 0 0
[pid=26181/tid=26210] ppid=26179 vsize=34250048 CPUtime=4917.98 cores=0-31
/proc/26181/task/26210/stat : 26210 (plingeling) R 26179 26181 26155 0 -1 4202560 1660152 1250 0 0 489294 2504 0 0 25 0 33 0 677721008 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4364645 0 0 4096 17442 0 0 0 -1 3 0 0 0
[pid=26181/tid=26211] ppid=26179 vsize=34250048 CPUtime=4938.08 cores=0-31
/proc/26181/task/26211/stat : 26211 (plingeling) R 26179 26181 26155 0 -1 4202560 1056102 1250 0 0 492649 1159 0 0 25 0 33 0 677721009 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4264940 0 0 4096 17442 0 0 0 -1 5 0 0 0
[pid=26181/tid=26212] ppid=26179 vsize=34250048 CPUtime=4929.66 cores=0-31
/proc/26181/task/26212/stat : 26212 (plingeling) R 26179 26181 26155 0 -1 4202560 1600663 1250 0 0 490525 2441 0 0 25 0 33 0 677721010 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 30 0 0 0
[pid=26181/tid=26213] ppid=26179 vsize=34250048 CPUtime=4934.03 cores=0-31
/proc/26181/task/26213/stat : 26213 (plingeling) R 26179 26181 26155 0 -1 4202560 1017989 1250 0 0 491903 1500 0 0 25 0 33 0 677721010 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 12 0 0 0
[pid=26181/tid=26214] ppid=26179 vsize=34250048 CPUtime=4927.3 cores=0-31
/proc/26181/task/26214/stat : 26214 (plingeling) R 26179 26181 26155 0 -1 4202560 1839711 1250 0 0 489888 2842 0 0 25 0 33 0 677721011 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 4 0 0 0
[pid=26181/tid=26215] ppid=26179 vsize=34250048 CPUtime=4930.91 cores=0-31
/proc/26181/task/26215/stat : 26215 (plingeling) R 26179 26181 26155 0 -1 4202560 1584865 1250 0 0 490604 2487 0 0 25 0 33 0 677721012 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 7 0 0 0
[pid=26181/tid=26216] ppid=26179 vsize=34250048 CPUtime=4932.71 cores=0-31
/proc/26181/task/26216/stat : 26216 (plingeling) R 26179 26181 26155 0 -1 4202560 1757467 1250 0 0 490764 2507 0 0 25 0 33 0 677721013 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4250967 0 0 4096 17442 0 0 0 -1 25 0 0 0
[pid=26181/tid=26217] ppid=26179 vsize=34250048 CPUtime=4928.45 cores=0-31
/proc/26181/task/26217/stat : 26217 (plingeling) R 26179 26181 26155 0 -1 4202560 984493 1250 0 0 491555 1290 0 0 25 0 33 0 677721013 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236784 0 0 4096 17442 0 0 0 -1 29 0 0 0
[pid=26181/tid=26218] ppid=26179 vsize=34250048 CPUtime=4920.69 cores=0-31
/proc/26181/task/26218/stat : 26218 (plingeling) R 26179 26181 26155 0 -1 4202560 1692992 1250 0 0 489112 2957 0 0 25 0 33 0 677721014 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4331796 0 0 4096 17442 0 0 0 -1 26 0 0 0
[pid=26181/tid=26219] ppid=26179 vsize=34250048 CPUtime=4931.69 cores=0-31
/proc/26181/task/26219/stat : 26219 (plingeling) R 26179 26181 26155 0 -1 4202560 1269272 1250 0 0 491145 2024 0 0 25 0 33 0 677721015 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236778 0 0 4096 17442 0 0 0 -1 28 0 0 0
[pid=26181/tid=26220] ppid=26179 vsize=34250048 CPUtime=4938.49 cores=0-31
/proc/26181/task/26220/stat : 26220 (plingeling) R 26179 26181 26155 0 -1 4202560 1259920 1250 0 0 492237 1612 0 0 25 0 33 0 677721017 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4236603 0 0 4096 17442 0 0 0 -1 17 0 0 0
[pid=26181/tid=26221] ppid=26179 vsize=34250048 CPUtime=4931.73 cores=0-31
/proc/26181/task/26221/stat : 26221 (plingeling) R 26179 26181 26155 0 -1 4202560 1045713 1250 0 0 491653 1520 0 0 25 0 33 0 677721017 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4277545 0 0 4096 17442 0 0 0 -1 6 0 0 0
[pid=26181/tid=26222] ppid=26179 vsize=34250048 CPUtime=4928.79 cores=0-31
/proc/26181/task/26222/stat : 26222 (plingeling) R 26179 26181 26155 0 -1 4202560 1554044 1250 0 0 490504 2375 0 0 25 0 33 0 677721019 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4364607 0 0 4096 17442 0 0 0 -1 27 0 0 0
[pid=26181/tid=26223] ppid=26179 vsize=34250048 CPUtime=4932.5 cores=0-31
/proc/26181/task/26223/stat : 26223 (plingeling) R 26179 26181 26155 0 -1 4202560 1996036 1250 0 0 489872 3378 0 0 25 0 33 0 677721020 35072049152 8293487 268435456000 4194304 4417452 140735443751376 18446744073709551615 4354497 0 0 4096 17442 0 0 0 -1 16 0 0 0
Current children cumulated CPU time (s) 157892
Current children cumulated vsize (KiB) 34250048

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)

Sending SIGKILL to -26181

Solver just ended. Dumping a history of the last processes samples

[startup+5100.7 s]
/proc/loadavg: 32.08 32.02 31.85 2/416 26377
/proc/meminfo: memFree=229350280/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=0 CPUtime=157893 cores=0-31
/proc/26181/stat : 26181 (plingeling) Z 26179 26181 26155 0 -1 4203548 51026808 1250 0 0 15714958 74379 0 0 18 0 2 0 677706235 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26181/statm: 0 0 0 0 0 0 0
[pid=26181/tid=26198] ppid=26179 vsize=0 CPUtime=4932.39 cores=0-31
/proc/26181/task/26198/stat : 26198 (plingeling) R 26179 26181 26155 0 -1 4203588 1539439 1250 0 0 490715 2524 0 0 25 0 2 0 677721001 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 19 0 0 0
Current children cumulated CPU time (s) 157893
Current children cumulated vsize (KiB) 0

[startup+5101.5 s]
/proc/loadavg: 32.08 32.02 31.85 2/416 26377
/proc/meminfo: memFree=229350280/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=0 CPUtime=157894 cores=0-31
/proc/26181/stat : 26181 (plingeling) Z 26179 26181 26155 0 -1 4203548 51026808 1250 0 0 15714958 74456 0 0 18 0 2 0 677706235 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26181/statm: 0 0 0 0 0 0 0
[pid=26181/tid=26198] ppid=26179 vsize=0 CPUtime=4933.16 cores=0-31
/proc/26181/task/26198/stat : 26198 (plingeling) R 26179 26181 26155 0 -1 4203588 1539439 1250 0 0 490715 2601 0 0 25 0 2 0 677721001 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 19 0 0 0
Current children cumulated CPU time (s) 157894
Current children cumulated vsize (KiB) 0

[startup+5101.92 s]
/proc/loadavg: 32.08 32.02 31.85 2/416 26377
/proc/meminfo: memFree=249374368/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=0 CPUtime=157895 cores=0-31
/proc/26181/stat : 26181 (plingeling) Z 26179 26181 26155 0 -1 4203548 51026808 1250 0 0 15714958 74498 0 0 18 0 2 0 677706235 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26181/statm: 0 0 0 0 0 0 0
[pid=26181/tid=26198] ppid=26179 vsize=0 CPUtime=4933.58 cores=0-31
/proc/26181/task/26198/stat : 26198 (plingeling) R 26179 26181 26155 0 -1 4203588 1539439 1250 0 0 490715 2643 0 0 25 0 2 0 677721001 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 19 0 0 0
Current children cumulated CPU time (s) 157895
Current children cumulated vsize (KiB) 0

[startup+5102.1 s]
/proc/loadavg: 32.08 32.02 31.85 2/416 26377
/proc/meminfo: memFree=249374368/264257624 swapFree=67111528/67111528
[pid=26181] ppid=26179 vsize=0 CPUtime=157895 cores=0-31
/proc/26181/stat : 26181 (plingeling) Z 26179 26181 26155 0 -1 4203548 51026808 1250 0 0 15714958 74516 0 0 18 0 2 0 677706235 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 17 4 0 0 0
/proc/26181/statm: 0 0 0 0 0 0 0
[pid=26181/tid=26198] ppid=26179 vsize=0 CPUtime=4933.76 cores=0-31
/proc/26181/task/26198/stat : 26198 (plingeling) R 26179 26181 26155 0 -1 4203588 1539439 1250 0 0 490715 2661 0 0 25 0 2 0 677721001 0 0 268435456000 0 0 0 0 0 0 0 4096 0 18446744073709551615 0 0 -1 19 0 0 0
Current children cumulated CPU time (s) 157895
Current children cumulated vsize (KiB) 0

Child ended because it received signal 15 (SIGTERM)
Real time (s): 5102.11
CPU time (s): 157895
CPU user time (s): 157150
CPU system time (s): 745.181
CPU usage (%): 3094.69
Max. virtual memory (cumulated for all children) (KiB): 39735764

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 157150
system time used= 745.181
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 51028058
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 22315
involuntary context switches= 400966

runsolver used 26.126 second user time and 119.179 second system time

The end

Launcher Data 

Begin job on node200 at 2011-05-17 21:56:38
IDJOB=3372471
IDBENCH=82404
IDSOLVER=1852
FILE ID=node200/3372471-1305662191
RUNJOBID= node200-1305662187-26169
PBS_JOBID= 13357370
Free space on /tmp= 61888 MiB

SOLVER NAME= Plingeling 587f (fixed)
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.040-NOTKNOWN.cnf
COMMAND LINE= plingeling -t NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3372471-1305662191/watcher-3372471-1305662191 -o /tmp/evaluation-result-3372471-1305662191/solver-3372471-1305662191 -C 160000 -W 5100 -M 266338304  plingeling -t 32 HOME/instance-3372471-1305662191.cnf

TIME LIMIT= 160000 seconds
MEMORY LIMIT= 266338304 MiB
NBCORE= 32

MD5SUM BENCH= 6c2d25eda70423643946d76935ceb950
RANDOM SEED=980188430

node200.alineos.net Linux 2.6.18-194.el5 #1 SMP Fri Apr 2 14:58:14 EDT 2010

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 32
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 64
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 0
cpu cores	: 8
apicid		: 96
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 34
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 66
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 1
cpu cores	: 8
apicid		: 98
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 36
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 68
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 2
cpu cores	: 8
apicid		: 100
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.02
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.17
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 38
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 70
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3989.98
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 3
cpu cores	: 8
apicid		: 102
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 16
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.04
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 17
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 48
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 18
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 80
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 19
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 8
cpu cores	: 8
apicid		: 112
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 20
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 21
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 50
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.07
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 22
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 82
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 23
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 9
cpu cores	: 8
apicid		: 114
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.23
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 24
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 25
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 52
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 26
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 84
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.08
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 27
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 10
cpu cores	: 8
apicid		: 116
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 28
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 0
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 29
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 1
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 54
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 30
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 2
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 86
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.05
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]

processor	: 31
vendor_id	: GenuineIntel
cpu family	: 6
model		: 46
model name	: Intel(R) Xeon(R) CPU           X7550  @ 2.00GHz
stepping	: 6
cpu MHz		: 1995.049
cache size	: 18432 KB
physical id	: 3
siblings	: 8
core id		: 11
cpu cores	: 8
apicid		: 118
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr sse4_1 sse4_2 popcnt lahf_lm
bogomips	: 3990.06
clflush size	: 64
cache_alignment	: 64
address sizes	: 44 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     264257624 kB
MemFree:      255363768 kB
Buffers:        449404 kB
Cached:        8043848 kB
SwapCached:          0 kB
Active:        7561916 kB
Inactive:      1000268 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     264257624 kB
LowFree:      255363768 kB
SwapTotal:    67111528 kB
SwapFree:     67111528 kB
Dirty:          165644 kB
Writeback:           0 kB
AnonPages:       68732 kB
Mapped:          14480 kB
Slab:           252948 kB
PageTables:       4380 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  199240340 kB
Committed_AS:   189764 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296372 kB
VmallocChunk: 34359440603 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 61888 MiB
End job on node200 at 2011-05-17 23:21:40