Trace number 3387273

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
minisat hack: minisat_psm 2011-04-01? (TO) 5000.01 4999.93

General information on the benchmark

Nameapplication/rintanen/SATPlanning/
transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.020-NOTKNOWN.cnf
MD5SUM5d1253e2dcc0e3e55992672756beadbc
Bench CategoryAPPLICATION (applications instances)
Best result obtained on this benchmarkSAT
Best CPU time to get the best result obtained on this benchmark2556.24
Satisfiable
(Un)Satisfiability was proved
Number of variables756832
Number of clauses4116966
Sum of the clauses size9765566
Maximum clause length1178
Minimum clause length1
Number of clauses of size 1846
Number of clauses of size 24082840
Number of clauses of size 3640
Number of clauses of size 4800
Number of clauses of size 5800
Number of clauses of size over 531040

Solver Data 

0.00/0.00	c Parsing...
1.29/1.30	c ==============================================================================
1.29/1.30	c |           |     ORIGINAL     |              LEARNT              |          |
1.29/1.30	c | Conflicts | Clauses Literals |   Limit Clauses Literals  Lit/Cl | Progress |
1.29/1.30	c ==============================================================================
1.29/1.30	c |         0 | 3028353  7134510 |  908505       0        0     nan |  0.000 % |
3.09/3.26	c |         0 | 2570144  6111727 |      --       0       --      -- |     --   | -156456/-307529
3.09/4.84	c ==============================================================================
3.09/4.84	c Result  :   #vars: 365932   #clauses: 2570144   #literals: 6111727
3.09/4.84	c CPU time:   2.98655 s
3.09/4.84	c ==============================================================================

Verifier Data 

ERROR: Unexpected answer ! (SATISFIABLE/UNSATISFIABLE expected)
Got answer: <no 's ' line found>

Watcher Data 

runsolver Copyright (C) 2010-2011 Olivier ROUSSEL

This is runsolver version 3.3.0 (svn: 961)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-3387273-1305599435/watcher-3387273-1305599435 -o /tmp/evaluation-result-3387273-1305599435/solver-3387273-1305599435 -C 5000 -W 5100 -M 15500 ./esg_static.sh HOME/instance-3387273-1305599435.cnf 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 5000 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 5030 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 5100 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 1.14 1.04 1.07 2/536 1415
/proc/meminfo: memFree=18664044/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 328 0 0 0 0 0 0 0 21 0 1 0 652360056 8908800 250 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/1415/statm: 2175 250 209 178 0 68 0

[startup+0.0814079 s]
/proc/loadavg: 1.14 1.04 1.07 2/536 1415
/proc/meminfo: memFree=18664044/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 328 0 0 0 0 0 0 0 21 0 1 0 652360056 8908800 250 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/1415/statm: 2175 250 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.100383 s]
/proc/loadavg: 1.14 1.04 1.07 2/536 1415
/proc/meminfo: memFree=18664044/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 328 0 0 0 0 0 0 0 21 0 1 0 652360056 8908800 250 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/1415/statm: 2175 250 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.300351 s]
/proc/loadavg: 1.14 1.04 1.07 2/536 1415
/proc/meminfo: memFree=18664044/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 328 0 0 0 0 0 0 0 21 0 1 0 652360056 8908800 250 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/1415/statm: 2175 250 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+0.70028 s]
/proc/loadavg: 1.14 1.04 1.07 2/536 1415
/proc/meminfo: memFree=18664044/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 328 0 0 0 0 0 0 0 21 0 1 0 652360056 8908800 250 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/1415/statm: 2175 250 209 178 0 68 0
Current children cumulated CPU time (s) 0
Current children cumulated vsize (KiB) 8700

[startup+1.50114 s]
/proc/loadavg: 1.14 1.04 1.07 3/538 1417
/proc/meminfo: memFree=18279884/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 328 0 0 0 0 0 0 0 21 0 1 0 652360056 8908800 250 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/1415/statm: 2175 250 209 178 0 68 0
[pid=1416] ppid=1415 vsize=342812 CPUtime=1.49 cores=1,3,5,7
/proc/1416/stat : 1416 (SatELite_releas) R 1415 1415 31886 0 -1 4202496 74612 0 0 0 132 17 0 0 25 0 1 0 652360056 351039488 74567 33554432000 134512640 135144242 4290736304 18446744073709551615 134551334 0 0 4096 3 0 0 0 17 3 0 0 0
/proc/1416/statm: 85703 74567 71 155 0 85546 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 351512

[startup+3.10085 s]
/proc/loadavg: 1.14 1.04 1.07 3/538 1417
/proc/meminfo: memFree=18108776/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=0 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 328 0 0 0 0 0 0 0 21 0 1 0 652360056 8908800 250 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 1 0 0 0
/proc/1415/statm: 2175 250 209 178 0 68 0
[pid=1416] ppid=1415 vsize=401452 CPUtime=3.09 cores=1,3,5,7
/proc/1416/stat : 1416 (SatELite_releas) R 1415 1415 31886 0 -1 4202496 121457 0 0 0 285 24 0 0 25 0 1 0 652360056 411086848 86583 33554432000 134512640 135144242 4290736304 18446744073709551615 134585430 0 0 4096 3 0 0 0 17 3 0 0 0
/proc/1416/statm: 100363 86583 77 155 0 100206 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 410152

[startup+6.30029 s]
/proc/loadavg: 1.53 1.12 1.09 3/538 1419
/proc/meminfo: memFree=18355100/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=200316 CPUtime=1.85 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 47180 0 0 0 179 6 0 0 20 0 1 0 652360500 205123584 38988 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209216 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 50079 38988 251 16 0 46721 0
Current children cumulated CPU time (s) 6.27
Current children cumulated vsize (KiB) 209016

[startup+12.7201 s]
/proc/loadavg: 1.73 1.17 1.11 3/538 1419
/proc/meminfo: memFree=18313432/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=205376 CPUtime=8.27 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 52075 0 0 0 821 6 0 0 25 0 1 0 652360500 210305024 43874 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209909 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 51344 43874 251 16 0 47986 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 214076

[startup+25.5008 s]
/proc/loadavg: 1.79 1.21 1.12 3/538 1419
/proc/meminfo: memFree=18290748/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=258640 CPUtime=21.05 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 85407 0 0 0 2095 10 0 0 25 0 1 0 652360500 264847360 48757 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209105 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 64660 48757 252 16 0 61302 0
Current children cumulated CPU time (s) 25.47
Current children cumulated vsize (KiB) 267340

[startup+51.1013 s]
/proc/loadavg: 1.86 1.27 1.14 3/538 1420
/proc/meminfo: memFree=18277732/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=260224 CPUtime=46.66 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 153446 0 0 0 4647 19 0 0 25 0 1 0 652360500 266469376 50570 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209075 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 65056 50570 252 16 0 61698 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 268924

[startup+102.301 s]
/proc/loadavg: 1.94 1.38 1.19 3/538 1421
/proc/meminfo: memFree=18271080/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=263972 CPUtime=97.85 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 324244 0 0 0 9747 38 0 0 25 0 1 0 652360500 270307328 49406 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209339 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 65993 49406 252 16 0 62635 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 272672

[startup+162.32 s]
/proc/loadavg: 2.05 1.52 1.24 3/538 1423
/proc/meminfo: memFree=18259760/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=267704 CPUtime=157.88 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 488277 0 0 0 15731 57 0 0 25 0 1 0 652360500 274128896 50769 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209354 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 66926 50769 252 16 0 63568 0
Current children cumulated CPU time (s) 162.3
Current children cumulated vsize (KiB) 276404

[startup+222.301 s]
/proc/loadavg: 2.05 1.62 1.29 3/538 1425
/proc/meminfo: memFree=18199836/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=270500 CPUtime=217.86 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 663262 0 0 0 21708 78 0 0 25 0 1 0 652360500 276992000 55964 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209075 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 67625 55964 252 16 0 64267 0
Current children cumulated CPU time (s) 222.28
Current children cumulated vsize (KiB) 279200

[startup+282.301 s]
/proc/loadavg: 2.02 1.69 1.33 3/538 1426
/proc/meminfo: memFree=18183172/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=354164 CPUtime=277.86 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 794938 0 0 0 27693 93 0 0 25 0 1 0 652360500 362663936 62931 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209058 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 88541 62931 252 16 0 85183 0
Current children cumulated CPU time (s) 282.28
Current children cumulated vsize (KiB) 362864

[startup+342.3 s]
/proc/loadavg: 2.00 1.74 1.37 3/538 1428
/proc/meminfo: memFree=18188108/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=275280 CPUtime=337.86 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 961622 0 0 0 33675 111 0 0 25 0 1 0 652360500 281886720 58994 33554432000 4194304 4256717 140734787301552 18446744073709551615 4213469 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 68820 58994 252 16 0 65462 0
Current children cumulated CPU time (s) 342.28
Current children cumulated vsize (KiB) 283980

[startup+402.3 s]
/proc/loadavg: 2.15 1.82 1.42 3/538 1429
/proc/meminfo: memFree=18158304/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7

################
# More data... #
################

/proc/1419/statm: 190141 138625 252 16 0 186783 0
Current children cumulated CPU time (s) 4302.39
Current children cumulated vsize (KiB) 769264

[startup+4362.3 s]
/proc/loadavg: 2.00 2.03 2.00 3/538 1643
/proc/meminfo: memFree=18075400/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=760564 CPUtime=4357.97 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10165241 0 0 0 434527 1270 0 0 25 0 1 0 652360500 778817536 131866 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209204 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190141 131866 252 16 0 186783 0
Current children cumulated CPU time (s) 4362.39
Current children cumulated vsize (KiB) 769264

[startup+4422.3 s]
/proc/loadavg: 2.00 2.02 2.00 3/538 1645
/proc/meminfo: memFree=18060712/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=760564 CPUtime=4417.97 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10284788 0 0 0 440512 1285 0 0 25 0 1 0 652360500 778817536 138871 33554432000 4194304 4256717 140734787301552 18446744073709551615 4212739 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190141 138871 252 16 0 186783 0
Current children cumulated CPU time (s) 4422.39
Current children cumulated vsize (KiB) 769264

[startup+4482.3 s]
/proc/loadavg: 2.00 2.02 2.00 3/538 1646
/proc/meminfo: memFree=18038324/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=760564 CPUtime=4477.97 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10408155 0 0 0 446495 1302 0 0 25 0 1 0 652360500 778817536 141675 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209204 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190141 141675 252 16 0 186783 0
Current children cumulated CPU time (s) 4482.39
Current children cumulated vsize (KiB) 769264

[startup+4542.3 s]
/proc/loadavg: 2.00 2.01 2.00 3/538 1648
/proc/meminfo: memFree=18042000/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=761824 CPUtime=4537.97 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10533890 0 0 0 452479 1318 0 0 25 0 1 0 652360500 780107776 138415 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209058 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190456 138415 252 16 0 187098 0
Current children cumulated CPU time (s) 4542.39
Current children cumulated vsize (KiB) 770524

[startup+4602.3 s]
/proc/loadavg: 2.23 2.05 2.02 3/538 1649
/proc/meminfo: memFree=18080632/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=546900 CPUtime=4597.98 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10646540 0 0 0 458465 1333 0 0 25 0 1 0 652360500 560025600 128670 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209204 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 136725 128670 252 16 0 133367 0
Current children cumulated CPU time (s) 4602.4
Current children cumulated vsize (KiB) 555600

[startup+4662.3 s]
/proc/loadavg: 2.20 2.07 2.02 3/538 1651
/proc/meminfo: memFree=18079844/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=761824 CPUtime=4657.98 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10766478 0 0 0 464450 1348 0 0 25 0 1 0 652360500 780107776 131768 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209909 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190456 131768 252 16 0 187098 0
Current children cumulated CPU time (s) 4662.4
Current children cumulated vsize (KiB) 770524

[startup+4722.3 s]
/proc/loadavg: 2.08 2.06 2.01 3/538 1653
/proc/meminfo: memFree=17919348/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=763352 CPUtime=4717.97 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10804090 0 0 0 470445 1352 0 0 25 0 1 0 652360500 781672448 169338 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209216 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190838 169338 252 16 0 187480 0
Current children cumulated CPU time (s) 4722.39
Current children cumulated vsize (KiB) 772052

[startup+4782.32 s]
/proc/loadavg: 2.02 2.04 2.01 3/538 1654
/proc/meminfo: memFree=17912360/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=763352 CPUtime=4778 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 10940106 0 0 0 476430 1370 0 0 25 0 1 0 652360500 781672448 170574 33554432000 4194304 4256717 140734787301552 18446744073709551615 4208998 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190838 170574 252 16 0 187480 0
Current children cumulated CPU time (s) 4782.42
Current children cumulated vsize (KiB) 772052

[startup+4842.3 s]
/proc/loadavg: 2.01 2.03 2.00 3/538 1656
/proc/meminfo: memFree=17969708/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=763352 CPUtime=4837.98 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 11076983 0 0 0 482411 1387 0 0 25 0 1 0 652360500 781672448 156752 33554432000 4194304 4256717 140734787301552 18446744073709551615 4213733 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190838 156752 252 16 0 187480 0
Current children cumulated CPU time (s) 4842.4
Current children cumulated vsize (KiB) 772052

[startup+4902.3 s]
/proc/loadavg: 2.00 2.02 2.00 3/538 1657
/proc/meminfo: memFree=18040952/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=763352 CPUtime=4897.98 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 11191795 0 0 0 488397 1401 0 0 25 0 1 0 652360500 781672448 139075 33554432000 4194304 4256717 140734787301552 18446744073709551615 4213501 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 190838 139075 252 16 0 187480 0
Current children cumulated CPU time (s) 4902.4
Current children cumulated vsize (KiB) 772052

[startup+4962.3 s]
/proc/loadavg: 2.00 2.02 2.00 3/538 1659
/proc/meminfo: memFree=18042276/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=764628 CPUtime=4957.98 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 11307327 0 0 0 494382 1416 0 0 25 0 1 0 652360500 782979072 141482 33554432000 4194304 4256717 140734787301552 18446744073709551615 4209126 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 191157 141482 252 16 0 187799 0
Current children cumulated CPU time (s) 4962.4
Current children cumulated vsize (KiB) 773328



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+4999.9 s]
/proc/loadavg: 2.00 2.02 2.00 3/538 1660
/proc/meminfo: memFree=18041744/32951124 swapFree=51873468/67111528
[pid=1415] ppid=1413 vsize=8700 CPUtime=4.42 cores=1,3,5,7
/proc/1415/stat : 1415 (esg_static.sh) S 1413 1415 31886 0 -1 4202496 353 153237 0 0 0 0 406 36 18 0 1 0 652360056 8908800 252 33554432000 4194304 4922060 140736095021360 18446744073709551615 212039474309 0 65536 4100 65538 18446744071562232103 0 0 17 3 0 0 0
/proc/1415/statm: 2175 252 209 178 0 68 0
[pid=1419] ppid=1415 vsize=764628 CPUtime=4995.59 cores=1,3,5,7
/proc/1419/stat : 1419 (minisat_static) R 1415 1415 31886 0 -1 4202496 11426003 0 0 0 498129 1430 0 0 25 0 1 0 652360500 782979072 136142 33554432000 4194304 4256717 140734787301552 18446744073709551615 4219248 0 0 4096 8388610 0 0 0 17 5 0 0 0
/proc/1419/statm: 191157 136142 252 16 0 187799 0
Current children cumulated CPU time (s) 5000.01
Current children cumulated vsize (KiB) 773328

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended. Dumping a history of the last processes samples

Child ended because it received signal 15 (SIGTERM)

# WARNING:
# CPU time reported by wait4() is probably wrong !
# wait4(...,&childrusage) returns 1415 and gives
#  childrusage.ru_utime.tv_sec=4
#  childrusage.ru_utime.tv_usec=71381
#  childrusage.ru_stime.tv_sec=0
#  childrusage.ru_stime.tv_usec=366944
# CPU time returned by wait4() is 4.43833
# while last known CPU time is 5000.01
#
# Solver probably didn't/couldn't wait for its children
# Using CPU time of the last sample as value...

Real time (s): 4999.93
CPU time (s): 5000.01
CPU user time (s): 4985.35
CPU system time (s): 14.66
CPU usage (%): 100.002
Max. virtual memory (cumulated for all children) (KiB): 1680100

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 4.07138
system time used= 0.366944
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 153590
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9
involuntary context switches= 27

runsolver used 25.9491 second user time and 60.9927 second system time

The end

Launcher Data 

Begin job on node146 at 2011-05-17 04:30:38
IDJOB=3387273
IDBENCH=82387
IDSOLVER=1874
FILE ID=node146/3387273-1305599435
RUNJOBID= node146-1305599435-1395
PBS_JOBID= 13350332
Free space on /tmp= 73200 MiB

SOLVER NAME= minisat hack: minisat_psm 2011-04-01
BENCH NAME= SAT11/application/rintanen/SATPlanning/transport-transport-three-cities-sequential-14nodes-1000size-4degree-100mindistance-4trucks-14packages-2008seed.020-NOTKNOWN.cnf
COMMAND LINE= ./esg_static.sh BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-3387273-1305599435/watcher-3387273-1305599435 -o /tmp/evaluation-result-3387273-1305599435/solver-3387273-1305599435 -C 5000 -W 5100 -M 15500  ./esg_static.sh HOME/instance-3387273-1305599435.cnf

TIME LIMIT= 5000 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 5d1253e2dcc0e3e55992672756beadbc
RANDOM SEED=1951091992

node146.alineos.net Linux 2.6.18-164.el5 #1 SMP Thu Sep 3 03:28:30 EDT 2009

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5333.66
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.86
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
cpu MHz		: 2666.830
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx rdtscp lm constant_tsc ida nonstop_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr popcnt lahf_lm
bogomips	: 5332.75
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management: [8]


/proc/meminfo:
MemTotal:     32951124 kB
MemFree:      18664324 kB
Buffers:        235364 kB
Cached:        1611756 kB
SwapCached:      56140 kB
Active:       11142852 kB
Inactive:      2852068 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:     32951124 kB
LowFree:      18664324 kB
SwapTotal:    67111528 kB
SwapFree:     51873468 kB
Dirty:          141288 kB
Writeback:           0 kB
AnonPages:    12098504 kB
Mapped:          30456 kB
Slab:           123180 kB
PageTables:      97752 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  83587088 kB
Committed_AS: 160822468 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    296008 kB
VmallocChunk: 34359440975 kB
HugePages_Total:     0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 73184 MiB
End job on node146 at 2011-05-17 05:54:00