Trace number 1087500

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
Mistral-option 1.314UNSAT 323.136 324.742

DiagnosticValue
ASSIGNMENTS0
CHECKS38032

General information on the benchmark

Namecsp/fapp/fapp29/
normalized-fapp29-2900-3.xml
MD5SUMa235a933b5e2cb4afa8ed79f71c05679
Bench Category2-ARY-INT (binary constraints in intension)
Best result obtained on this benchmarkUNSAT
Best CPU time to get the best result obtained on this benchmark17.5323
Satisfiable
(Un)Satisfiability was proved
Number of variables2900
Number of constraints67898
Maximum constraint arity2
Maximum domain size998
Number of constraints which are defined in extension0
Number of constraints which are defined in intension67898
Global constraints used (with number of constraints)

Solver Data (download as text)

0.20/0.22	c mistral-option version 1.313
322.88/324.54	s UNSATISFIABLE
322.88/324.54	d CHECKS 38032
322.88/324.54	d ASSIGNMENTS 0

Verifier Data (download as text)

No possible verification on an UNSAT instance

Watcher Data (download as text)

runsolver version 3.2.5 (c) roussel@cril.univ-artois.fr

command line: /home/evaluation/evaluation/pub/bin/runsolver --timestamp -w /tmp/evaluation-result-1087500-1215244423/watcher-1087500-1215244423 -o /tmp/evaluation-result-1087500-1215244423/solver-1087500-1215244423 -C 1800 -W 2200 -M 900 --output-limit 1,15 HOME/xsolve HOME/instance-1087500-1215244423.xml -heu dyn -sac 3 -ds 2 

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 1800 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 1830 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2200 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 921600 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 972800 KiB
Solver output will be limited to a maximum of 15728640 bytes. The first 1048576 bytes and the last 14680064 bytes will be preserved
Current StackSize limit: 10240 KiB


[startup+0 s]
/proc/loadavg: 2.65 2.37 2.17 4/80 22067
/proc/meminfo: memFree=951760/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=18572 CPUtime=0
/proc/22067/stat : 22067 (runsolver) R 22065 22067 21239 0 -1 4194368 15 0 0 0 0 0 0 0 20 0 1 0 2003416600 19017728 292 996147200 4194304 4296836 548682068416 18446744073709551615 221236489511 0 0 4096 24578 0 0 0 17 1 0 0
/proc/22067/statm: 4643 292 257 25 0 2626 0

[startup+0.218715 s]
/proc/loadavg: 2.65 2.37 2.17 4/80 22067
/proc/meminfo: memFree=951760/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=8960 CPUtime=0.2
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 1557 0 0 0 20 0 0 0 18 0 1 0 2003416600 9175040 1523 996147200 134512640 135239763 4294956192 18446744073709551615 4294960144 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 2240 1523 379 177 0 1143 0
Current children cumulated CPU time (s) 0.2
Current children cumulated vsize (KiB) 8960

[startup+0.301732 s]
/proc/loadavg: 2.65 2.37 2.17 4/80 22067
/proc/meminfo: memFree=951760/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=10168 CPUtime=0.29
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 1832 0 0 0 28 1 0 0 19 0 1 0 2003416600 10412032 1798 996147200 134512640 135239763 4294956192 18446744073709551615 8873595 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 2542 1798 379 177 0 1445 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 10168

[startup+0.401749 s]
/proc/loadavg: 2.65 2.37 2.17 4/80 22067
/proc/meminfo: memFree=951760/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=11476 CPUtime=0.39
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 2160 0 0 0 38 1 0 0 20 0 1 0 2003416600 11751424 2126 996147200 134512640 135239763 4294956192 18446744073709551615 8873595 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 2869 2127 379 177 0 1772 0
Current children cumulated CPU time (s) 0.39
Current children cumulated vsize (KiB) 11476

[startup+0.701804 s]
/proc/loadavg: 2.65 2.37 2.17 4/80 22067
/proc/meminfo: memFree=951760/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=15436 CPUtime=0.69
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 3195 0 0 0 67 2 0 0 23 0 1 0 2003416600 15806464 3095 996147200 134512640 135239763 4294956192 18446744073709551615 8873595 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 3859 3095 379 177 0 2762 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 15436

[startup+1.50296 s]
/proc/loadavg: 2.60 2.36 2.17 3/81 22068
/proc/meminfo: memFree=934472/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=25712 CPUtime=1.48
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 5856 0 0 0 145 3 0 0 25 0 1 0 2003416600 26329088 5626 996147200 134512640 135239763 4294956192 18446744073709551615 8873595 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 6428 5626 379 177 0 5331 0
Current children cumulated CPU time (s) 1.48
Current children cumulated vsize (KiB) 25712

[startup+3.10126 s]
/proc/loadavg: 2.60 2.36 2.17 3/81 22068
/proc/meminfo: memFree=920904/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=45632 CPUtime=3.07
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 11087 0 0 0 301 6 0 0 25 0 1 0 2003416600 46727168 10501 996147200 134512640 135239763 4294956192 18446744073709551615 4294960144 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 11408 10501 379 177 0 10311 0
Current children cumulated CPU time (s) 3.07
Current children cumulated vsize (KiB) 45632

[startup+6.30186 s]
/proc/loadavg: 2.60 2.36 2.17 3/81 22068
/proc/meminfo: memFree=850952/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=6.25
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26183 0 0 0 611 14 0 0 25 0 1 0 2003416600 132415488 25499 996147200 134512640 135239763 4294956192 18446744073709551615 134543192 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25499 396 177 0 31231 0
Current children cumulated CPU time (s) 6.25
Current children cumulated vsize (KiB) 129312

[startup+12.7011 s]
/proc/loadavg: 2.51 2.35 2.17 3/81 22068
/proc/meminfo: memFree=850888/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=12.62
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 1248 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134532191 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 12.62
Current children cumulated vsize (KiB) 129312

[startup+25.5065 s]
/proc/loadavg: 2.43 2.34 2.17 3/81 22068
/proc/meminfo: memFree=850888/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=25.36
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 2522 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134535399 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 25.36
Current children cumulated vsize (KiB) 129312

[startup+51.1013 s]
/proc/loadavg: 2.26 2.30 2.16 3/81 22068
/proc/meminfo: memFree=850952/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=50.82
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 5068 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134535381 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 50.82
Current children cumulated vsize (KiB) 129312

[startup+102.302 s]
/proc/loadavg: 2.11 2.25 2.15 3/81 22068
/proc/meminfo: memFree=850952/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=101.77
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 10163 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134532177 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 101.77
Current children cumulated vsize (KiB) 129312

[startup+162.301 s]
/proc/loadavg: 2.04 2.20 2.14 3/81 22068
/proc/meminfo: memFree=851016/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=161.48
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 16134 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134535329 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 161.48
Current children cumulated vsize (KiB) 129312

[startup+222.302 s]
/proc/loadavg: 2.01 2.17 2.13 4/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=221.19
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 22105 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 135095110 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 221.19
Current children cumulated vsize (KiB) 129312

[startup+282.302 s]
/proc/loadavg: 2.00 2.13 2.12 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=280.89
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 28075 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134543109 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 280.89
Current children cumulated vsize (KiB) 129312

Solver just ended. Dumping a history of the last processes samples

[startup+307.302 s]
/proc/loadavg: 2.00 2.12 2.11 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=305.77
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 30563 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 135096898 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 305.77
Current children cumulated vsize (KiB) 129312

[startup+313.706 s]
/proc/loadavg: 2.00 2.12 2.11 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=312.14
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 31200 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134543111 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 312.14
Current children cumulated vsize (KiB) 129312

[startup+320.102 s]
/proc/loadavg: 2.00 2.12 2.11 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=318.5
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 31836 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134543127 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 318.5
Current children cumulated vsize (KiB) 129312

[startup+323.302 s]
/proc/loadavg: 2.00 2.11 2.11 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=321.69
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 32155 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134543109 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 321.69
Current children cumulated vsize (KiB) 129312

[startup+324.102 s]
/proc/loadavg: 2.00 2.11 2.11 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=322.49
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 32235 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134539608 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 322.49
Current children cumulated vsize (KiB) 129312

[startup+324.502 s]
/proc/loadavg: 2.00 2.11 2.11 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=129312 CPUtime=322.88
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26184 0 0 0 32274 14 0 0 25 0 1 0 2003416600 132415488 25500 996147200 134512640 135239763 4294956192 18446744073709551615 134534752 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32328 25500 397 177 0 31231 0
Current children cumulated CPU time (s) 322.88
Current children cumulated vsize (KiB) 129312

[startup+324.707 s]
/proc/loadavg: 2.00 2.11 2.11 3/81 22068
/proc/meminfo: memFree=851080/2055920 swapFree=4180412/4192956
[pid=22067] ppid=22065 vsize=128260 CPUtime=323.09
/proc/22067/stat : 22067 (xsolve) R 22065 22067 21239 0 -1 4194304 26198 0 0 0 32295 14 0 0 25 0 1 0 2003416600 131338240 25379 996147200 134512640 135239763 4294956192 18446744073709551615 8871477 0 0 4096 0 0 0 0 17 1 0 0
/proc/22067/statm: 32065 25379 411 177 0 30968 0
Current children cumulated CPU time (s) 323.09
Current children cumulated vsize (KiB) 128260

Child status: 0
Real time (s): 324.742
CPU time (s): 323.136
CPU user time (s): 322.97
CPU system time (s): 0.165974
CPU usage (%): 99.5055
Max. virtual memory (cumulated for all children) (KiB): 129312

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 322.97
system time used= 0.165974
maximum resident set size= 0
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 26199
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7
involuntary context switches= 3695

runsolver used 0.359945 second user time and 0.830873 second system time

The end

Launcher Data (download as text)

Begin job on node21 at 2008-07-05 09:53:43
IDJOB=1087500
IDBENCH=58000
IDSOLVER=357
FILE ID=node21/1087500-1215244423
PBS_JOBID= 7881986
Free space on /tmp= 66468 MiB

SOLVER NAME= Mistral-option 1.314
BENCH NAME= CPAI08/csp/fapp/fapp29/normalized-fapp29-2900-3.xml
COMMAND LINE= HOME/xsolve BENCHNAME -heu dyn -sac 3 -ds 2 
RUNSOLVER COMMAND LINE= /home/evaluation/evaluation/pub/bin/runsolver --timestamp -w /tmp/evaluation-result-1087500-1215244423/watcher-1087500-1215244423 -o /tmp/evaluation-result-1087500-1215244423/solver-1087500-1215244423 -C 1800 -W 2200 -M 900 --output-limit 1,15  HOME/xsolve HOME/instance-1087500-1215244423.xml -heu dyn -sac 3 -ds 2 

TIME LIMIT= 1800 seconds
MEMORY LIMIT= 900 MiB

MD5SUM BENCH= a235a933b5e2cb4afa8ed79f71c05679
RANDOM SEED=1590544811

node21.alineos.net Linux 2.6.9-22.EL.rootsmp #1 SMP Mon Oct 3 08:59:52 CEST 2005

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.263
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5914.62
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 15
model		: 4
model name	:                   Intel(R) Xeon(TM) CPU 3.00GHz
stepping	: 3
cpu MHz		: 3000.263
cache size	: 2048 KB
fpu		: yes
fpu_exception	: yes
cpuid level	: 5
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm pni monitor ds_cpl cid cx16 xtpr
bogomips	: 5586.94
clflush size	: 64
cache_alignment	: 128
address sizes	: 36 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:      2055920 kB
MemFree:        952240 kB
Buffers:        102092 kB
Cached:         444820 kB
SwapCached:       6472 kB
Active:         704168 kB
Inactive:       329116 kB
HighTotal:           0 kB
HighFree:            0 kB
LowTotal:      2055920 kB
LowFree:        952240 kB
SwapTotal:     4192956 kB
SwapFree:      4180412 kB
Dirty:           10584 kB
Writeback:           0 kB
Mapped:         498744 kB
Slab:            54644 kB
Committed_AS:  4343968 kB
PageTables:       2756 kB
VmallocTotal: 536870911 kB
VmallocUsed:    264952 kB
VmallocChunk: 536605679 kB
HugePages_Total:     0
HugePages_Free:      0
Hugepagesize:     2048 kB

Free space on /tmp at the end= 66468 MiB
End job on node21 at 2008-07-05 09:59:08