Trace number 4409167

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (TO) 2400.1 2400.11

General information on the benchmark

NameGolombRuler/GolombRuler-a3-s1/
GolombRuler-28-a3.xml
MD5SUM7bf94b2c859f2895a81dfe97f17444ea
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark935
Best CPU time to get the best result obtained on this benchmark2400.46
Satisfiable
(Un)Satisfiability was proved
Number of variables812
Number of constraints381
Number of domains2
Minimum domain size1000
Maximum domain size1001
Distribution of domain sizes[{"size":1000,"count":378},{"size":1001,"count":28}]
Minimum variable degree0
Maximum variable degree29
Distribution of variable degrees[{"degree":0,"count":406},{"degree":2,"count":378},{"degree":28,"count":26},{"degree":29,"count":2}]
Minimum constraint arity1
Maximum constraint arity378
Distribution of constraint arities[{"arity":1,"count":1},{"arity":3,"count":378},{"arity":28,"count":1},{"arity":378,"count":1}]
Number of extensional constraints0
Number of intensional constraints379
Distribution of constraint types[{"type":"intension","count":379},{"type":"allDifferent","count":1},{"type":"ordered","count":1}]
Optimization problemYES
Type of objectivemin VAR

Solver Data 

0.10/0.13	c
0.10/0.13	c This is cosoco 2.00 --  
0.10/0.13	c
0.10/0.13	c command line: cosoco -lc -model -restarts -stick HOME/instance-4409167-1569798098.xml 
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
0.10/0.13	0
2400.09/2400.11	c enable optimization
2400.09/2400.11	c ========================================[ Problem Statistics ]===========================================
2400.09/2400.11	c |                                                                                                       
2400.09/2400.11	c |  Parse time        :         0.12 s 
2400.09/2400.11	c |
2400.09/2400.11	c |               Variables: 812
2400.09/2400.11	c |            Domain Sizes: 1000..1001
2400.09/2400.11	c |
2400.09/2400.11	c |             Constraints: 408
2400.09/2400.11	c |                   Arity: 1..378  -- Unary: 2  -- Binary: 27  -- Ternary: 378
2400.09/2400.11	c | 
2400.09/2400.11	c |                   Types: 
2400.09/2400.11	c |                          (X - Y < k): 27
2400.09/2400.11	c |                          All Different: 1
2400.09/2400.11	c |                          Intension: 1
2400.09/2400.11	c |                          Sum: 378
2400.09/2400.11	c |                          Unary: 1
2400.09/2400.11	c |                          
2400.09/2400.11	c |               Objective: Minimize Intension
2400.09/2400.11	
2400.09/2400.11	c =========================================================================================================
2400.09/2400.11	
2400.09/2400.11	c *** INTERRUPTED ***
2400.09/2400.11	c call to solver        : 0
2400.09/2400.11	c
2400.09/2400.11	c restarts              : 48          
2400.09/2400.11	c conflicts             : 96538        (40 /sec)
2400.09/2400.11	c decisions             : 267950       (112 /sec)
2400.09/2400.11	c propagations          : 3133111906  (1305421 /sec)
2400.09/2400.11	c root propagations     : 7636        
2400.09/2400.11	c filter calls          : 293243316      (122181 /sec)
2400.09/2400.11	c useless filter calls  : 54968899       (18 %)
2400.09/2400.11	c Memory used           : 37.98 MB
2400.09/2400.11	c Best bound time (wc)  : -1.32531e+09 s
2400.09/2400.11	c CPU time              : 2400.08 s
2400.09/2400.11	c real time             : 2400.11 s
2400.09/2400.11	
2400.09/2400.11	c No Bound found!
2400.09/2400.11	
2400.09/2400.11	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16375 MiB (15222 MiB free)
  memory of node 1: 16384 MiB (15537 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409167-1569798098/watcher-4409167-1569798098 -o /tmp/evaluation-result-4409167-1569798098/solver-4409167-1569798098 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node120-1325301813-18515 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4409167-1569798098.xml 

running on 4 cores: 4-7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=20382, runsolver pid=20379

[startup+0.10016 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 20383
/proc/meminfo: memFree=31488628/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=37104 memory=13448 CPUtime=0.1 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 3543 0 0 0 9 1 0 0 20 0 1 0 555815839 37994496 3362 33554432000 4194304 6211132 140720947708480 140720947705016 140595624659941 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9276 3362 821 493 0 2581 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 37104 KiB
Current children cumulated memory: 13448 KiB

[startup+0.210958 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 20383
/proc/meminfo: memFree=31487140/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38148 memory=14624 CPUtime=0.21 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 3866 0 0 0 20 1 0 0 20 0 1 0 555815839 39063552 3656 33554432000 4194304 6211132 140720947708480 140720947705016 5193760 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9537 3656 838 493 0 2842 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 38148 KiB
Current children cumulated memory: 14624 KiB

[startup+0.310544 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 20383
/proc/meminfo: memFree=31487140/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38148 memory=14624 CPUtime=0.31 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 3867 0 0 0 30 1 0 0 20 0 1 0 555815839 39063552 3656 33554432000 4194304 6211132 140720947708480 140720947706744 5193760 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9537 3656 838 493 0 2842 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 38148 KiB
Current children cumulated memory: 14624 KiB

[startup+0.700301 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 20383
/proc/meminfo: memFree=31486772/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=14888 CPUtime=0.69 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 3937 0 0 0 68 1 0 0 20 0 1 0 555815839 39370752 3722 33554432000 4194304 6211132 140720947708480 140720947706744 5193760 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3722 881 493 0 2917 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 14888 KiB

[startup+1.5003 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 20383
/proc/meminfo: memFree=31486772/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=14888 CPUtime=1.49 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 3941 0 0 0 148 1 0 0 20 0 1 0 555815839 39370752 3722 33554432000 4194304 6211132 140720947708480 140720947705576 5150109 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3722 881 493 0 2917 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 14888 KiB

[startup+3.11037 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 20383
/proc/meminfo: memFree=31487136/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15116 CPUtime=3.1 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 5729 0 0 0 309 1 0 0 20 0 1 0 555815839 39370752 3779 33554432000 4194304 6211132 140720947708480 140720947705576 5149718 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3779 881 493 0 2917 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15116 KiB

[startup+6.30029 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 20384
/proc/meminfo: memFree=31486936/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15116 CPUtime=6.29 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 7524 0 0 0 628 1 0 0 20 0 1 0 555815839 39370752 3779 33554432000 4194304 6211132 140720947708480 140720947705576 5079049 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3779 881 493 0 2917 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15116 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 20384
/proc/meminfo: memFree=31486936/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15116 CPUtime=12.69 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 9306 0 0 0 1268 1 0 0 20 0 1 0 555815839 39370752 3779 33554432000 4194304 6211132 140720947708480 140720947705576 5149721 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3779 881 493 0 2917 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15116 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 20384
/proc/meminfo: memFree=31486192/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15116 CPUtime=25.48 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 12881 0 0 0 2547 1 0 0 20 0 1 0 555815839 39370752 3779 33554432000 4194304 6211132 140720947708480 140720947705576 5193760 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3779 881 493 0 2917 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15116 KiB

[startup+51.1103 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 20386
/proc/meminfo: memFree=31486516/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15116 CPUtime=51.09 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 13130 0 0 0 5108 1 0 0 20 0 1 0 555815839 39370752 3779 33554432000 4194304 6211132 140720947708480 140720947705576 5079432 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3779 881 493 0 2917 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15116 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 20390
/proc/meminfo: memFree=31486304/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15120 CPUtime=102.27 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 13381 0 0 0 10226 1 0 0 20 0 1 0 555815839 39370752 3780 33554432000 4194304 6211132 140720947708480 140720947705576 5149709 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3780 881 493 0 2917 0
Current children cumulated CPU time: 102.27 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15120 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 20394
/proc/meminfo: memFree=31486768/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15120 CPUtime=162.27 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 13625 0 0 0 16226 1 0 0 20 0 1 0 555815839 39370752 3780 33554432000 4194304 6211132 140720947708480 140720947705576 5149706 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3780 881 493 0 2917 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15120 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 20401
/proc/meminfo: memFree=31486028/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15120 CPUtime=222.26 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 13867 0 0 0 22225 1 0 0 20 0 1 0 555815839 39370752 3780 33554432000 4194304 6211132 140720947708480 140720947705576 5149680 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3780 881 493 0 2917 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15120 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 20405
/proc/meminfo: memFree=31486244/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15120 CPUtime=282.26 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 14111 0 0 0 28225 1 0 0 20 0 1 0 555815839 39370752 3780 33554432000 4194304 6211132 140720947708480 140720947705576 5149706 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3780 881 493 0 2917 0
Current children cumulated CPU time: 282.26 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15120 KiB

[startup+342.3 s]
/proc/loadavg: 1.96 2.00 2.04 3/199 20409
/proc/meminfo: memFree=31486716/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15120 CPUtime=342.26 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 14355 0 0 0 34225 1 0 0 20 0 1 0 555815839 39370752 3780 33554432000 4194304 6211132 140720947708480 140720947705576 5149721 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3780 881 493 0 2917 0
Current children cumulated CPU time: 342.26 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15120 KiB

[startup+402.3 s]
/proc/loadavg: 1.98 2.00 2.04 3/200 20416
/proc/meminfo: memFree=31485032/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15120 CPUtime=402.26 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 14597 0 0 0 40225 1 0 0 20 0 1 0 555815839 39370752 3780 33554432000 4194304 6211132 140720947708480 140720947705576 5193760 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3780 881 493 0 2917 0
Current children cumulated CPU time: 402.26 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15120 KiB

[startup+462.3 s]
/proc/loadavg: 1.99 2.00 2.04 3/199 20420
/proc/meminfo: memFree=31485660/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38448 memory=15120 CPUtime=462.26 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 14841 0 0 0 46225 1 0 0 20 0 1 0 555815839 39370752 3780 33554432000 4194304 6211132 140720947708480 140720947705576 5149706 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9612 3780 881 493 0 2917 0
Current children cumulated CPU time: 462.26 s
Current children cumulated vsize: 38448 KiB
Current children cumulated memory: 15120 KiB


################
# More data... #
################

/proc/20382/statm: 9656 3797 881 493 0 2961 0
Current children cumulated CPU time: 1662.27 s
Current children cumulated vsize: 38624 KiB
Current children cumulated memory: 15188 KiB

[startup+1722.3 s]
/proc/loadavg: 2.03 2.06 2.05 3/200 20603
/proc/meminfo: memFree=31504148/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38624 memory=15188 CPUtime=1722.27 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 20035 0 0 0 172226 1 0 0 20 0 1 0 555815839 39550976 3797 33554432000 4194304 6211132 140720947708480 140720947705784 5150084 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9656 3797 881 493 0 2961 0
Current children cumulated CPU time: 1722.27 s
Current children cumulated vsize: 38624 KiB
Current children cumulated memory: 15188 KiB

[startup+1782.3 s]
/proc/loadavg: 2.06 2.07 2.05 3/200 20607
/proc/meminfo: memFree=31504260/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38624 memory=15188 CPUtime=1782.27 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 20275 0 0 0 178226 1 0 0 20 0 1 0 555815839 39550976 3797 33554432000 4194304 6211132 140720947708480 140720947705784 5193760 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9656 3797 881 493 0 2961 0
Current children cumulated CPU time: 1782.27 s
Current children cumulated vsize: 38624 KiB
Current children cumulated memory: 15188 KiB

[startup+1842.3 s]
/proc/loadavg: 2.02 2.05 2.05 3/201 20613
/proc/meminfo: memFree=31503344/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38624 memory=15188 CPUtime=1842.27 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 20515 0 0 0 184226 1 0 0 20 0 1 0 555815839 39550976 3797 33554432000 4194304 6211132 140720947708480 140720947705784 5149709 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9656 3797 881 493 0 2961 0
Current children cumulated CPU time: 1842.27 s
Current children cumulated vsize: 38624 KiB
Current children cumulated memory: 15188 KiB

[startup+1902.3 s]
/proc/loadavg: 2.01 2.04 2.05 3/200 20617
/proc/meminfo: memFree=31503840/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38624 memory=15188 CPUtime=1902.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 20755 0 0 0 190226 2 0 0 20 0 1 0 555815839 39550976 3797 33554432000 4194304 6211132 140720947708480 140720947705784 5193760 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9656 3797 881 493 0 2961 0
Current children cumulated CPU time: 1902.28 s
Current children cumulated vsize: 38624 KiB
Current children cumulated memory: 15188 KiB

[startup+1962.3 s]
/proc/loadavg: 2.07 2.05 2.05 3/200 20621
/proc/meminfo: memFree=31504208/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38624 memory=15204 CPUtime=1962.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 21064 0 0 0 196226 2 0 0 20 0 1 0 555815839 39550976 3801 33554432000 4194304 6211132 140720947708480 140720947705784 5149714 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9656 3801 881 493 0 2961 0
Current children cumulated CPU time: 1962.28 s
Current children cumulated vsize: 38624 KiB
Current children cumulated memory: 15204 KiB

[startup+2022.3 s]
/proc/loadavg: 2.03 2.04 2.05 3/200 20628
/proc/meminfo: memFree=31502932/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38756 memory=15280 CPUtime=2022.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 21367 0 0 0 202226 2 0 0 20 0 1 0 555815839 39686144 3820 33554432000 4194304 6211132 140720947708480 140720947705704 5075678 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9689 3820 881 493 0 2994 0
Current children cumulated CPU time: 2022.28 s
Current children cumulated vsize: 38756 KiB
Current children cumulated memory: 15280 KiB

[startup+2082.3 s]
/proc/loadavg: 2.01 2.04 2.05 3/199 20632
/proc/meminfo: memFree=31503312/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38756 memory=15344 CPUtime=2082.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 21667 0 0 0 208226 2 0 0 20 0 1 0 555815839 39686144 3836 33554432000 4194304 6211132 140720947708480 140720947705704 5193760 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9689 3836 881 493 0 2994 0
Current children cumulated CPU time: 2082.28 s
Current children cumulated vsize: 38756 KiB
Current children cumulated memory: 15344 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 20636
/proc/meminfo: memFree=31503304/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38756 memory=15344 CPUtime=2142.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 21936 0 0 0 214226 2 0 0 20 0 1 0 555815839 39686144 3836 33554432000 4194304 6211132 140720947708480 140720947705704 5149706 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9689 3836 881 493 0 2994 0
Current children cumulated CPU time: 2142.28 s
Current children cumulated vsize: 38756 KiB
Current children cumulated memory: 15344 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/200 20643
/proc/meminfo: memFree=31503248/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38756 memory=15344 CPUtime=2202.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 22231 0 0 0 220226 2 0 0 20 0 1 0 555815839 39686144 3836 33554432000 4194304 6211132 140720947708480 140720947705704 5078752 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9689 3836 881 493 0 2994 0
Current children cumulated CPU time: 2202.28 s
Current children cumulated vsize: 38756 KiB
Current children cumulated memory: 15344 KiB

[startup+2262.3 s]
/proc/loadavg: 2.07 2.04 2.05 3/199 20647
/proc/meminfo: memFree=31503800/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38756 memory=15344 CPUtime=2262.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 22509 0 0 0 226226 2 0 0 20 0 1 0 555815839 39686144 3836 33554432000 4194304 6211132 140720947708480 140720947705704 5149714 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9689 3836 881 493 0 2994 0
Current children cumulated CPU time: 2262.28 s
Current children cumulated vsize: 38756 KiB
Current children cumulated memory: 15344 KiB

[startup+2322.3 s]
/proc/loadavg: 2.03 2.03 2.05 3/200 20659
/proc/meminfo: memFree=31503104/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38756 memory=15344 CPUtime=2322.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 22791 0 0 0 232226 2 0 0 20 0 1 0 555815839 39686144 3836 33554432000 4194304 6211132 140720947708480 140720947705704 5193760 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9689 3836 881 493 0 2994 0
Current children cumulated CPU time: 2322.28 s
Current children cumulated vsize: 38756 KiB
Current children cumulated memory: 15344 KiB

[startup+2382.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/202 20688
/proc/meminfo: memFree=31502392/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38888 memory=15344 CPUtime=2382.28 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 23130 0 0 0 238226 2 0 0 20 0 1 0 555815839 39821312 3836 33554432000 4194304 6211132 140720947708480 140720947705704 5078762 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9722 3836 881 493 0 3027 0
Current children cumulated CPU time: 2382.28 s
Current children cumulated vsize: 38888 KiB
Current children cumulated memory: 15344 KiB



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+2400.11 s]
/proc/loadavg: 2.01 2.02 2.05 3/201 20688
/proc/meminfo: memFree=31502548/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38888 memory=15484 CPUtime=2400.09 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 23418 0 0 0 240007 2 0 0 20 0 1 0 555815839 39821312 3871 33554432000 4194304 6211132 140720947708480 140720947705704 5079374 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9722 3871 881 493 0 3027 0
Current children cumulated CPU time: 2400.09 s
Current children cumulated vsize: 38888 KiB
Current children cumulated memory: 15484 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2400.11 s]
# the end of solver process 20382 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2400.08 s, system=0.020999 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

[startup+2400.11 s]
/proc/loadavg: 2.01 2.02 2.05 3/201 20688
/proc/meminfo: memFree=31502548/32770688 swapFree=66927776/67108860
[pid=20382] ppid=20379 vsize=38888 memory=15484 CPUtime=2400.09 cores=4-7
/proc/20382/stat : 20382 (cosoco) R 20379 20382 18483 0 -1 4202496 23418 0 0 0 240007 2 0 0 20 0 1 0 555815839 39821312 3871 33554432000 4194304 6211132 140720947708480 140720947705704 5079374 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 11517952 140720947713035 140720947713137 140720947713137 140720947716046 0
/proc/20382/statm: 9722 3871 881 493 0 3027 0
Current children cumulated CPU time: 2400.09 s
Current children cumulated vsize: 38888 KiB
Current children cumulated memory: 15484 KiB

Child status: 1
Real time (s): 2400.11
CPU time (s): 2400.1
CPU user time (s): 2400.08
CPU system time (s): 0.020999
CPU usage (%): 99.9995
Max. virtual memory (cumulated for all children) (KiB): 38888
Max. memory (cumulated for all children) (KiB): 15484

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2400.08
system time used= 0.020999
maximum resident set size= 15548
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 23469
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 19
involuntary context switches= 795


# summary of solver processes directly reported to runsolver:
#   pid: 20382
#   total CPU time (s): 2400.1
#   total CPU user time (s): 2400.08
#   total CPU system time (s): 0.020999

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.33394 second user time and 18.6552 second system time

The end

Launcher Data 

Begin job on node120 at 2011-12-31 07:21:58
IDJOB=4409167
IDBENCH=126211
IDSOLVER=2876
FILE ID=node120/4409167-1569798098
RUNJOBID= node120-1325301813-18515
PBS_JOBID= 21705832
Free space on /tmp= 61840 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/GolombRuler/GolombRuler-a3-s1/GolombRuler-28-a3.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409167-1569798098/watcher-4409167-1569798098 -o /tmp/evaluation-result-4409167-1569798098/solver-4409167-1569798098 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node120-1325301813-18515 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4409167-1569798098.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 7bf94b2c859f2895a81dfe97f17444ea
RANDOM SEED=545763179

node120.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5329.69
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770688 kB
MemFree:        31498572 kB
MemAvailable:   31758152 kB
Buffers:          203708 kB
Cached:           385980 kB
SwapCached:        68828 kB
Active:           360696 kB
Inactive:         343292 kB
Active(anon):      65352 kB
Inactive(anon):   130284 kB
Active(file):     295344 kB
Inactive(file):   213008 kB
Unevictable:      107072 kB
Mlocked:          107072 kB
SwapTotal:      67108860 kB
SwapFree:       66927776 kB
Dirty:              2836 kB
Writeback:             0 kB
AnonPages:        180796 kB
Mapped:            29220 kB
Shmem:             72460 kB
Slab:             164220 kB
SReclaimable:     134128 kB
SUnreclaim:        30092 kB
KernelStack:        3792 kB
PageTables:         5620 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494204 kB
Committed_AS:     617344 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350212 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    131072 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      146880 kB
DirectMap2M:    33398784 kB

Free space on /tmp at the end= 61836 MiB
End job on node120 at 2011-12-31 08:01:58