Trace number 4240442

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
R1: cosoco 1.1? (TO) 2400.09 2400.41

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-07-350-100.xml
MD5SUMcd1777af07bd314664d86e97c163b75e
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark22
Best CPU time to get the best result obtained on this benchmark19965.9
Satisfiable
(Un)Satisfiability was proved
Number of variables2451
Number of constraints29
Number of domains2
Minimum domain size2
Maximum domain size334
Distribution of domain sizes[{"size":2,"count":2450},{"size":334,"count":1}]
Minimum variable degree8
Maximum variable degree22
Distribution of variable degrees[{"degree":8,"count":2450},{"degree":22,"count":1}]
Minimum constraint arity350
Maximum constraint arity2450
Distribution of constraint arities[{"arity":350,"count":7},{"arity":701,"count":21},{"arity":2450,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":28}]
Optimization problemYES
Type of objectivemin VAR

Solver Data 

0.19/0.20	c best=?? lower=0 upper=333

Verifier Data 

One s line expected

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4240442-1500636670/watcher-4240442-1500636670 -o /tmp/evaluation-result-4240442-1500636670/solver-4240442-1500636670 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1500623787-5215 --watchdog 2560 HOME/cosoco -lc -model -restarts HOME/instance-4240442-1500636670.xml 

pid=8031
running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 1.95 2.00 2.04 3/186 8034
/proc/meminfo: memFree=30731248/32770624 swapFree=6216/755916
[pid=8034] ppid=8031 vsize=27120 CPUtime=0 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 986 0 0 0 0 0 0 0 20 0 1 0 18374759 27770880 804 33554432000 4194304 5970956 140726766852864 140726766852056 140217376356312 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 6780 804 703 434 0 144 0

[startup+0.100163 s]
/proc/loadavg: 1.95 2.00 2.04 3/186 8034
/proc/meminfo: memFree=30731248/32770624 swapFree=6216/755916
[pid=8034] ppid=8031 vsize=30252 CPUtime=0.09 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 1850 0 0 0 9 0 0 0 20 0 1 0 18374759 30978048 1654 33554432000 4194304 5970956 140726766852864 140726766845864 5386458 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 7563 1654 801 434 0 927 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 30252

[startup+0.200315 s]
/proc/loadavg: 1.95 2.00 2.04 3/186 8034
/proc/meminfo: memFree=30731248/32770624 swapFree=6216/755916
[pid=8034] ppid=8031 vsize=36772 CPUtime=0.19 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 3508 0 0 0 19 0 0 0 20 0 1 0 18374759 37654528 3318 33554432000 4194304 5970956 140726766852864 140726766845864 140217366932453 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 9226 3318 803 434 0 2590 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 36772

[startup+0.30031 s]
/proc/loadavg: 1.95 2.00 2.04 3/186 8034
/proc/meminfo: memFree=30731248/32770624 swapFree=6216/755916
[pid=8034] ppid=8031 vsize=39016 CPUtime=0.28 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 4078 0 0 0 28 0 0 0 20 0 1 0 18374759 39952384 3912 33554432000 4194304 5970956 140726766852864 140726766849832 4964777 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 9754 3912 841 434 0 3118 0
Current children cumulated CPU time (s) 0.28
Current children cumulated vsize (KiB) 39016

[startup+0.700216 s]
/proc/loadavg: 1.95 2.00 2.04 3/186 8034
/proc/meminfo: memFree=30731248/32770624 swapFree=6216/755916
[pid=8034] ppid=8031 vsize=40204 CPUtime=0.69 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 4395 0 0 0 68 1 0 0 20 0 1 0 18374759 41168896 4176 33554432000 4194304 5970956 140726766852864 140726766849832 4964777 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 10051 4176 841 434 0 3415 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 40204

[startup+1.50022 s]
/proc/loadavg: 1.95 2.00 2.04 3/187 8035
/proc/meminfo: memFree=30706688/32770624 swapFree=6216/755944
[pid=8034] ppid=8031 vsize=44692 CPUtime=1.49 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 5516 0 0 0 148 1 0 0 20 0 1 0 18374759 45764608 5298 33554432000 4194304 5970956 140726766852864 140726766849832 4964777 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 11173 5298 841 434 0 4537 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 44692

[startup+3.10022 s]
/proc/loadavg: 1.95 2.00 2.04 3/187 8035
/proc/meminfo: memFree=30695512/32770624 swapFree=6216/755944
[pid=8034] ppid=8031 vsize=53668 CPUtime=3.09 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 8479 0 0 0 308 1 0 0 20 0 1 0 18374759 54956032 7546 33554432000 4194304 5970956 140726766852864 140726766849832 140217366932453 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 13417 7546 841 434 0 6781 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 53668

[startup+6.30022 s]
/proc/loadavg: 1.95 2.00 2.04 3/187 8035
/proc/meminfo: memFree=30660884/32770624 swapFree=6216/755952
[pid=8034] ppid=8031 vsize=69640 CPUtime=6.29 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 13524 0 0 0 627 2 0 0 20 0 1 0 18374759 71311360 11581 33554432000 4194304 5970956 140726766852864 140726766849832 4964777 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 17410 11581 842 434 0 10774 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 69640

[startup+12.7002 s]
/proc/loadavg: 1.96 2.00 2.04 3/187 8035
/proc/meminfo: memFree=30589416/32770624 swapFree=6216/755956
[pid=8034] ppid=8031 vsize=105544 CPUtime=12.69 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 24337 0 0 0 1265 4 0 0 20 0 1 0 18374759 108077056 20548 33554432000 4194304 5970956 140726766852864 140726766849832 140217366928190 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 26386 20548 842 434 0 19750 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 105544

[startup+25.5004 s]
/proc/loadavg: 1.97 2.00 2.04 3/187 8035
/proc/meminfo: memFree=30483456/32770624 swapFree=6216/755976
[pid=8034] ppid=8031 vsize=136828 CPUtime=25.49 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 59008 0 0 0 2543 6 0 0 20 0 1 0 18374759 140111872 28395 33554432000 4194304 5970956 140726766852864 140726766849832 4964452 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 34207 28395 843 434 0 27571 0
Current children cumulated CPU time (s) 25.49
Current children cumulated vsize (KiB) 136828

[startup+51.1004 s]
/proc/loadavg: 1.98 2.00 2.04 3/187 8035
/proc/meminfo: memFree=30346484/32770624 swapFree=6216/755992
[pid=8034] ppid=8031 vsize=136960 CPUtime=51.08 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 67025 0 0 0 5102 6 0 0 20 0 1 0 18374759 140247040 28403 33554432000 4194304 5970956 140726766852864 140726766849896 5098954 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 34240 28403 843 434 0 27604 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 136960

[startup+102.3 s]
/proc/loadavg: 2.06 2.02 2.05 3/187 8036
/proc/meminfo: memFree=30069120/32770624 swapFree=6216/756016
[pid=8034] ppid=8031 vsize=137356 CPUtime=102.28 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 81748 0 0 0 10221 7 0 0 20 0 1 0 18374759 140652544 28503 33554432000 4194304 5970956 140726766852864 140726766849896 4908985 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 34339 28503 843 434 0 27703 0
Current children cumulated CPU time (s) 102.28
Current children cumulated vsize (KiB) 137356

[startup+162.3 s]
/proc/loadavg: 2.02 2.01 2.05 3/187 8037
/proc/meminfo: memFree=29748724/32770624 swapFree=6216/756048
[pid=8034] ppid=8031 vsize=138148 CPUtime=162.27 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 107643 0 0 0 16218 9 0 0 20 0 1 0 18374759 141463552 28729 33554432000 4194304 5970956 140726766852864 140726766849896 5070437 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 34537 28729 843 434 0 27901 0
Current children cumulated CPU time (s) 162.27
Current children cumulated vsize (KiB) 138148

[startup+222.306 s]
/proc/loadavg: 2.01 2.01 2.05 3/187 8038
/proc/meminfo: memFree=29421236/32770624 swapFree=6216/756076
[pid=8034] ppid=8031 vsize=139868 CPUtime=222.27 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 121906 0 0 0 22217 10 0 0 20 0 1 0 18374759 143224832 29143 33554432000 4194304 5970956 140726766852864 140726766849896 5070523 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 34967 29143 843 434 0 28331 0
Current children cumulated CPU time (s) 222.27
Current children cumulated vsize (KiB) 139868

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 8039
/proc/meminfo: memFree=29100764/32770624 swapFree=6216/756108
[pid=8034] ppid=8031 vsize=142396 CPUtime=282.25 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 135901 0 0 0 28214 11 0 0 20 0 1 0 18374759 145813504 29754 33554432000 4194304 5970956 140726766852864 140726766849896 4964452 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 35599 29754 843 434 0 28963 0
Current children cumulated CPU time (s) 282.25
Current children cumulated vsize (KiB) 142396

[startup+342.3 s]
/proc/loadavg: 2.16 2.06 2.06 3/187 8040
/proc/meminfo: memFree=28770540/32770624 swapFree=6216/756136
[pid=8034] ppid=8031 vsize=145760 CPUtime=342.24 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 149614 0 0 0 34212 12 0 0 20 0 1 0 18374759 149258240 30616 33554432000 4194304 5970956 140726766852864 140726766849896 5070523 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 36440 30616 843 434 0 29804 0
Current children cumulated CPU time (s) 342.24
Current children cumulated vsize (KiB) 145760

[startup+402.3 s]
/proc/loadavg: 2.06 2.05 2.05 3/187 8041
/proc/meminfo: memFree=28450252/32770624 swapFree=6216/756168
[pid=8034] ppid=8031 vsize=147220 CPUtime=402.23 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 162060 0 0 0 40211 12 0 0 20 0 1 0 18374759 150753280 30943 33554432000 4194304 5970956 140726766852864 140726766849736 4961400 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 36805 30943 843 434 0 30169 0
Current children cumulated CPU time (s) 402.23
Current children cumulated vsize (KiB) 147220

[startup+462.3 s]
/proc/loadavg: 2.02 2.04 2.05 3/187 8042
/proc/meminfo: memFree=28125308/32770624 swapFree=6216/756196
[pid=8034] ppid=8031 vsize=147484 CPUtime=462.22 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 169528 0 0 0 46210 12 0 0 20 0 1 0 18374759 151023616 31019 33554432000 4194304 5970956 140726766852864 140726766849736 4961400 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 36871 31019 843 434 0 30235 0
Current children cumulated CPU time (s) 462.22
Current children cumulated vsize (KiB) 147484

[startup+522.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/187 8043
/proc/meminfo: memFree=27804176/32770624 swapFree=6216/756228
[pid=8034] ppid=8031 vsize=147884 CPUtime=522.22 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 177018 0 0 0 52209 13 0 0 20 0 1 0 18374759 151433216 31134 33554432000 4194304 5970956 140726766852864 140726766849736 4961400 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 36971 31134 843 434 0 30335 0
Current children cumulated CPU time (s) 522.22
Current children cumulated vsize (KiB) 147884

[startup+582.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/187 8051
/proc/meminfo: memFree=27479140/32770624 swapFree=6216/756272
[pid=8034] ppid=8031 vsize=148284 CPUtime=582.21 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 184527 0 0 0 58208 13 0 0 20 0 1 0 18374759 151842816 31249 33554432000 4194304 5970956 140726766852864 140726766849736 4964768 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37071 31249 843 434 0 30435 0
Current children cumulated CPU time (s) 582.21
Current children cumulated vsize (KiB) 148284


################
# More data... #
################


[startup+1482.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/187 8074
/proc/meminfo: memFree=25814696/32770624 swapFree=6216/756744
[pid=8034] ppid=8031 vsize=149616 CPUtime=1482.09 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 367827 0 0 0 148187 22 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4964768 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1482.09
Current children cumulated vsize (KiB) 149616

[startup+1542.31 s]
/proc/loadavg: 2.00 2.02 2.05 3/187 8075
/proc/meminfo: memFree=25814844/32770624 swapFree=6216/756776
[pid=8034] ppid=8031 vsize=149616 CPUtime=1542.09 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 380396 0 0 0 154187 22 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4964452 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1542.09
Current children cumulated vsize (KiB) 149616

[startup+1602.3 s]
/proc/loadavg: 2.08 2.04 2.05 3/187 8076
/proc/meminfo: memFree=25814620/32770624 swapFree=6216/756808
[pid=8034] ppid=8031 vsize=149616 CPUtime=1602.08 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 392979 0 0 0 160185 23 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 5098954 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1602.08
Current children cumulated vsize (KiB) 149616

[startup+1662.3 s]
/proc/loadavg: 2.03 2.03 2.05 3/187 8078
/proc/meminfo: memFree=25814280/32770624 swapFree=6216/756840
[pid=8034] ppid=8031 vsize=149616 CPUtime=1662.07 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 405595 0 0 0 166184 23 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4908971 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1662.07
Current children cumulated vsize (KiB) 149616

[startup+1722.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/187 8079
/proc/meminfo: memFree=25813944/32770624 swapFree=6216/756876
[pid=8034] ppid=8031 vsize=149616 CPUtime=1722.06 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 418216 0 0 0 172182 24 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 5098954 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1722.06
Current children cumulated vsize (KiB) 149616

[startup+1782.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/188 8088
/proc/meminfo: memFree=25812816/32770624 swapFree=6216/756912
[pid=8034] ppid=8031 vsize=149616 CPUtime=1782.06 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 430872 0 0 0 178181 25 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 5098948 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1782.06
Current children cumulated vsize (KiB) 149616

[startup+1842.3 s]
/proc/loadavg: 2.15 2.06 2.06 3/189 8112
/proc/meminfo: memFree=25812592/32770624 swapFree=6216/756952
[pid=8034] ppid=8031 vsize=149616 CPUtime=1842.05 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 443528 0 0 0 184180 25 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4964452 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1842.05
Current children cumulated vsize (KiB) 149616

[startup+1902.3 s]
/proc/loadavg: 2.05 2.05 2.05 3/189 8113
/proc/meminfo: memFree=25813064/32770624 swapFree=6216/756980
[pid=8034] ppid=8031 vsize=149616 CPUtime=1902.04 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 456222 0 0 0 190178 26 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4964424 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1902.04
Current children cumulated vsize (KiB) 149616

[startup+1962.3 s]
/proc/loadavg: 2.02 2.04 2.05 3/189 8114
/proc/meminfo: memFree=25813628/32770624 swapFree=6216/757004
[pid=8034] ppid=8031 vsize=149616 CPUtime=1962.03 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 468918 0 0 0 196177 26 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4908988 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 1962.03
Current children cumulated vsize (KiB) 149616

[startup+2022.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/189 8115
/proc/meminfo: memFree=25813096/32770624 swapFree=6216/757048
[pid=8034] ppid=8031 vsize=149616 CPUtime=2022.02 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 481647 0 0 0 202175 27 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 5070480 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 2022.02
Current children cumulated vsize (KiB) 149616

[startup+2082.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/188 8116
/proc/meminfo: memFree=25813640/32770624 swapFree=6216/757076
[pid=8034] ppid=8031 vsize=149616 CPUtime=2082.02 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 494380 0 0 0 208175 27 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4961411 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 2082.02
Current children cumulated vsize (KiB) 149616

[startup+2142.3 s]
/proc/loadavg: 2.03 2.04 2.05 3/187 8117
/proc/meminfo: memFree=25814052/32770624 swapFree=6216/757104
[pid=8034] ppid=8031 vsize=149616 CPUtime=2142.01 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 507146 0 0 0 214173 28 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4908985 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 2142.01
Current children cumulated vsize (KiB) 149616

[startup+2202.31 s]
/proc/loadavg: 2.01 2.03 2.05 3/187 8118
/proc/meminfo: memFree=25814324/32770624 swapFree=6216/757140
[pid=8034] ppid=8031 vsize=149616 CPUtime=2202 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 519933 0 0 0 220171 29 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 5098954 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 2202
Current children cumulated vsize (KiB) 149616

[startup+2262.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/187 8119
/proc/meminfo: memFree=25814436/32770624 swapFree=6216/757168
[pid=8034] ppid=8031 vsize=149616 CPUtime=2262 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 532725 0 0 0 226170 30 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 4961411 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 2262
Current children cumulated vsize (KiB) 149616

[startup+2322.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/187 8120
/proc/meminfo: memFree=25813496/32770624 swapFree=6216/757204
[pid=8034] ppid=8031 vsize=149616 CPUtime=2321.98 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 545548 0 0 0 232168 30 0 0 20 0 1 0 18374759 153206784 31581 33554432000 4194304 5970956 140726766852864 140726766849752 5098948 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31581 843 434 0 30768 0
Current children cumulated CPU time (s) 2321.98
Current children cumulated vsize (KiB) 149616

[startup+2382.3 s]
/proc/loadavg: 2.04 2.03 2.05 3/188 8190
/proc/meminfo: memFree=31028252/32770624 swapFree=6216/757268
[pid=8034] ppid=8031 vsize=149616 CPUtime=2381.97 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 558397 0 0 0 238167 30 0 0 20 0 1 0 18374759 153206784 31582 33554432000 4194304 5970956 140726766852864 140726766849752 5069735 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31582 843 434 0 30768 0
Current children cumulated CPU time (s) 2381.97
Current children cumulated vsize (KiB) 149616



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+2400.4 s]
/proc/loadavg: 2.03 2.03 2.05 3/187 8190
/proc/meminfo: memFree=31023856/32770624 swapFree=6216/757296
[pid=8034] ppid=8031 vsize=149616 CPUtime=2400.08 cores=1,3,5,7
/proc/8034/stat : 8034 (cosoco) R 8031 8034 5183 0 -1 4202496 560257 0 0 0 239977 31 0 0 20 0 1 0 18374759 153206784 31582 33554432000 4194304 5970956 140726766852864 140726766849752 4961400 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8070896 8075904 9789440 140726766855361 140726766855491 140726766855491 140726766858190 0
/proc/8034/statm: 37404 31582 843 434 0 30768 0
Current children cumulated CPU time (s) 2400.08
Current children cumulated vsize (KiB) 149616

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended.
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 8034 sig 9
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 2400.41
CPU time (s): 2400.09
CPU user time (s): 2399.77
CPU system time (s): 0.314958
CPU usage (%): 99.9867
Max. virtual memory (cumulated for all children) (KiB): 149616

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2399.77
system time used= 0.314958
maximum resident set size= 126372
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 560286
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 9
involuntary context switches= 121067

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.30421 second user time and 9.35455 second system time

The end

Launcher Data 

Begin job on node103 at 2017-07-21 13:31:10
IDJOB=4240442
IDBENCH=122949
IDSOLVER=2620
FILE ID=node103/4240442-1500636670
RUNJOBID= node103-1500623787-5215
PBS_JOBID= 20623024
Free space on /tmp= 62016 MiB

SOLVER NAME= cosoco 1.1
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-07-350-100.xml
COMMAND LINE= DIR/cosoco -lc -model -restarts BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4240442-1500636670/watcher-4240442-1500636670 -o /tmp/evaluation-result-4240442-1500636670/solver-4240442-1500636670 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1500623787-5215 --watchdog 2560  HOME/cosoco -lc -model -restarts HOME/instance-4240442-1500636670.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cd1777af07bd314664d86e97c163b75e
RANDOM SEED=1184167087

node103.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2133.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.65
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        30731812 kB
MemAvailable:   31533916 kB
Buffers:          163444 kB
Cached:           927112 kB
SwapCached:            0 kB
Active:           843804 kB
Inactive:         772948 kB
Active(anon):     529164 kB
Inactive(anon):    17032 kB
Active(file):     314640 kB
Inactive(file):   755916 kB
Unevictable:        6216 kB
Mlocked:            6216 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:              2452 kB
Writeback:             0 kB
AnonPages:        598212 kB
Mapped:            35360 kB
Shmem:             17044 kB
Slab:             120268 kB
SReclaimable:      94768 kB
SUnreclaim:        25500 kB
KernelStack:        3360 kB
PageTables:         6248 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     804668 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     75776 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 62016 MiB
End job on node103 at 2017-07-21 14:11:10