Trace number 4386788

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
BTD 19.07.01? (TO) 2519.91 2520

General information on the benchmark

NameCarSequencing/CarSequencing-m1-jcr/
CarSequencing-16-81.xml
MD5SUMe1683218a1e3c524b21c5eaa634d1705
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark27.9455
Satisfiable
(Un)Satisfiability was proved
Number of variables600
Number of constraints737
Number of domains2
Minimum domain size2
Maximum domain size26
Distribution of domain sizes[{"size":2,"count":500},{"size":26,"count":100}]
Minimum variable degree2
Maximum variable degree52
Distribution of variable degrees[{"degree":2,"count":100},{"degree":3,"count":5},{"degree":4,"count":5},{"degree":5,"count":6},{"degree":6,"count":10},{"degree":7,"count":10},{"degree":8,"count":18},{"degree":9,"count":18},{"degree":10,"count":18},{"degree":11,"count":18},{"degree":12,"count":18},{"degree":13,"count":18},{"degree":14,"count":18},{"degree":15,"count":18},{"degree":16,"count":18},{"degree":17,"count":18},{"degree":18,"count":18},{"degree":19,"count":18},{"degree":20,"count":18},{"degree":21,"count":18},{"degree":22,"count":20},{"degree":23,"count":20},{"degree":24,"count":20},{"degree":25,"count":20},{"degree":26,"count":10},"...",{"degree":28,"count":8}, {"degree":29,"count":8}, {"degree":30,"count":9}, {"degree":31,"count":9}, {"degree":32,"count":22}, {"degree":33,"count":5}, {"degree":34,"count":6}, {"degree":35,"count":6}, {"degree":36,"count":7}, {"degree":37,"count":2}, {"degree":38,"count":2}, {"degree":39,"count":2}, {"degree":40,"count":2}, {"degree":41,"count":2}, {"degree":42,"count":2}, {"degree":43,"count":2}, {"degree":44,"count":2}, {"degree":45,"count":2}, {"degree":46,"count":2}, {"degree":47,"count":2}, {"degree":48,"count":2}, {"degree":49,"count":2}, {"degree":50,"count":2}, {"degree":51,"count":3}, {"degree":52,"count":3}]
Minimum constraint arity2
Maximum constraint arity100
Distribution of constraint arities[{"arity":2,"count":99},{"arity":3,"count":196},{"arity":4,"count":1},{"arity":5,"count":194},{"arity":6,"count":101},{"arity":7,"count":1},{"arity":8,"count":1},{"arity":10,"count":4},{"arity":12,"count":1},{"arity":13,"count":1},{"arity":14,"count":1},{"arity":15,"count":2},{"arity":16,"count":2},{"arity":18,"count":1},{"arity":19,"count":2},{"arity":20,"count":3},{"arity":22,"count":3},{"arity":24,"count":1},{"arity":25,"count":4},{"arity":26,"count":1},{"arity":28,"count":3},{"arity":30,"count":3},{"arity":31,"count":2},{"arity":32,"count":1},{"arity":34,"count":3},"...",{"arity":67,"count":2}, {"arity":68,"count":1}, {"arity":70,"count":5}, {"arity":72,"count":1}, {"arity":73,"count":2}, {"arity":74,"count":1}, {"arity":75,"count":2}, {"arity":76,"count":3}, {"arity":78,"count":1}, {"arity":79,"count":2}, {"arity":80,"count":3}, {"arity":82,"count":3}, {"arity":84,"count":1}, {"arity":85,"count":4}, {"arity":86,"count":1}, {"arity":88,"count":3}, {"arity":90,"count":3}, {"arity":91,"count":2}, {"arity":92,"count":1}, {"arity":94,"count":3}, {"arity":95,"count":2}, {"arity":96,"count":1}, {"arity":97,"count":2}, {"arity":98,"count":1}, {"arity":100,"count":6}]
Number of extensional constraints100
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":100},{"type":"sum","count":636},{"type":"cardinality","count":1}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.00	c Version 19.07.01

Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (8026 MiB free)
  memory of node 1: 16384 MiB (12985 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4386788-1568281019/watcher-4386788-1568281019 -o /tmp/evaluation-result-4386788-1568281019/solver-4386788-1568281019 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node103-1568280854-6243 --watchdog 2580 BTD 2520 HOME/instance-4386788-1568281019.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=6400, runsolver pid=6397
Current StackSize limit: 8192 KiB


[startup+0.100162 s]*
/proc/loadavg: 1.57 0.56 0.24 3/199 6401
/proc/meminfo: memFree=21509576/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38404 memory=8076 CPUtime=0.1 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 2207 0 0 0 10 0 0 0 20 0 1 0 404355016 39325696 2019 33554432000 4194304 36700936 140734208632016 140734208632016 4789268 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9601 2019 502 7937 0 1629 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 38404 KiB
Current children cumulated memory: 8076 KiB

[startup+0.20971 s]*
/proc/loadavg: 1.57 0.56 0.24 3/199 6401
/proc/meminfo: memFree=21509080/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38596 memory=8340 CPUtime=0.21 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 2242 0 0 0 21 0 0 0 20 0 1 0 404355016 39522304 2085 33554432000 4194304 36700936 140734208632016 140734208632016 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9649 2085 515 7937 0 1677 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 38596 KiB
Current children cumulated memory: 8340 KiB

[startup+0.308958 s]*
/proc/loadavg: 1.57 0.56 0.24 3/199 6401
/proc/meminfo: memFree=21508336/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38596 memory=8340 CPUtime=0.31 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 2259 0 0 0 31 0 0 0 20 0 1 0 404355016 39522304 2085 33554432000 4194304 36700936 140734208632016 140734208632016 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9649 2085 515 7937 0 1677 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 38596 KiB
Current children cumulated memory: 8340 KiB

[startup+0.700277 s]
/proc/loadavg: 1.57 0.56 0.24 3/199 6401
/proc/meminfo: memFree=21507220/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38728 memory=8340 CPUtime=0.69 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 2265 0 0 0 69 0 0 0 20 0 1 0 404355016 39657472 2085 33554432000 4194304 36700936 140734208632016 140734208631064 4645776 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9682 2085 515 7937 0 1710 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 38728 KiB
Current children cumulated memory: 8340 KiB

[startup+1.50028 s]
/proc/loadavg: 1.57 0.56 0.24 3/199 6401
/proc/meminfo: memFree=21504056/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38728 memory=8340 CPUtime=1.49 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 2271 0 0 0 149 0 0 0 20 0 1 0 404355016 39657472 2085 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9682 2085 515 7937 0 1710 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 38728 KiB
Current children cumulated memory: 8340 KiB

[startup+3.10026 s]
/proc/loadavg: 1.57 0.56 0.24 3/199 6401
/proc/meminfo: memFree=21495444/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38728 memory=8648 CPUtime=3.09 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 3204 0 0 0 308 1 0 0 20 0 1 0 404355016 39657472 2162 33554432000 4194304 36700936 140734208632016 140734208631064 4645791 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9682 2162 527 7937 0 1710 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 38728 KiB
Current children cumulated memory: 8648 KiB

[startup+6.30025 s]
/proc/loadavg: 1.60 0.59 0.25 3/199 6401
/proc/meminfo: memFree=21479340/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38860 memory=8672 CPUtime=6.29 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 4133 0 0 0 627 2 0 0 20 0 1 0 404355016 39792640 2168 33554432000 4194304 36700936 140734208632016 140734208631064 4787909 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9715 2168 527 7937 0 1743 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 38860 KiB
Current children cumulated memory: 8672 KiB

[startup+12.7003 s]
/proc/loadavg: 1.63 0.61 0.26 3/200 6403
/proc/meminfo: memFree=21442232/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38860 memory=8780 CPUtime=12.69 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 5111 0 0 0 1264 5 0 0 20 0 1 0 404355016 39792640 2195 33554432000 4194304 36700936 140734208632016 140734208631064 4645776 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9715 2195 527 7937 0 1743 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 38860 KiB
Current children cumulated memory: 8780 KiB

[startup+25.5003 s]*
/proc/loadavg: 1.72 0.68 0.29 3/199 6404
/proc/meminfo: memFree=21378240/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38992 memory=8836 CPUtime=25.49 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 6915 0 0 0 2539 10 0 0 20 0 1 0 404355016 39927808 2209 33554432000 4194304 36700936 140734208632016 140734208631064 4930444 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9748 2209 527 7937 0 1776 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 38992 KiB
Current children cumulated memory: 8836 KiB

[startup+51.1003 s]
/proc/loadavg: 1.81 0.79 0.33 3/199 6404
/proc/meminfo: memFree=21266744/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38992 memory=8864 CPUtime=51.09 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 7998 0 0 0 5089 20 0 0 20 0 1 0 404355016 39927808 2216 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9748 2216 527 7937 0 1776 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 38992 KiB
Current children cumulated memory: 8864 KiB

[startup+102.3 s]
/proc/loadavg: 1.92 0.97 0.42 3/201 6411
/proc/meminfo: memFree=21066932/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=38992 memory=8904 CPUtime=102.29 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 8973 0 0 0 10192 37 0 0 20 0 1 0 404355016 39927808 2226 33554432000 4194304 36700936 140734208632016 140734208631064 4787711 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9748 2226 527 7937 0 1776 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 38992 KiB
Current children cumulated memory: 8904 KiB

[startup+162.3 s]
/proc/loadavg: 1.97 1.16 0.52 3/200 6415
/proc/meminfo: memFree=20863344/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39124 memory=8964 CPUtime=162.28 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 9986 0 0 0 16162 66 0 0 20 0 1 0 404355016 40062976 2241 33554432000 4194304 36700936 140734208632016 140734208631064 4468772 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9781 2241 527 7937 0 1809 0
Current children cumulated CPU time: 162.28 s
Current children cumulated vsize: 39124 KiB
Current children cumulated memory: 8964 KiB

[startup+222.3 s]
/proc/loadavg: 1.99 1.31 0.61 3/200 6419
/proc/meminfo: memFree=20677324/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39124 memory=9020 CPUtime=222.28 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 10937 0 0 0 22129 99 0 0 20 0 1 0 404355016 40062976 2255 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9781 2255 527 7937 0 1809 0
Current children cumulated CPU time: 222.28 s
Current children cumulated vsize: 39124 KiB
Current children cumulated memory: 9020 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 1.44 0.70 3/201 6432
/proc/meminfo: memFree=20501824/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39124 memory=9020 CPUtime=282.28 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 11938 0 0 0 28103 125 0 0 20 0 1 0 404355016 40062976 2255 33554432000 4194304 36700936 140734208632016 140734208631064 4468772 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9781 2255 527 7937 0 1809 0
Current children cumulated CPU time: 282.28 s
Current children cumulated vsize: 39124 KiB
Current children cumulated memory: 9020 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 1.54 0.78 3/200 6436
/proc/meminfo: memFree=20334932/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39124 memory=9048 CPUtime=342.27 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 12951 0 0 0 34078 149 0 0 20 0 1 0 404355016 40062976 2262 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9781 2262 527 7937 0 1809 0
Current children cumulated CPU time: 342.27 s
Current children cumulated vsize: 39124 KiB
Current children cumulated memory: 9048 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 1.62 0.86 3/199 6440
/proc/meminfo: memFree=20175604/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39124 memory=9048 CPUtime=402.27 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 13835 0 0 0 40049 178 0 0 20 0 1 0 404355016 40062976 2262 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9781 2262 527 7937 0 1809 0
Current children cumulated CPU time: 402.27 s
Current children cumulated vsize: 39124 KiB
Current children cumulated memory: 9048 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 1.69 0.93 3/200 6446
/proc/meminfo: memFree=20019232/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39124 memory=9048 CPUtime=462.27 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 14695 0 0 0 46022 205 0 0 20 0 1 0 404355016 40062976 2262 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9781 2262 527 7937 0 1809 0
Current children cumulated CPU time: 462.27 s
Current children cumulated vsize: 39124 KiB
Current children cumulated memory: 9048 KiB


################
# More data... #
################

/proc/6400/statm: 9814 2280 528 7937 0 1842 0
Current children cumulated CPU time: 1782.22 s
Current children cumulated vsize: 39256 KiB
Current children cumulated memory: 9120 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 1.78 3/199 6593
/proc/meminfo: memFree=15639472/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39256 memory=9120 CPUtime=1842.23 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 35005 0 0 0 183327 896 0 0 20 0 1 0 404355016 40198144 2280 33554432000 4194304 36700936 140734208632016 140734208631064 4788192 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9814 2280 528 7937 0 1842 0
Current children cumulated CPU time: 1842.23 s
Current children cumulated vsize: 39256 KiB
Current children cumulated memory: 9120 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 1.79 3/200 6599
/proc/meminfo: memFree=15487184/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39256 memory=9120 CPUtime=1902.22 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 35880 0 0 0 189301 921 0 0 20 0 1 0 404355016 40198144 2280 33554432000 4194304 36700936 140734208632016 140734208631064 4462456 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9814 2280 528 7937 0 1842 0
Current children cumulated CPU time: 1902.22 s
Current children cumulated vsize: 39256 KiB
Current children cumulated memory: 9120 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 1.80 3/199 6603
/proc/meminfo: memFree=15337908/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39256 memory=9120 CPUtime=1962.21 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 36754 0 0 0 195269 952 0 0 20 0 1 0 404355016 40198144 2280 33554432000 4194304 36700936 140734208632016 140734208631064 4645835 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9814 2280 528 7937 0 1842 0
Current children cumulated CPU time: 1962.21 s
Current children cumulated vsize: 39256 KiB
Current children cumulated memory: 9120 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 1.81 3/199 6607
/proc/meminfo: memFree=15189024/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2022.21 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 37717 0 0 0 201244 977 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2022.21 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 1.82 3/202 6622
/proc/meminfo: memFree=15040552/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2082.21 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 38541 0 0 0 207216 1005 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2082.21 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 1.83 3/201 6626
/proc/meminfo: memFree=14894604/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2142.21 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 39356 0 0 0 213191 1030 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2142.21 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 1.85 3/201 6630
/proc/meminfo: memFree=14749108/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2202.2 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 40171 0 0 0 219165 1055 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2202.2 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 1.86 3/202 6636
/proc/meminfo: memFree=14603516/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2262.2 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 40987 0 0 0 225139 1081 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2262.2 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 1.87 3/201 6640
/proc/meminfo: memFree=14457992/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2322.21 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 41802 0 0 0 231113 1108 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 4788174 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2322.21 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 1.87 3/199 6644
/proc/meminfo: memFree=10706444/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2382.2 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 42616 0 0 0 237086 1134 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2382.2 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.01 1.88 3/200 6650
/proc/meminfo: memFree=8413604/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2442.2 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 43434 0 0 0 243059 1161 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2442.2 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.01 1.88 3/199 6715
/proc/meminfo: memFree=22087392/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2502.2 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 44249 0 0 0 249031 1189 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2502.2 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]
/proc/loadavg: 2.00 2.01 1.89 3/199 6715
/proc/meminfo: memFree=22087456/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2519.9 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 44754 0 0 0 250794 1196 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2519.9 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520 s]
# the end of solver process 6400 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2507.94 s, system=11.9646 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

[startup+2520 s]
/proc/loadavg: 2.00 2.01 1.89 3/199 6715
/proc/meminfo: memFree=22087456/32770624 swapFree=66997112/67108860
[pid=6400] ppid=6397 vsize=39388 memory=9196 CPUtime=2519.9 cores=1,3,5,7
/proc/6400/stat : 6400 (BTD) R 6397 6400 6211 0 -1 4202496 44754 0 0 0 250794 1196 0 0 20 0 1 0 404355016 40333312 2299 33554432000 4194304 36700936 140734208632016 140734208631064 8340231 0 0 4096 0 0 0 0 17 3 0 0 0 0 0 38800048 38936640 48431104 140734208640040 140734208640116 140734208640116 140734208643025 0
/proc/6400/statm: 9847 2299 528 7937 0 1875 0
Current children cumulated CPU time: 2519.9 s
Current children cumulated vsize: 39388 KiB
Current children cumulated memory: 9196 KiB

Child ended because it received signal 15 (SIGTERM)
Real time (s): 2520
CPU time (s): 2519.91
CPU user time (s): 2507.94
CPU system time (s): 11.9646
CPU usage (%): 99.9962
Max. virtual memory (cumulated for all children) (KiB): 39388
Max. memory (cumulated for all children) (KiB): 9196

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2507.94
system time used= 11.9646
maximum resident set size= 9196
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 44754
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3
involuntary context switches= 632


# summary of solver processes directly reported to runsolver:
#   pid: 6400
#   total CPU time (s): 2519.91
#   total CPU user time (s): 2507.94
#   total CPU system time (s): 11.9646

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.77733 second user time and 16.0399 second system time

The end

Launcher Data 

Begin job on node103 at 2019-09-12 11:36:00
IDJOB=4386788
IDBENCH=128090
IDSOLVER=2822
FILE ID=node103/4386788-1568281019
RUNJOBID= node103-1568280854-6243
PBS_JOBID= 21701091
Free space on /tmp= 52636 MiB

SOLVER NAME= BTD 19.07.01
BENCH NAME= XCSP17/CarSequencing/CarSequencing-m1-jcr/CarSequencing-16-81.xml
COMMAND LINE= BTD TIMELIMIT BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4386788-1568281019/watcher-4386788-1568281019 -o /tmp/evaluation-result-4386788-1568281019/solver-4386788-1568281019 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node103-1568280854-6243 --watchdog 2580  BTD 2520 HOME/instance-4386788-1568281019.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= e1683218a1e3c524b21c5eaa634d1705
RANDOM SEED=641112728

node103.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        21516288 kB
MemAvailable:   31052416 kB
Buffers:          100332 kB
Cached:          9665228 kB
SwapCached:         8920 kB
Active:          2727212 kB
Inactive:        7853696 kB
Active(anon):     809568 kB
Inactive(anon):    47120 kB
Active(file):    1917644 kB
Inactive(file):  7806576 kB
Unevictable:      108584 kB
Mlocked:          108584 kB
SwapTotal:      67108860 kB
SwapFree:       66997112 kB
Dirty:             61384 kB
Writeback:             0 kB
AnonPages:        924256 kB
Mapped:            44264 kB
Shmem:             32812 kB
Slab:             269132 kB
SReclaimable:     240368 kB
SUnreclaim:        28764 kB
KernelStack:        3760 kB
PageTables:         7112 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    1332232 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     94208 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 52604 MiB
End job on node103 at 2019-09-12 12:18:01