Trace number 4398487

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.O parallel? (TO) 2520.27 2523.25

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_20_8.xml
MD5SUM6b176121d3671d41526f7ff611fbc0d5
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark202
Best CPU time to get the best result obtained on this benchmark20086.7
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":11,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15350 MiB free)
  memory of node 1: 16384 MiB (15465 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4398487-1568614178/watcher-4398487-1568614178 -o /tmp/evaluation-result-4398487-1568614178/solver-4398487-1568614178 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node134-1568611094-22102 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4398487-1568614178.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=22557, runsolver pid=22554
Current StackSize limit: 8192 KiB


[startup+0.100153 s]*
/proc/loadavg: 3.29 4.64 6.44 2/197 22558
/proc/meminfo: memFree=31522244/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=66824 memory=42704 CPUtime=0.1 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 10873 0 0 0 9 1 0 0 20 0 1 0 437668351 68427776 10676 33554432000 4194304 6211092 140724655011232 140724655007768 139877015927781 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 16706 10676 754 493 0 10011 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 66824 KiB
Current children cumulated memory: 42704 KiB

[startup+0.209397 s]*
/proc/loadavg: 3.29 4.64 6.44 2/197 22558
/proc/meminfo: memFree=31491440/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=93216 memory=69368 CPUtime=0.21 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 17513 0 0 0 19 2 0 0 20 0 1 0 437668351 95453184 17342 33554432000 4194304 6211092 140724655011232 140724655008504 139877015927781 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 23304 17342 809 493 0 16609 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 93216 KiB
Current children cumulated memory: 69368 KiB

[startup+0.309028 s]*
/proc/loadavg: 3.29 4.64 6.44 2/197 22558
/proc/meminfo: memFree=31489952/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=94664 memory=70712 CPUtime=0.3 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 17891 0 0 0 28 2 0 0 20 0 1 0 437668351 96935936 17678 33554432000 4194304 6211092 140724655011232 140724655008504 5443103 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 23666 17678 819 493 0 16971 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 94664 KiB
Current children cumulated memory: 70712 KiB

[startup+0.700321 s]
/proc/loadavg: 3.29 4.64 6.44 2/197 22558
/proc/meminfo: memFree=31488572/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=98632 memory=74672 CPUtime=0.69 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 18884 0 0 0 67 2 0 0 20 0 1 0 437668351 100999168 18668 33554432000 4194304 6211092 140724655011232 140724654984792 5452234 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 24658 18668 819 493 0 17963 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 98632 KiB
Current children cumulated memory: 74672 KiB

[startup+1.50029 s]
/proc/loadavg: 3.29 4.64 6.44 3/197 22558
/proc/meminfo: memFree=31483152/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=107084 memory=83100 CPUtime=1.48 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 20958 0 0 0 146 2 0 0 20 0 1 0 437668351 109654016 20775 33554432000 4194304 6211092 140724655011232 140724654984792 5443042 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 26771 20775 819 493 0 20076 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 107084 KiB
Current children cumulated memory: 83100 KiB

[startup+3.10929 s]
/proc/loadavg: 3.10 4.58 6.41 2/197 22558
/proc/meminfo: memFree=31464896/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=123900 memory=99596 CPUtime=3.1 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 25369 0 0 0 306 4 0 0 20 0 1 0 437668351 126873600 24899 33554432000 4194304 6211092 140724655011232 140724654984792 139877016848272 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 30975 24899 819 493 0 24280 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 123900 KiB
Current children cumulated memory: 99596 KiB

[startup+6.30028 s]
/proc/loadavg: 3.10 4.58 6.41 2/197 22558
/proc/meminfo: memFree=31432600/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=157444 memory=132780 CPUtime=6.28 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 33867 0 0 0 623 5 0 0 20 0 1 0 437668351 161222656 33195 33554432000 4194304 6211092 140724655011232 140724654984792 139877015927781 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 39361 33195 819 493 0 32666 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 157444 KiB
Current children cumulated memory: 132780 KiB

[startup+12.7003 s]
/proc/loadavg: 2.78 4.46 6.35 2/197 22560
/proc/meminfo: memFree=31370536/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=223300 memory=199040 CPUtime=12.68 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 50661 0 0 0 1260 8 0 0 20 0 1 0 437668351 228659200 49760 33554432000 4194304 6211092 140724655011232 140724654984792 5451552 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 55825 49760 819 493 0 49130 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 223300 KiB
Current children cumulated memory: 199040 KiB

[startup+25.5003 s]
/proc/loadavg: 2.59 4.37 6.30 2/197 22560
/proc/meminfo: memFree=31236068/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=357004 memory=331736 CPUtime=25.48 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 84170 0 0 0 2532 16 0 0 20 0 1 0 437668351 365572096 82934 33554432000 4194304 6211092 140724655011232 140724654984792 5451576 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 89251 82934 819 493 0 82556 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 357004 KiB
Current children cumulated memory: 331736 KiB

[startup+51.109 s]
/proc/loadavg: 2.05 4.10 6.16 2/197 22561
/proc/meminfo: memFree=30965164/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=626820 memory=599144 CPUtime=51.07 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 149544 0 0 0 5082 25 0 0 20 0 1 0 437668351 641863680 149786 33554432000 4194304 6211092 140724655011232 140724654984792 5446248 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 156705 149786 819 493 0 150010 0
Current children cumulated CPU time: 51.07 s
Current children cumulated vsize: 626820 KiB
Current children cumulated memory: 599144 KiB

[startup+102.3 s]
/proc/loadavg: 1.42 3.58 5.87 2/198 22572
/proc/meminfo: memFree=30433772/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=1170676 memory=1130984 CPUtime=102.22 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 279446 0 0 0 10180 42 0 0 20 0 1 0 437668351 1198772224 282746 33554432000 4194304 6211092 140724655011232 140724654984792 5446238 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 292669 282746 819 493 0 285974 0
Current children cumulated CPU time: 102.22 s
Current children cumulated vsize: 1170676 KiB
Current children cumulated memory: 1130984 KiB

[startup+162.3 s]
/proc/loadavg: 1.15 3.11 5.56 2/198 22575
/proc/meminfo: memFree=29806808/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=1801028 memory=1756344 CPUtime=162.19 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 431762 0 0 0 16157 62 0 0 20 0 1 0 437668351 1844252672 439086 33554432000 4194304 6211092 140724655011232 140724654984792 5013258 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 450257 439086 819 493 0 443562 0
Current children cumulated CPU time: 162.19 s
Current children cumulated vsize: 1801028 KiB
Current children cumulated memory: 1756344 KiB

[startup+222.3 s]
/proc/loadavg: 1.06 2.72 5.27 2/197 22578
/proc/meminfo: memFree=29179732/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=2408228 memory=2383948 CPUtime=222.17 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 583633 0 0 0 22134 83 0 0 20 0 1 0 437668351 2466025472 595987 33554432000 4194304 6211092 140724655011232 140724654984792 5446226 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 602057 595987 819 493 0 595362 0
Current children cumulated CPU time: 222.17 s
Current children cumulated vsize: 2408228 KiB
Current children cumulated memory: 2383948 KiB

[startup+282.3 s]
/proc/loadavg: 1.02 2.41 5.01 2/196 22583
/proc/meminfo: memFree=28552140/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=3052076 memory=3011548 CPUtime=282.14 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 736010 0 0 0 28110 104 0 0 20 0 1 0 437668351 3125325824 752821 33554432000 4194304 6211092 140724655011232 140724654984792 5453912 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 763019 752887 819 493 0 756324 0
Current children cumulated CPU time: 282.14 s
Current children cumulated vsize: 3052076 KiB
Current children cumulated memory: 3011548 KiB

[startup+342.3 s]
/proc/loadavg: 1.04 2.17 4.76 2/196 22587
/proc/meminfo: memFree=27926304/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=3713056 memory=3634544 CPUtime=342.1 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 887849 0 0 0 34087 123 0 0 20 0 1 0 437668351 3802169344 908636 33554432000 4194304 6211092 140724655011232 140724654984792 5443111 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 928264 908636 819 493 0 921569 0
Current children cumulated CPU time: 342.1 s
Current children cumulated vsize: 3713056 KiB
Current children cumulated memory: 3634544 KiB

[startup+402.3 s]
/proc/loadavg: 1.02 1.95 4.53 2/195 22590
/proc/meminfo: memFree=27298564/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=4318672 memory=4261536 CPUtime=402.07 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 1039639 0 0 0 40064 143 0 0 20 0 1 0 437668351 4422320128 1065384 33554432000 4194304 6211092 140724655011232 140724654984792 139877016851699 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 1079668 1065384 819 493 0 1072973 0
Current children cumulated CPU time: 402.07 s
Current children cumulated vsize: 4318672 KiB
Current children cumulated memory: 4261536 KiB

[startup+462.3 s]
/proc/loadavg: 1.01 1.78 4.30 2/195 22595
/proc/meminfo: memFree=26673352/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=4924156 memory=4885488 CPUtime=462.04 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 1191234 0 0 0 46039 165 0 0 20 0 1 0 437668351 5042335744 1221372 33554432000 4194304 6211092 140724655011232 140724654984792 5451561 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 1231039 1221372 819 493 0 1224344 0
Current children cumulated CPU time: 462.04 s
Current children cumulated vsize: 4924156 KiB
Current children cumulated memory: 4885488 KiB


################
# More data... #
################

Current children cumulated vsize: 20102008 KiB
Current children cumulated memory: 19856208 KiB

[startup+1962.3 s]
/proc/loadavg: 1.09 1.12 1.72 2/197 22718
/proc/meminfo: memFree=11044684/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=20704984 memory=20479708 CPUtime=1961.29 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 4978500 0 0 0 195164 965 0 0 20 0 1 0 437668351 21201903616 5119927 33554432000 4194304 6211092 140724655011232 140724654984792 5451434 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 5176246 5119927 819 493 0 5169551 0
Current children cumulated CPU time: 1961.29 s
Current children cumulated vsize: 20704984 KiB
Current children cumulated memory: 20479708 KiB

[startup+2022.3 s]
/proc/loadavg: 1.03 1.10 1.67 2/197 22721
/proc/meminfo: memFree=10422544/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=21307564 memory=21100636 CPUtime=2021.26 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 5129528 0 0 0 201114 1012 0 0 20 0 1 0 437668351 21818945536 5275159 33554432000 4194304 6211092 140724655011232 140724654984792 5451416 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 5326891 5275159 819 493 0 5320196 0
Current children cumulated CPU time: 2021.26 s
Current children cumulated vsize: 21307564 KiB
Current children cumulated memory: 21100636 KiB

[startup+2082.3 s]
/proc/loadavg: 1.01 1.08 1.63 2/197 22726
/proc/meminfo: memFree=9798404/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=21910144 memory=21723872 CPUtime=2081.23 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 5280575 0 0 0 207065 1058 0 0 20 0 1 0 437668351 22435987456 5430968 33554432000 4194304 6211092 140724655011232 140724654984792 5446279 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 5477536 5430968 819 493 0 5470841 0
Current children cumulated CPU time: 2081.23 s
Current children cumulated vsize: 21910144 KiB
Current children cumulated memory: 21723872 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.07 1.59 2/197 22730
/proc/meminfo: memFree=9175608/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=22512988 memory=22345016 CPUtime=2141.2 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 5431657 0 0 0 213019 1101 0 0 20 0 1 0 437668351 23053299712 5586254 33554432000 4194304 6211092 140724655011232 140724654984792 5446606 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 5628247 5586254 819 493 0 5621552 0
Current children cumulated CPU time: 2141.2 s
Current children cumulated vsize: 22512988 KiB
Current children cumulated memory: 22345016 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.06 1.55 2/195 22733
/proc/meminfo: memFree=8551228/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=23115304 memory=22968000 CPUtime=2201.18 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 5582711 0 0 0 218968 1150 0 0 20 0 1 0 437668351 23670071296 5742000 33554432000 4194304 6211092 140724655011232 140724654984792 5013008 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 5778826 5742000 819 493 0 5772131 0
Current children cumulated CPU time: 2201.18 s
Current children cumulated vsize: 23115304 KiB
Current children cumulated memory: 22968000 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.04 1.52 2/195 22738
/proc/meminfo: memFree=7929940/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=23717752 memory=23588616 CPUtime=2261.15 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 5733695 0 0 0 224913 1202 0 0 20 0 1 0 437668351 24286978048 5897154 33554432000 4194304 6211092 140724655011232 140724654984792 139877025872710 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 5929438 5897154 819 493 0 5922743 0
Current children cumulated CPU time: 2261.15 s
Current children cumulated vsize: 23717752 KiB
Current children cumulated memory: 23588616 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.04 1.49 2/195 22741
/proc/meminfo: memFree=7305920/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=24320068 memory=24211600 CPUtime=2321.12 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 5884727 0 0 0 230862 1250 0 0 20 0 1 0 437668351 24903749632 6052900 33554432000 4194304 6211092 140724655011232 140724654984792 5446248 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6080017 6052900 819 493 0 6073322 0
Current children cumulated CPU time: 2321.12 s
Current children cumulated vsize: 24320068 KiB
Current children cumulated memory: 24211600 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.03 1.46 2/195 22744
/proc/meminfo: memFree=6684104/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=24922120 memory=24832228 CPUtime=2381.09 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 6035716 0 0 0 236808 1301 0 0 20 0 1 0 437668351 25520250880 6208057 33554432000 4194304 6211092 140724655011232 140724654984792 139877015927781 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6230530 6208057 819 493 0 6223835 0
Current children cumulated CPU time: 2381.09 s
Current children cumulated vsize: 24922120 KiB
Current children cumulated memory: 24832228 KiB

[startup+2442.3 s]
/proc/loadavg: 0.93 1.01 1.42 2/194 22749
/proc/meminfo: memFree=6060796/32770624 swapFree=66917048/67108860
[pid=22557] ppid=22554 vsize=25524172 memory=25454672 CPUtime=2441.06 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 6186616 0 0 0 242753 1353 0 0 20 0 1 0 437668351 26136752128 6363668 33554432000 4194304 6211092 140724655011232 140724654984792 5454176 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6381043 6363668 819 493 0 6374348 0
Current children cumulated CPU time: 2441.06 s
Current children cumulated vsize: 25524172 KiB
Current children cumulated memory: 25454672 KiB

[startup+2502.3 s]
/proc/loadavg: 0.98 1.01 1.40 2/195 22760
/proc/meminfo: memFree=5438548/32770624 swapFree=66917052/67108860
[pid=22557] ppid=22554 vsize=26126092 memory=26075036 CPUtime=2501.03 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 6337544 0 0 0 248694 1409 0 0 20 0 1 0 437668351 26753118208 6518759 33554432000 4194304 6211092 140724655011232 140724654984792 139877016851947 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6531523 6518759 819 493 0 6524828 0
Current children cumulated CPU time: 2501.03 s
Current children cumulated vsize: 26126092 KiB
Current children cumulated memory: 26075036 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 0.98 1.01 1.39 2/195 22761
/proc/meminfo: memFree=5251220/32770624 swapFree=66917052/67108860
[pid=22557] ppid=22554 vsize=26303896 memory=26258768 CPUtime=2518.74 cores=0-7
/proc/22557/stat : 22557 (cosoco) R 22554 22557 22075 0 -1 4202496 6382127 0 0 0 250451 1423 0 0 20 0 1 0 437668351 26935189504 6564692 33554432000 4194304 6211092 140724655011232 140724654984792 4982624 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6575974 6564692 819 493 0 6569279 0
Current children cumulated CPU time: 2518.74 s
Current children cumulated vsize: 26303896 KiB
Current children cumulated memory: 26258768 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.25 s]
# the end of solver process 22557 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2504.78 s, system=15.4879 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 0.98 1.01 1.39 2/195 22761
/proc/meminfo: memFree=5251220/32770624 swapFree=66917052/67108860
[pid=22557] ppid=22554 vsize=26303896 memory=26258768 CPUtime=2518.84 cores=0-7
/proc/22557/stat : 22557 (cosoco) S 22554 22557 22075 0 -1 512 6382133 0 0 0 250461 1423 0 0 20 0 1 0 437668351 26935189504 6564692 33554432000 4194304 6211092 140724655011232 140724654984792 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6575974 6564692 819 493 0 6569279 0
Current children cumulated CPU time: 2518.84 s
Current children cumulated vsize: 26303896 KiB
Current children cumulated memory: 26258768 KiB

[startup+2521.6 s]
/proc/loadavg: 0.98 1.01 1.39 2/196 22762
/proc/meminfo: memFree=4762768/32770624 swapFree=66917052/67108860
[pid=22557] ppid=22554 vsize=26303896 memory=26258768 CPUtime=2518.97 cores=0-7
/proc/22557/stat : 22557 (cosoco) S 22554 22557 22075 0 -1 512 6382133 0 0 0 250473 1424 0 0 20 0 1 0 437668351 26935189504 6564692 33554432000 4194304 6211092 140724655011232 140724654984792 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6575974 6564692 819 493 0 6569279 0
Current children cumulated CPU time: 2518.97 s
Current children cumulated vsize: 26303896 KiB
Current children cumulated memory: 26258768 KiB

[startup+2522 s]*
/proc/loadavg: 0.98 1.01 1.39 1/196 22762
/proc/meminfo: memFree=4264412/32770624 swapFree=66917052/67108860
[pid=22557] ppid=22554 vsize=26303896 memory=26258768 CPUtime=2519.03 cores=0-7
/proc/22557/stat : 22557 (cosoco) S 22554 22557 22075 0 -1 512 6382133 0 0 0 250478 1425 0 0 20 0 1 0 437668351 26935189504 6564692 33554432000 4194304 6211092 140724655011232 140724654984792 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 17735680 140724655019025 140724655019117 140724655019117 140724655022030 0
/proc/22557/statm: 6575974 6564692 819 493 0 6569279 0
Current children cumulated CPU time: 2519.03 s
Current children cumulated vsize: 26303896 KiB
Current children cumulated memory: 26258768 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.25
CPU time (s): 2520.27
CPU user time (s): 2504.78
CPU system time (s): 15.4879
CPU usage (%): 99.8821
Max. virtual memory (cumulated for all children) (KiB): 26303896
Max. memory (cumulated for all children) (KiB): 26258768

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2504.78
system time used= 15.4879
maximum resident set size= 26258956
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6382133
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 124946
involuntary context switches= 3647


# summary of solver processes directly reported to runsolver:
#   pid: 22557
#   total CPU time (s): 2520.27
#   total CPU user time (s): 2504.78
#   total CPU system time (s): 15.4879

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.44528 second user time and 16.7723 second system time

The end

Launcher Data 

Begin job on node134 at 2019-09-16 08:08:57
IDJOB=4398487
IDBENCH=139577
IDSOLVER=2851
FILE ID=node134/4398487-1568614178
RUNJOBID= node134-1568611094-22102
PBS_JOBID= 21703452
Free space on /tmp= 59600 MiB

SOLVER NAME= cosoco 2.O parallel
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_20_8.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4398487-1568614178/watcher-4398487-1568614178 -o /tmp/evaluation-result-4398487-1568614178/solver-4398487-1568614178 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node134-1568611094-22102 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4398487-1568614178.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 6b176121d3671d41526f7ff611fbc0d5
RANDOM SEED=1344458176

node134.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.76
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31556704 kB
MemAvailable:   31793500 kB
Buffers:          130760 kB
Cached:           388452 kB
SwapCached:        90380 kB
Active:           453076 kB
Inactive:         159060 kB
Active(anon):        872 kB
Inactive(anon):   140572 kB
Active(file):     452204 kB
Inactive(file):    18488 kB
Unevictable:      108536 kB
Mlocked:          108536 kB
SwapTotal:      67108860 kB
SwapFree:       66917048 kB
Dirty:             10336 kB
Writeback:             0 kB
AnonPages:        112552 kB
Mapped:            47072 kB
Shmem:             39656 kB
Slab:             194780 kB
SReclaimable:     163876 kB
SUnreclaim:        30904 kB
KernelStack:        4096 kB
PageTables:         5028 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     505988 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      118144 kB
DirectMap2M:    33427456 kB

Free space on /tmp at the end= 59116 MiB
End job on node134 at 2019-09-16 08:51:00