Trace number 4409140

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1514.29 1516.37

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_3_12.xml
MD5SUM75a96ae289dcd0162feddbff51d0d425
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark1336
Best CPU time to get the best result obtained on this benchmark1997.76
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":28,"count":111},{"size":38,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15112 MiB free)
  memory of node 1: 16384 MiB (15172 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409140-1569812665/watcher-4409140-1569812665 -o /tmp/evaluation-result-4409140-1569812665/solver-4409140-1569812665 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node105-1569808860-29381 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4409140-1569812665.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=29761, runsolver pid=29758

[startup+0.10011 s]*
/proc/loadavg: 1.81 1.50 1.73 3/199 29762
/proc/meminfo: memFree=31000296/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=39024 memory=15100 CPUtime=0.09 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 3985 0 0 0 9 0 0 0 20 0 1 0 557523919 39960576 3775 33554432000 4194304 6211132 140734943335024 140734943330968 139709304957758 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 9756 3775 809 493 0 3061 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 39024 KiB
Current children cumulated memory: 15100 KiB

[startup+0.209563 s]*
/proc/loadavg: 1.81 1.50 1.73 3/200 29764
/proc/meminfo: memFree=30998808/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=40728 memory=17004 CPUtime=0.21 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 4418 0 0 0 20 1 0 0 20 0 1 0 557523919 41705472 4251 33554432000 4194304 6211132 140734943335024 140734943308584 139709304957758 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 10182 4251 819 493 0 3487 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 40728 KiB
Current children cumulated memory: 17004 KiB

[startup+0.309071 s]*
/proc/loadavg: 1.81 1.50 1.73 3/200 29764
/proc/meminfo: memFree=30997692/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=41716 memory=17796 CPUtime=0.31 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 4675 0 0 0 30 1 0 0 20 0 1 0 557523919 42717184 4449 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 10429 4449 819 493 0 3734 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 41716 KiB
Current children cumulated memory: 17796 KiB

[startup+0.700229 s]
/proc/loadavg: 1.81 1.50 1.73 4/200 29764
/proc/meminfo: memFree=30995664/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=45712 memory=21996 CPUtime=0.69 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 5664 0 0 0 68 1 0 0 20 0 1 0 557523919 46809088 5499 33554432000 4194304 6211132 140734943335024 140734943308584 5452189 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 11428 5499 819 493 0 4733 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45712 KiB
Current children cumulated memory: 21996 KiB

[startup+1.50024 s]
/proc/loadavg: 1.83 1.51 1.73 3/200 29764
/proc/meminfo: memFree=30990880/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=54080 memory=30056 CPUtime=1.49 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 7731 0 0 0 148 1 0 0 20 0 1 0 557523919 55377920 7514 33554432000 4194304 6211132 140734943335024 140734943308584 5443090 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 13520 7514 819 493 0 6825 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 54080 KiB
Current children cumulated memory: 30056 KiB

[startup+3.10934 s]
/proc/loadavg: 1.83 1.51 1.73 3/200 29764
/proc/meminfo: memFree=30970296/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=70896 memory=46816 CPUtime=3.1 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 12061 0 0 0 308 2 0 0 20 0 1 0 557523919 72597504 11704 33554432000 4194304 6211132 140734943335024 140734943308584 5452189 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 17724 11704 819 493 0 11029 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 70896 KiB
Current children cumulated memory: 46816 KiB

[startup+6.30022 s]
/proc/loadavg: 1.84 1.52 1.73 3/200 29764
/proc/meminfo: memFree=30939512/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=104440 memory=79736 CPUtime=6.29 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 20476 0 0 0 626 3 0 0 20 0 1 0 557523919 106946560 19934 33554432000 4194304 6211132 140734943335024 140734943308584 139709305882512 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 26110 19934 819 493 0 19415 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 104440 KiB
Current children cumulated memory: 79736 KiB

[startup+12.7002 s]
/proc/loadavg: 1.85 1.52 1.73 3/200 29764
/proc/meminfo: memFree=30877424/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=169900 memory=145996 CPUtime=12.69 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 37182 0 0 0 1263 6 0 0 20 0 1 0 557523919 173977600 36499 33554432000 4194304 6211132 140734943335024 140734943308584 5446189 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 42475 36499 819 493 0 35780 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 169900 KiB
Current children cumulated memory: 145996 KiB

[startup+25.5002 s]
/proc/loadavg: 1.88 1.54 1.74 3/200 29828
/proc/meminfo: memFree=30709760/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=303208 memory=277956 CPUtime=25.48 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 70523 0 0 0 2538 10 0 0 20 0 1 0 557523919 310484992 69489 33554432000 4194304 6211132 140734943335024 140734943308584 5012962 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 75802 69489 819 493 0 69107 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 303208 KiB
Current children cumulated memory: 277956 KiB

[startup+51.1092 s]
/proc/loadavg: 1.92 1.58 1.75 3/200 29830
/proc/meminfo: memFree=30170336/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=571704 memory=543596 CPUtime=51.08 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 135566 0 0 0 5088 20 0 0 20 0 1 0 557523919 585424896 135899 33554432000 4194304 6211132 140734943335024 140734943308584 5451564 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 142926 135899 819 493 0 136231 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 571704 KiB
Current children cumulated memory: 543596 KiB

[startup+102.3 s]
/proc/loadavg: 1.97 1.65 1.76 3/200 29834
/proc/meminfo: memFree=29111596/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=1113052 memory=1073096 CPUtime=102.25 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 264931 0 0 0 10187 38 0 0 20 0 1 0 557523919 1139765248 268274 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 278263 268274 819 493 0 271568 0
Current children cumulated CPU time: 102.25 s
Current children cumulated vsize: 1113052 KiB
Current children cumulated memory: 1073096 KiB

[startup+162.3 s]
/proc/loadavg: 1.99 1.71 1.78 3/200 29838
/proc/meminfo: memFree=27861176/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=1744592 memory=1701328 CPUtime=162.25 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 417407 0 0 0 16164 61 0 0 20 0 1 0 557523919 1786462208 425332 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 436148 425332 819 493 0 429453 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 1744592 KiB
Current children cumulated memory: 1701328 KiB

[startup+222.3 s]
/proc/loadavg: 2.22 1.81 1.80 3/199 29842
/proc/meminfo: memFree=26612264/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=2351132 memory=2326396 CPUtime=222.24 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 569060 0 0 0 22144 80 0 0 20 0 1 0 557523919 2407559168 581599 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 587783 581599 819 493 0 581088 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 2351132 KiB
Current children cumulated memory: 2326396 KiB

[startup+282.3 s]
/proc/loadavg: 2.32 1.93 1.85 3/199 29848
/proc/meminfo: memFree=25362388/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=2994584 memory=2953428 CPUtime=282.23 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 721254 0 0 0 28124 99 0 0 20 0 1 0 557523919 3066454016 738357 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 748646 738357 819 493 0 741951 0
Current children cumulated CPU time: 282.23 s
Current children cumulated vsize: 2994584 KiB
Current children cumulated memory: 2953428 KiB

[startup+342.3 s]
/proc/loadavg: 2.12 1.94 1.86 3/198 29852
/proc/meminfo: memFree=24114728/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=3656224 memory=3577680 CPUtime=342.23 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 873281 0 0 0 34102 121 0 0 20 0 1 0 557523919 3743973376 894420 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 914056 894420 819 493 0 907361 0
Current children cumulated CPU time: 342.23 s
Current children cumulated vsize: 3656224 KiB
Current children cumulated memory: 3577680 KiB

[startup+402.3 s]
/proc/loadavg: 2.04 1.96 1.87 3/198 29856
/proc/meminfo: memFree=22866032/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=4262368 memory=4204924 CPUtime=402.21 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 1025075 0 0 0 40079 142 0 0 20 0 1 0 557523919 4364664832 1051231 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 1065592 1051231 819 493 0 1058897 0
Current children cumulated CPU time: 402.21 s
Current children cumulated vsize: 4262368 KiB
Current children cumulated memory: 4204924 KiB

[startup+462.3 s]
/proc/loadavg: 2.02 1.96 1.87 3/199 29870
/proc/meminfo: memFree=21616552/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=4868512 memory=4829396 CPUtime=462.21 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 1176775 0 0 0 46056 165 0 0 20 0 1 0 557523919 4985356288 1207349 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 1217128 1207349 819 493 0 1210433 0
Current children cumulated CPU time: 462.21 s
Current children cumulated vsize: 4868512 KiB
Current children cumulated memory: 4829396 KiB


################
# More data... #
################

Current children cumulated vsize: 9317284 KiB
Current children cumulated memory: 9207116 KiB

[startup+942.3 s]
/proc/loadavg: 2.00 1.99 1.92 3/198 29906
/proc/meminfo: memFree=11634764/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=9923032 memory=9833244 CPUtime=942.15 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 2390895 0 0 0 93850 365 0 0 20 0 1 0 557523919 10161184768 2458311 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 2480758 2458311 819 493 0 2474063 0
Current children cumulated CPU time: 942.15 s
Current children cumulated vsize: 9923032 KiB
Current children cumulated memory: 9833244 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 1.99 1.93 3/198 29912
/proc/meminfo: memFree=10390360/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=10528384 memory=10459260 CPUtime=1002.14 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 2542456 0 0 0 99821 393 0 0 20 0 1 0 557523919 10781065216 2614815 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 2632096 2614815 819 493 0 2625401 0
Current children cumulated CPU time: 1002.14 s
Current children cumulated vsize: 10528384 KiB
Current children cumulated memory: 10459260 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 1.99 1.93 3/200 29925
/proc/meminfo: memFree=9142548/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=11133736 memory=11083124 CPUtime=1062.14 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 2694053 0 0 0 105791 423 0 0 20 0 1 0 557523919 11400945664 2770781 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 2783434 2770781 819 493 0 2776739 0
Current children cumulated CPU time: 1062.14 s
Current children cumulated vsize: 11133736 KiB
Current children cumulated memory: 11083124 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 1.99 1.94 3/200 29929
/proc/meminfo: memFree=7893916/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=11739220 memory=11708984 CPUtime=1122.13 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 2845678 0 0 0 111763 450 0 0 20 0 1 0 557523919 12020961280 2927246 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 2934805 2927246 819 493 0 2928110 0
Current children cumulated CPU time: 1122.13 s
Current children cumulated vsize: 11739220 KiB
Current children cumulated memory: 11708984 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 1.99 1.94 3/200 29935
/proc/meminfo: memFree=6648656/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=12531296 memory=12332068 CPUtime=1182.12 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 2997589 0 0 0 117731 481 0 0 20 0 1 0 557523919 12832047104 3083017 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3132824 3083017 819 493 0 3126129 0
Current children cumulated CPU time: 1182.12 s
Current children cumulated vsize: 12531296 KiB
Current children cumulated memory: 12332068 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 1.99 1.95 3/200 29939
/proc/meminfo: memFree=5402928/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=13136516 memory=12957672 CPUtime=1242.11 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 3149165 0 0 0 123700 511 0 0 20 0 1 0 557523919 13451792384 3239418 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3284129 3239418 819 493 0 3277434 0
Current children cumulated CPU time: 1242.11 s
Current children cumulated vsize: 13136516 KiB
Current children cumulated memory: 12957672 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 1.99 1.95 3/200 29943
/proc/meminfo: memFree=4156596/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=13741868 memory=13581492 CPUtime=1302.11 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 3300739 0 0 0 129668 543 0 0 20 0 1 0 557523919 14071672832 3395373 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3435467 3395373 819 493 0 3428772 0
Current children cumulated CPU time: 1302.11 s
Current children cumulated vsize: 13741868 KiB
Current children cumulated memory: 13581492 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 1.99 1.95 3/198 29950
/proc/meminfo: memFree=2910528/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=14346956 memory=14207092 CPUtime=1362.1 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 3452279 0 0 0 135635 575 0 0 20 0 1 0 557523919 14691282944 3551773 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3586739 3551773 819 493 0 3580044 0
Current children cumulated CPU time: 1362.1 s
Current children cumulated vsize: 14346956 KiB
Current children cumulated memory: 14207092 KiB

[startup+1422.3 s]
/proc/loadavg: 2.08 2.00 1.96 3/198 29954
/proc/meminfo: memFree=1664216/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=14952044 memory=14832692 CPUtime=1422.09 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 3603797 0 0 0 141603 606 0 0 20 0 1 0 557523919 15310893056 3708173 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3738011 3708173 819 493 0 3731316 0
Current children cumulated CPU time: 1422.09 s
Current children cumulated vsize: 14952044 KiB
Current children cumulated memory: 14832692 KiB

[startup+1482.3 s]
/proc/loadavg: 2.03 2.00 1.96 3/198 29958
/proc/meminfo: memFree=418404/32770624 swapFree=66957812/67108860
[pid=29761] ppid=29758 vsize=15556868 memory=15455788 CPUtime=1482.08 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 3755312 0 0 0 147569 639 0 0 20 0 1 0 557523919 15930232832 3863947 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3889217 3863947 819 493 0 3882522 0
Current children cumulated CPU time: 1482.08 s
Current children cumulated vsize: 15556868 KiB
Current children cumulated memory: 15455788 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1513.6 s]
/proc/loadavg: 2.02 2.00 1.96 3/198 29961
/proc/meminfo: memFree=181852/32770624 swapFree=66957932/67108860
[pid=29761] ppid=29758 vsize=15872216 memory=15780640 CPUtime=1513.38 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) R 29758 29761 29350 0 -1 4202496 3836108 0 0 0 150681 657 0 0 20 0 1 0 557523919 16253149184 3945160 33554432000 4194304 6211132 140734943335024 140734943308584 139709304962021 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3968054 3945160 819 493 0 3961359 0
Current children cumulated CPU time: 1513.38 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15780640 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1516.37 s]
# the end of solver process 29761 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1506.92 s, system=7.36287 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1514.4 s]
/proc/loadavg: 2.02 2.00 1.96 2/199 29962
/proc/meminfo: memFree=188312/32770624 swapFree=66957932/67108860
[pid=29761] ppid=29758 vsize=15872216 memory=15780640 CPUtime=1513.38 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) S 29758 29761 29350 0 -1 512 3836113 0 0 0 150681 657 0 0 20 0 1 0 557523919 16253149184 3945160 33554432000 4194304 6211132 140734943335024 140734943308584 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3968054 3945160 819 493 0 3961359 0
Current children cumulated CPU time: 1513.38 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15780640 KiB

[startup+1515.2 s]
/proc/loadavg: 2.02 2.00 1.96 3/199 29962
/proc/meminfo: memFree=201092/32770624 swapFree=66957932/67108860
[pid=29761] ppid=29758 vsize=15872216 memory=15780640 CPUtime=1513.47 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) S 29758 29761 29350 0 -1 512 3836113 0 0 0 150689 658 0 0 20 0 1 0 557523919 16253149184 3945160 33554432000 4194304 6211132 140734943335024 140734943308584 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3968054 3945160 819 493 0 3961359 0
Current children cumulated CPU time: 1513.47 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15780640 KiB

[startup+1515.6 s]
/proc/loadavg: 2.02 2.00 1.96 3/199 29962
/proc/meminfo: memFree=201092/32770624 swapFree=66957932/67108860
[pid=29761] ppid=29758 vsize=15872216 memory=15780640 CPUtime=1513.5 cores=0,2,4,6
/proc/29761/stat : 29761 (cosoco) S 29758 29761 29350 0 -1 512 3836113 0 0 0 150692 658 0 0 20 0 1 0 557523919 16253149184 3945160 33554432000 4194304 6211132 140734943335024 140734943308584 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 9961472 140734943339531 140734943339633 140734943339633 140734943342542 0
/proc/29761/statm: 3968054 3945160 819 493 0 3961359 0
Current children cumulated CPU time: 1513.5 s
Current children cumulated vsize: 15872216 KiB
Current children cumulated memory: 15780640 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1516.37
CPU time (s): 1514.29
CPU user time (s): 1506.92
CPU system time (s): 7.36287
CPU usage (%): 99.8622
Max. virtual memory (cumulated for all children) (KiB): 15872216
Max. memory (cumulated for all children) (KiB): 15780640

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1506.92
system time used= 7.36287
maximum resident set size= 15780668
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3836113
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 52888
involuntary context switches= 2324


# summary of solver processes directly reported to runsolver:
#   pid: 29761
#   total CPU time (s): 1514.29
#   total CPU user time (s): 1506.92
#   total CPU system time (s): 7.36287

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.48298 second user time and 10.2349 second system time

The end

Launcher Data 

Begin job on node105 at 2019-09-30 04:02:41
IDJOB=4409140
IDBENCH=139399
IDSOLVER=2876
FILE ID=node105/4409140-1569812665
RUNJOBID= node105-1569808860-29381
PBS_JOBID= 21705841
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_3_12.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409140-1569812665/watcher-4409140-1569812665 -o /tmp/evaluation-result-4409140-1569812665/solver-4409140-1569812665 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node105-1569808860-29381 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4409140-1569812665.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 75a96ae289dcd0162feddbff51d0d425
RANDOM SEED=1191659641

node105.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.00
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31011880 kB
MemAvailable:   31830148 kB
Buffers:          222716 kB
Cached:           934960 kB
SwapCached:        48112 kB
Active:           508432 kB
Inactive:         700892 kB
Active(anon):      19540 kB
Inactive(anon):   111736 kB
Active(file):     488892 kB
Inactive(file):   589156 kB
Unevictable:      110664 kB
Mlocked:          110664 kB
SwapTotal:      67108860 kB
SwapFree:       66957812 kB
Dirty:              2932 kB
Writeback:             0 kB
AnonPages:        126332 kB
Mapped:            27444 kB
Shmem:             70752 kB
Slab:             140972 kB
SReclaimable:     112108 kB
SUnreclaim:        28864 kB
KernelStack:        3760 kB
PageTables:         5144 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     562776 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     77824 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 61824 MiB
End job on node105 at 2019-09-30 04:27:57