Trace number 4408475

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1559.3 1561.33

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_5_11.xml
MD5SUMbb782f0c2876622a4bcf03fe44f48018
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark374
Best CPU time to get the best result obtained on this benchmark19959.2
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":18,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (13470 MiB free)
  memory of node 1: 16384 MiB (14042 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408475-1569811465/watcher-4408475-1569811465 -o /tmp/evaluation-result-4408475-1569811465/solver-4408475-1569811465 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node145-1569787350-23555 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408475-1569811465.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=28451, runsolver pid=28448

[startup+0.100162 s]*
/proc/loadavg: 4.17 2.78 2.32 3/201 28452
/proc/meminfo: memFree=28162212/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=39160 memory=15368 CPUtime=0.1 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 4008 0 0 0 10 0 0 0 20 0 1 0 557082882 40099840 3842 33554432000 4194304 6211132 140735733991808 140735733988344 140615742751872 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 9790 3842 809 493 0 3095 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 39160 KiB
Current children cumulated memory: 15368 KiB

[startup+0.209678 s]*
/proc/loadavg: 4.08 2.78 2.32 3/201 28452
/proc/meminfo: memFree=28160352/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=40736 memory=17008 CPUtime=0.21 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 4420 0 0 0 21 0 0 0 20 0 1 0 557082882 41713664 4252 33554432000 4194304 6211132 140735733991808 140735733988344 140615736634176 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 10184 4252 819 493 0 3489 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 40736 KiB
Current children cumulated memory: 17008 KiB

[startup+0.308984 s]*
/proc/loadavg: 4.08 2.78 2.32 3/201 28452
/proc/meminfo: memFree=28159608/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=41720 memory=17800 CPUtime=0.3 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 4671 0 0 0 30 0 0 0 20 0 1 0 557082882 42721280 4450 33554432000 4194304 6211132 140735733991808 140735733988344 5453958 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 10430 4450 819 493 0 3735 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 41720 KiB
Current children cumulated memory: 17800 KiB

[startup+0.700271 s]
/proc/loadavg: 4.08 2.78 2.32 3/201 28452
/proc/meminfo: memFree=28157376/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=45584 memory=21736 CPUtime=0.69 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 5636 0 0 0 68 1 0 0 20 0 1 0 557082882 46678016 5434 33554432000 4194304 6211132 140735733991808 140735733988344 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 11396 5434 819 493 0 4701 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45584 KiB
Current children cumulated memory: 21736 KiB

[startup+1.50029 s]
/proc/loadavg: 4.08 2.78 2.32 3/201 28452
/proc/meminfo: memFree=28152304/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=53688 memory=29792 CPUtime=1.49 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 7652 0 0 0 148 1 0 0 20 0 1 0 557082882 54976512 7448 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 13422 7448 819 493 0 6727 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 53688 KiB
Current children cumulated memory: 29792 KiB

[startup+3.10931 s]
/proc/loadavg: 4.08 2.78 2.32 3/201 28452
/proc/meminfo: memFree=28132224/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=70108 memory=46016 CPUtime=3.1 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 11877 0 0 0 308 2 0 0 20 0 1 0 557082882 71790592 11504 33554432000 4194304 6211132 140735733991808 140735733965880 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 17527 11504 819 493 0 10832 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 70108 KiB
Current children cumulated memory: 46016 KiB

[startup+6.30028 s]
/proc/loadavg: 3.91 2.77 2.32 3/201 28452
/proc/meminfo: memFree=28101800/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=102992 memory=78168 CPUtime=6.29 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 20099 0 0 0 625 4 0 0 20 0 1 0 557082882 105463808 19542 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 25748 19542 819 493 0 19053 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 102992 KiB
Current children cumulated memory: 78168 KiB

[startup+12.7003 s]
/proc/loadavg: 3.76 2.76 2.32 3/201 28452
/proc/meminfo: memFree=28041136/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=167132 memory=143116 CPUtime=12.69 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 36317 0 0 0 1263 6 0 0 20 0 1 0 557082882 171143168 35779 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 41783 35779 819 493 0 35088 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 167132 KiB
Current children cumulated memory: 143116 KiB

[startup+25.5003 s]
/proc/loadavg: 3.49 2.73 2.31 3/201 28453
/proc/meminfo: memFree=27908432/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=298064 memory=273244 CPUtime=25.48 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 68415 0 0 0 2538 10 0 0 20 0 1 0 557082882 305217536 68311 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 74516 68311 819 493 0 67821 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 298064 KiB
Current children cumulated memory: 273244 KiB

[startup+51.1093 s]
/proc/loadavg: 2.90 2.66 2.31 3/201 28455
/proc/meminfo: memFree=27645316/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=561412 memory=531808 CPUtime=51.08 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 132538 0 0 0 5090 18 0 0 20 0 1 0 557082882 574885888 132952 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 140353 132952 819 493 0 133658 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 561412 KiB
Current children cumulated memory: 531808 KiB

[startup+102.3 s]
/proc/loadavg: 2.39 2.56 2.29 3/201 28460
/proc/meminfo: memFree=27127148/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=1074476 memory=1050556 CPUtime=102.26 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 258675 0 0 0 10191 35 0 0 20 0 1 0 557082882 1100263424 262639 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 268619 262639 819 493 0 261924 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 1074476 KiB
Current children cumulated memory: 1050556 KiB

[startup+162.3 s]
/proc/loadavg: 2.14 2.46 2.27 3/202 28466
/proc/meminfo: memFree=26518660/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=1704072 memory=1659684 CPUtime=162.25 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 406864 0 0 0 16170 55 0 0 20 0 1 0 557082882 1744969728 414921 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 426018 414921 819 493 0 419323 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 1704072 KiB
Current children cumulated memory: 1659684 KiB

[startup+222.3 s]
/proc/loadavg: 2.05 2.38 2.26 3/201 28470
/proc/meminfo: memFree=25907860/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=2292792 memory=2266800 CPUtime=222.23 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 554046 0 0 0 22153 70 0 0 20 0 1 0 557082882 2347819008 566700 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 573198 566700 819 493 0 566503 0
Current children cumulated CPU time: 222.23 s
Current children cumulated vsize: 2292792 KiB
Current children cumulated memory: 2266800 KiB

[startup+282.3 s]
/proc/loadavg: 2.02 2.31 2.24 3/201 28474
/proc/meminfo: memFree=25299836/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=2918424 memory=2874276 CPUtime=282.22 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 701291 0 0 0 28130 92 0 0 20 0 1 0 557082882 2988466176 718569 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 729606 718569 819 493 0 722911 0
Current children cumulated CPU time: 282.22 s
Current children cumulated vsize: 2918424 KiB
Current children cumulated memory: 2874276 KiB

[startup+342.3 s]
/proc/loadavg: 2.01 2.25 2.22 3/201 28541
/proc/meminfo: memFree=24977404/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=3507012 memory=3481064 CPUtime=342.21 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 848450 0 0 0 34109 112 0 0 20 0 1 0 557082882 3591180288 870266 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 876753 870266 819 493 0 870058 0
Current children cumulated CPU time: 342.21 s
Current children cumulated vsize: 3507012 KiB
Current children cumulated memory: 3481064 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.21 2.21 3/200 28553
/proc/meminfo: memFree=23742476/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=4150964 memory=4090380 CPUtime=402.19 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 996171 0 0 0 40089 130 0 0 20 0 1 0 557082882 4250587136 1022595 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 1037741 1022595 819 493 0 1031046 0
Current children cumulated CPU time: 402.19 s
Current children cumulated vsize: 4150964 KiB
Current children cumulated memory: 4090380 KiB

[startup+462.3 s]
/proc/loadavg: 2.04 2.18 2.20 3/201 28558
/proc/meminfo: memFree=22510528/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=4739552 memory=4697348 CPUtime=462.18 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 1143359 0 0 0 46067 151 0 0 20 0 1 0 557082882 4853301248 1174337 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 1184888 1174337 819 493 0 1178193 0
Current children cumulated CPU time: 462.18 s
Current children cumulated vsize: 4739552 KiB
Current children cumulated memory: 4697348 KiB


################
# More data... #
################

Current children cumulated vsize: 9652568 KiB
Current children cumulated memory: 9553524 KiB

[startup+1002.3 s]
/proc/loadavg: 2.02 2.05 2.13 3/201 28611
/proc/meminfo: memFree=11432384/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=10240364 memory=10161856 CPUtime=1002.07 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 2468140 0 0 0 99831 376 0 0 20 0 1 0 557082882 10486132736 2540464 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 2560091 2540464 819 493 0 2553396 0
Current children cumulated CPU time: 1002.07 s
Current children cumulated vsize: 10240364 KiB
Current children cumulated memory: 10161856 KiB

[startup+1062.3 s]
/proc/loadavg: 2.01 2.04 2.12 3/202 28639
/proc/meminfo: memFree=10202164/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=10828028 memory=10767904 CPUtime=1062.05 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 2615148 0 0 0 105800 405 0 0 20 0 1 0 557082882 11087900672 2691976 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 2707007 2691976 819 493 0 2700312 0
Current children cumulated CPU time: 1062.05 s
Current children cumulated vsize: 10828028 KiB
Current children cumulated memory: 10767904 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.04 2.11 3/201 28643
/proc/meminfo: memFree=8973624/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=11416088 memory=11374436 CPUtime=1122.04 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 2762236 0 0 0 111770 434 0 0 20 0 1 0 557082882 11690074112 2843609 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 2854022 2843609 819 493 0 2847327 0
Current children cumulated CPU time: 1122.04 s
Current children cumulated vsize: 11416088 KiB
Current children cumulated memory: 11374436 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.03 2.11 3/201 28647
/proc/meminfo: memFree=7743484/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=12190476 memory=11980028 CPUtime=1182.03 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 2909634 0 0 0 117735 468 0 0 20 0 1 0 557082882 12483047424 2995007 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3047619 2995007 819 493 0 3040924 0
Current children cumulated CPU time: 1182.03 s
Current children cumulated vsize: 12190476 KiB
Current children cumulated memory: 11980028 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.02 2.10 3/202 28653
/proc/meminfo: memFree=6513440/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=12778140 memory=12587880 CPUtime=1242.02 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 3057119 0 0 0 123702 500 0 0 20 0 1 0 557082882 13084815360 3146970 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3194535 3146970 819 493 0 3187840 0
Current children cumulated CPU time: 1242.02 s
Current children cumulated vsize: 12778140 KiB
Current children cumulated memory: 12587880 KiB

[startup+1302.3 s]
/proc/loadavg: 2.06 2.04 2.10 3/199 28657
/proc/meminfo: memFree=5283860/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=13365672 memory=13193884 CPUtime=1302 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 3204066 0 0 0 129670 530 0 0 20 0 1 0 557082882 13686448128 3298471 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3341418 3298471 819 493 0 3334723 0
Current children cumulated CPU time: 1302 s
Current children cumulated vsize: 13365672 KiB
Current children cumulated memory: 13193884 KiB

[startup+1362.3 s]
/proc/loadavg: 2.02 2.03 2.10 3/199 28661
/proc/meminfo: memFree=4056760/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=13952940 memory=13799624 CPUtime=1361.99 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 3350982 0 0 0 135639 560 0 0 20 0 1 0 557082882 14287810560 3449906 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3488235 3449906 819 493 0 3481540 0
Current children cumulated CPU time: 1361.99 s
Current children cumulated vsize: 13952940 KiB
Current children cumulated memory: 13799624 KiB

[startup+1422.3 s]
/proc/loadavg: 2.09 2.04 2.10 3/200 28667
/proc/meminfo: memFree=2826048/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=14540076 memory=14407144 CPUtime=1421.97 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 3497850 0 0 0 141606 591 0 0 20 0 1 0 557082882 14889037824 3601786 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3635019 3601786 819 493 0 3628324 0
Current children cumulated CPU time: 1421.97 s
Current children cumulated vsize: 14540076 KiB
Current children cumulated memory: 14407144 KiB

[startup+1482.3 s]
/proc/loadavg: 2.03 2.03 2.09 3/199 28671
/proc/meminfo: memFree=1598676/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=15126684 memory=15012292 CPUtime=1481.96 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 3644622 0 0 0 147568 628 0 0 20 0 1 0 557082882 15489724416 3753073 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3781671 3753073 819 493 0 3774976 0
Current children cumulated CPU time: 1481.96 s
Current children cumulated vsize: 15126684 KiB
Current children cumulated memory: 15012292 KiB

[startup+1542.3 s]
/proc/loadavg: 2.08 2.04 2.09 3/199 28675
/proc/meminfo: memFree=370032/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=15713424 memory=15617504 CPUtime=1541.95 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 3791379 0 0 0 153532 663 0 0 20 0 1 0 557082882 16090546176 3904376 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3928356 3904376 819 493 0 3921661 0
Current children cumulated CPU time: 1541.95 s
Current children cumulated vsize: 15713424 KiB
Current children cumulated memory: 15617504 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1558.6 s]
/proc/loadavg: 2.06 2.04 2.09 3/199 28677
/proc/meminfo: memFree=202400/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=15872748 memory=15782832 CPUtime=1558.24 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) R 28448 28451 23523 0 -1 4202496 3831741 0 0 0 155152 672 0 0 20 0 1 0 557082882 16253693952 3945708 33554432000 4194304 6211132 140735733991808 140735733965368 140615726689253 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3968187 3945708 819 493 0 3961492 0
Current children cumulated CPU time: 1558.24 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782832 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1561.33 s]
# the end of solver process 28451 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1551.81 s, system=7.49311 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1560 s]*
/proc/loadavg: 2.06 2.04 2.09 3/200 28678
/proc/meminfo: memFree=199120/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=15872748 memory=15782832 CPUtime=1558.48 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) S 28448 28451 23523 0 -1 512 3831746 0 0 0 155174 674 0 0 20 0 1 0 557082882 16253693952 3945708 33554432000 4194304 6211132 140735733991808 140735733965368 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3968187 3945708 819 493 0 3961492 0
Current children cumulated CPU time: 1558.48 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782832 KiB

[startup+1560.4 s]
/proc/loadavg: 2.06 2.04 2.09 3/200 28678
/proc/meminfo: memFree=199120/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=15872748 memory=15782832 CPUtime=1558.54 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) S 28448 28451 23523 0 -1 512 3831746 0 0 0 155179 675 0 0 20 0 1 0 557082882 16253693952 3945708 33554432000 4194304 6211132 140735733991808 140735733965368 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3968187 3945708 819 493 0 3961492 0
Current children cumulated CPU time: 1558.54 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782832 KiB

[startup+1560.6 s]
/proc/loadavg: 2.06 2.04 2.09 3/200 28678
/proc/meminfo: memFree=199120/32770624 swapFree=66966920/67108860
[pid=28451] ppid=28448 vsize=15872748 memory=15782832 CPUtime=1558.56 cores=1,3,5,7
/proc/28451/stat : 28451 (cosoco) S 28448 28451 23523 0 -1 512 3831746 0 0 0 155180 676 0 0 20 0 1 0 557082882 16253693952 3945708 33554432000 4194304 6211132 140735733991808 140735733965368 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 33361920 140735733994507 140735733994609 140735733994609 140735733997518 0
/proc/28451/statm: 3968187 3945708 819 493 0 3961492 0
Current children cumulated CPU time: 1558.56 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782832 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1561.33
CPU time (s): 1559.3
CPU user time (s): 1551.81
CPU system time (s): 7.49311
CPU usage (%): 99.8696
Max. virtual memory (cumulated for all children) (KiB): 15872748
Max. memory (cumulated for all children) (KiB): 15782832

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1551.81
system time used= 7.49311
maximum resident set size= 15783012
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3831746
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 129387
involuntary context switches= 1254


# summary of solver processes directly reported to runsolver:
#   pid: 28451
#   total CPU time (s): 1559.3
#   total CPU user time (s): 1551.81
#   total CPU system time (s): 7.49311

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.1349 second user time and 7.95648 second system time

The end

Launcher Data 

Begin job on node145 at 2019-09-30 04:43:42
IDJOB=4408475
IDBENCH=139384
IDSOLVER=2874
FILE ID=node145/4408475-1569811465
RUNJOBID= node145-1569787350-23555
PBS_JOBID= 21705820
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_5_11.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408475-1569811465/watcher-4408475-1569811465 -o /tmp/evaluation-result-4408475-1569811465/solver-4408475-1569811465 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node145-1569787350-23555 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408475-1569811465.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= bb782f0c2876622a4bcf03fe44f48018
RANDOM SEED=1445533790

node145.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        28173804 kB
MemAvailable:   31399668 kB
Buffers:          240448 kB
Cached:          3309168 kB
SwapCached:        32684 kB
Active:          1754944 kB
Inactive:        2256996 kB
Active(anon):     448372 kB
Inactive(anon):    96120 kB
Active(file):    1306572 kB
Inactive(file):  2160876 kB
Unevictable:      108080 kB
Mlocked:          108080 kB
SwapTotal:      67108860 kB
SwapFree:       66966920 kB
Dirty:              6784 kB
Writeback:             0 kB
AnonPages:        538360 kB
Mapped:            28612 kB
Shmem:             73436 kB
Slab:             178960 kB
SReclaimable:     148504 kB
SUnreclaim:        30456 kB
KernelStack:        3504 kB
PageTables:         6220 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     993048 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    489472 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 61524 MiB
End job on node145 at 2019-09-30 05:09:44