Trace number 4390428

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1522.84 1524.59

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_19_19.xml
MD5SUM96fe93baa9d8ae993bfc516e2d84a024
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark0
Best CPU time to get the best result obtained on this benchmark1288.59
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":10,"count":111},{"size":48,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (9003 MiB free)
  memory of node 1: 16384 MiB (14991 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390428-1568355143/watcher-4390428-1568355143 -o /tmp/evaluation-result-4390428-1568355143/solver-4390428-1568355143 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1568329709-13076 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390428-1568355143.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=17422, runsolver pid=17419
Current StackSize limit: 8192 KiB


[startup+0.100117 s]*
/proc/loadavg: 2.03 2.00 2.04 3/201 17423
/proc/meminfo: memFree=24555888/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=39148 memory=15348 CPUtime=0.1 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 4006 0 0 0 10 0 0 0 20 0 1 0 411767350 40087552 3837 33554432000 4194304 6210612 140735732052432 140735732048968 5729883 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 9787 3837 809 493 0 3092 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 39148 KiB
Current children cumulated memory: 15348 KiB

[startup+0.20981 s]*
/proc/loadavg: 2.03 2.00 2.04 4/201 17423
/proc/meminfo: memFree=24553580/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=40720 memory=16996 CPUtime=0.21 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 4431 0 0 0 21 0 0 0 20 0 1 0 411767350 41697280 4249 33554432000 4194304 6210612 140735732052432 140735732025992 140662176998610 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 10180 4249 819 493 0 3485 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 40720 KiB
Current children cumulated memory: 16996 KiB

[startup+0.309034 s]*
/proc/loadavg: 2.03 2.00 2.04 3/201 17423
/proc/meminfo: memFree=24552340/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=41708 memory=18052 CPUtime=0.31 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 4688 0 0 0 31 0 0 0 20 0 1 0 411767350 42708992 4513 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 10427 4513 819 493 0 3732 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 41708 KiB
Current children cumulated memory: 18052 KiB

[startup+0.700274 s]
/proc/loadavg: 2.03 2.00 2.04 3/201 17423
/proc/meminfo: memFree=24548868/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=45836 memory=21988 CPUtime=0.69 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 5681 0 0 0 69 0 0 0 20 0 1 0 411767350 46936064 5497 33554432000 4194304 6210612 140735732052432 140735732025992 5456150 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 11459 5497 819 493 0 4764 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45836 KiB
Current children cumulated memory: 21988 KiB

[startup+1.50029 s]
/proc/loadavg: 2.03 2.00 2.04 3/201 17423
/proc/meminfo: memFree=24539568/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=54204 memory=30312 CPUtime=1.49 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 7755 0 0 0 148 1 0 0 20 0 1 0 411767350 55504896 7578 33554432000 4194304 6210612 140735732052432 140735732025992 5442936 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 13551 7578 819 493 0 6856 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 54204 KiB
Current children cumulated memory: 30312 KiB

[startup+3.10928 s]
/proc/loadavg: 2.03 2.00 2.04 3/202 17425
/proc/meminfo: memFree=24497820/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=71020 memory=46804 CPUtime=3.1 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 12099 0 0 0 309 1 0 0 20 0 1 0 411767350 72724480 11701 33554432000 4194304 6210612 140735732052432 140735732025992 140662176995235 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 17755 11701 819 493 0 11060 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 71020 KiB
Current children cumulated memory: 46804 KiB

[startup+6.30027 s]
/proc/loadavg: 1.95 1.99 2.03 3/201 17425
/proc/meminfo: memFree=24437156/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=104696 memory=79984 CPUtime=6.29 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 20547 0 0 0 627 2 0 0 20 0 1 0 411767350 107208704 19996 33554432000 4194304 6210612 140735732052432 140735732025992 140662176998406 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 26174 19996 819 493 0 19479 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 104696 KiB
Current children cumulated memory: 79984 KiB

[startup+12.7003 s]
/proc/loadavg: 1.95 1.99 2.03 3/201 17426
/proc/meminfo: memFree=24314376/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=170420 memory=146516 CPUtime=12.69 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 37318 0 0 0 1264 5 0 0 20 0 1 0 411767350 174510080 36629 33554432000 4194304 6210612 140735732052432 140735732025992 5452074 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 42605 36629 819 493 0 35910 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 170420 KiB
Current children cumulated memory: 146516 KiB

[startup+25.5003 s]
/proc/loadavg: 1.96 1.99 2.03 3/201 17426
/proc/meminfo: memFree=24047136/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=304388 memory=280236 CPUtime=25.49 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 70592 0 0 0 2540 9 0 0 20 0 1 0 411767350 311693312 70059 33554432000 4194304 6210612 140735732052432 140735732025992 5452055 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 76097 70059 819 493 0 69402 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 304388 KiB
Current children cumulated memory: 280236 KiB

[startup+51.1094 s]
/proc/loadavg: 1.98 1.99 2.03 3/202 17428
/proc/meminfo: memFree=23513096/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=573940 memory=544912 CPUtime=51.09 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 135953 0 0 0 5092 17 0 0 20 0 1 0 411767350 587714560 136228 33554432000 4194304 6210612 140735732052432 140735732025992 5446752 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 143485 136228 819 493 0 136790 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 573940 KiB
Current children cumulated memory: 544912 KiB

[startup+102.3 s]
/proc/loadavg: 1.99 1.99 2.03 3/201 17432
/proc/meminfo: memFree=22467484/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=1115684 memory=1075968 CPUtime=102.28 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 265771 0 0 0 10196 32 0 0 20 0 1 0 411767350 1142460416 268992 33554432000 4194304 6210612 140735732052432 140735732025992 5478203 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 278921 268992 819 493 0 272226 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 1115684 KiB
Current children cumulated memory: 1075968 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 1.99 2.03 3/201 17436
/proc/meminfo: memFree=21235676/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=1743396 memory=1699452 CPUtime=162.28 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 417070 0 0 0 16173 55 0 0 20 0 1 0 411767350 1785237504 424863 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 435849 424863 819 493 0 429154 0
Current children cumulated CPU time: 162.28 s
Current children cumulated vsize: 1743396 KiB
Current children cumulated memory: 1699452 KiB

[startup+222.3 s]
/proc/loadavg: 2.04 2.00 2.04 3/200 17440
/proc/meminfo: memFree=20005084/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=2346504 memory=2321088 CPUtime=222.27 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 567858 0 0 0 22153 74 0 0 20 0 1 0 411767350 2402820096 580272 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 586626 580272 819 493 0 579931 0
Current children cumulated CPU time: 222.27 s
Current children cumulated vsize: 2346504 KiB
Current children cumulated memory: 2321088 KiB

[startup+282.3 s]
/proc/loadavg: 2.01 2.00 2.04 3/199 17446
/proc/meminfo: memFree=18772728/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=2986392 memory=2944560 CPUtime=282.27 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 719358 0 0 0 28134 93 0 0 20 0 1 0 411767350 3058065408 736140 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 746598 736140 819 493 0 739903 0
Current children cumulated CPU time: 282.27 s
Current children cumulated vsize: 2986392 KiB
Current children cumulated memory: 2944560 KiB

[startup+342.3 s]
/proc/loadavg: 2.08 2.02 2.05 3/199 17450
/proc/meminfo: memFree=17542072/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=3644732 memory=3565924 CPUtime=342.27 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 870910 0 0 0 34112 115 0 0 20 0 1 0 411767350 3732205568 891481 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 911183 891481 819 493 0 904488 0
Current children cumulated CPU time: 342.27 s
Current children cumulated vsize: 3644732 KiB
Current children cumulated memory: 3565924 KiB

[startup+402.3 s]
/proc/loadavg: 2.56 2.18 2.10 3/199 17454
/proc/meminfo: memFree=16311476/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=4247708 memory=4189628 CPUtime=402.26 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 1022244 0 0 0 40089 137 0 0 20 0 1 0 411767350 4349652992 1047407 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 1061927 1047407 819 493 0 1055232 0
Current children cumulated CPU time: 402.26 s
Current children cumulated vsize: 4247708 KiB
Current children cumulated memory: 4189628 KiB

[startup+462.3 s]
/proc/loadavg: 2.21 2.15 2.09 3/199 17460
/proc/meminfo: memFree=15083552/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=4850552 memory=4810756 CPUtime=462.25 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 1173464 0 0 0 46066 159 0 0 20 0 1 0 411767350 4966965248 1202689 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 1212638 1202689 819 493 0 1205943 0
Current children cumulated CPU time: 462.25 s
Current children cumulated vsize: 4850552 KiB
Current children cumulated memory: 4810756 KiB


################
# More data... #
################

Current children cumulated vsize: 9878672 KiB
Current children cumulated memory: 9788176 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.08 2.09 3/199 17512
/proc/meminfo: memFree=4026240/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=10480856 memory=10408828 CPUtime=1002.21 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 2537204 0 0 0 99840 381 0 0 20 0 1 0 411767350 10732396544 2602207 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 2620214 2602207 819 493 0 2613519 0
Current children cumulated CPU time: 1002.21 s
Current children cumulated vsize: 10480856 KiB
Current children cumulated memory: 10408828 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.06 2.08 3/199 17516
/proc/meminfo: memFree=2799164/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=11082644 memory=11031016 CPUtime=1062.21 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 2688697 0 0 0 105812 409 0 0 20 0 1 0 411767350 11348627456 2757754 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 2770661 2757754 819 493 0 2763966 0
Current children cumulated CPU time: 1062.21 s
Current children cumulated vsize: 11082644 KiB
Current children cumulated memory: 11031016 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.05 2.08 3/199 17527
/proc/meminfo: memFree=1571188/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=11684300 memory=11651284 CPUtime=1122.21 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 2839994 0 0 0 111783 438 0 0 20 0 1 0 411767350 11964723200 2912821 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 2921075 2912821 819 493 0 2914380 0
Current children cumulated CPU time: 1122.21 s
Current children cumulated vsize: 11684300 KiB
Current children cumulated memory: 11651284 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.04 2.07 3/199 17533
/proc/meminfo: memFree=345304/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=12472812 memory=12272848 CPUtime=1182.21 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 2991665 0 0 0 117753 468 0 0 20 0 1 0 411767350 12772159488 3068212 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3118203 3068212 819 493 0 3111508 0
Current children cumulated CPU time: 1182.21 s
Current children cumulated vsize: 12472812 KiB
Current children cumulated memory: 12272848 KiB

[startup+1242.3 s]
/proc/loadavg: 2.06 2.05 2.07 3/199 17537
/proc/meminfo: memFree=198188/32770624 swapFree=66997372/67108860
[pid=17422] ppid=17419 vsize=13073544 memory=12891840 CPUtime=1242.2 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 3142716 0 0 0 123713 507 0 0 20 0 1 0 411767350 13387309056 3222960 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3268386 3222960 819 493 0 3261691 0
Current children cumulated CPU time: 1242.2 s
Current children cumulated vsize: 13073544 KiB
Current children cumulated memory: 12891840 KiB

[startup+1302.3 s]
/proc/loadavg: 2.12 2.07 2.08 3/201 17608
/proc/meminfo: memFree=16023736/32770624 swapFree=66892776/67108860
[pid=17422] ppid=17419 vsize=13673616 memory=13512568 CPUtime=1302.19 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 3293619 0 0 0 129672 547 0 0 20 0 1 0 411767350 14001782784 3352370 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3418404 3352370 819 493 0 3411709 0
Current children cumulated CPU time: 1302.19 s
Current children cumulated vsize: 13673616 KiB
Current children cumulated memory: 13512568 KiB

[startup+1362.3 s]
/proc/loadavg: 2.05 2.06 2.07 3/201 17614
/proc/meminfo: memFree=15400748/32770624 swapFree=66892776/67108860
[pid=17422] ppid=17419 vsize=14275140 memory=14132572 CPUtime=1362.2 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 3444876 0 0 0 135639 581 0 0 20 0 1 0 411767350 14617743360 3507371 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3568785 3507371 819 493 0 3562090 0
Current children cumulated CPU time: 1362.2 s
Current children cumulated vsize: 14275140 KiB
Current children cumulated memory: 14132572 KiB

[startup+1422.3 s]
/proc/loadavg: 2.02 2.05 2.06 3/201 17618
/proc/meminfo: memFree=14777272/32770624 swapFree=66892776/67108860
[pid=17422] ppid=17419 vsize=14876400 memory=14754356 CPUtime=1422.19 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 3596087 0 0 0 141604 615 0 0 20 0 1 0 411767350 15233433600 3662817 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3719100 3662817 819 493 0 3712405 0
Current children cumulated CPU time: 1422.19 s
Current children cumulated vsize: 14876400 KiB
Current children cumulated memory: 14754356 KiB

[startup+1482.3 s]
/proc/loadavg: 2.01 2.04 2.06 3/201 17622
/proc/meminfo: memFree=14156328/32770624 swapFree=66892776/67108860
[pid=17422] ppid=17419 vsize=15477528 memory=15373832 CPUtime=1482.18 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 3747240 0 0 0 147566 652 0 0 20 0 1 0 411767350 15848988672 3817686 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3869382 3817686 819 493 0 3862687 0
Current children cumulated CPU time: 1482.18 s
Current children cumulated vsize: 15477528 KiB
Current children cumulated memory: 15373832 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1521.7 s]
/proc/loadavg: 2.00 2.04 2.05 3/201 17626
/proc/meminfo: memFree=13752764/32770624 swapFree=66892776/67108860
[pid=17422] ppid=17419 vsize=15872208 memory=15780756 CPUtime=1521.58 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 4202496 3846456 0 0 0 151481 677 0 0 20 0 1 0 411767350 16253140992 3919417 33554432000 4194304 6210612 140735732052432 140735732025992 140662176074725 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3968052 3919417 819 493 0 3961357 0
Current children cumulated CPU time: 1521.58 s
Current children cumulated vsize: 15872208 KiB
Current children cumulated memory: 15780756 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1524.59 s]
# the end of solver process 17422 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1515.05 s, system=7.79252 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1522.4 s]
/proc/loadavg: 2.00 2.04 2.05 2/202 17627
/proc/meminfo: memFree=13667452/32770624 swapFree=66892784/67108860
[pid=17422] ppid=17419 vsize=15872604 memory=15781212 CPUtime=1521.76 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) S 17419 17422 13044 0 -1 512 3870392 0 1122 0 151486 690 0 0 20 0 1 0 411767350 16253546496 3944495 33554432000 4194304 6210612 140735732052432 140735732025992 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 26 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3968151 3944495 820 493 0 3961456 0
Current children cumulated CPU time: 1521.76 s
Current children cumulated vsize: 15872604 KiB
Current children cumulated memory: 15781212 KiB

[startup+1523.2 s]
/proc/loadavg: 2.00 2.04 2.05 2/202 17627
/proc/meminfo: memFree=13166228/32770624 swapFree=66892784/67108860
[pid=17422] ppid=17419 vsize=15872604 memory=15781212 CPUtime=1521.87 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) S 17419 17422 13044 0 -1 512 3870392 0 1122 0 151497 690 0 0 20 0 1 0 411767350 16253546496 3944495 33554432000 4194304 6210612 140735732052432 140735732025992 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 26 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3968151 3944495 820 493 0 3961456 0
Current children cumulated CPU time: 1521.87 s
Current children cumulated vsize: 15872604 KiB
Current children cumulated memory: 15781212 KiB

[startup+1523.6 s]
/proc/loadavg: 2.00 2.04 2.05 2/202 17627
/proc/meminfo: memFree=13166228/32770624 swapFree=66892784/67108860
[pid=17422] ppid=17419 vsize=15872604 memory=15781212 CPUtime=1521.93 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) S 17419 17422 13044 0 -1 512 3870392 0 1122 0 151503 690 0 0 20 0 1 0 411767350 16253546496 3944495 33554432000 4194304 6210612 140735732052432 140735732025992 5188952 0 16384 4096 8405026 0 0 0 17 1 0 0 26 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3968151 3944495 820 493 0 3961456 0
Current children cumulated CPU time: 1521.93 s
Current children cumulated vsize: 15872604 KiB
Current children cumulated memory: 15781212 KiB

[startup+1523.7 s]
/proc/loadavg: 2.00 2.04 2.05 2/202 17627
/proc/meminfo: memFree=13166228/32770624 swapFree=66892784/67108860
[pid=17422] ppid=17419 vsize=15872604 memory=15781212 CPUtime=1521.95 cores=1,3,5,7
/proc/17422/stat : 17422 (cosoco) R 17419 17422 13044 0 -1 512 3870392 0 1122 0 151505 690 0 0 20 0 1 0 411767350 16253546496 3944495 33554432000 4194304 6210612 140735732052432 140735732025992 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 26 0 0 8308220 8313664 13533184 140735732057099 140735732057201 140735732057201 140735732060110 0
/proc/17422/statm: 3968151 3944495 820 493 0 3961456 0
Current children cumulated CPU time: 1521.95 s
Current children cumulated vsize: 15872604 KiB
Current children cumulated memory: 15781212 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1524.59
CPU time (s): 1522.84
CPU user time (s): 1515.05
CPU system time (s): 7.79252
CPU usage (%): 99.8857
Max. virtual memory (cumulated for all children) (KiB): 15872604
Max. memory (cumulated for all children) (KiB): 15781212

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1515.05
system time used= 7.79252
maximum resident set size= 15778044
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3870392
page faults= 1122
swaps= 0
block input operations= 71672
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 108817
involuntary context switches= 1894


# summary of solver processes directly reported to runsolver:
#   pid: 17422
#   total CPU time (s): 1522.84
#   total CPU user time (s): 1515.05
#   total CPU system time (s): 7.79252

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 4.70761 second user time and 10.8571 second system time

The end

Launcher Data 

Begin job on node103 at 2019-09-13 08:11:24
IDJOB=4390428
IDBENCH=139614
IDSOLVER=2831
FILE ID=node103/4390428-1568355143
RUNJOBID= node103-1568329709-13076
PBS_JOBID= 21701158
Free space on /tmp= 57264 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_19_19.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390428-1568355143/watcher-4390428-1568355143 -o /tmp/evaluation-result-4390428-1568355143/solver-4390428-1568355143 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1568329709-13076 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390428-1568355143.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 96fe93baa9d8ae993bfc516e2d84a024
RANDOM SEED=1092435030

node103.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.74
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        24571168 kB
MemAvailable:   28875208 kB
Buffers:          134460 kB
Cached:          4424756 kB
SwapCached:         8796 kB
Active:          4695112 kB
Inactive:        2859992 kB
Active(anon):    2981408 kB
Inactive(anon):    55912 kB
Active(file):    1713704 kB
Inactive(file):  2804080 kB
Unevictable:      106480 kB
Mlocked:          106480 kB
SwapTotal:      67108860 kB
SwapFree:       66997372 kB
Dirty:              2724 kB
Writeback:             0 kB
AnonPages:       3094740 kB
Mapped:            45408 kB
Shmem:             32816 kB
Slab:             234108 kB
SReclaimable:     204180 kB
SUnreclaim:        29928 kB
KernelStack:        3584 kB
PageTables:        11528 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    3524576 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    446464 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 56804 MiB
End job on node103 at 2019-09-13 08:36:50