Trace number 4390430

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1514.44 1516.85

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_9_11.xml
MD5SUMb3f80a69476de872363c069f96d9d14d
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark382
Best CPU time to get the best result obtained on this benchmark2400.08
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":15,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15509 MiB free)
  memory of node 1: 16384 MiB (13319 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390430-1568355454/watcher-4390430-1568355454 -o /tmp/evaluation-result-4390430-1568355454/solver-4390430-1568355454 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node145-1568328837-13395 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390430-1568355454.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=17966, runsolver pid=17963
Current StackSize limit: 8192 KiB


[startup+0.100152 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29508524/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=39160 memory=15372 CPUtime=0.1 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 4009 0 0 0 10 0 0 0 20 0 1 0 411481965 40099840 3843 33554432000 4194304 6210612 140727417566112 140727417561864 5817725 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 9790 3843 809 493 0 3095 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 39160 KiB
Current children cumulated memory: 15372 KiB

[startup+0.209805 s]*
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29506712/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=40736 memory=17012 CPUtime=0.2 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 4432 0 0 0 20 0 0 0 20 0 1 0 411481965 41713664 4253 33554432000 4194304 6210612 140727417566112 140727417561864 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 10184 4253 819 493 0 3489 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 40736 KiB
Current children cumulated memory: 17012 KiB

[startup+0.309018 s]*
/proc/loadavg: 2.00 2.01 2.05 5/199 17967
/proc/meminfo: memFree=29505596/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=41720 memory=18068 CPUtime=0.3 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 4691 0 0 0 30 0 0 0 20 0 1 0 411481965 42721280 4517 33554432000 4194304 6210612 140727417566112 140727417561864 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 10430 4517 819 493 0 3735 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 41720 KiB
Current children cumulated memory: 18068 KiB

[startup+0.700273 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29503364/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=45848 memory=22000 CPUtime=0.69 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 5682 0 0 0 68 1 0 0 20 0 1 0 411481965 46948352 5500 33554432000 4194304 6210612 140727417566112 140727417539672 5440937 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 11462 5500 819 493 0 4767 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45848 KiB
Current children cumulated memory: 22000 KiB

[startup+1.50029 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29498528/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=54216 memory=30324 CPUtime=1.49 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 7757 0 0 0 148 1 0 0 20 0 1 0 411481965 55517184 7581 33554432000 4194304 6210612 140727417566112 140727417539672 5452055 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 13554 7581 819 493 0 6859 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 54216 KiB
Current children cumulated memory: 30324 KiB

[startup+3.1095 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29477696/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=71032 memory=46812 CPUtime=3.1 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 12103 0 0 0 309 1 0 0 20 0 1 0 411481965 72736768 11703 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 17758 11703 819 493 0 11063 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 71032 KiB
Current children cumulated memory: 46812 KiB

[startup+6.30024 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29446348/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=104708 memory=80000 CPUtime=6.28 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 20548 0 0 0 626 2 0 0 20 0 1 0 411481965 107220992 20000 33554432000 4194304 6210612 140727417566112 140727417539672 5439634 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 26177 20000 819 493 0 19482 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 104708 KiB
Current children cumulated memory: 80000 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29384604/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=170432 memory=146528 CPUtime=12.68 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 37321 0 0 0 1264 4 0 0 20 0 1 0 411481965 174522368 36632 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 42608 36632 819 493 0 35913 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 170432 KiB
Current children cumulated memory: 146528 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17967
/proc/meminfo: memFree=29249212/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=304400 memory=279292 CPUtime=25.48 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 70846 0 0 0 2540 8 0 0 20 0 1 0 411481965 311705600 69823 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 76100 69823 819 493 0 69405 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 304400 KiB
Current children cumulated memory: 279292 KiB

[startup+51.1095 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17970
/proc/meminfo: memFree=28976160/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=574876 memory=547320 CPUtime=51.08 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 136910 0 0 0 5091 17 0 0 20 0 1 0 411481965 588673024 136830 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 143719 136830 819 493 0 137024 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 574876 KiB
Current children cumulated memory: 547320 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17974
/proc/meminfo: memFree=28444864/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=1119260 memory=1079432 CPUtime=102.24 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 266931 0 0 0 10190 34 0 0 20 0 1 0 411481965 1146122240 269858 33554432000 4194304 6210612 140727417566112 140727417539672 5454008 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 279815 269858 819 493 0 273120 0
Current children cumulated CPU time: 102.24 s
Current children cumulated vsize: 1119260 KiB
Current children cumulated memory: 1079432 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 17978
/proc/meminfo: memFree=27817604/32770624 swapFree=66957396/67108860
[pid=17966] ppid=17963 vsize=1749876 memory=1705360 CPUtime=162.21 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 419288 0 0 0 16166 55 0 0 20 0 1 0 411481965 1791873024 426340 33554432000 4194304 6210612 140727417566112 140727417539672 5446483 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 437469 426340 819 493 0 430774 0
Current children cumulated CPU time: 162.21 s
Current children cumulated vsize: 1749876 KiB
Current children cumulated memory: 1705360 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 17992
/proc/meminfo: memFree=27188860/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=2356020 memory=2331944 CPUtime=222.17 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 570828 0 0 0 22142 75 0 0 20 0 1 0 411481965 2412564480 582986 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 589005 582986 819 493 0 582310 0
Current children cumulated CPU time: 222.17 s
Current children cumulated vsize: 2356020 KiB
Current children cumulated memory: 2331944 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 17996
/proc/meminfo: memFree=26563092/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=2998944 memory=2956664 CPUtime=282.13 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 722928 0 0 0 28119 94 0 0 20 0 1 0 411481965 3070918656 739166 33554432000 4194304 6210612 140727417566112 140727417539672 139707837877117 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 749736 739166 819 493 0 743041 0
Current children cumulated CPU time: 282.13 s
Current children cumulated vsize: 2998944 KiB
Current children cumulated memory: 2956664 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 18000
/proc/meminfo: memFree=25937120/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=3660320 memory=3581888 CPUtime=342.1 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 874765 0 0 0 34095 115 0 0 20 0 1 0 411481965 3748167680 895472 33554432000 4194304 6210612 140727417566112 140727417539672 139707828856051 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 915080 895472 819 493 0 908385 0
Current children cumulated CPU time: 342.1 s
Current children cumulated vsize: 3660320 KiB
Current children cumulated memory: 3581888 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 18006
/proc/meminfo: memFree=25310312/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=4266332 memory=4209376 CPUtime=402.07 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 1026645 0 0 0 40072 135 0 0 20 0 1 0 411481965 4368723968 1052344 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 1066583 1052344 819 493 0 1059888 0
Current children cumulated CPU time: 402.07 s
Current children cumulated vsize: 4266332 KiB
Current children cumulated memory: 4209376 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 18010
/proc/meminfo: memFree=24682088/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=4872080 memory=4833716 CPUtime=462.03 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 1178224 0 0 0 46049 154 0 0 20 0 1 0 411481965 4989009920 1208429 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 1218020 1208429 819 493 0 1211325 0
Current children cumulated CPU time: 462.03 s
Current children cumulated vsize: 4872080 KiB
Current children cumulated memory: 4833716 KiB


################
# More data... #
################

Current children cumulated vsize: 9923960 KiB
Current children cumulated memory: 9834952 KiB

[startup+1002.3 s]
/proc/loadavg: 2.01 2.08 2.07 3/199 18122
/proc/meminfo: memFree=19027520/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=10529180 memory=10458716 CPUtime=1001.78 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 2543327 0 0 0 99803 375 0 0 20 0 1 0 411481965 10781880320 2614679 33554432000 4194304 6210612 140727417566112 140727417539672 5442946 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 2632295 2614679 819 493 0 2625600 0
Current children cumulated CPU time: 1001.78 s
Current children cumulated vsize: 10529180 KiB
Current children cumulated memory: 10458716 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.07 2.07 3/199 18126
/proc/meminfo: memFree=18400276/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=11134400 memory=11084156 CPUtime=1061.75 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 2694842 0 0 0 105771 404 0 0 20 0 1 0 411481965 11401625600 2771039 33554432000 4194304 6210612 140727417566112 140727417539672 5440420 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 2783600 2771039 819 493 0 2776905 0
Current children cumulated CPU time: 1061.75 s
Current children cumulated vsize: 11134400 KiB
Current children cumulated memory: 11084156 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.05 2.06 3/200 18132
/proc/meminfo: memFree=17775008/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=11739356 memory=11707760 CPUtime=1121.74 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 2846398 0 0 0 111741 433 0 0 20 0 1 0 411481965 12021100544 2926940 33554432000 4194304 6210612 140727417566112 140727417539672 139707828852657 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 2934839 2926940 819 493 0 2928144 0
Current children cumulated CPU time: 1121.74 s
Current children cumulated vsize: 11739356 KiB
Current children cumulated memory: 11707760 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 18136
/proc/meminfo: memFree=17149324/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=12531036 memory=12332300 CPUtime=1181.72 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 2998755 0 0 0 117709 463 0 0 20 0 1 0 411481965 12831780864 3083075 33554432000 4194304 6210612 140727417566112 140727417539672 5453736 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3132759 3083075 819 493 0 3126064 0
Current children cumulated CPU time: 1181.72 s
Current children cumulated vsize: 12531036 KiB
Current children cumulated memory: 12332300 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 18140
/proc/meminfo: memFree=16522168/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=13135860 memory=12957844 CPUtime=1241.69 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 3150241 0 0 0 123675 494 0 0 20 0 1 0 411481965 13451120640 3239461 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3283965 3239461 819 493 0 3277270 0
Current children cumulated CPU time: 1241.69 s
Current children cumulated vsize: 13135860 KiB
Current children cumulated memory: 12957844 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 18146
/proc/meminfo: memFree=15898660/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=13740684 memory=13581080 CPUtime=1301.67 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 3301710 0 0 0 129642 525 0 0 20 0 1 0 411481965 14070460416 3395270 33554432000 4194304 6210612 140727417566112 140727417539672 5442853 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3435171 3395270 819 493 0 3428476 0
Current children cumulated CPU time: 1301.67 s
Current children cumulated vsize: 13740684 KiB
Current children cumulated memory: 13581080 KiB

[startup+1362.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 18150
/proc/meminfo: memFree=15271816/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=14344848 memory=14205568 CPUtime=1361.64 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 3453029 0 0 0 135603 561 0 0 20 0 1 0 411481965 14689124352 3551392 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3586212 3551392 819 493 0 3579517 0
Current children cumulated CPU time: 1361.64 s
Current children cumulated vsize: 14344848 KiB
Current children cumulated memory: 14205568 KiB

[startup+1422.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/200 18162
/proc/meminfo: memFree=14648108/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=14948748 memory=14828012 CPUtime=1421.61 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 3604269 0 0 0 141562 599 0 0 20 0 1 0 411481965 15307517952 3707003 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3737187 3707003 819 493 0 3730492 0
Current children cumulated CPU time: 1421.61 s
Current children cumulated vsize: 14948748 KiB
Current children cumulated memory: 14828012 KiB

[startup+1482.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 18168
/proc/meminfo: memFree=14022708/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=15552648 memory=15452292 CPUtime=1481.59 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 3755565 0 0 0 147522 637 0 0 20 0 1 0 411481965 15925911552 3863073 33554432000 4194304 6210612 140727417566112 140727417539672 5440950 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3888162 3863073 819 493 0 3881467 0
Current children cumulated CPU time: 1481.59 s
Current children cumulated vsize: 15552648 KiB
Current children cumulated memory: 15452292 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1514.11 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 18168
/proc/meminfo: memFree=13689724/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=15872748 memory=15782584 CPUtime=1513.38 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 4202496 3835697 0 0 0 150680 658 0 0 20 0 1 0 411481965 16253693952 3945646 33554432000 4194304 6210612 140727417566112 140727417539672 139707827932133 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3968187 3945646 819 493 0 3961492 0
Current children cumulated CPU time: 1513.38 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782584 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1516.85 s]
# the end of solver process 17966 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1507.07 s, system=7.37779 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1515.2 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 18169
/proc/meminfo: memFree=13209204/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=15872748 memory=15782584 CPUtime=1513.56 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 512 3835701 0 0 0 150694 662 0 0 20 0 1 0 411481965 16253693952 3945646 33554432000 4194304 6210612 140727417566112 140727417539672 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3968187 3945646 819 493 0 3961492 0
Current children cumulated CPU time: 1513.56 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782584 KiB

[startup+1515.6 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 18169
/proc/meminfo: memFree=13209204/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=15872748 memory=15782584 CPUtime=1513.62 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) S 17963 17966 13363 0 -1 512 3835701 0 0 0 150699 663 0 0 20 0 1 0 411481965 16253693952 3945646 33554432000 4194304 6210612 140727417566112 140727417539672 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3968187 3945646 819 493 0 3961492 0
Current children cumulated CPU time: 1513.62 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782584 KiB

[startup+1516 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 18169
/proc/meminfo: memFree=12687268/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=15872748 memory=15782584 CPUtime=1513.68 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) R 17963 17966 13363 0 -1 512 3835701 0 0 0 150705 663 0 0 20 0 1 0 411481965 16253693952 3945646 33554432000 4194304 6210612 140727417566112 140727417539672 5188952 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3968187 3945646 819 493 0 3961492 0
Current children cumulated CPU time: 1513.68 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782584 KiB

[startup+1516.11 s]
/proc/loadavg: 2.00 2.01 2.05 3/201 18169
/proc/meminfo: memFree=12687268/32770624 swapFree=66957400/67108860
[pid=17966] ppid=17963 vsize=15872748 memory=15782584 CPUtime=1513.7 cores=1,3,5,7
/proc/17966/stat : 17966 (cosoco) S 17963 17966 13363 0 -1 512 3835701 0 0 0 150706 664 0 0 20 0 1 0 411481965 16253693952 3945646 33554432000 4194304 6210612 140727417566112 140727417539672 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 23588864 140727417574411 140727417574513 140727417574513 140727417577422 0
/proc/17966/statm: 3968187 3945646 819 493 0 3961492 0
Current children cumulated CPU time: 1513.7 s
Current children cumulated vsize: 15872748 KiB
Current children cumulated memory: 15782584 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1516.85
CPU time (s): 1514.44
CPU user time (s): 1507.07
CPU system time (s): 7.37779
CPU usage (%): 99.8415
Max. virtual memory (cumulated for all children) (KiB): 15872748
Max. memory (cumulated for all children) (KiB): 15782584

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1507.07
system time used= 7.37779
maximum resident set size= 15782780
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3835701
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 131834
involuntary context switches= 1895


# summary of solver processes directly reported to runsolver:
#   pid: 17966
#   total CPU time (s): 1514.44
#   total CPU user time (s): 1507.07
#   total CPU system time (s): 7.37779

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.71323 second user time and 10.0318 second system time

The end

Launcher Data 

Begin job on node145 at 2019-09-13 08:16:53
IDJOB=4390430
IDBENCH=139658
IDSOLVER=2831
FILE ID=node145/4390430-1568355454
RUNJOBID= node145-1568328837-13395
PBS_JOBID= 21701155
Free space on /tmp= 55184 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_9_11.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390430-1568355454/watcher-4390430-1568355454 -o /tmp/evaluation-result-4390430-1568355454/solver-4390430-1568355454 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node145-1568328837-13395 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390430-1568355454.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b3f80a69476de872363c069f96d9d14d
RANDOM SEED=1671250385

node145.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.59
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        29520156 kB
MemAvailable:   31850076 kB
Buffers:           83696 kB
Cached:          2524208 kB
SwapCached:        41308 kB
Active:          1335916 kB
Inactive:        1320604 kB
Active(anon):      11224 kB
Inactive(anon):    78772 kB
Active(file):    1324692 kB
Inactive(file):  1241832 kB
Unevictable:      110276 kB
Mlocked:          110276 kB
SwapTotal:      67108860 kB
SwapFree:       66957396 kB
Dirty:              2732 kB
Writeback:             0 kB
AnonPages:        119084 kB
Mapped:            44308 kB
Shmem:             32344 kB
Slab:             186336 kB
SReclaimable:     158464 kB
SUnreclaim:        27872 kB
KernelStack:        3520 kB
PageTables:         5220 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     534132 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     71680 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      120192 kB
DirectMap2M:    33425408 kB

Free space on /tmp at the end= 54672 MiB
End job on node145 at 2019-09-13 08:42:10