Trace number 4397190

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1562.07 1563.63

General information on the benchmark

NameSteelMillSlab/
SteelMillSlab-bench_16_2.xml
MD5SUM8e5b61791e512734038a023d36a5924c
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark69
Best CPU time to get the best result obtained on this benchmark20045.8
Satisfiable
(Un)Satisfiability was proved
Number of variables12654
Number of constraints466
Number of domains4
Minimum domain size2
Maximum domain size111
Distribution of domain sizes[{"size":2,"count":12321},{"size":10,"count":111},{"size":51,"count":111},{"size":111,"count":111}]
Minimum variable degree2
Maximum variable degree114
Distribution of variable degrees[{"degree":2,"count":12432},{"degree":3,"count":111},{"degree":112,"count":78},{"degree":113,"count":24},{"degree":114,"count":9}]
Minimum constraint arity2
Maximum constraint arity112
Distribution of constraint arities[{"arity":2,"count":132},{"arity":111,"count":112},{"arity":112,"count":222}]
Number of extensional constraints0
Number of intensional constraints132
Distribution of constraint types[{"type":"intension","count":132},{"type":"sum","count":112},{"type":"element","count":111},{"type":"channel","count":111}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (11734 MiB free)
  memory of node 1: 16384 MiB (15515 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4397190-1568594232/watcher-4397190-1568594232 -o /tmp/evaluation-result-4397190-1568594232/solver-4397190-1568594232 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node123-1568594193-24804 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4397190-1568594232.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=24833, runsolver pid=24830
Current StackSize limit: 8192 KiB


[startup+0.100108 s]*
/proc/loadavg: 1.00 1.15 1.57 3/199 24834
/proc/meminfo: memFree=27883144/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=39020 memory=15224 CPUtime=0.1 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3989 0 0 0 10 0 0 0 20 0 1 0 435688041 39956480 3806 33554432000 4194304 6211092 140727366575408 140727366571944 140201116302309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 9755 3806 809 493 0 3060 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 39020 KiB
Current children cumulated memory: 15224 KiB

[startup+0.209394 s]*
/proc/loadavg: 1.00 1.15 1.57 3/199 24834
/proc/meminfo: memFree=27879456/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=40596 memory=17008 CPUtime=0.21 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 4417 0 0 0 21 0 0 0 20 0 1 0 435688041 41570304 4252 33554432000 4194304 6211092 140727366575408 140727366571944 140201116302309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 10149 4252 819 493 0 3454 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 40596 KiB
Current children cumulated memory: 17008 KiB

[startup+0.309038 s]*
/proc/loadavg: 1.00 1.15 1.57 3/199 24834
/proc/meminfo: memFree=27876780/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=41712 memory=17800 CPUtime=0.31 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 4670 0 0 0 31 0 0 0 20 0 1 0 435688041 42713088 4450 33554432000 4194304 6211092 140727366575408 140727366548968 140201116302309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 10428 4450 819 493 0 3733 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 41712 KiB
Current children cumulated memory: 17800 KiB

[startup+0.700229 s]
/proc/loadavg: 1.00 1.15 1.57 3/199 24834
/proc/meminfo: memFree=27873184/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=45576 memory=21732 CPUtime=0.69 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 5640 0 0 0 69 0 0 0 20 0 1 0 435688041 46669824 5433 33554432000 4194304 6211092 140727366575408 140727366548968 140201116302309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 11394 5433 819 493 0 4699 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 45576 KiB
Current children cumulated memory: 21732 KiB

[startup+1.50024 s]
/proc/loadavg: 1.08 1.17 1.57 3/199 24834
/proc/meminfo: memFree=27863512/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=53812 memory=29788 CPUtime=1.49 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 7667 0 0 0 148 1 0 0 20 0 1 0 435688041 55103488 7447 33554432000 4194304 6211092 140727366575408 140727366548968 5455650 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 13453 7447 819 493 0 6758 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 53812 KiB
Current children cumulated memory: 29788 KiB

[startup+3.10916 s]
/proc/loadavg: 1.08 1.17 1.57 3/199 24834
/proc/meminfo: memFree=27822536/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=70364 memory=46136 CPUtime=3.1 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 11914 0 0 0 309 1 0 0 20 0 1 0 435688041 72052736 11534 33554432000 4194304 6211092 140727366575408 140727366548968 5451532 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 17591 11534 819 493 0 10896 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 70364 KiB
Current children cumulated memory: 46136 KiB

[startup+6.30022 s]
/proc/loadavg: 1.16 1.18 1.57 3/199 24834
/proc/meminfo: memFree=27761296/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=103248 memory=78524 CPUtime=6.29 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 20170 0 0 0 627 2 0 0 20 0 1 0 435688041 105725952 19631 33554432000 4194304 6211092 140727366575408 140727366548968 5440716 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 25812 19631 819 493 0 19117 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 103248 KiB
Current children cumulated memory: 78524 KiB

[startup+12.7002 s]
/proc/loadavg: 1.22 1.19 1.58 3/199 24834
/proc/meminfo: memFree=27638884/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=167388 memory=143352 CPUtime=12.69 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 36540 0 0 0 1265 4 0 0 20 0 1 0 435688041 171405312 35838 33554432000 4194304 6211092 140727366575408 140727366548968 140201116302309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 41847 35838 819 493 0 35152 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 167388 KiB
Current children cumulated memory: 143352 KiB

[startup+25.5002 s]
/proc/loadavg: 1.34 1.22 1.58 3/199 24834
/proc/meminfo: memFree=27373252/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=298320 memory=272928 CPUtime=25.49 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 69264 0 0 0 2541 8 0 0 20 0 1 0 435688041 305479680 68232 33554432000 4194304 6211092 140727366575408 140727366548968 5452226 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 74580 68232 819 493 0 67885 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 298320 KiB
Current children cumulated memory: 272928 KiB

[startup+51.1092 s]
/proc/loadavg: 1.54 1.28 1.59 3/199 24836
/proc/meminfo: memFree=26840896/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=561536 memory=533560 CPUtime=51.1 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 133072 0 0 0 5093 17 0 0 20 0 1 0 435688041 575012864 133390 33554432000 4194304 6211092 140727366575408 140727366548968 5452157 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 140384 133390 819 493 0 133689 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 561536 KiB
Current children cumulated memory: 533560 KiB

[startup+102.3 s]
/proc/loadavg: 1.74 1.38 1.61 3/199 24840
/proc/meminfo: memFree=25799992/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=1074336 memory=1050172 CPUtime=102.29 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 259313 0 0 0 10196 33 0 0 20 0 1 0 435688041 1100120064 262543 33554432000 4194304 6211092 140727366575408 140727366548968 5012962 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 268584 262543 819 493 0 261889 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 1074336 KiB
Current children cumulated memory: 1050172 KiB

[startup+162.3 s]
/proc/loadavg: 1.86 1.47 1.62 3/200 24845
/proc/meminfo: memFree=24569424/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=1703668 memory=1658604 CPUtime=162.29 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 407346 0 0 0 16176 53 0 0 20 0 1 0 435688041 1744556032 414651 33554432000 4194304 6211092 140727366575408 140727366548968 5451572 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 425917 414651 819 493 0 419222 0
Current children cumulated CPU time: 162.29 s
Current children cumulated vsize: 1703668 KiB
Current children cumulated memory: 1658604 KiB

[startup+222.3 s]
/proc/loadavg: 1.95 1.57 1.65 3/202 24859
/proc/meminfo: memFree=23343064/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=2291860 memory=2265192 CPUtime=222.3 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 554395 0 0 0 22158 72 0 0 20 0 1 0 435688041 2346864640 566298 33554432000 4194304 6211092 140727366575408 140727366548968 140201117222816 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 572965 566298 819 493 0 566270 0
Current children cumulated CPU time: 222.3 s
Current children cumulated vsize: 2291860 KiB
Current children cumulated memory: 2265192 KiB

[startup+282.3 s]
/proc/loadavg: 1.90 1.63 1.66 3/201 24863
/proc/meminfo: memFree=22118384/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=2916964 memory=2871848 CPUtime=282.3 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 702168 0 0 0 28140 90 0 0 20 0 1 0 435688041 2986971136 717962 33554432000 4194304 6211092 140727366575408 140727366548968 5479619 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 729241 717962 819 493 0 722546 0
Current children cumulated CPU time: 282.3 s
Current children cumulated vsize: 2916964 KiB
Current children cumulated memory: 2871848 KiB

[startup+342.3 s]
/proc/loadavg: 1.92 1.68 1.68 3/201 24867
/proc/meminfo: memFree=20887088/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=3505024 memory=3480432 CPUtime=342.29 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 849542 0 0 0 34122 107 0 0 20 0 1 0 435688041 3589144576 870108 33554432000 4194304 6211092 140727366575408 140727366548968 5451484 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 876256 870108 819 493 0 869561 0
Current children cumulated CPU time: 342.29 s
Current children cumulated vsize: 3505024 KiB
Current children cumulated memory: 3480432 KiB

[startup+402.3 s]
/proc/loadavg: 1.97 1.74 1.70 3/202 24873
/proc/meminfo: memFree=19658048/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=4148448 memory=4086988 CPUtime=402.3 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 997434 0 0 0 40101 129 0 0 20 0 1 0 435688041 4248010752 1021747 33554432000 4194304 6211092 140727366575408 140727366548968 140201126247246 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 1037112 1021747 819 493 0 1030417 0
Current children cumulated CPU time: 402.3 s
Current children cumulated vsize: 4148448 KiB
Current children cumulated memory: 4086988 KiB

[startup+462.3 s]
/proc/loadavg: 1.99 1.79 1.72 3/200 24877
/proc/meminfo: memFree=18429944/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=4736376 memory=4693268 CPUtime=462.3 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 1145103 0 0 0 46080 150 0 0 20 0 1 0 435688041 4850049024 1173317 33554432000 4194304 6211092 140727366575408 140727366548968 140201117226194 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 1184094 1173317 819 493 0 1177399 0
Current children cumulated CPU time: 462.3 s
Current children cumulated vsize: 4736376 KiB
Current children cumulated memory: 4693268 KiB


################
# More data... #
################

Current children cumulated vsize: 9645036 KiB
Current children cumulated memory: 9547308 KiB

[startup+1002.3 s]
/proc/loadavg: 2.21 2.08 1.88 3/199 24926
/proc/meminfo: memFree=7377208/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=10232304 memory=10152960 CPUtime=1002.31 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 2474458 0 0 0 99869 362 0 0 20 0 1 0 435688041 10477879296 2538240 33554432000 4194304 6211092 140727366575408 140727366548968 5451484 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 2558076 2538240 819 493 0 2551381 0
Current children cumulated CPU time: 1002.31 s
Current children cumulated vsize: 10232304 KiB
Current children cumulated memory: 10152960 KiB

[startup+1062.3 s]
/proc/loadavg: 2.08 2.07 1.89 3/199 24930
/proc/meminfo: memFree=6150004/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=10819440 memory=10758660 CPUtime=1062.31 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 2622272 0 0 0 105840 391 0 0 20 0 1 0 435688041 11079106560 2689665 33554432000 4194304 6211092 140727366575408 140727366548968 140201116302309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 2704860 2689665 819 493 0 2698165 0
Current children cumulated CPU time: 1062.31 s
Current children cumulated vsize: 10819440 KiB
Current children cumulated memory: 10758660 KiB

[startup+1122.3 s]
/proc/loadavg: 2.54 2.21 1.95 3/199 24935
/proc/meminfo: memFree=4924236/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=11406576 memory=11364200 CPUtime=1122.31 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 2769916 0 0 0 111812 419 0 0 20 0 1 0 435688041 11680333824 2841050 33554432000 4194304 6211092 140727366575408 140727366548968 140201116302309 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 2851644 2841050 819 493 0 2844949 0
Current children cumulated CPU time: 1122.31 s
Current children cumulated vsize: 11406576 KiB
Current children cumulated memory: 11364200 KiB

[startup+1182.3 s]
/proc/loadavg: 2.16 2.15 1.95 3/199 24941
/proc/meminfo: memFree=3696388/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=11993580 memory=11969744 CPUtime=1182.32 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 2917550 0 0 0 117784 448 0 0 20 0 1 0 435688041 12281425920 2992436 33554432000 4194304 6211092 140727366575408 140727366548968 140201117222829 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 2998395 2992436 819 493 0 2991700 0
Current children cumulated CPU time: 1182.32 s
Current children cumulated vsize: 11993580 KiB
Current children cumulated memory: 11969744 KiB

[startup+1242.3 s]
/proc/loadavg: 2.06 2.13 1.95 3/199 24945
/proc/meminfo: memFree=2473120/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=12767440 memory=12576308 CPUtime=1242.32 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3065487 0 0 0 123755 477 0 0 20 0 1 0 435688041 13073858560 3144077 33554432000 4194304 6211092 140727366575408 140727366548968 140201126247246 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3191860 3144077 819 493 0 3185165 0
Current children cumulated CPU time: 1242.32 s
Current children cumulated vsize: 12767440 KiB
Current children cumulated memory: 12576308 KiB

[startup+1302.3 s]
/proc/loadavg: 2.02 2.10 1.95 3/199 24950
/proc/meminfo: memFree=1249788/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=13354180 memory=13181536 CPUtime=1302.31 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3213049 0 0 0 129724 507 0 0 20 0 1 0 435688041 13674680320 3295384 33554432000 4194304 6211092 140727366575408 140727366548968 140201117225986 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3338545 3295384 819 493 0 3331850 0
Current children cumulated CPU time: 1302.31 s
Current children cumulated vsize: 13354180 KiB
Current children cumulated memory: 13181536 KiB

[startup+1362.3 s]
/proc/loadavg: 2.01 2.08 1.95 3/199 24956
/proc/meminfo: memFree=194996/32770624 swapFree=66999956/67108860
[pid=24833] ppid=24830 vsize=13940788 memory=13788644 CPUtime=1362.32 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3360564 0 0 0 135695 537 0 0 20 0 1 0 435688041 14275366912 3447161 33554432000 4194304 6211092 140727366575408 140727366548968 5451482 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3485197 3447161 819 493 0 3478502 0
Current children cumulated CPU time: 1362.32 s
Current children cumulated vsize: 13940788 KiB
Current children cumulated memory: 13788644 KiB

[startup+1422.3 s]
/proc/loadavg: 1.93 2.05 1.95 3/201 24968
/proc/meminfo: memFree=202432/32770624 swapFree=66627240/67108860
[pid=24833] ppid=24830 vsize=14523964 memory=14387836 CPUtime=1422.32 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3507211 0 0 0 141655 577 0 0 20 0 1 0 435688041 14872539136 3504357 33554432000 4194304 6211092 140727366575408 140727366548968 140201116298046 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3630991 3504357 222 493 0 3624296 0
Current children cumulated CPU time: 1422.32 s
Current children cumulated vsize: 14523964 KiB
Current children cumulated memory: 14387836 KiB

[startup+1482.3 s]
/proc/loadavg: 2.05 2.07 1.96 3/202 24994
/proc/meminfo: memFree=191432/32770624 swapFree=65920256/67108860
[pid=24833] ppid=24830 vsize=15106876 memory=14988172 CPUtime=1482.31 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3654034 0 0 0 147611 620 0 0 20 0 1 0 435688041 15469441024 3477682 33554432000 4194304 6211092 140727366575408 140727366548968 140201126247242 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3776719 3477682 222 493 0 3770024 0
Current children cumulated CPU time: 1482.31 s
Current children cumulated vsize: 15106876 KiB
Current children cumulated memory: 14988172 KiB

[startup+1542.3 s]
/proc/loadavg: 2.45 2.14 1.99 2/199 25010
/proc/meminfo: memFree=15886348/32770624 swapFree=65436340/67108860
[pid=24833] ppid=24830 vsize=15691504 memory=15592388 CPUtime=1542.31 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3801361 0 0 0 153573 658 0 0 20 0 1 0 435688041 16068100096 3507745 33554432000 4194304 6211092 140727366575408 140727366548968 5452145 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3922876 3507745 222 493 0 3916181 0
Current children cumulated CPU time: 1542.31 s
Current children cumulated vsize: 15691504 KiB
Current children cumulated memory: 15592388 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1560.8 s]
/proc/loadavg: 2.35 2.13 1.99 3/203 25065
/proc/meminfo: memFree=16251704/32770624 swapFree=65436348/67108860
[pid=24833] ppid=24830 vsize=15872476 memory=15779320 CPUtime=1560.81 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) R 24830 24833 24772 0 -1 4202496 3846925 0 0 0 155414 667 0 0 20 0 1 0 435688041 16253415424 3554478 33554432000 4194304 6211092 140727366575408 140727366548968 5446267 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3968119 3554478 222 493 0 3961424 0
Current children cumulated CPU time: 1560.81 s
Current children cumulated vsize: 15872476 KiB
Current children cumulated memory: 15779320 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1563.63 s]
# the end of solver process 24833 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1554.27 s, system=7.80157 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1561.6 s]
/proc/loadavg: 2.35 2.13 1.99 2/204 25066
/proc/meminfo: memFree=16240940/32770624 swapFree=65436356/67108860
[pid=24833] ppid=24830 vsize=15872740 memory=15779880 CPUtime=1560.95 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) D 24830 24833 24772 0 -1 512 3855483 0 1243 0 155418 677 0 0 20 0 1 0 435688041 16253685760 3564309 33554432000 4194304 6211092 140727366575408 140727366548968 5189768 0 16384 4096 8405026 18446744071580419444 0 0 17 5 0 0 66 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3968185 3564309 231 493 0 3961490 0
Current children cumulated CPU time: 1560.95 s
Current children cumulated vsize: 15872740 KiB
Current children cumulated memory: 15779880 KiB

[startup+1562.4 s]
/proc/loadavg: 2.48 2.17 2.00 4/204 25067
/proc/meminfo: memFree=16007928/32770624 swapFree=65436356/67108860
[pid=24833] ppid=24830 vsize=15872740 memory=15779880 CPUtime=1561.14 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) D 24830 24833 24772 0 -1 512 3871459 0 3614 0 155422 692 0 0 20 0 1 0 435688041 16253685760 3582657 33554432000 4194304 6211092 140727366575408 140727366548968 5189768 0 16384 4096 8405026 18446744071580419444 0 0 17 5 0 0 127 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3968185 3582657 231 493 0 3961490 0
Current children cumulated CPU time: 1561.14 s
Current children cumulated vsize: 15872740 KiB
Current children cumulated memory: 15779880 KiB

[startup+1562.8 s]
/proc/loadavg: 2.48 2.17 2.00 4/204 25067
/proc/meminfo: memFree=16007928/32770624 swapFree=65436356/67108860
[pid=24833] ppid=24830 vsize=15872740 memory=15779880 CPUtime=1561.22 cores=1,3,5,7
/proc/24833/stat : 24833 (cosoco) D 24830 24833 24772 0 -1 512 3878948 0 4727 0 155426 696 0 0 20 0 1 0 435688041 16253685760 3591237 33554432000 4194304 6211092 140727366575408 140727366548968 5189768 0 16384 4096 8405026 18446744071580419444 0 0 17 5 0 0 158 0 0 8312316 8317760 16453632 140727366579211 140727366579313 140727366579313 140727366582222 0
/proc/24833/statm: 3968185 3591237 231 493 0 3961490 0
Current children cumulated CPU time: 1561.22 s
Current children cumulated vsize: 15872740 KiB
Current children cumulated memory: 15779880 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1563.63
CPU time (s): 1562.07
CPU user time (s): 1554.27
CPU system time (s): 7.80157
CPU usage (%): 99.9
Max. virtual memory (cumulated for all children) (KiB): 15872740
Max. memory (cumulated for all children) (KiB): 15779880

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1554.27
system time used= 7.80157
maximum resident set size= 14365156
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3878948
page faults= 4728
swaps= 0
block input operations= 307624
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 4879
involuntary context switches= 549


# summary of solver processes directly reported to runsolver:
#   pid: 24833
#   total CPU time (s): 1562.07
#   total CPU user time (s): 1554.27
#   total CPU system time (s): 7.80157

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.38429 second user time and 10.5243 second system time

The end

Launcher Data 

Begin job on node123 at 2019-09-16 02:36:33
IDJOB=4397190
IDBENCH=139548
IDSOLVER=2848
FILE ID=node123/4397190-1568594232
RUNJOBID= node123-1568594193-24804
PBS_JOBID= 21703348
Free space on /tmp= 59184 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/SteelMillSlab/SteelMillSlab-bench_16_2.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4397190-1568594232/watcher-4397190-1568594232 -o /tmp/evaluation-result-4397190-1568594232/solver-4397190-1568594232 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node123-1568594193-24804 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4397190-1568594232.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8e5b61791e512734038a023d36a5924c
RANDOM SEED=1939549204

node123.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        27904368 kB
MemAvailable:   31863024 kB
Buffers:          170852 kB
Cached:          4061068 kB
SwapCached:         7360 kB
Active:          1326944 kB
Inactive:        2918176 kB
Active(anon):       5796 kB
Inactive(anon):    56876 kB
Active(file):    1321148 kB
Inactive(file):  2861300 kB
Unevictable:      108052 kB
Mlocked:          108052 kB
SwapTotal:      67108860 kB
SwapFree:       66999956 kB
Dirty:              5440 kB
Writeback:             0 kB
AnonPages:        114908 kB
Mapped:            47540 kB
Shmem:             40884 kB
Slab:             213756 kB
SReclaimable:     184088 kB
SUnreclaim:        29668 kB
KernelStack:        3280 kB
PageTables:         6288 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     536840 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 59036 MiB
End job on node123 at 2019-09-16 03:02:38