Trace number 4408274

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1177.86 1180.15

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-015-006.xml
MD5SUMd279c4dbde03937b121f919805027b3b
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark3.09037
Satisfiable
(Un)Satisfiability was proved
Number of variables151
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size14
Distribution of domain sizes[{"size":2,"count":150},{"size":14,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":150},{"degree":46,"count":1}]
Minimum constraint arity15
Maximum constraint arity150
Distribution of constraint arities[{"arity":15,"count":10},{"arity":31,"count":45},{"arity":150,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15307 MiB free)
  memory of node 1: 16384 MiB (15444 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408274-1569791758/watcher-4408274-1569791758 -o /tmp/evaluation-result-4408274-1569791758/solver-4408274-1569791758 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node104-1569783168-25106 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408274-1569791758.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=25961, runsolver pid=25958

[startup+0.100105 s]*
/proc/loadavg: 2.08 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31489104/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=28720 memory=5008 CPUtime=0.1 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 1457 0 0 0 10 0 0 0 20 0 1 0 555432103 29409280 1252 33554432000 4194304 6211132 140730573091392 140730573087928 140531105643493 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 7180 1252 846 493 0 485 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28720 KiB
Current children cumulated memory: 5008 KiB

[startup+0.209619 s]*
/proc/loadavg: 2.08 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31487696/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=30248 memory=6348 CPUtime=0.21 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 1810 0 0 0 21 0 0 0 20 0 1 0 555432103 30973952 1587 33554432000 4194304 6211132 140730573091392 140730573087928 140531105643493 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 7562 1587 846 493 0 867 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30248 KiB
Current children cumulated memory: 6348 KiB

[startup+0.309061 s]*
/proc/loadavg: 2.08 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31487088/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=31496 memory=7680 CPUtime=0.31 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 2128 0 0 0 31 0 0 0 20 0 1 0 555432103 32251904 1920 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 7874 1920 846 493 0 1179 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 31496 KiB
Current children cumulated memory: 7680 KiB

[startup+0.700238 s]
/proc/loadavg: 2.08 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31484484/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=36696 memory=12772 CPUtime=0.69 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 3368 0 0 0 69 0 0 0 20 0 1 0 555432103 37576704 3193 33554432000 4194304 6211132 140730573091392 140730573079832 140531115588456 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 9174 3193 846 493 0 2479 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 36696 KiB
Current children cumulated memory: 12772 KiB

[startup+1.50026 s]
/proc/loadavg: 2.08 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31477912/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=46708 memory=23164 CPUtime=1.49 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 5973 0 0 0 148 1 0 0 20 0 1 0 555432103 47828992 5791 33554432000 4194304 6211132 140730573091392 140730573079320 5013034 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 11677 5791 846 493 0 4982 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 46708 KiB
Current children cumulated memory: 23164 KiB

[startup+3.10958 s]
/proc/loadavg: 2.08 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31452024/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=68092 memory=44132 CPUtime=3.1 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 11238 0 0 0 308 2 0 0 20 0 1 0 555432103 69726208 11033 33554432000 4194304 6211132 140730573091392 140730573079320 5452266 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 17023 11033 846 493 0 10328 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 68092 KiB
Current children cumulated memory: 44132 KiB

[startup+6.30033 s]
/proc/loadavg: 2.08 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31413076/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=111104 memory=85632 CPUtime=6.29 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 21692 0 0 0 626 3 0 0 20 0 1 0 555432103 113770496 21408 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 27776 21408 846 493 0 21081 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 111104 KiB
Current children cumulated memory: 85632 KiB

[startup+12.7003 s]
/proc/loadavg: 2.07 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31335548/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=198944 memory=170568 CPUtime=12.68 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 41429 0 0 0 1263 5 0 0 20 0 1 0 555432103 203718656 42642 33554432000 4194304 6211132 140730573091392 140730573079320 5451615 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 49736 42642 846 493 0 43041 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 198944 KiB
Current children cumulated memory: 170568 KiB

[startup+25.5003 s]
/proc/loadavg: 2.05 2.03 1.97 3/199 25962
/proc/meminfo: memFree=31165660/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=361204 memory=336744 CPUtime=25.48 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 79949 0 0 0 2539 9 0 0 20 0 1 0 555432103 369872896 84186 33554432000 4194304 6211132 140730573091392 140730573079320 140531106564032 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 90301 84186 846 493 0 83606 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 361204 KiB
Current children cumulated memory: 336744 KiB

[startup+51.1097 s]
/proc/loadavg: 2.04 2.03 1.97 3/198 25965
/proc/meminfo: memFree=30823664/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=705168 memory=672112 CPUtime=51.09 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 156648 0 0 0 5089 20 0 0 20 0 1 0 555432103 722092032 168028 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 176292 168028 846 493 0 169597 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 705168 KiB
Current children cumulated memory: 672112 KiB

[startup+102.3 s]
/proc/loadavg: 2.02 2.02 1.97 3/198 25969
/proc/meminfo: memFree=30152872/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=1403756 memory=1341520 CPUtime=102.27 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 309161 0 0 0 10185 42 0 0 20 0 1 0 555432103 1437446144 335314 33554432000 4194304 6211132 140730573091392 140730573079320 140531106567758 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 350939 335314 846 493 0 344244 0
Current children cumulated CPU time: 102.27 s
Current children cumulated vsize: 1403756 KiB
Current children cumulated memory: 1341520 KiB

[startup+162.3 s]
/proc/loadavg: 2.01 2.02 1.97 3/198 25973
/proc/meminfo: memFree=29363176/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=2201364 memory=2129036 CPUtime=162.23 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 487154 0 0 0 16156 67 0 0 20 0 1 0 555432103 2254196736 532259 33554432000 4194304 6211132 140730573091392 140730573079320 140531106563984 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 550341 532259 846 493 0 543646 0
Current children cumulated CPU time: 162.23 s
Current children cumulated vsize: 2201364 KiB
Current children cumulated memory: 2129036 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 1.97 3/197 25979
/proc/meminfo: memFree=28579324/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=3039160 memory=2914064 CPUtime=222.21 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 664549 0 0 0 22129 92 0 0 20 0 1 0 555432103 3112099840 728516 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 759790 728516 846 493 0 753095 0
Current children cumulated CPU time: 222.21 s
Current children cumulated vsize: 3039160 KiB
Current children cumulated memory: 2914064 KiB

[startup+282.3 s]
/proc/loadavg: 2.12 2.04 1.98 3/199 25992
/proc/meminfo: memFree=27792868/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=3745228 memory=3695948 CPUtime=282.17 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 841145 0 0 0 28100 117 0 0 20 0 1 0 555432103 3835113472 923987 33554432000 4194304 6211132 140730573091392 140730573079320 140531105639230 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 936307 923987 846 493 0 929612 0
Current children cumulated CPU time: 282.17 s
Current children cumulated vsize: 3745228 KiB
Current children cumulated memory: 3695948 KiB

[startup+342.3 s]
/proc/loadavg: 2.04 2.04 1.98 3/199 25996
/proc/meminfo: memFree=27006612/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=4648956 memory=4481164 CPUtime=342.14 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 1018617 0 0 0 34073 141 0 0 20 0 1 0 555432103 4760530944 1120291 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 1162239 1120291 846 493 0 1155544 0
Current children cumulated CPU time: 342.14 s
Current children cumulated vsize: 4648956 KiB
Current children cumulated memory: 4481164 KiB

[startup+402.3 s]
/proc/loadavg: 2.02 2.03 1.98 3/199 26002
/proc/meminfo: memFree=26220356/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=5355156 memory=5265292 CPUtime=402.11 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 1195281 0 0 0 40045 166 0 0 20 0 1 0 555432103 5483679744 1316323 33554432000 4194304 6211132 140730573091392 140730573079320 140531106567758 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 1338789 1316323 846 493 0 1332094 0
Current children cumulated CPU time: 402.11 s
Current children cumulated vsize: 5355156 KiB
Current children cumulated memory: 5265292 KiB

[startup+462.3 s]
/proc/loadavg: 2.01 2.03 1.98 3/199 26006
/proc/meminfo: memFree=25434836/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=6341240 memory=6049480 CPUtime=462.08 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 1372527 0 0 0 46015 193 0 0 20 0 1 0 555432103 6493429760 1512370 33554432000 4194304 6211132 140730573091392 140730573079320 5455666 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 1585310 1512370 846 493 0 1578615 0
Current children cumulated CPU time: 462.08 s
Current children cumulated vsize: 6341240 KiB
Current children cumulated memory: 6049480 KiB


################
# More data... #
################

Current children cumulated vsize: 7769648 KiB
Current children cumulated memory: 7618568 KiB

[startup+642.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/198 26020
/proc/meminfo: memFree=23076404/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=8499264 memory=8403320 CPUtime=641.98 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 1903016 0 0 0 63925 273 0 0 20 0 1 0 555432103 8703246336 2100830 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 2124816 2100830 846 493 0 2118121 0
Current children cumulated CPU time: 641.98 s
Current children cumulated vsize: 8499264 KiB
Current children cumulated memory: 8403320 KiB

[startup+702.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/198 26024
/proc/meminfo: memFree=22292712/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=9624568 memory=9185832 CPUtime=701.96 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 2079831 0 0 0 69893 303 0 0 20 0 1 0 555432103 9855557632 2296458 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 2406142 2296458 846 493 0 2399447 0
Current children cumulated CPU time: 701.96 s
Current children cumulated vsize: 9624568 KiB
Current children cumulated memory: 9185832 KiB

[startup+762.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/198 26030
/proc/meminfo: memFree=21508864/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=10329712 memory=9968744 CPUtime=761.95 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 2256241 0 0 0 75862 333 0 0 20 0 1 0 555432103 10577625088 2492186 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 2582428 2492186 846 493 0 2575733 0
Current children cumulated CPU time: 761.95 s
Current children cumulated vsize: 10329712 KiB
Current children cumulated memory: 9968744 KiB

[startup+822.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/198 26033
/proc/meminfo: memFree=20724428/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=11034460 memory=10753432 CPUtime=821.92 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 2432597 0 0 0 81827 365 0 0 20 0 1 0 555432103 11299287040 2688358 33554432000 4194304 6211132 140730573091392 140730573079320 5013001 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 2758615 2688358 846 493 0 2751920 0
Current children cumulated CPU time: 821.92 s
Current children cumulated vsize: 11034460 KiB
Current children cumulated memory: 10753432 KiB

[startup+882.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26105
/proc/meminfo: memFree=19487812/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=11776120 memory=11536204 CPUtime=881.89 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 2609511 0 0 0 87795 394 0 0 20 0 1 0 555432103 12058746880 2884051 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 2944030 2884051 846 493 0 2937335 0
Current children cumulated CPU time: 881.89 s
Current children cumulated vsize: 11776120 KiB
Current children cumulated memory: 11536204 KiB

[startup+942.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26111
/proc/meminfo: memFree=18700856/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=12480868 memory=12318852 CPUtime=941.87 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 2785838 0 0 0 93759 428 0 0 20 0 1 0 555432103 12780408832 3079713 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3120217 3079713 846 493 0 3113522 0
Current children cumulated CPU time: 941.87 s
Current children cumulated vsize: 12480868 KiB
Current children cumulated memory: 12318852 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26115
/proc/meminfo: memFree=17917352/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=13185484 memory=13101128 CPUtime=1001.85 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 2962157 0 0 0 99728 457 0 0 20 0 1 0 555432103 13501935616 3275282 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3296371 3275282 846 493 0 3289676 0
Current children cumulated CPU time: 1001.85 s
Current children cumulated vsize: 13185484 KiB
Current children cumulated memory: 13101128 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26119
/proc/meminfo: memFree=17131244/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=14520600 memory=13885504 CPUtime=1061.83 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 3138971 0 0 0 105693 490 0 0 20 0 1 0 555432103 14869094400 3471376 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3630150 3471376 846 493 0 3623455 0
Current children cumulated CPU time: 1061.83 s
Current children cumulated vsize: 14520600 KiB
Current children cumulated memory: 13885504 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26125
/proc/meminfo: memFree=16344288/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=15225084 memory=14665736 CPUtime=1121.81 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 3315263 0 0 0 111657 524 0 0 20 0 1 0 555432103 15590486016 3666434 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3806271 3666434 846 493 0 3799576 0
Current children cumulated CPU time: 1121.81 s
Current children cumulated vsize: 15225084 KiB
Current children cumulated memory: 14665736 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1177.4 s]
/proc/loadavg: 2.00 2.01 1.98 3/198 26130
/proc/meminfo: memFree=15623720/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=15872280 memory=15386636 CPUtime=1176.89 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) R 25958 25961 25075 0 -1 4202496 3477219 0 0 0 117133 556 0 0 20 0 1 0 555432103 16253214720 3846659 33554432000 4194304 6211132 140730573091392 140730573079320 140531105643493 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3968070 3846659 846 493 0 3961375 0
Current children cumulated CPU time: 1176.89 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15386636 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1180.15 s]
# the end of solver process 25961 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1171.55 s, system=6.31121 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1177.6 s]
/proc/loadavg: 2.00 2.01 1.98 3/198 26130
/proc/meminfo: memFree=15623720/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=15872280 memory=15386636 CPUtime=1176.89 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) S 25958 25961 25075 0 -1 512 3477224 0 0 0 117133 556 0 0 20 0 1 0 555432103 16253214720 3846659 33554432000 4194304 6211132 140730573091392 140730573079320 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3968070 3846659 846 493 0 3961375 0
Current children cumulated CPU time: 1176.89 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15386636 KiB

[startup+1178.4 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26131
/proc/meminfo: memFree=15553976/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=15872280 memory=15386636 CPUtime=1176.97 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) S 25958 25961 25075 0 -1 512 3477224 0 0 0 117141 556 0 0 20 0 1 0 555432103 16253214720 3846659 33554432000 4194304 6211132 140730573091392 140730573079320 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3968070 3846659 846 493 0 3961375 0
Current children cumulated CPU time: 1176.97 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15386636 KiB

[startup+1179.2 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26131
/proc/meminfo: memFree=15042096/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=15872280 memory=15386636 CPUtime=1177.08 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) S 25958 25961 25075 0 -1 512 3477224 0 0 0 117152 556 0 0 20 0 1 0 555432103 16253214720 3846659 33554432000 4194304 6211132 140730573091392 140730573079320 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3968070 3846659 846 493 0 3961375 0
Current children cumulated CPU time: 1177.08 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15386636 KiB

[startup+1179.4 s]
/proc/loadavg: 2.00 2.01 1.98 3/199 26131
/proc/meminfo: memFree=15042096/32770624 swapFree=66986440/67108860
[pid=25961] ppid=25958 vsize=15872280 memory=15386636 CPUtime=1177.11 cores=0,2,4,6
/proc/25961/stat : 25961 (cosoco) S 25958 25961 25075 0 -1 512 3477224 0 0 0 117155 556 0 0 20 0 1 0 555432103 16253214720 3846659 33554432000 4194304 6211132 140730573091392 140730573079320 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 33226752 140730573095947 140730573096049 140730573096049 140730573098958 0
/proc/25961/statm: 3968070 3846659 846 493 0 3961375 0
Current children cumulated CPU time: 1177.11 s
Current children cumulated vsize: 15872280 KiB
Current children cumulated memory: 15386636 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1180.15
CPU time (s): 1177.86
CPU user time (s): 1171.55
CPU system time (s): 6.31121
CPU usage (%): 99.8064
Max. virtual memory (cumulated for all children) (KiB): 15872280
Max. memory (cumulated for all children) (KiB): 15386636

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1171.55
system time used= 6.31121
maximum resident set size= 15386816
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3477224
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 94845
involuntary context switches= 1388


# summary of solver processes directly reported to runsolver:
#   pid: 25961
#   total CPU time (s): 1177.86
#   total CPU user time (s): 1171.55
#   total CPU system time (s): 6.31121

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 4.53497 second user time and 8.1212 second system time

The end

Launcher Data 

Begin job on node104 at 2019-09-29 22:05:31
IDJOB=4408274
IDBENCH=122976
IDSOLVER=2874
FILE ID=node104/4408274-1569791758
RUNJOBID= node104-1569783168-25106
PBS_JOBID= 21705839
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-015-006.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408274-1569791758/watcher-4408274-1569791758 -o /tmp/evaluation-result-4408274-1569791758/solver-4408274-1569791758 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node104-1569783168-25106 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408274-1569791758.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d279c4dbde03937b121f919805027b3b
RANDOM SEED=2091154819

node104.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5334.09
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31490432 kB
MemAvailable:   31689836 kB
Buffers:          190136 kB
Cached:           338976 kB
SwapCached:        22632 kB
Active:           444036 kB
Inactive:         264536 kB
Active(anon):     170508 kB
Inactive(anon):    91040 kB
Active(file):     273528 kB
Inactive(file):   173496 kB
Unevictable:      109640 kB
Mlocked:          109640 kB
SwapTotal:      67108860 kB
SwapFree:       66986440 kB
Dirty:              2928 kB
Writeback:             0 kB
AnonPages:        267640 kB
Mapped:            27264 kB
Shmem:             73632 kB
Slab:             165652 kB
SReclaimable:     136428 kB
SUnreclaim:        29224 kB
KernelStack:        3552 kB
PageTables:         5356 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     712440 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    215040 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 61656 MiB
End job on node104 at 2019-09-29 22:25:12