Trace number 4389865

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1305.05 1306.88

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-031-009.xml
MD5SUMc781bc55b7ee4c4bb57d86a03397642c
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark3.79055
Satisfiable
(Un)Satisfiability was proved
Number of variables311
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size30
Distribution of domain sizes[{"size":2,"count":310},{"size":30,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":310},{"degree":46,"count":1}]
Minimum constraint arity31
Maximum constraint arity310
Distribution of constraint arities[{"arity":31,"count":10},{"arity":63,"count":45},{"arity":310,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (5898 MiB free)
  memory of node 1: 16384 MiB (11950 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4389865-1568322081/watcher-4389865-1568322081 -o /tmp/evaluation-result-4389865-1568322081/solver-4389865-1568322081 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node123-1568311188-14001 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4389865-1568322081.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=15920, runsolver pid=15917

[startup+0.100154 s]*
/proc/loadavg: 2.02 2.06 2.06 3/199 15921
/proc/meminfo: memFree=18275064/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=28736 memory=5008 CPUtime=0.1 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 1465 0 0 0 10 0 0 0 20 0 1 0 408472959 29425664 1252 33554432000 4194304 6210612 140734313621504 140734313615824 5442952 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 7184 1252 844 493 0 489 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28736 KiB
Current children cumulated memory: 5008 KiB

[startup+0.209246 s]*
/proc/loadavg: 2.02 2.06 2.06 3/199 15921
/proc/meminfo: memFree=18273124/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=30148 memory=6328 CPUtime=0.21 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 1809 0 0 0 21 0 0 0 20 0 1 0 408472959 30871552 1582 33554432000 4194304 6210612 140734313621504 140734313603288 140713852154714 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 7537 1582 845 493 0 842 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30148 KiB
Current children cumulated memory: 6328 KiB

[startup+0.308843 s]*
/proc/loadavg: 2.02 2.06 2.06 3/199 15921
/proc/meminfo: memFree=18271264/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=31288 memory=7716 CPUtime=0.3 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 2094 0 0 0 30 0 0 0 20 0 1 0 408472959 32038912 1929 33554432000 4194304 6210612 140734313621504 140734313603288 5446100 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 7822 1929 845 493 0 1127 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 31288 KiB
Current children cumulated memory: 7716 KiB

[startup+0.70028 s]
/proc/loadavg: 2.02 2.06 2.06 3/199 15921
/proc/meminfo: memFree=18267544/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=36092 memory=12360 CPUtime=0.69 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 3251 0 0 0 69 0 0 0 20 0 1 0 408472959 36958208 3090 33554432000 4194304 6210612 140734313621504 140734313603288 140713851731658 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 9023 3090 845 493 0 2328 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 36092 KiB
Current children cumulated memory: 12360 KiB

[startup+1.50028 s]
/proc/loadavg: 2.02 2.06 2.06 4/199 15921
/proc/meminfo: memFree=18257872/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=45616 memory=21860 CPUtime=1.49 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 5667 0 0 0 148 1 0 0 20 0 1 0 408472959 46710784 5465 33554432000 4194304 6210612 140734313621504 140734313603288 140713851226942 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 11404 5465 845 493 0 4709 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 45616 KiB
Current children cumulated memory: 21860 KiB

[startup+3.10931 s]
/proc/loadavg: 2.02 2.06 2.06 4/199 15921
/proc/meminfo: memFree=18218580/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=65164 memory=41224 CPUtime=3.1 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 10567 0 0 0 309 1 0 0 20 0 1 0 408472959 66727936 10306 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 16291 10306 845 493 0 9596 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 65164 KiB
Current children cumulated memory: 41224 KiB

[startup+6.30028 s]
/proc/loadavg: 2.02 2.06 2.06 4/199 15921
/proc/meminfo: memFree=18159712/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=104392 memory=79900 CPUtime=6.29 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 20247 0 0 0 627 2 0 0 20 0 1 0 408472959 106897408 19975 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 26098 19975 845 493 0 19403 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 104392 KiB
Current children cumulated memory: 79900 KiB

[startup+12.7003 s]
/proc/loadavg: 2.02 2.06 2.06 4/199 15921
/proc/meminfo: memFree=18041832/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=183612 memory=157212 CPUtime=12.68 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 39110 0 0 0 1264 4 0 0 20 0 1 0 408472959 188018688 39303 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 45903 39303 845 493 0 39208 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 183612 KiB
Current children cumulated memory: 157212 KiB

[startup+25.5003 s]
/proc/loadavg: 2.01 2.06 2.05 4/199 15943
/proc/meminfo: memFree=17785524/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=336048 memory=311932 CPUtime=25.49 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 76869 0 0 0 2539 10 0 0 20 0 1 0 408472959 344113152 77983 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 84012 77983 845 493 0 77317 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 336048 KiB
Current children cumulated memory: 311932 KiB

[startup+51.1094 s]
/proc/loadavg: 2.01 2.05 2.05 4/199 15946
/proc/meminfo: memFree=17271848/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=649636 memory=620948 CPUtime=51.1 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 151631 0 0 0 5089 21 0 0 20 0 1 0 408472959 665227264 155237 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 162409 155237 845 493 0 155714 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 649636 KiB
Current children cumulated memory: 620948 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.04 2.05 4/199 15949
/proc/meminfo: memFree=16267908/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=1281424 memory=1240484 CPUtime=102.28 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 300361 0 0 0 10190 38 0 0 20 0 1 0 408472959 1312178176 310121 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 320356 310121 845 493 0 313661 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 1281424 KiB
Current children cumulated memory: 1240484 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.04 2.05 4/199 15953
/proc/meminfo: memFree=15083468/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=2011612 memory=1965980 CPUtime=162.29 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 474127 0 0 0 16170 59 0 0 20 0 1 0 408472959 2059890688 491495 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 502903 491495 845 493 0 496208 0
Current children cumulated CPU time: 162.29 s
Current children cumulated vsize: 2011612 KiB
Current children cumulated memory: 1965980 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.03 2.05 4/199 15959
/proc/meminfo: memFree=13901544/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=2759352 memory=2689396 CPUtime=222.28 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 647884 0 0 0 22147 81 0 0 20 0 1 0 408472959 2825576448 672349 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 689871 672349 845 493 0 683176 0
Current children cumulated CPU time: 222.28 s
Current children cumulated vsize: 2759352 KiB
Current children cumulated memory: 2689396 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.03 2.05 4/199 15963
/proc/meminfo: memFree=12720672/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=3452604 memory=3414580 CPUtime=282.27 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 821029 0 0 0 28123 104 0 0 20 0 1 0 408472959 3535466496 853645 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 863151 853645 845 493 0 856456 0
Current children cumulated CPU time: 282.27 s
Current children cumulated vsize: 3452604 KiB
Current children cumulated memory: 3414580 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.02 2.05 4/199 15967
/proc/meminfo: memFree=11539064/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=4227196 memory=4138456 CPUtime=342.28 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 994513 0 0 0 34100 128 0 0 20 0 1 0 408472959 4328648704 1034614 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 1056799 1034614 845 493 0 1050104 0
Current children cumulated CPU time: 342.28 s
Current children cumulated vsize: 4227196 KiB
Current children cumulated memory: 4138456 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 15973
/proc/meminfo: memFree=10356048/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=4918612 memory=4863364 CPUtime=402.28 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 1167681 0 0 0 40075 153 0 0 20 0 1 0 408472959 5036658688 1215841 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 1229653 1215841 845 493 0 1222958 0
Current children cumulated CPU time: 402.28 s
Current children cumulated vsize: 4918612 KiB
Current children cumulated memory: 4863364 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 15977
/proc/meminfo: memFree=9173056/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=5612452 memory=5588268 CPUtime=462.27 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 1340839 0 0 0 46049 178 0 0 20 0 1 0 408472959 5747150848 1397067 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 1403113 1397067 845 493 0 1396418 0
Current children cumulated CPU time: 462.27 s
Current children cumulated vsize: 5612452 KiB
Current children cumulated memory: 5588268 KiB


################
# More data... #
################

Current children cumulated vsize: 8692384 KiB
Current children cumulated memory: 8485648 KiB

[startup+762.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/201 16010
/proc/meminfo: memFree=3262756/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=9383932 memory=9209000 CPUtime=762.26 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 2207580 0 0 0 75916 310 0 0 20 0 1 0 408472959 9609146368 2302250 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 2345983 2302250 845 493 0 2339288 0
Current children cumulated CPU time: 762.26 s
Current children cumulated vsize: 9383932 KiB
Current children cumulated memory: 9209000 KiB

[startup+822.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/201 16014
/proc/meminfo: memFree=2077752/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=10080340 memory=9936036 CPUtime=822.26 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 2380802 0 0 0 81887 339 0 0 20 0 1 0 408472959 10322268160 2484009 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 2520085 2484009 845 493 0 2513390 0
Current children cumulated CPU time: 822.26 s
Current children cumulated vsize: 10080340 KiB
Current children cumulated memory: 9936036 KiB

[startup+882.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 16018
/proc/meminfo: memFree=898236/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=10770964 memory=10659536 CPUtime=882.26 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 2553714 0 0 0 87856 370 0 0 20 0 1 0 408472959 11029467136 2664818 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 2692741 2664818 845 493 0 2686046 0
Current children cumulated CPU time: 882.26 s
Current children cumulated vsize: 10770964 KiB
Current children cumulated memory: 10659536 KiB

[startup+942.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 16024
/proc/meminfo: memFree=187424/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=11461192 memory=11380120 CPUtime=942.26 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 2728461 0 0 0 93824 402 0 0 20 0 1 0 408472959 11736260608 2845030 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 2865298 2845030 845 493 0 2858603 0
Current children cumulated CPU time: 942.26 s
Current children cumulated vsize: 11461192 KiB
Current children cumulated memory: 11380120 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 16028
/proc/meminfo: memFree=207472/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=12150496 memory=12102492 CPUtime=1002.26 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 2905099 0 0 0 99782 444 0 0 20 0 1 0 408472959 12442107904 3025623 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3037624 3025623 845 493 0 3030929 0
Current children cumulated CPU time: 1002.26 s
Current children cumulated vsize: 12150496 KiB
Current children cumulated memory: 12102492 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 16032
/proc/meminfo: memFree=194480/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=13119816 memory=12824688 CPUtime=1062.26 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 3078237 0 0 0 105739 487 0 0 20 0 1 0 408472959 13434691584 3206172 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3279954 3206172 845 493 0 3273259 0
Current children cumulated CPU time: 1062.26 s
Current children cumulated vsize: 13119816 KiB
Current children cumulated memory: 12824688 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/199 16038
/proc/meminfo: memFree=186164/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=13812848 memory=13544100 CPUtime=1122.26 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 3250576 0 0 0 111669 557 0 0 20 0 1 0 408472959 14144356352 3386025 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3453212 3386025 845 493 0 3446517 0
Current children cumulated CPU time: 1122.26 s
Current children cumulated vsize: 13812848 KiB
Current children cumulated memory: 13544100 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/200 16050
/proc/meminfo: memFree=206376/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=14494232 memory=14258272 CPUtime=1182.25 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 3422188 0 0 0 117566 659 0 0 20 0 1 0 408472959 14842093568 3564568 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3623558 3564568 845 493 0 3616863 0
Current children cumulated CPU time: 1182.25 s
Current children cumulated vsize: 14494232 KiB
Current children cumulated memory: 14258272 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/200 16054
/proc/meminfo: memFree=190336/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=15173240 memory=14967976 CPUtime=1242.25 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 3593191 0 0 0 123450 775 0 0 20 0 1 0 408472959 15537397760 3741994 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3793310 3741994 845 493 0 3786615 0
Current children cumulated CPU time: 1242.25 s
Current children cumulated vsize: 15173240 KiB
Current children cumulated memory: 14967976 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 2.05 4/200 16060
/proc/meminfo: memFree=180880/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=15852908 memory=15680496 CPUtime=1302.25 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 3764774 0 0 0 129338 887 0 0 20 0 1 0 408472959 16233377792 3920124 33554432000 4194304 6210612 140734313621504 140734313603288 140713851231205 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3963227 3920124 845 493 0 3956532 0
Current children cumulated CPU time: 1302.25 s
Current children cumulated vsize: 15852908 KiB
Current children cumulated memory: 15680496 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1304 s]*
/proc/loadavg: 2.00 2.01 2.05 4/200 16060
/proc/meminfo: memFree=197004/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=15872180 memory=15699600 CPUtime=1303.96 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) R 15917 15920 13969 0 -1 4202496 3769612 0 0 0 129505 891 0 0 20 0 1 0 408472959 16253112320 3924900 33554432000 4194304 6210612 140734313621504 140734313603288 5442952 0 0 4096 8405026 0 0 0 17 1 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3968045 3924900 845 493 0 3961350 0
Current children cumulated CPU time: 1303.96 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15699600 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1306.88 s]
# the end of solver process 15920 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1295.23 s, system=9.81107 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1304.8 s]
/proc/loadavg: 2.00 2.01 2.05 4/200 16060
/proc/meminfo: memFree=197004/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=15872180 memory=15699600 CPUtime=1304 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) S 15917 15920 13969 0 -1 512 3769619 0 0 0 129509 891 0 0 20 0 1 0 408472959 16253112320 3924900 33554432000 4194304 6210612 140734313621504 140734313603288 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 1 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3968045 3924900 845 493 0 3961350 0
Current children cumulated CPU time: 1304 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15699600 KiB

[startup+1305.6 s]
/proc/loadavg: 2.00 2.01 2.05 4/201 16061
/proc/meminfo: memFree=197212/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=15872180 memory=15699600 CPUtime=1304.11 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) S 15917 15920 13969 0 -1 512 3769619 0 0 0 129518 893 0 0 20 0 1 0 408472959 16253112320 3924900 33554432000 4194304 6210612 140734313621504 140734313603288 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3968045 3924900 845 493 0 3961350 0
Current children cumulated CPU time: 1304.11 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15699600 KiB

[startup+1306 s]*
/proc/loadavg: 2.00 2.01 2.05 3/201 16061
/proc/meminfo: memFree=184436/32770624 swapFree=66999904/67108860
[pid=15920] ppid=15917 vsize=15872180 memory=15699600 CPUtime=1304.16 cores=1,3,5,7
/proc/15920/stat : 15920 (cosoco) S 15917 15920 13969 0 -1 512 3769619 0 0 0 129523 893 0 0 20 0 1 0 408472959 16253112320 3924900 33554432000 4194304 6210612 140734313621504 140734313603288 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8308220 8313664 17268736 140734313628683 140734313628785 140734313628785 140734313631694 0
/proc/15920/statm: 3968045 3924900 845 493 0 3961350 0
Current children cumulated CPU time: 1304.16 s
Current children cumulated vsize: 15872180 KiB
Current children cumulated memory: 15699600 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1306.88
CPU time (s): 1305.05
CPU user time (s): 1295.23
CPU system time (s): 9.81107
CPU usage (%): 99.8595
Max. virtual memory (cumulated for all children) (KiB): 15872180
Max. memory (cumulated for all children) (KiB): 15699600

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1295.23
system time used= 9.81107
maximum resident set size= 15699796
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3769619
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 88017
involuntary context switches= 1548


# summary of solver processes directly reported to runsolver:
#   pid: 15920
#   total CPU time (s): 1305.05
#   total CPU user time (s): 1295.23
#   total CPU system time (s): 9.81107

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 4.63483 second user time and 8.6875 second system time

The end

Launcher Data 

Begin job on node123 at 2019-09-12 23:00:42
IDJOB=4389865
IDBENCH=122975
IDSOLVER=2829
FILE ID=node123/4389865-1568322081
RUNJOBID= node123-1568311188-14001
PBS_JOBID= 21701126
Free space on /tmp= 56796 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-031-009.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4389865-1568322081/watcher-4389865-1568322081 -o /tmp/evaluation-result-4389865-1568322081/solver-4389865-1568322081 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node123-1568311188-14001 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4389865-1568322081.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= c781bc55b7ee4c4bb57d86a03397642c
RANDOM SEED=438726964

node123.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.49
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        18277996 kB
MemAvailable:   29229172 kB
Buffers:          202656 kB
Cached:         10921612 kB
SwapCached:         7400 kB
Active:          4212124 kB
Inactive:        9545664 kB
Active(anon):    2630320 kB
Inactive(anon):    44484 kB
Active(file):    1581804 kB
Inactive(file):  9501180 kB
Unevictable:      107520 kB
Mlocked:          107520 kB
SwapTotal:      67108860 kB
SwapFree:       66999904 kB
Dirty:              2748 kB
Writeback:             0 kB
AnonPages:       2821072 kB
Mapped:            44600 kB
Shmem:             32668 kB
Slab:             326052 kB
SReclaimable:     296652 kB
SUnreclaim:        29400 kB
KernelStack:        3520 kB
PageTables:        11196 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    3150560 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    620544 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 56448 MiB
End job on node123 at 2019-09-12 23:22:29