Trace number 4397057

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 1710.36 1712.31

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-038-010.xml
MD5SUM8e449228b7b3cf049a5fccac2aa13fe6
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark4.37968
Satisfiable
(Un)Satisfiability was proved
Number of variables381
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size37
Distribution of domain sizes[{"size":2,"count":380},{"size":37,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":380},{"degree":46,"count":1}]
Minimum constraint arity38
Maximum constraint arity380
Distribution of constraint arities[{"arity":38,"count":10},{"arity":77,"count":45},{"arity":380,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (2426 MiB free)
  memory of node 1: 16384 MiB (13123 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4397057-1568577114/watcher-4397057-1568577114 -o /tmp/evaluation-result-4397057-1568577114/solver-4397057-1568577114 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node119-1568572745-9395 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4397057-1568577114.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=10441, runsolver pid=10438
Current StackSize limit: 8192 KiB


[startup+0.10016 s]*
/proc/loadavg: 2.68 2.18 2.06 3/203 10442
/proc/meminfo: memFree=16293360/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=28284 memory=4744 CPUtime=0.09 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 1372 0 1 0 9 0 0 0 20 0 1 0 433626344 28962816 1186 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 7071 1186 845 493 0 376 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 28284 KiB
Current children cumulated memory: 4744 KiB

[startup+0.209841 s]*
/proc/loadavg: 2.68 2.18 2.06 3/203 10442
/proc/meminfo: memFree=16293740/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=29380 memory=5800 CPUtime=0.19 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 1622 0 1 0 19 0 0 0 20 0 1 0 433626344 30085120 1450 33554432000 4194304 6211092 140734925421088 140734925400184 5441103 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 7345 1450 846 493 0 650 0
Current children cumulated CPU time: 0.19 s
Current children cumulated vsize: 29380 KiB
Current children cumulated memory: 5800 KiB

[startup+0.309732 s]*
/proc/loadavg: 2.68 2.18 2.06 3/203 10442
/proc/meminfo: memFree=16294096/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=30212 memory=6592 CPUtime=0.29 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 1846 0 1 0 29 0 0 0 20 0 1 0 433626344 30937088 1648 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 7553 1648 846 493 0 858 0
Current children cumulated CPU time: 0.29 s
Current children cumulated vsize: 30212 KiB
Current children cumulated memory: 6592 KiB

[startup+0.700349 s]
/proc/loadavg: 2.68 2.18 2.06 3/203 10442
/proc/meminfo: memFree=16290764/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=33720 memory=9896 CPUtime=0.66 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 2690 0 1 0 65 1 0 0 20 0 1 0 433626344 34529280 2474 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 8430 2474 846 493 0 1735 0
Current children cumulated CPU time: 0.66 s
Current children cumulated vsize: 33720 KiB
Current children cumulated memory: 9896 KiB

[startup+1.50027 s]
/proc/loadavg: 2.68 2.18 2.06 3/203 10443
/proc/meminfo: memFree=16283972/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=41192 memory=17436 CPUtime=1.47 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 4524 0 1 0 145 2 0 0 20 0 1 0 433626344 42180608 4359 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 10298 4359 846 493 0 3603 0
Current children cumulated CPU time: 1.47 s
Current children cumulated vsize: 41192 KiB
Current children cumulated memory: 17436 KiB

[startup+3.10939 s]
/proc/loadavg: 2.68 2.18 2.06 3/202 10445
/proc/meminfo: memFree=16736520/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=55744 memory=32224 CPUtime=3.08 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 8271 0 1 0 305 3 0 0 20 0 1 0 433626344 57081856 8056 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 13936 8056 846 493 0 7241 0
Current children cumulated CPU time: 3.08 s
Current children cumulated vsize: 55744 KiB
Current children cumulated memory: 32224 KiB

[startup+6.30027 s]
/proc/loadavg: 2.63 2.18 2.06 3/202 10445
/proc/meminfo: memFree=16686388/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=85460 memory=61428 CPUtime=6.27 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 15668 0 1 0 623 4 0 0 20 0 1 0 433626344 87511040 15357 33554432000 4194304 6211092 140734925421088 140734925400184 5446912 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 21365 15357 846 493 0 14670 0
Current children cumulated CPU time: 6.27 s
Current children cumulated vsize: 85460 KiB
Current children cumulated memory: 61428 KiB

[startup+12.7003 s]
/proc/loadavg: 2.58 2.18 2.06 3/202 10445
/proc/meminfo: memFree=16586072/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=145296 memory=120356 CPUtime=12.67 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 30459 0 1 0 1261 6 0 0 20 0 1 0 433626344 148783104 30089 33554432000 4194304 6211092 140734925421088 140734925400184 139834420716391 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 36324 30089 846 493 0 29629 0
Current children cumulated CPU time: 12.67 s
Current children cumulated vsize: 145296 KiB
Current children cumulated memory: 120356 KiB

[startup+25.5003 s]
/proc/loadavg: 2.45 2.17 2.06 3/202 10445
/proc/meminfo: memFree=16367592/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=265972 memory=238424 CPUtime=25.47 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 59517 0 1 0 2538 9 0 0 20 0 1 0 433626344 272355328 59606 33554432000 4194304 6211092 140734925421088 140734925400184 5441082 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 66493 59606 846 493 0 59798 0
Current children cumulated CPU time: 25.47 s
Current children cumulated vsize: 265972 KiB
Current children cumulated memory: 238424 KiB

[startup+51.1095 s]
/proc/loadavg: 2.30 2.15 2.06 3/202 10448
/proc/meminfo: memFree=15926320/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=498580 memory=474152 CPUtime=51.08 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 116502 0 1 0 5091 17 0 0 20 0 1 0 433626344 510545920 118538 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 124645 118538 846 493 0 117950 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 498580 KiB
Current children cumulated memory: 474152 KiB

[startup+102.3 s]
/proc/loadavg: 2.13 2.13 2.05 3/202 10452
/proc/meminfo: memFree=15061964/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=976092 memory=946592 CPUtime=102.27 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 230040 0 1 0 10194 33 0 0 20 0 1 0 433626344 999518208 236648 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 244023 236648 846 493 0 237328 0
Current children cumulated CPU time: 102.27 s
Current children cumulated vsize: 976092 KiB
Current children cumulated memory: 946592 KiB

[startup+162.3 s]
/proc/loadavg: 2.05 2.11 2.05 3/202 10456
/proc/meminfo: memFree=14049216/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=1528320 memory=1498660 CPUtime=162.27 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 362445 0 1 0 16177 50 0 0 20 0 1 0 433626344 1564999680 374665 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 382080 374665 846 493 0 375385 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 1528320 KiB
Current children cumulated memory: 1498660 KiB

[startup+222.3 s]
/proc/loadavg: 2.02 2.09 2.05 3/202 10461
/proc/meminfo: memFree=13036452/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=2093000 memory=2051012 CPUtime=222.26 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 494908 0 1 0 22159 67 0 0 20 0 1 0 433626344 2143232000 512753 33554432000 4194304 6211092 140734925421088 140734925400184 139834419788606 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 523250 512753 846 493 0 516555 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 2093000 KiB
Current children cumulated memory: 2051012 KiB

[startup+282.3 s]
/proc/loadavg: 2.01 2.07 2.05 3/201 10466
/proc/meminfo: memFree=12026168/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=2676212 memory=2603356 CPUtime=282.26 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 627402 0 1 0 28141 85 0 0 20 0 1 0 433626344 2740441088 650839 33554432000 4194304 6211092 140734925421088 140734925400184 5013034 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 669053 650839 846 493 0 662358 0
Current children cumulated CPU time: 282.26 s
Current children cumulated vsize: 2676212 KiB
Current children cumulated memory: 2603356 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.06 2.05 3/199 10470
/proc/meminfo: memFree=11011700/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=3204512 memory=3155908 CPUtime=342.26 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 759482 0 1 0 34125 101 0 0 20 0 1 0 433626344 3281420288 788977 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 801128 788977 846 493 0 794433 0
Current children cumulated CPU time: 342.26 s
Current children cumulated vsize: 3204512 KiB
Current children cumulated memory: 3155908 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.05 2.05 3/199 10476
/proc/meminfo: memFree=9999344/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=3731588 memory=3707212 CPUtime=402.25 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 891835 0 1 0 40104 121 0 0 20 0 1 0 433626344 3821146112 926803 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 932897 926803 846 493 0 926202 0
Current children cumulated CPU time: 402.25 s
Current children cumulated vsize: 3731588 KiB
Current children cumulated memory: 3707212 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 10481
/proc/meminfo: memFree=8985304/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=4342884 memory=4261116 CPUtime=462.26 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 1024305 0 1 0 46084 142 0 0 20 0 1 0 433626344 4447113216 1065279 33554432000 4194304 6211092 140734925421088 140734925400184 5441082 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 1085721 1065279 846 493 0 1079026 0
Current children cumulated CPU time: 462.26 s
Current children cumulated vsize: 4342884 KiB
Current children cumulated memory: 4261116 KiB


################
# More data... #
################

Current children cumulated vsize: 10447248 KiB
Current children cumulated memory: 10324488 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/199 10759
/proc/meminfo: memFree=17163824/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=10972872 memory=10874712 CPUtime=1182.13 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 2606719 0 1 0 117756 457 0 0 20 0 1 0 433626344 11236220928 2718678 33554432000 4194304 6211092 140734925421088 140734925400184 139834420713396 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 2743218 2718678 846 493 0 2736523 0
Current children cumulated CPU time: 1182.13 s
Current children cumulated vsize: 10972872 KiB
Current children cumulated memory: 10874712 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/199 10763
/proc/meminfo: memFree=16611704/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=11498496 memory=11425148 CPUtime=1242.13 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 2738374 0 1 0 123723 490 0 0 20 0 1 0 433626344 11774459904 2856287 33554432000 4194304 6211092 140734925421088 140734925400184 139834420717104 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 2874624 2856287 846 493 0 2867929 0
Current children cumulated CPU time: 1242.13 s
Current children cumulated vsize: 11498496 KiB
Current children cumulated memory: 11425148 KiB

[startup+1302.3 s]
/proc/loadavg: 2.41 2.13 2.08 3/199 10769
/proc/meminfo: memFree=16059844/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=12028716 memory=11976028 CPUtime=1302.11 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 2870172 0 1 0 129690 521 0 0 20 0 1 0 433626344 12317405184 2994007 33554432000 4194304 6211092 140734925421088 140734925400184 5013073 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3007179 2994007 846 493 0 3000484 0
Current children cumulated CPU time: 1302.11 s
Current children cumulated vsize: 12028716 KiB
Current children cumulated memory: 11976028 KiB

[startup+1362.3 s]
/proc/loadavg: 2.15 2.11 2.08 3/199 10773
/proc/meminfo: memFree=15510188/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=12554208 memory=12524772 CPUtime=1362.1 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 3001990 0 1 0 135660 550 0 0 20 0 1 0 433626344 12855508992 3131193 33554432000 4194304 6211092 140734925421088 140734925400184 139834419792869 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3138552 3131193 846 493 0 3131857 0
Current children cumulated CPU time: 1362.1 s
Current children cumulated vsize: 12554208 KiB
Current children cumulated memory: 12524772 KiB

[startup+1422.3 s]
/proc/loadavg: 2.09 2.10 2.08 3/199 10777
/proc/meminfo: memFree=14956868/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=13360244 memory=13077240 CPUtime=1422.09 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 3134181 0 1 0 141629 580 0 0 20 0 1 0 433626344 13680889856 3269310 33554432000 4194304 6211092 140734925421088 140734925400184 5452207 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3340061 3269310 846 493 0 3333366 0
Current children cumulated CPU time: 1422.09 s
Current children cumulated vsize: 13360244 KiB
Current children cumulated memory: 13077240 KiB

[startup+1482.3 s]
/proc/loadavg: 2.03 2.08 2.07 3/199 10783
/proc/meminfo: memFree=14405460/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=13885736 memory=13627280 CPUtime=1482.08 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 3265787 0 1 0 147596 612 0 0 20 0 1 0 433626344 14218993664 3406820 33554432000 4194304 6211092 140734925421088 140734925400184 5446630 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3471434 3406820 846 493 0 3464739 0
Current children cumulated CPU time: 1482.08 s
Current children cumulated vsize: 13885736 KiB
Current children cumulated memory: 13627280 KiB

[startup+1542.3 s]
/proc/loadavg: 2.01 2.07 2.07 3/199 10787
/proc/meminfo: memFree=13854428/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=14410832 memory=14177452 CPUtime=1542.07 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 3397379 0 1 0 153555 652 0 0 20 0 1 0 433626344 14756691968 3544363 33554432000 4194304 6211092 140734925421088 140734925400184 5442964 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3602708 3544363 846 493 0 3596013 0
Current children cumulated CPU time: 1542.07 s
Current children cumulated vsize: 14410832 KiB
Current children cumulated memory: 14177452 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.06 2.06 3/199 10791
/proc/meminfo: memFree=13303372/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=14935532 memory=14726704 CPUtime=1602.06 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 3528818 0 1 0 159516 690 0 0 20 0 1 0 433626344 15293984768 3681676 33554432000 4194304 6211092 140734925421088 140734925400184 139834419788606 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3733883 3681676 846 493 0 3727188 0
Current children cumulated CPU time: 1602.06 s
Current children cumulated vsize: 14935532 KiB
Current children cumulated memory: 14726704 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.05 2.05 3/199 10797
/proc/meminfo: memFree=12751828/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=15459968 memory=15276220 CPUtime=1662.04 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 3660267 0 1 0 165472 732 0 0 20 0 1 0 433626344 15831007232 3819055 33554432000 4194304 6211092 140734925421088 140734925400184 5456298 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3864992 3819055 846 493 0 3858297 0
Current children cumulated CPU time: 1662.04 s
Current children cumulated vsize: 15459968 KiB
Current children cumulated memory: 15276220 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1709.5 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 10801
/proc/meminfo: memFree=12321396/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=15872600 memory=15707584 CPUtime=1709.22 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) R 10438 10441 9363 0 -1 4202496 3763667 0 1 0 170158 764 0 0 20 0 1 0 433626344 16253542400 3926896 33554432000 4194304 6211092 140734925421088 140734925400184 5451572 0 0 4096 8405026 0 0 0 17 3 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3968150 3926896 846 493 0 3961455 0
Current children cumulated CPU time: 1709.22 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15707584 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1712.31 s]
# the end of solver process 10441 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1701.88 s, system=8.47806 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1710.4 s]
/proc/loadavg: 2.00 2.04 2.05 3/200 10802
/proc/meminfo: memFree=12045700/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=15872600 memory=15707584 CPUtime=1709.37 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) S 10438 10441 9363 0 -1 512 3763671 0 1 0 170171 766 0 0 20 0 1 0 433626344 16253542400 3926896 33554432000 4194304 6211092 140734925421088 140734925400184 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3968150 3926896 846 493 0 3961455 0
Current children cumulated CPU time: 1709.37 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15707584 KiB

[startup+1711.2 s]
/proc/loadavg: 2.00 2.04 2.05 4/200 10802
/proc/meminfo: memFree=11480120/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=15872600 memory=15707584 CPUtime=1709.49 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) S 10438 10441 9363 0 -1 512 3763671 0 1 0 170183 766 0 0 20 0 1 0 433626344 16253542400 3926896 33554432000 4194304 6211092 140734925421088 140734925400184 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3968150 3926896 846 493 0 3961455 0
Current children cumulated CPU time: 1709.49 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15707584 KiB

[startup+1711.4 s]
/proc/loadavg: 2.00 2.04 2.05 4/200 10802
/proc/meminfo: memFree=11480120/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=15872600 memory=15707584 CPUtime=1709.53 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) S 10438 10441 9363 0 -1 512 3763671 0 1 0 170186 767 0 0 20 0 1 0 433626344 16253542400 3926896 33554432000 4194304 6211092 140734925421088 140734925400184 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3968150 3926896 846 493 0 3961455 0
Current children cumulated CPU time: 1709.53 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15707584 KiB

[startup+1711.5 s]
/proc/loadavg: 2.00 2.04 2.05 4/200 10802
/proc/meminfo: memFree=11480120/32770624 swapFree=66939884/67108860
[pid=10441] ppid=10438 vsize=15872600 memory=15707584 CPUtime=1709.55 cores=1,3,5,7
/proc/10441/stat : 10441 (cosoco) S 10438 10441 9363 0 -1 512 3763671 0 1 0 170188 767 0 0 20 0 1 0 433626344 16253542400 3926896 33554432000 4194304 6211092 140734925421088 140734925400184 5189768 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 1 0 0 8312316 8317760 28717056 140734925427723 140734925427825 140734925427825 140734925430734 0
/proc/10441/statm: 3968150 3926896 846 493 0 3961455 0
Current children cumulated CPU time: 1709.55 s
Current children cumulated vsize: 15872600 KiB
Current children cumulated memory: 15707584 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1712.31
CPU time (s): 1710.36
CPU user time (s): 1701.88
CPU system time (s): 8.47806
CPU usage (%): 99.8863
Max. virtual memory (cumulated for all children) (KiB): 15872600
Max. memory (cumulated for all children) (KiB): 15707584

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1701.88
system time used= 8.47806
maximum resident set size= 15707644
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3763671
page faults= 1
swaps= 0
block input operations= 9936
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 137295
involuntary context switches= 1712


# summary of solver processes directly reported to runsolver:
#   pid: 10441
#   total CPU time (s): 1710.36
#   total CPU user time (s): 1701.88
#   total CPU system time (s): 8.47806

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.97537 second user time and 11.5593 second system time

The end

Launcher Data 

Begin job on node119 at 2019-09-15 21:51:15
IDJOB=4397057
IDBENCH=122994
IDSOLVER=2848
FILE ID=node119/4397057-1568577114
RUNJOBID= node119-1568572745-9395
PBS_JOBID= 21703330
Free space on /tmp= 60900 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-038-010.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4397057-1568577114/watcher-4397057-1568577114 -o /tmp/evaluation-result-4397057-1568577114/solver-4397057-1568577114 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node119-1568572745-9395 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4397057-1568577114.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8e449228b7b3cf049a5fccac2aa13fe6
RANDOM SEED=420041215

node119.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        15818860 kB
MemAvailable:   20539428 kB
Buffers:          133812 kB
Cached:          4860752 kB
SwapCached:        61096 kB
Active:         13212068 kB
Inactive:        3094340 kB
Active(anon):   10743228 kB
Inactive(anon):   618240 kB
Active(file):    2468840 kB
Inactive(file):  2476100 kB
Unevictable:      106932 kB
Mlocked:          106932 kB
SwapTotal:      67108860 kB
SwapFree:       66939884 kB
Dirty:              3140 kB
Writeback:         81392 kB
AnonPages:      11363240 kB
Mapped:            50316 kB
Shmem:             41008 kB
Slab:             222212 kB
SReclaimable:     182928 kB
SUnreclaim:        39284 kB
KernelStack:        3728 kB
PageTables:        27296 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:   11783132 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:   2029568 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      126336 kB
DirectMap2M:    33419264 kB

Free space on /tmp at the end= 60832 MiB
End job on node119 at 2019-09-15 22:19:47