Trace number 4409031

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0? (MO) 2104.77 2106.9

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-04-350-100.xml
MD5SUM05a37308653f1418d8a9d31e27700b02
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark9
Best CPU time to get the best result obtained on this benchmark10.3495
Satisfiable
(Un)Satisfiability was proved
Number of variables1401
Number of constraints11
Number of domains2
Minimum domain size2
Maximum domain size342
Distribution of domain sizes[{"size":2,"count":1400},{"size":342,"count":1}]
Minimum variable degree5
Maximum variable degree7
Distribution of variable degrees[{"degree":5,"count":1400},{"degree":7,"count":1}]
Minimum constraint arity350
Maximum constraint arity1400
Distribution of constraint arities[{"arity":350,"count":4},{"arity":701,"count":6},{"arity":1400,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":10}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14485 MiB free)
  memory of node 1: 16384 MiB (14629 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409031-1569792426/watcher-4409031-1569792426 -o /tmp/evaluation-result-4409031-1569792426/solver-4409031-1569792426 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node124-1569787351-8456 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4409031-1569792426.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=9393, runsolver pid=9390
Current StackSize limit: 8192 KiB


[startup+0.100151 s]*
/proc/loadavg: 2.16 2.03 2.01 3/199 9394
/proc/meminfo: memFree=29811916/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=29216 memory=5636 CPUtime=0.1 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 1608 0 0 0 10 0 0 0 20 0 1 0 555182607 29917184 1409 33554432000 4194304 6211132 140724100389136 140724100385672 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 7304 1409 846 493 0 609 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 29216 KiB
Current children cumulated memory: 5636 KiB

[startup+0.209539 s]*
/proc/loadavg: 2.16 2.03 2.01 3/199 9394
/proc/meminfo: memFree=29811172/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=30008 memory=6428 CPUtime=0.21 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 1814 0 0 0 21 0 0 0 20 0 1 0 555182607 30728192 1607 33554432000 4194304 6211132 140724100389136 140724100385672 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 7502 1607 846 493 0 807 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 30008 KiB
Current children cumulated memory: 6428 KiB

[startup+0.308915 s]*
/proc/loadavg: 2.16 2.03 2.01 3/199 9394
/proc/meminfo: memFree=29810428/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=30800 memory=7220 CPUtime=0.31 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 2002 0 0 0 31 0 0 0 20 0 1 0 555182607 31539200 1805 33554432000 4194304 6211132 140724100389136 140724100385672 140406862334824 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 7700 1805 846 493 0 1005 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30800 KiB
Current children cumulated memory: 7220 KiB

[startup+0.700269 s]
/proc/loadavg: 2.16 2.03 2.01 3/199 9394
/proc/meminfo: memFree=29808940/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=33704 memory=10124 CPUtime=0.69 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 2726 0 0 0 69 0 0 0 20 0 1 0 555182607 34512896 2531 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 8426 2531 846 493 0 1731 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33704 KiB
Current children cumulated memory: 10124 KiB

[startup+1.50027 s]
/proc/loadavg: 2.16 2.03 2.01 3/199 9394
/proc/meminfo: memFree=29805280/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=39644 memory=16196 CPUtime=1.49 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 4235 0 0 0 149 0 0 0 20 0 1 0 555182607 40595456 4049 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 9911 4049 846 493 0 3216 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 39644 KiB
Current children cumulated memory: 16196 KiB

[startup+3.10934 s]
/proc/loadavg: 2.16 2.03 2.01 3/199 9394
/proc/meminfo: memFree=29789904/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=51852 memory=28376 CPUtime=3.1 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 7456 0 0 0 309 1 0 0 20 0 1 0 555182607 53096448 7094 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 12963 7094 846 493 0 6268 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 51852 KiB
Current children cumulated memory: 28376 KiB

[startup+6.30027 s]
/proc/loadavg: 2.14 2.03 2.01 3/199 9394
/proc/meminfo: memFree=29767648/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=75868 memory=52420 CPUtime=6.29 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 13640 0 0 0 627 2 0 0 20 0 1 0 555182607 77688832 13105 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 18967 13105 846 493 0 12272 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 75868 KiB
Current children cumulated memory: 52420 KiB

[startup+12.7003 s]
/proc/loadavg: 2.13 2.03 2.01 3/200 9397
/proc/meminfo: memFree=29721876/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=124232 memory=100456 CPUtime=12.69 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 25866 0 0 0 1266 3 0 0 20 0 1 0 555182607 127213568 25114 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 31058 25114 846 493 0 24363 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 124232 KiB
Current children cumulated memory: 100456 KiB

[startup+25.5003 s]
/proc/loadavg: 2.10 2.03 2.01 3/199 9397
/proc/meminfo: memFree=29624464/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=220480 memory=197004 CPUtime=25.49 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 50318 0 0 0 2542 7 0 0 20 0 1 0 555182607 225771520 49251 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 55120 49251 846 493 0 48425 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 220480 KiB
Current children cumulated memory: 197004 KiB

[startup+51.1094 s]
/proc/loadavg: 2.07 2.03 2.01 3/199 9397
/proc/meminfo: memFree=29427984/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=413740 memory=389840 CPUtime=51.1 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 98724 0 0 0 5096 14 0 0 20 0 1 0 555182607 423669760 97460 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 103435 97460 846 493 0 96740 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 413740 KiB
Current children cumulated memory: 389840 KiB

[startup+102.3 s]
/proc/loadavg: 2.03 2.02 2.01 3/200 9404
/proc/meminfo: memFree=29042428/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=800260 memory=775696 CPUtime=102.27 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 195278 0 0 0 10198 29 0 0 20 0 1 0 555182607 819466240 193924 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 200065 193924 846 493 0 193370 0
Current children cumulated CPU time: 102.27 s
Current children cumulated vsize: 800260 KiB
Current children cumulated memory: 775696 KiB

[startup+162.3 s]
/proc/loadavg: 2.01 2.02 2.01 3/199 9408
/proc/meminfo: memFree=28589012/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=1252540 memory=1227664 CPUtime=162.27 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 308300 0 0 0 16181 46 0 0 20 0 1 0 555182607 1282600960 306916 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 313135 306916 846 493 0 306440 0
Current children cumulated CPU time: 162.27 s
Current children cumulated vsize: 1252540 KiB
Current children cumulated memory: 1227664 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/200 9484
/proc/meminfo: memFree=27899392/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=1705900 memory=1681360 CPUtime=222.26 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 421220 0 0 0 22165 61 0 0 20 0 1 0 555182607 1746841600 420340 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 426475 420340 846 493 0 419780 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 1705900 KiB
Current children cumulated memory: 1681360 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/201 9490
/proc/meminfo: memFree=27000532/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=2155888 memory=2131744 CPUtime=282.24 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 533763 0 0 0 28147 77 0 0 20 0 1 0 555182607 2207629312 532936 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 538972 532936 846 493 0 532277 0
Current children cumulated CPU time: 282.24 s
Current children cumulated vsize: 2155888 KiB
Current children cumulated memory: 2131744 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/201 9495
/proc/meminfo: memFree=26099560/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=2610868 memory=2583668 CPUtime=342.24 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 646767 0 0 0 34128 96 0 0 20 0 1 0 555182607 2673528832 645917 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 652717 645917 846 493 0 646022 0
Current children cumulated CPU time: 342.24 s
Current children cumulated vsize: 2610868 KiB
Current children cumulated memory: 2583668 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/201 9499
/proc/meminfo: memFree=25198888/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=3060988 memory=3035932 CPUtime=402.23 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 759426 0 0 0 40109 114 0 0 20 0 1 0 555182607 3134451712 758983 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 765247 758983 846 493 0 758552 0
Current children cumulated CPU time: 402.23 s
Current children cumulated vsize: 3060988 KiB
Current children cumulated memory: 3035932 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/202 9505
/proc/meminfo: memFree=24298824/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=3518268 memory=3487456 CPUtime=462.22 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 872583 0 0 0 46089 133 0 0 20 0 1 0 555182607 3602706432 871864 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 879567 871864 846 493 0 872872 0
Current children cumulated CPU time: 462.22 s
Current children cumulated vsize: 3518268 KiB
Current children cumulated memory: 3487456 KiB


################
# More data... #
################

Current children cumulated vsize: 11192764 KiB
Current children cumulated memory: 11167172 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.00 2.01 3/200 9604
/proc/meminfo: memFree=8065228/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=11642488 memory=11618628 CPUtime=1542.06 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 2907408 0 0 0 153644 562 0 0 20 0 1 0 555182607 11921907712 2904657 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 2910622 2904657 846 493 0 2903927 0
Current children cumulated CPU time: 1542.06 s
Current children cumulated vsize: 11642488 KiB
Current children cumulated memory: 11618628 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.00 2.01 3/199 9620
/proc/meminfo: memFree=7162744/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=12116420 memory=12070276 CPUtime=1602.04 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3020420 0 0 0 159620 584 0 0 20 0 1 0 555182607 12407214080 3017569 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3029105 3017569 846 493 0 3022410 0
Current children cumulated CPU time: 1602.04 s
Current children cumulated vsize: 12116420 KiB
Current children cumulated memory: 12070276 KiB

[startup+1662.3 s]
/proc/loadavg: 1.95 1.98 2.00 3/200 9625
/proc/meminfo: memFree=6262712/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=12565484 memory=12521400 CPUtime=1662.03 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3133289 0 0 0 165593 610 0 0 20 0 1 0 555182607 12867055616 3130350 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3141371 3130350 846 493 0 3134676 0
Current children cumulated CPU time: 1662.03 s
Current children cumulated vsize: 12565484 KiB
Current children cumulated memory: 12521400 KiB

[startup+1722.3 s]
/proc/loadavg: 2.02 2.00 2.01 3/201 9632
/proc/meminfo: memFree=5365200/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=13014944 memory=12973052 CPUtime=1722.03 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3246229 0 0 0 171565 638 0 0 20 0 1 0 555182607 13327302656 3243263 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3253736 3243263 846 493 0 3247041 0
Current children cumulated CPU time: 1722.03 s
Current children cumulated vsize: 13014944 KiB
Current children cumulated memory: 12973052 KiB

[startup+1782.3 s]
/proc/loadavg: 2.01 2.00 2.01 3/200 9636
/proc/meminfo: memFree=4465876/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=13464404 memory=13424440 CPUtime=1782.02 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3359202 0 0 0 177536 666 0 0 20 0 1 0 555182607 13787549696 3356110 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3366101 3356110 846 493 0 3359406 0
Current children cumulated CPU time: 1782.02 s
Current children cumulated vsize: 13464404 KiB
Current children cumulated memory: 13424440 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.00 2.01 3/200 9640
/proc/meminfo: memFree=3567628/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=13913468 memory=13875564 CPUtime=1842.01 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3472059 0 0 0 183507 694 0 0 20 0 1 0 555182607 14247391232 3468891 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3478367 3468891 846 493 0 3471672 0
Current children cumulated CPU time: 1842.01 s
Current children cumulated vsize: 13913468 KiB
Current children cumulated memory: 13875564 KiB

[startup+1902.3 s]
/proc/loadavg: 1.88 1.97 2.00 3/201 9647
/proc/meminfo: memFree=2670504/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=14362268 memory=14326424 CPUtime=1902.01 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3584853 0 0 0 189479 722 0 0 20 0 1 0 555182607 14706962432 3581606 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3590567 3581606 846 493 0 3583872 0
Current children cumulated CPU time: 1902.01 s
Current children cumulated vsize: 14362268 KiB
Current children cumulated memory: 14326424 KiB

[startup+1962.3 s]
/proc/loadavg: 1.95 1.98 2.00 3/199 9651
/proc/meminfo: memFree=1768836/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=14811332 memory=14777604 CPUtime=1962 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3697817 0 0 0 195450 750 0 0 20 0 1 0 555182607 15166803968 3694401 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3702833 3694401 846 493 0 3696138 0
Current children cumulated CPU time: 1962 s
Current children cumulated vsize: 14811332 KiB
Current children cumulated memory: 14777604 KiB

[startup+2022.3 s]
/proc/loadavg: 1.98 1.98 2.00 3/200 9663
/proc/meminfo: memFree=867216/32770624 swapFree=66976804/67108860
[pid=9393] ppid=9390 vsize=15260264 memory=15228464 CPUtime=2021.98 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3810627 0 0 0 201416 782 0 0 20 0 1 0 555182607 15626510336 3807116 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3815066 3807116 846 493 0 3808371 0
Current children cumulated CPU time: 2021.98 s
Current children cumulated vsize: 15260264 KiB
Current children cumulated memory: 15228464 KiB

[startup+2082.3 s]
/proc/loadavg: 1.99 1.99 2.00 3/201 9694
/proc/meminfo: memFree=193688/32770624 swapFree=66976664/67108860
[pid=9393] ppid=9390 vsize=15708800 memory=15679016 CPUtime=2081.97 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3923383 0 0 0 207380 817 0 0 20 0 1 0 555182607 16085811200 3919754 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3927200 3919754 846 493 0 3920505 0
Current children cumulated CPU time: 2081.97 s
Current children cumulated vsize: 15708800 KiB
Current children cumulated memory: 15679016 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+2104.2 s]
/proc/loadavg: 2.00 1.99 2.00 3/200 9694
/proc/meminfo: memFree=195412/32770624 swapFree=66976580/67108860
[pid=9393] ppid=9390 vsize=15872612 memory=15842908 CPUtime=2103.88 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) R 9390 9393 8424 0 -1 4202496 3964680 0 0 0 209559 829 0 0 20 0 1 0 555182607 16253554688 3960727 33554432000 4194304 6211132 140724100389136 140724100248424 140406852389861 0 0 4096 8405026 0 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3968153 3960727 846 493 0 3961458 0
Current children cumulated CPU time: 2103.88 s
Current children cumulated vsize: 15872612 KiB
Current children cumulated memory: 15842908 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2106.9 s]
# the end of solver process 9393 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2095.77 s, system=9.00132 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2105.6 s]
/proc/loadavg: 2.08 2.00 2.00 3/201 9695
/proc/meminfo: memFree=190300/32770624 swapFree=66976580/67108860
[pid=9393] ppid=9390 vsize=15872616 memory=15842908 CPUtime=2103.99 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) S 9390 9393 8424 0 -1 512 3964685 0 0 0 209570 829 0 0 20 0 1 0 555182607 16253558784 3960727 33554432000 4194304 6211132 140724100389136 140724100248424 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3968154 3960727 846 493 0 3961459 0
Current children cumulated CPU time: 2103.99 s
Current children cumulated vsize: 15872616 KiB
Current children cumulated memory: 15842908 KiB

[startup+2106 s]*
/proc/loadavg: 2.08 2.00 2.00 3/201 9695
/proc/meminfo: memFree=202996/32770624 swapFree=66976580/67108860
[pid=9393] ppid=9390 vsize=15872616 memory=15842908 CPUtime=2104.03 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) S 9390 9393 8424 0 -1 512 3964685 0 0 0 209574 829 0 0 20 0 1 0 555182607 16253558784 3960727 33554432000 4194304 6211132 140724100389136 140724100248424 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3968154 3960727 846 493 0 3961459 0
Current children cumulated CPU time: 2104.03 s
Current children cumulated vsize: 15872616 KiB
Current children cumulated memory: 15842908 KiB

[startup+2106.2 s]
/proc/loadavg: 2.08 2.00 2.00 3/201 9695
/proc/meminfo: memFree=202996/32770624 swapFree=66976580/67108860
[pid=9393] ppid=9390 vsize=15872616 memory=15842908 CPUtime=2104.06 cores=1,3,5,7
/proc/9393/stat : 9393 (cosoco) S 9390 9393 8424 0 -1 512 3964685 0 0 0 209576 830 0 0 20 0 1 0 555182607 16253558784 3960727 33554432000 4194304 6211132 140724100389136 140724100248424 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 3 0 0 0 0 0 8312316 8317760 19775488 140724100391947 140724100392049 140724100392049 140724100394958 0
/proc/9393/statm: 3968154 3960727 846 493 0 3961459 0
Current children cumulated CPU time: 2104.06 s
Current children cumulated vsize: 15872616 KiB
Current children cumulated memory: 15842908 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2106.9
CPU time (s): 2104.77
CPU user time (s): 2095.77
CPU system time (s): 9.00132
CPU usage (%): 99.8986
Max. virtual memory (cumulated for all children) (KiB): 15872616
Max. memory (cumulated for all children) (KiB): 15842908

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2095.77
system time used= 9.00132
maximum resident set size= 15843100
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3964685
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 78772
involuntary context switches= 1740


# summary of solver processes directly reported to runsolver:
#   pid: 9393
#   total CPU time (s): 2104.77
#   total CPU user time (s): 2095.77
#   total CPU system time (s): 9.00132

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.35293 second user time and 14.1457 second system time

The end

Launcher Data 

Begin job on node124 at 2019-09-29 23:26:24
IDJOB=4409031
IDBENCH=122932
IDSOLVER=2876
FILE ID=node124/4409031-1569792426
RUNJOBID= node124-1569787351-8456
PBS_JOBID= 21705829
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-04-350-100.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409031-1569792426/watcher-4409031-1569792426 -o /tmp/evaluation-result-4409031-1569792426/solver-4409031-1569792426 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node124-1569787351-8456 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4409031-1569792426.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 05a37308653f1418d8a9d31e27700b02
RANDOM SEED=236993948

node124.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.47
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        29814188 kB
MemAvailable:   31844288 kB
Buffers:          210640 kB
Cached:          2158624 kB
SwapCached:        29076 kB
Active:          1610508 kB
Inactive:         795532 kB
Active(anon):      26460 kB
Inactive(anon):    92752 kB
Active(file):    1584048 kB
Inactive(file):   702780 kB
Unevictable:      107652 kB
Mlocked:          107652 kB
SwapTotal:      67108860 kB
SwapFree:       66976804 kB
Dirty:              2812 kB
Writeback:             0 kB
AnonPages:         69464 kB
Mapped:            28072 kB
Shmem:             73680 kB
Slab:             147212 kB
SReclaimable:     118212 kB
SUnreclaim:        29000 kB
KernelStack:        3760 kB
PageTables:         5836 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     569528 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     22528 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      112000 kB
DirectMap2M:    33433600 kB

Free space on /tmp at the end= 61724 MiB
End job on node124 at 2019-09-30 00:01:31