Trace number 4390277

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2? (MO) 1696.24 1698.44

General information on the benchmark

NameOpd/Opd-sum-small/
Opd-10-038-010.xml
MD5SUM8e449228b7b3cf049a5fccac2aa13fe6
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark2
Best CPU time to get the best result obtained on this benchmark4.37498
Satisfiable
(Un)Satisfiability was proved
Number of variables381
Number of constraints56
Number of domains2
Minimum domain size2
Maximum domain size37
Distribution of domain sizes[{"size":2,"count":380},{"size":37,"count":1}]
Minimum variable degree11
Maximum variable degree46
Distribution of variable degrees[{"degree":11,"count":380},{"degree":46,"count":1}]
Minimum constraint arity38
Maximum constraint arity380
Distribution of constraint arities[{"arity":38,"count":10},{"arity":77,"count":45},{"arity":380,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":55}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (12126 MiB free)
  memory of node 1: 16384 MiB (3452 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4390277-1568337335/watcher-4390277-1568337335 -o /tmp/evaluation-result-4390277-1568337335/solver-4390277-1568337335 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node139-1568325638-27860 --watchdog 2560 cosoco -lc -model -restarts -stick HOME/instance-4390277-1568337335.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=29964, runsolver pid=29961
Current StackSize limit: 8192 KiB


[startup+0.100192 s]*
/proc/loadavg: 2.32 2.10 2.07 3/199 29965
/proc/meminfo: memFree=15952048/32770624 swapFree=64675640/67108860
[pid=29964] ppid=29961 vsize=28416 memory=4740 CPUtime=0.1 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 1407 0 1 0 10 0 0 0 20 0 1 0 410001174 29097984 1185 33554432000 4194304 6210612 140721931096080 140721931095272 5440420 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 7104 1185 844 493 0 409 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 28416 KiB
Current children cumulated memory: 4740 KiB

[startup+0.281032 s]*
/proc/loadavg: 2.32 2.10 2.07 3/199 29965
/proc/meminfo: memFree=15953388/32770624 swapFree=64675640/67108860
[pid=29964] ppid=29961 vsize=30212 memory=6588 CPUtime=0.28 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 1828 0 1 0 28 0 0 0 20 0 1 0 410001174 30937088 1647 33554432000 4194304 6210612 140721931096080 140721931095272 139904886647781 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 7553 1647 845 493 0 858 0
Current children cumulated CPU time: 0.28 s
Current children cumulated vsize: 30212 KiB
Current children cumulated memory: 6588 KiB

[startup+0.309679 s]*
/proc/loadavg: 2.32 2.10 2.07 3/199 29965
/proc/meminfo: memFree=15953016/32770624 swapFree=64675640/67108860
[pid=29964] ppid=29961 vsize=30476 memory=6852 CPUtime=0.31 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 1892 0 1 0 31 0 0 0 20 0 1 0 410001174 31207424 1713 33554432000 4194304 6210612 140721931096080 140721931095272 139904886647781 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 7619 1713 845 493 0 924 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30476 KiB
Current children cumulated memory: 6852 KiB

[startup+0.70038 s]
/proc/loadavg: 2.32 2.10 2.07 3/199 29965
/proc/meminfo: memFree=16192656/32770624 swapFree=64675640/67108860
[pid=29964] ppid=29961 vsize=33984 memory=10420 CPUtime=0.69 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 2775 0 1 0 69 0 0 0 20 0 1 0 410001174 34799616 2605 33554432000 4194304 6210612 140721931096080 140721931075176 5452082 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 8496 2605 845 493 0 1801 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33984 KiB
Current children cumulated memory: 10420 KiB

[startup+1.50035 s]
/proc/loadavg: 2.32 2.10 2.07 3/198 29965
/proc/meminfo: memFree=16191884/32770624 swapFree=64675716/67108860
[pid=29964] ppid=29961 vsize=41588 memory=17696 CPUtime=1.48 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 4622 0 1 0 148 0 0 0 20 0 1 0 410001174 42586112 4424 33554432000 4194304 6210612 140721931096080 140721931075176 5012488 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 10397 4424 845 493 0 3702 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 41588 KiB
Current children cumulated memory: 17696 KiB

[startup+3.10947 s]
/proc/loadavg: 2.32 2.10 2.07 3/198 29968
/proc/meminfo: memFree=16398232/32770624 swapFree=64676008/67108860
[pid=29964] ppid=29961 vsize=56912 memory=32580 CPUtime=3.1 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 8399 0 1 0 309 1 0 0 20 0 1 0 410001174 58277888 8145 33554432000 4194304 6210612 140721931096080 140721931075176 139904886647781 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 14228 8145 845 493 0 7533 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 56912 KiB
Current children cumulated memory: 32580 KiB

[startup+6.30033 s]
/proc/loadavg: 2.13 2.07 2.06 3/198 29968
/proc/meminfo: memFree=16347700/32770624 swapFree=64676008/67108860
[pid=29964] ppid=29961 vsize=86120 memory=62220 CPUtime=6.28 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 15845 0 1 0 627 1 0 0 20 0 1 0 410001174 88186880 15555 33554432000 4194304 6210612 140721931096080 140721931075176 5452047 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 21530 15555 845 493 0 14835 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 86120 KiB
Current children cumulated memory: 62220 KiB

[startup+12.7003 s]
/proc/loadavg: 2.12 2.06 2.05 2/193 29969
/proc/meminfo: memFree=29619136/32770624 swapFree=66931936/67108860
[pid=29964] ppid=29961 vsize=145956 memory=121136 CPUtime=12.68 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 30631 0 1 0 1265 3 0 0 20 0 1 0 410001174 149458944 30284 33554432000 4194304 6210612 140721931096080 140721931075176 139904886647781 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 36489 30284 845 493 0 29794 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 145956 KiB
Current children cumulated memory: 121136 KiB

[startup+25.5003 s]
/proc/loadavg: 2.22 2.09 2.06 3/198 30035
/proc/meminfo: memFree=29712704/32770624 swapFree=66932116/67108860
[pid=29964] ppid=29961 vsize=267292 memory=239748 CPUtime=25.48 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 59836 0 1 0 2541 7 0 0 20 0 1 0 410001174 273707008 59937 33554432000 4194304 6210612 140721931096080 140721931075176 5453752 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 66823 59937 845 493 0 60128 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 267292 KiB
Current children cumulated memory: 239748 KiB

[startup+51.1094 s]
/proc/loadavg: 2.15 2.09 2.06 3/198 30036
/proc/meminfo: memFree=29172472/32770624 swapFree=66932116/67108860
[pid=29964] ppid=29961 vsize=500692 memory=476348 CPUtime=51.07 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 117538 0 1 0 5092 15 0 0 20 0 1 0 410001174 512708608 119087 33554432000 4194304 6210612 140721931096080 140721931075176 5455493 0 0 4096 8405026 0 0 0 17 0 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 125173 119087 845 493 0 118478 0
Current children cumulated CPU time: 51.07 s
Current children cumulated vsize: 500692 KiB
Current children cumulated memory: 476348 KiB

[startup+102.3 s]
/proc/loadavg: 2.01 2.06 2.05 3/198 30039
/proc/meminfo: memFree=28113516/32770624 swapFree=66932116/67108860
[pid=29964] ppid=29961 vsize=980184 memory=950116 CPUtime=102.25 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 231434 0 1 0 10192 33 0 0 20 0 1 0 410001174 1003708416 237529 33554432000 4194304 6210612 140721931096080 140721931075176 139904886647781 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 245046 237529 845 493 0 238351 0
Current children cumulated CPU time: 102.25 s
Current children cumulated vsize: 980184 KiB
Current children cumulated memory: 950116 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 30046
/proc/meminfo: memFree=26866248/32770624 swapFree=66932116/67108860
[pid=29964] ppid=29961 vsize=1536636 memory=1508712 CPUtime=162.23 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 364930 0 1 0 16171 52 0 0 20 0 1 0 410001174 1573515264 377178 33554432000 4194304 6210612 140721931096080 140721931075176 139904886647781 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 384159 377178 845 493 0 377464 0
Current children cumulated CPU time: 162.23 s
Current children cumulated vsize: 1536636 KiB
Current children cumulated memory: 1508712 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/200 30056
/proc/meminfo: memFree=25613240/32770624 swapFree=66932120/67108860
[pid=29964] ppid=29961 vsize=2105804 memory=2065528 CPUtime=222.21 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 498526 0 1 0 22152 69 0 0 20 0 1 0 410001174 2156343296 516382 33554432000 4194304 6210612 140721931096080 140721931075176 5446483 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 526451 516382 845 493 0 519756 0
Current children cumulated CPU time: 222.21 s
Current children cumulated vsize: 2105804 KiB
Current children cumulated memory: 2065528 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 30060
/proc/meminfo: memFree=24367608/32770624 swapFree=66932120/67108860
[pid=29964] ppid=29961 vsize=2693636 memory=2622384 CPUtime=282.19 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 632169 0 1 0 28131 88 0 0 20 0 1 0 410001174 2758283264 655596 33554432000 4194304 6210612 140721931096080 140721931075176 5454016 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 673409 655596 845 493 0 666714 0
Current children cumulated CPU time: 282.19 s
Current children cumulated vsize: 2693636 KiB
Current children cumulated memory: 2622384 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/200 30066
/proc/meminfo: memFree=23118472/32770624 swapFree=66932120/67108860
[pid=29964] ppid=29961 vsize=3226292 memory=3179216 CPUtime=342.18 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 765310 0 1 0 34115 103 0 0 20 0 1 0 410001174 3303723008 794804 33554432000 4194304 6210612 140721931096080 140721931075176 139904887573911 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 806573 794804 845 493 0 799878 0
Current children cumulated CPU time: 342.18 s
Current children cumulated vsize: 3226292 KiB
Current children cumulated memory: 3179216 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/200 30070
/proc/meminfo: memFree=21873644/32770624 swapFree=66932120/67108860
[pid=29964] ppid=29961 vsize=3840240 memory=3734016 CPUtime=402.16 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 898492 0 1 0 40094 122 0 0 20 0 1 0 410001174 3932405760 933504 33554432000 4194304 6210612 140721931096080 140721931075176 5446488 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 960060 933504 845 493 0 953365 0
Current children cumulated CPU time: 402.16 s
Current children cumulated vsize: 3840240 KiB
Current children cumulated memory: 3734016 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/199 30074
/proc/meminfo: memFree=20626752/32770624 swapFree=66932120/67108860
[pid=29964] ppid=29961 vsize=4373244 memory=4291036 CPUtime=462.15 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 1031732 0 1 0 46076 139 0 0 20 0 1 0 410001174 4478201856 1072759 33554432000 4194304 6210612 140721931096080 140721931075176 139904886653284 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 1093311 1072759 845 493 0 1086616 0
Current children cumulated CPU time: 462.15 s
Current children cumulated vsize: 4373244 KiB
Current children cumulated memory: 4291036 KiB


################
# More data... #
################

Current children cumulated vsize: 11056296 KiB
Current children cumulated memory: 10961520 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/197 30185
/proc/meminfo: memFree=4397744/32770624 swapFree=66932172/67108860
[pid=29964] ppid=29961 vsize=11586276 memory=11516844 CPUtime=1241.94 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 2760792 0 1 0 123766 428 0 0 20 0 1 0 410001174 11864346624 2879211 33554432000 4194304 6210612 140721931096080 140721931075176 139904896592710 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 2896569 2879211 845 493 0 2889874 0
Current children cumulated CPU time: 1241.94 s
Current children cumulated vsize: 11586276 KiB
Current children cumulated memory: 11516844 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/197 30189
/proc/meminfo: memFree=3154060/32770624 swapFree=66932172/67108860
[pid=29964] ppid=29961 vsize=12120984 memory=12072164 CPUtime=1301.92 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 2893693 0 1 0 129737 455 0 0 20 0 1 0 410001174 12411887616 3018041 33554432000 4194304 6210612 140721931096080 140721931075176 5446110 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3030246 3018041 845 493 0 3023551 0
Current children cumulated CPU time: 1301.92 s
Current children cumulated vsize: 12120984 KiB
Current children cumulated memory: 12072164 KiB

[startup+1362.3 s]
/proc/loadavg: 2.05 2.03 2.05 3/196 30193
/proc/meminfo: memFree=1903684/32770624 swapFree=66932172/67108860
[pid=29964] ppid=29961 vsize=12931112 memory=12628752 CPUtime=1361.9 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 3026990 0 1 0 135705 485 0 0 20 0 1 0 410001174 13241458688 3157188 33554432000 4194304 6210612 140721931096080 140721931075176 5012637 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3232778 3157188 845 493 0 3226083 0
Current children cumulated CPU time: 1361.9 s
Current children cumulated vsize: 12931112 KiB
Current children cumulated memory: 12628752 KiB

[startup+1422.3 s]
/proc/loadavg: 2.34 2.12 2.08 3/199 30273
/proc/meminfo: memFree=16765276/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=13460036 memory=13180184 CPUtime=1421.89 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 3159470 0 1 0 141676 513 0 0 20 0 1 0 410001174 13783076864 3295046 33554432000 4194304 6210612 140721931096080 140721931075176 139904887571290 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3365009 3295046 845 493 0 3358314 0
Current children cumulated CPU time: 1421.89 s
Current children cumulated vsize: 13460036 KiB
Current children cumulated memory: 13180184 KiB

[startup+1482.3 s]
/proc/loadavg: 2.13 2.10 2.07 3/199 30277
/proc/meminfo: memFree=16209656/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=13989752 memory=13734976 CPUtime=1481.87 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 3292251 0 1 0 147646 541 0 0 20 0 1 0 410001174 14325506048 3433744 33554432000 4194304 6210612 140721931096080 140721931075176 5446119 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3497438 3433744 845 493 0 3490743 0
Current children cumulated CPU time: 1481.87 s
Current children cumulated vsize: 13989752 KiB
Current children cumulated memory: 13734976 KiB

[startup+1542.3 s]
/proc/loadavg: 2.05 2.08 2.06 3/199 30281
/proc/meminfo: memFree=15653760/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=14519072 memory=14291340 CPUtime=1541.85 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 3424907 0 1 0 153611 574 0 0 20 0 1 0 410001174 14867529728 3572835 33554432000 4194304 6210612 140721931096080 140721931075176 5452001 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3629768 3572835 845 493 0 3623073 0
Current children cumulated CPU time: 1541.85 s
Current children cumulated vsize: 14519072 KiB
Current children cumulated memory: 14291340 KiB

[startup+1602.3 s]
/proc/loadavg: 2.02 2.06 2.06 3/199 30287
/proc/meminfo: memFree=15080924/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15048524 memory=14845604 CPUtime=1601.84 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 3557549 0 1 0 159578 606 0 0 20 0 1 0 410001174 15409688576 3711401 33554432000 4194304 6210612 140721931096080 140721931075176 139904887571305 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3762131 3711401 845 493 0 3755436 0
Current children cumulated CPU time: 1601.84 s
Current children cumulated vsize: 15048524 KiB
Current children cumulated memory: 14845604 KiB

[startup+1662.3 s]
/proc/loadavg: 2.01 2.05 2.05 3/199 30291
/proc/meminfo: memFree=14525732/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15577712 memory=15399608 CPUtime=1661.82 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 3690177 0 1 0 165542 640 0 0 20 0 1 0 410001174 15951577088 3849902 33554432000 4194304 6210612 140721931096080 140721931075176 139904886647781 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3894428 3849902 845 493 0 3887733 0
Current children cumulated CPU time: 1661.82 s
Current children cumulated vsize: 15577712 KiB
Current children cumulated memory: 15399608 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1695.7 s]
/proc/loadavg: 2.00 2.05 2.05 3/198 30295
/proc/meminfo: memFree=14222184/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15872204 memory=15708656 CPUtime=1695.21 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 4202496 3763967 0 1 0 168861 660 0 0 20 0 1 0 410001174 16253136896 3927164 33554432000 4194304 6210612 140721931096080 140721931075176 139904887573919 0 0 4096 8405026 0 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3968051 3927164 845 493 0 3961356 0
Current children cumulated CPU time: 1695.21 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708656 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1698.44 s]
# the end of solver process 29964 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1688.88 s, system=7.36135 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1696 s]*
/proc/loadavg: 2.00 2.05 2.05 3/199 30296
/proc/meminfo: memFree=14059012/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15872204 memory=15708920 CPUtime=1695.25 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) S 29961 29964 27829 0 -1 512 3763972 0 1 0 168864 661 0 0 20 0 1 0 410001174 16253136896 3927230 33554432000 4194304 6210612 140721931096080 140721931075176 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3968051 3927230 845 493 0 3961356 0
Current children cumulated CPU time: 1695.25 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708920 KiB

[startup+1696.8 s]
/proc/loadavg: 2.00 2.05 2.05 3/199 30296
/proc/meminfo: memFree=14059012/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15872204 memory=15708920 CPUtime=1695.36 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) S 29961 29964 27829 0 -1 512 3763972 0 1 0 168875 661 0 0 20 0 1 0 410001174 16253136896 3927230 33554432000 4194304 6210612 140721931096080 140721931075176 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3968051 3927230 845 493 0 3961356 0
Current children cumulated CPU time: 1695.36 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708920 KiB

[startup+1697.2 s]
/proc/loadavg: 2.00 2.05 2.05 3/199 30296
/proc/meminfo: memFree=13533872/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15872204 memory=15708920 CPUtime=1695.43 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) S 29961 29964 27829 0 -1 512 3763972 0 1 0 168881 662 0 0 20 0 1 0 410001174 16253136896 3927230 33554432000 4194304 6210612 140721931096080 140721931075176 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3968051 3927230 845 493 0 3961356 0
Current children cumulated CPU time: 1695.43 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708920 KiB

[startup+1697.6 s]
/proc/loadavg: 2.00 2.05 2.05 3/199 30296
/proc/meminfo: memFree=13533872/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15872204 memory=15708920 CPUtime=1695.48 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) R 29961 29964 27829 0 -1 512 3763972 0 1 0 168886 662 0 0 20 0 1 0 410001174 16253136896 3927230 33554432000 4194304 6210612 140721931096080 140721931075176 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3968051 3927230 845 493 0 3961356 0
Current children cumulated CPU time: 1695.48 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708920 KiB

[startup+1697.7 s]
/proc/loadavg: 2.00 2.05 2.05 3/199 30296
/proc/meminfo: memFree=13533872/32770624 swapFree=66932180/67108860
[pid=29964] ppid=29961 vsize=15872204 memory=15708920 CPUtime=1695.49 cores=0,2,4,6
/proc/29964/stat : 29964 (cosoco) S 29961 29964 27829 0 -1 512 3763972 0 1 0 168887 662 0 0 20 0 1 0 410001174 16253136896 3927230 33554432000 4194304 6210612 140721931096080 140721931075176 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 20410368 140721931097099 140721931097201 140721931097201 140721931100110 0
/proc/29964/statm: 3968051 3927230 845 493 0 3961356 0
Current children cumulated CPU time: 1695.49 s
Current children cumulated vsize: 15872204 KiB
Current children cumulated memory: 15708920 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1698.44
CPU time (s): 1696.24
CPU user time (s): 1688.88
CPU system time (s): 7.36135
CPU usage (%): 99.8705
Max. virtual memory (cumulated for all children) (KiB): 15872204
Max. memory (cumulated for all children) (KiB): 15708920

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1688.88
system time used= 7.36135
maximum resident set size= 15708924
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3763972
page faults= 1
swaps= 0
block input operations= 8
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 127206
involuntary context switches= 2287


# summary of solver processes directly reported to runsolver:
#   pid: 29964
#   total CPU time (s): 1696.24
#   total CPU user time (s): 1688.88
#   total CPU system time (s): 7.36135

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.34743 second user time and 11.6096 second system time

The end

Launcher Data 

Begin job on node139 at 2019-09-13 03:17:17
IDJOB=4390277
IDBENCH=122994
IDSOLVER=2831
FILE ID=node139/4390277-1568337335
RUNJOBID= node139-1568325638-27860
PBS_JOBID= 21701151
Free space on /tmp= 55104 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Opd/Opd-sum-small/Opd-10-038-010.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4390277-1568337335/watcher-4390277-1568337335 -o /tmp/evaluation-result-4390277-1568337335/solver-4390277-1568337335 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node139-1568325638-27860 --watchdog 2560  cosoco -lc -model -restarts -stick HOME/instance-4390277-1568337335.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 8e449228b7b3cf049a5fccac2aa13fe6
RANDOM SEED=2139852419

node139.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.57
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        15953364 kB
MemAvailable:   18177140 kB
Buffers:           62404 kB
Cached:          2421192 kB
SwapCached:       127240 kB
Active:         12657272 kB
Inactive:        3482104 kB
Active(anon):   12348580 kB
Inactive(anon):  1348004 kB
Active(file):     308692 kB
Inactive(file):  2134100 kB
Unevictable:      110720 kB
Mlocked:          110720 kB
SwapTotal:      67108860 kB
SwapFree:       64675640 kB
Dirty:              2588 kB
Writeback:         44852 kB
AnonPages:      13640528 kB
Mapped:            39348 kB
Shmem:             32168 kB
Slab:             240960 kB
SReclaimable:     193640 kB
SUnreclaim:        47320 kB
KernelStack:        3472 kB
PageTables:        36544 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:   16308476 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    100352 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      107904 kB
DirectMap2M:    33437696 kB

Free space on /tmp at the end= 54840 MiB
End job on node139 at 2019-09-13 03:45:36