Trace number 4409758

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0 parallel? (TO) 2520.6 2522.83

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-09-350-100.xml
MD5SUM9444e44f99a0ffd374c39bd698e45a3a
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark48
Best CPU time to get the best result obtained on this benchmark2520.13
Satisfiable
(Un)Satisfiability was proved
Number of variables3151
Number of constraints46
Number of domains2
Minimum domain size2
Maximum domain size330
Distribution of domain sizes[{"size":2,"count":3150},{"size":330,"count":1}]
Minimum variable degree10
Maximum variable degree37
Distribution of variable degrees[{"degree":10,"count":3150},{"degree":37,"count":1}]
Minimum constraint arity350
Maximum constraint arity3150
Distribution of constraint arities[{"arity":350,"count":9},{"arity":701,"count":36},{"arity":3150,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":45}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15413 MiB free)
  memory of node 1: 16384 MiB (15746 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409758-1569820272/watcher-4409758-1569820272 -o /tmp/evaluation-result-4409758-1569820272/solver-4409758-1569820272 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node149-1569816999-10284 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4409758-1569820272.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=10733, runsolver pid=10730
Current StackSize limit: 8192 KiB


[startup+0.100053 s]*
/proc/loadavg: 7.88 8.23 8.07 2/192 10734
/proc/meminfo: memFree=31891752/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=43500 memory=19984 CPUtime=0.09 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 5178 0 0 0 9 0 0 0 20 0 1 0 558265782 44544000 4996 33554432000 4194304 6211132 140722048679520 140722048676056 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 10875 4996 846 493 0 4180 0
Current children cumulated CPU time: 0.09 s
Current children cumulated vsize: 43500 KiB
Current children cumulated memory: 19984 KiB

[startup+0.208778 s]*
/proc/loadavg: 7.88 8.23 8.07 2/192 10734
/proc/meminfo: memFree=31890636/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=44292 memory=20776 CPUtime=0.2 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 5383 0 0 0 20 0 0 0 20 0 1 0 558265782 45355008 5194 33554432000 4194304 6211132 140722048679520 140722048676056 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 11073 5194 846 493 0 4378 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 44292 KiB
Current children cumulated memory: 20776 KiB

[startup+0.308551 s]*
/proc/loadavg: 7.88 8.23 8.07 2/192 10734
/proc/meminfo: memFree=31889892/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=45084 memory=21568 CPUtime=0.3 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 5570 0 0 0 30 0 0 0 20 0 1 0 558265782 46166016 5392 33554432000 4194304 6211132 140722048679520 140722048676056 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 11271 5392 846 493 0 4576 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 45084 KiB
Current children cumulated memory: 21568 KiB

[startup+0.700326 s]
/proc/loadavg: 7.88 8.23 8.07 2/192 10734
/proc/meminfo: memFree=31888404/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=47856 memory=24472 CPUtime=0.68 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 6286 0 0 0 68 0 0 0 20 0 1 0 558265782 49004544 6118 33554432000 4194304 6211132 140722048679520 140722048676056 5454215 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 11964 6118 846 493 0 5269 0
Current children cumulated CPU time: 0.68 s
Current children cumulated vsize: 47856 KiB
Current children cumulated memory: 24472 KiB

[startup+1.50032 s]
/proc/loadavg: 7.88 8.23 8.07 2/192 10734
/proc/meminfo: memFree=31885064/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=53796 memory=30280 CPUtime=1.49 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 7777 0 0 0 148 1 0 0 20 0 1 0 558265782 55087104 7570 33554432000 4194304 6211132 140722048679520 140722048538808 5443143 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 13449 7570 846 493 0 6754 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 53796 KiB
Current children cumulated memory: 30280 KiB

[startup+3.10848 s]
/proc/loadavg: 7.88 8.23 8.07 2/192 10734
/proc/meminfo: memFree=31870556/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=65824 memory=42348 CPUtime=3.09 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 11581 0 0 0 308 1 0 0 20 0 1 0 558265782 67403776 10587 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 16456 10587 846 493 0 9761 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 65824 KiB
Current children cumulated memory: 42348 KiB

[startup+6.30032 s]
/proc/loadavg: 7.33 8.11 8.03 2/192 10734
/proc/meminfo: memFree=31848484/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=89576 memory=66000 CPUtime=6.29 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 18314 0 0 0 627 2 0 0 20 0 1 0 558265782 91725824 16500 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 22394 16500 846 493 0 15699 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 89576 KiB
Current children cumulated memory: 66000 KiB

[startup+12.7003 s]
/proc/loadavg: 6.82 7.99 7.99 2/193 10736
/proc/meminfo: memFree=31804052/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=137412 memory=113772 CPUtime=12.69 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 31016 0 0 0 1265 4 0 0 20 0 1 0 558265782 140709888 28443 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 34353 28443 846 493 0 27658 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 137412 KiB
Current children cumulated memory: 113772 KiB

[startup+25.5003 s]
/proc/loadavg: 5.53 7.65 7.88 2/192 10736
/proc/meminfo: memFree=31707228/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=232472 memory=208772 CPUtime=25.49 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 56414 0 0 0 2542 7 0 0 20 0 1 0 558265782 238051328 52193 33554432000 4194304 6211132 140722048679520 140722048538808 5451593 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 58118 52193 846 493 0 51423 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 232472 KiB
Current children cumulated memory: 208772 KiB

[startup+51.1085 s]
/proc/loadavg: 3.99 7.12 7.70 2/192 10737
/proc/meminfo: memFree=31512476/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=424676 memory=400780 CPUtime=51.09 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 104793 0 0 0 5095 14 0 0 20 0 1 0 558265782 434868224 100195 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 106169 100195 846 493 0 99474 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 424676 KiB
Current children cumulated memory: 400780 KiB

[startup+102.3 s]
/proc/loadavg: 2.30 6.17 7.35 2/193 10741
/proc/meminfo: memFree=31128832/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=810008 memory=785316 CPUtime=102.26 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 201256 0 0 0 10201 25 0 0 20 0 1 0 558265782 829448192 196329 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 202502 196329 846 493 0 195807 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 810008 KiB
Current children cumulated memory: 785316 KiB

[startup+162.3 s]
/proc/loadavg: 1.48 5.23 6.95 2/192 10744
/proc/meminfo: memFree=30677268/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=1261100 memory=1236228 CPUtime=162.25 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 313970 0 0 0 16186 39 0 0 20 0 1 0 558265782 1291366400 309057 33554432000 4194304 6211132 140722048679520 140722048538808 140000637435291 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 315275 309057 846 493 0 308580 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 1261100 KiB
Current children cumulated memory: 1236228 KiB

[startup+222.3 s]
/proc/loadavg: 1.17 4.46 6.58 2/192 10747
/proc/meminfo: memFree=30225788/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=1713404 memory=1687084 CPUtime=222.24 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 426718 0 0 0 22163 61 0 0 20 0 1 0 558265782 1754525696 421771 33554432000 4194304 6211132 140722048679520 140722048538808 5446292 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 428351 421771 846 493 0 421656 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 1713404 KiB
Current children cumulated memory: 1687084 KiB

[startup+282.3 s]
/proc/loadavg: 1.06 3.83 6.23 2/192 10752
/proc/meminfo: memFree=29773980/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=2162336 memory=2137996 CPUtime=282.23 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 539427 0 0 0 28147 76 0 0 20 0 1 0 558265782 2214232064 534499 33554432000 4194304 6211132 140722048679520 140722048538808 140000637429667 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 540584 534499 846 493 0 533889 0
Current children cumulated CPU time: 282.23 s
Current children cumulated vsize: 2162336 KiB
Current children cumulated memory: 2137996 KiB

[startup+342.3 s]
/proc/loadavg: 1.02 3.32 5.90 2/191 10755
/proc/meminfo: memFree=29320560/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=2616128 memory=2590644 CPUtime=342.22 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 652071 0 0 0 34129 93 0 0 20 0 1 0 558265782 2678915072 647661 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 654032 647661 846 493 0 647337 0
Current children cumulated CPU time: 342.22 s
Current children cumulated vsize: 2616128 KiB
Current children cumulated memory: 2590644 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.89 5.60 2/191 10758
/proc/meminfo: memFree=28870716/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=3065060 memory=3039744 CPUtime=402.21 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 764650 0 0 0 40114 107 0 0 20 0 1 0 558265782 3138621440 759936 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 766265 759936 846 493 0 759570 0
Current children cumulated CPU time: 402.21 s
Current children cumulated vsize: 3065060 KiB
Current children cumulated memory: 3039744 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.55 5.31 2/192 10763
/proc/meminfo: memFree=28418884/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=3521152 memory=3490476 CPUtime=462.2 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 877991 0 0 0 46097 123 0 0 20 0 1 0 558265782 3605659648 872619 33554432000 4194304 6211132 140722048679520 140722048538808 5453940 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 880288 872619 846 493 0 873593 0
Current children cumulated CPU time: 462.2 s
Current children cumulated vsize: 3521152 KiB
Current children cumulated memory: 3490476 KiB


################
# More data... #
################

Current children cumulated vsize: 14334528 KiB
Current children cumulated memory: 14298456 KiB

[startup+1962.3 s]
/proc/loadavg: 1.00 1.01 1.86 2/191 10879
/proc/meminfo: memFree=17139192/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=14782536 memory=14748504 CPUtime=1961.9 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 3720469 0 0 0 195497 693 0 0 20 0 1 0 558265782 15137316864 3687126 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 3695634 3687126 846 493 0 3688939 0
Current children cumulated CPU time: 1961.9 s
Current children cumulated vsize: 14782536 KiB
Current children cumulated memory: 14748504 KiB

[startup+2022.3 s]
/proc/loadavg: 1.00 1.01 1.80 2/192 10883
/proc/meminfo: memFree=16688216/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=15230412 memory=15198248 CPUtime=2021.89 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 3834419 0 0 0 201466 723 0 0 20 0 1 0 558265782 15595941888 3799562 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 3807603 3799562 846 493 0 3800908 0
Current children cumulated CPU time: 2021.89 s
Current children cumulated vsize: 15230412 KiB
Current children cumulated memory: 15198248 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.01 1.75 2/192 10888
/proc/meminfo: memFree=16237256/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=15678156 memory=15648256 CPUtime=2081.89 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 3948045 0 0 0 207436 753 0 0 20 0 1 0 558265782 16054431744 3912064 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 3919539 3912064 846 493 0 3912844 0
Current children cumulated CPU time: 2081.89 s
Current children cumulated vsize: 15678156 KiB
Current children cumulated memory: 15648256 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.01 1.71 2/191 10891
/proc/meminfo: memFree=15785896/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=16125372 memory=16097468 CPUtime=2141.87 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4061827 0 0 0 213400 787 0 0 20 0 1 0 558265782 16512380928 4024367 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4031343 4024367 846 493 0 4024648 0
Current children cumulated CPU time: 2141.87 s
Current children cumulated vsize: 16125372 KiB
Current children cumulated memory: 16097468 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.01 1.66 2/191 10894
/proc/meminfo: memFree=15335584/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=16572588 memory=16546684 CPUtime=2201.85 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4175631 0 0 0 219363 822 0 0 20 0 1 0 558265782 16970330112 4136671 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4143147 4136671 846 493 0 4136452 0
Current children cumulated CPU time: 2201.85 s
Current children cumulated vsize: 16572588 KiB
Current children cumulated memory: 16546684 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.62 2/192 10899
/proc/meminfo: memFree=14887612/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=17019936 memory=16994120 CPUtime=2261.84 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4289130 0 0 0 225326 858 0 0 20 0 1 0 558265782 17428414464 4248530 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4254984 4248530 846 493 0 4248289 0
Current children cumulated CPU time: 2261.84 s
Current children cumulated vsize: 17019936 KiB
Current children cumulated memory: 16994120 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.58 2/191 10902
/proc/meminfo: memFree=14437496/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=17467284 memory=17443332 CPUtime=2321.83 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4402938 0 0 0 231287 896 0 0 20 0 1 0 558265782 17886498816 4360833 33554432000 4194304 6211132 140722048679520 140722048538808 5443093 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4366821 4360833 846 493 0 4360126 0
Current children cumulated CPU time: 2321.83 s
Current children cumulated vsize: 17467284 KiB
Current children cumulated memory: 17443332 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.55 2/191 10912
/proc/meminfo: memFree=13986908/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=17951280 memory=17892268 CPUtime=2381.81 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4517139 0 0 0 237248 933 0 0 20 0 1 0 558265782 18382110720 4473067 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4487820 4473067 846 493 0 4481125 0
Current children cumulated CPU time: 2381.81 s
Current children cumulated vsize: 17951280 KiB
Current children cumulated memory: 17892268 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.51 2/192 10917
/proc/meminfo: memFree=13536680/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=18398628 memory=18341972 CPUtime=2441.8 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4630757 0 0 0 243208 972 0 0 20 0 1 0 558265782 18840195072 4585493 33554432000 4194304 6211132 140722048679520 140722048538808 5446206 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4599657 4585493 846 493 0 4592962 0
Current children cumulated CPU time: 2441.8 s
Current children cumulated vsize: 18398628 KiB
Current children cumulated memory: 18341972 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.48 2/191 10920
/proc/meminfo: memFree=13086772/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=18845580 memory=18790972 CPUtime=2501.79 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4744828 0 0 0 249170 1009 0 0 20 0 1 0 558265782 19297873920 4697743 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4711395 4697743 846 493 0 4704700 0
Current children cumulated CPU time: 2501.79 s
Current children cumulated vsize: 18845580 KiB
Current children cumulated memory: 18790972 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.47 2/191 10921
/proc/meminfo: memFree=12952744/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=18977448 memory=18922948 CPUtime=2519.49 cores=0-7
/proc/10733/stat : 10733 (cosoco) R 10730 10733 10257 0 -1 4202496 4778136 0 0 0 250929 1020 0 0 20 0 1 0 558265782 19432906752 4730737 33554432000 4194304 6211132 140722048679520 140722048538808 140000636509157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4744362 4730737 846 493 0 4737667 0
Current children cumulated CPU time: 2519.49 s
Current children cumulated vsize: 18977448 KiB
Current children cumulated memory: 18922948 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2522.83 s]
# the end of solver process 10733 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2509.54 s, system=11.0609 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.47 2/191 10921
/proc/meminfo: memFree=12952744/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=18977448 memory=18922948 CPUtime=2519.63 cores=0-7
/proc/10733/stat : 10733 (cosoco) S 10730 10733 10257 0 -1 512 4778142 0 0 0 250941 1022 0 0 20 0 1 0 558265782 19432906752 4730737 33554432000 4194304 6211132 140722048679520 140722048538808 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4744362 4730737 846 493 0 4737667 0
Current children cumulated CPU time: 2519.63 s
Current children cumulated vsize: 18977448 KiB
Current children cumulated memory: 18922948 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.47 2/192 10922
/proc/meminfo: memFree=12411716/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=18977448 memory=18922948 CPUtime=2519.75 cores=0-7
/proc/10733/stat : 10733 (cosoco) S 10730 10733 10257 0 -1 512 4778142 0 0 0 250952 1023 0 0 20 0 1 0 558265782 19432906752 4730737 33554432000 4194304 6211132 140722048679520 140722048538808 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4744362 4730737 846 493 0 4737667 0
Current children cumulated CPU time: 2519.75 s
Current children cumulated vsize: 18977448 KiB
Current children cumulated memory: 18922948 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.47 1/192 10922
/proc/meminfo: memFree=11997924/32770624 swapFree=66943312/67108860
[pid=10733] ppid=10730 vsize=18977448 memory=18922948 CPUtime=2519.78 cores=0-7
/proc/10733/stat : 10733 (cosoco) S 10730 10733 10257 0 -1 512 4778142 0 0 0 250954 1024 0 0 20 0 1 0 558265782 19432906752 4730737 33554432000 4194304 6211132 140722048679520 140722048538808 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 29368320 140722048685073 140722048685165 140722048685165 140722048688078 0
/proc/10733/statm: 4744362 4730737 846 493 0 4737667 0
Current children cumulated CPU time: 2519.78 s
Current children cumulated vsize: 18977448 KiB
Current children cumulated memory: 18922948 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2522.83
CPU time (s): 2520.6
CPU user time (s): 2509.54
CPU system time (s): 11.0609
CPU usage (%): 99.9117
Max. virtual memory (cumulated for all children) (KiB): 18977448
Max. memory (cumulated for all children) (KiB): 18922948

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2509.54
system time used= 11.0609
maximum resident set size= 18923132
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 4778142
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 112496
involuntary context switches= 2238


# summary of solver processes directly reported to runsolver:
#   pid: 10733
#   total CPU time (s): 2520.6
#   total CPU user time (s): 2509.54
#   total CPU system time (s): 11.0609

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.52997 second user time and 16.412 second system time

The end

Launcher Data 

Begin job on node149 at 2019-09-30 07:10:33
IDJOB=4409758
IDBENCH=122967
IDSOLVER=2877
FILE ID=node149/4409758-1569820272
RUNJOBID= node149-1569816999-10284
PBS_JOBID= 21705913
Free space on /tmp= 62032 MiB

SOLVER NAME= cosoco 2.0 parallel
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-09-350-100.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409758-1569820272/watcher-4409758-1569820272 -o /tmp/evaluation-result-4409758-1569820272/solver-4409758-1569820272 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node149-1569816999-10284 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4409758-1569820272.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= 9444e44f99a0ffd374c39bd698e45a3a
RANDOM SEED=1448225904

node149.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.81
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31908452 kB
MemAvailable:   31935572 kB
Buffers:          115856 kB
Cached:           203804 kB
SwapCached:         5480 kB
Active:           147608 kB
Inactive:         178100 kB
Active(anon):      12252 kB
Inactive(anon):    20424 kB
Active(file):     135356 kB
Inactive(file):   157676 kB
Unevictable:      109768 kB
Mlocked:          109768 kB
SwapTotal:      67108860 kB
SwapFree:       66943312 kB
Dirty:              2548 kB
Writeback:             0 kB
AnonPages:        112492 kB
Mapped:            26620 kB
Shmem:             17896 kB
Slab:             128360 kB
SReclaimable:      99848 kB
SUnreclaim:        28512 kB
KernelStack:        3344 kB
PageTables:         5340 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     539008 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      144768 kB
DirectMap2M:    33400832 kB

Free space on /tmp at the end= 61596 MiB
End job on node149 at 2019-09-30 07:52:36