Trace number 4409573

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco 2.0 parallel? (TO) 2520.8 2523.19

General information on the benchmark

NameOpd/Opd-sum-large/
Opd-15-035-010.xml
MD5SUMb4ffb396ba331e6a6f6e6b1bc4ae482f
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkOPT
Best value of the objective obtained on this benchmark3
Best CPU time to get the best result obtained on this benchmark6.18633
Satisfiable
(Un)Satisfiability was proved
Number of variables526
Number of constraints121
Number of domains2
Minimum domain size2
Maximum domain size33
Distribution of domain sizes[{"size":2,"count":525},{"size":33,"count":1}]
Minimum variable degree16
Maximum variable degree106
Distribution of variable degrees[{"degree":16,"count":525},{"degree":106,"count":1}]
Minimum constraint arity35
Maximum constraint arity525
Distribution of constraint arities[{"arity":35,"count":15},{"arity":71,"count":105},{"arity":525,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":120}]
Optimization problemYES
Type of objectivemin VAR

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15434 MiB free)
  memory of node 1: 16384 MiB (15582 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4409573-1569819969/watcher-4409573-1569819969 -o /tmp/evaluation-result-4409573-1569819969/solver-4409573-1569819969 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node127-1569817234-15805 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4409573-1569819969.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=16210, runsolver pid=16207
Current StackSize limit: 8192 KiB


[startup+0.100153 s]*
/proc/loadavg: 7.38 8.03 7.82 2/194 16211
/proc/meminfo: memFree=31757880/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=30924 memory=7384 CPUtime=0.1 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 2033 0 0 0 10 0 0 0 20 0 1 0 558265655 31666176 1846 33554432000 4194304 6211132 140727559063008 140727559059544 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 7731 1846 846 493 0 1036 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 30924 KiB
Current children cumulated memory: 7384 KiB

[startup+0.20911 s]*
/proc/loadavg: 7.38 8.03 7.82 2/194 16211
/proc/meminfo: memFree=31756860/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=32100 memory=8440 CPUtime=0.21 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 2307 0 0 0 21 0 0 0 20 0 1 0 558265655 32870400 2110 33554432000 4194304 6211132 140727559063008 140727559043256 5441125 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 8025 2110 846 493 0 1330 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 32100 KiB
Current children cumulated memory: 8440 KiB

[startup+0.308915 s]*
/proc/loadavg: 7.38 8.03 7.82 2/194 16211
/proc/meminfo: memFree=31756176/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=33064 memory=9496 CPUtime=0.31 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 2558 0 0 0 31 0 0 0 20 0 1 0 558265655 33857536 2374 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 8266 2374 846 493 0 1571 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 33064 KiB
Current children cumulated memory: 9496 KiB

[startup+0.700325 s]
/proc/loadavg: 7.38 8.03 7.82 2/194 16211
/proc/meminfo: memFree=31754668/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=36968 memory=13404 CPUtime=0.69 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 3524 0 0 0 69 0 0 0 20 0 1 0 558265655 37855232 3351 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 9242 3351 846 493 0 2547 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 36968 KiB
Current children cumulated memory: 13404 KiB

[startup+1.50032 s]
/proc/loadavg: 7.38 8.03 7.82 2/194 16211
/proc/meminfo: memFree=31750148/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=45232 memory=21472 CPUtime=1.48 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5545 0 0 0 148 0 0 0 20 0 1 0 558265655 46317568 5368 33554432000 4194304 6211132 140727559063008 140727559043256 140382305217943 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 11308 5368 846 493 0 4613 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 45232 KiB
Current children cumulated memory: 21472 KiB

[startup+3.1087 s]
/proc/loadavg: 7.38 8.03 7.82 2/194 16211
/proc/meminfo: memFree=31729924/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=61744 memory=37608 CPUtime=3.09 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 9729 0 0 0 308 1 0 0 20 0 1 0 558265655 63225856 9402 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 15436 9402 846 493 0 8741 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 61744 KiB
Current children cumulated memory: 37608 KiB

[startup+6.30032 s]
/proc/loadavg: 6.79 7.90 7.78 2/193 16211
/proc/meminfo: memFree=31700016/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=94900 memory=69968 CPUtime=6.29 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 17915 0 0 0 627 2 0 0 20 0 1 0 558265655 97177600 17492 33554432000 4194304 6211132 140727559063008 140727559043256 5456330 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 23725 17492 846 493 0 17030 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 94900 KiB
Current children cumulated memory: 69968 KiB

[startup+12.7003 s]
/proc/loadavg: 6.33 7.78 7.74 2/193 16211
/proc/meminfo: memFree=31639552/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=158704 memory=134384 CPUtime=12.69 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 34196 0 0 0 1265 4 0 0 20 0 1 0 558265655 162512896 33596 33554432000 4194304 6211132 140727559063008 140727559043256 140382305216065 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 39676 33596 846 493 0 32981 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 158704 KiB
Current children cumulated memory: 134384 KiB

[startup+25.5003 s]
/proc/loadavg: 5.15 7.45 7.63 2/193 16211
/proc/meminfo: memFree=31508532/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=290232 memory=263804 CPUtime=25.49 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 66291 0 0 0 2540 9 0 0 20 0 1 0 558265655 297197568 65951 33554432000 4194304 6211132 140727559063008 140727559043256 5443128 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 72558 65951 846 493 0 65863 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 290232 KiB
Current children cumulated memory: 263804 KiB

[startup+51.1086 s]
/proc/loadavg: 3.73 6.93 7.46 2/191 16214
/proc/meminfo: memFree=31245080/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=555564 memory=522336 CPUtime=51.1 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 129029 0 0 0 5093 17 0 0 20 0 1 0 558265655 568897536 130584 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 138891 130584 846 493 0 132196 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 555564 KiB
Current children cumulated memory: 522336 KiB

[startup+102.3 s]
/proc/loadavg: 2.19 6.02 7.12 2/191 16216
/proc/meminfo: memFree=30725452/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=1066864 memory=1042000 CPUtime=102.27 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 253401 0 0 0 10195 32 0 0 20 0 1 0 558265655 1092468736 260500 33554432000 4194304 6211132 140727559063008 140727559043256 5451516 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 266716 260500 846 493 0 260021 0
Current children cumulated CPU time: 102.27 s
Current children cumulated vsize: 1066864 KiB
Current children cumulated memory: 1042000 KiB

[startup+162.3 s]
/proc/loadavg: 1.44 5.10 6.74 2/191 16219
/proc/meminfo: memFree=30119004/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=1708512 memory=1648384 CPUtime=162.25 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 399360 0 0 0 16174 51 0 0 20 0 1 0 558265655 1749516288 412096 33554432000 4194304 6211132 140727559063008 140727559043256 5454215 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 427128 412096 846 493 0 420433 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 1708512 KiB
Current children cumulated memory: 1648384 KiB

[startup+222.3 s]
/proc/loadavg: 1.16 4.36 6.38 2/191 16222
/proc/meminfo: memFree=29508456/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=2288580 memory=2257620 CPUtime=222.23 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 544531 0 0 0 22156 67 0 0 20 0 1 0 558265655 2343505920 564405 33554432000 4194304 6211132 140727559063008 140727559043256 140382305216051 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 572145 564405 846 493 0 565450 0
Current children cumulated CPU time: 222.23 s
Current children cumulated vsize: 2288580 KiB
Current children cumulated memory: 2257620 KiB

[startup+282.3 s]
/proc/loadavg: 1.06 3.75 6.04 2/191 16234
/proc/meminfo: memFree=28900100/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=2924236 memory=2864532 CPUtime=282.22 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 690120 0 0 0 28136 86 0 0 20 0 1 0 558265655 2994417664 716133 33554432000 4194304 6211132 140727559063008 140727559043256 140382314236750 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 731059 716133 846 493 0 724364 0
Current children cumulated CPU time: 282.22 s
Current children cumulated vsize: 2924236 KiB
Current children cumulated memory: 2864532 KiB

[startup+342.3 s]
/proc/loadavg: 1.02 3.25 5.72 2/191 16237
/proc/meminfo: memFree=28292692/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=3504672 memory=3471412 CPUtime=342.2 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 835272 0 0 0 34115 105 0 0 20 0 1 0 558265655 3588784128 867853 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 876168 867853 846 493 0 869473 0
Current children cumulated CPU time: 342.2 s
Current children cumulated vsize: 3504672 KiB
Current children cumulated memory: 3471412 KiB

[startup+402.3 s]
/proc/loadavg: 1.01 2.84 5.43 2/191 16240
/proc/meminfo: memFree=27684196/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=4167724 memory=4079424 CPUtime=402.19 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 980734 0 0 0 40091 128 0 0 20 0 1 0 558265655 4267749376 1019856 33554432000 4194304 6211132 140727559063008 140727559043256 5013088 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 1041931 1019856 846 493 0 1035236 0
Current children cumulated CPU time: 402.19 s
Current children cumulated vsize: 4167724 KiB
Current children cumulated memory: 4079424 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.50 5.15 2/191 16245
/proc/meminfo: memFree=27076332/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=4747732 memory=4688256 CPUtime=462.17 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 1125929 0 0 0 46068 149 0 0 20 0 1 0 558265655 4861677568 1172064 33554432000 4194304 6211132 140727559063008 140727559043256 5452185 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 1186933 1172064 846 493 0 1180238 0
Current children cumulated CPU time: 462.17 s
Current children cumulated vsize: 4747732 KiB
Current children cumulated memory: 4688256 KiB


################
# More data... #
################

Current children cumulated vsize: 19664468 KiB
Current children cumulated memory: 19245132 KiB

[startup+1962.3 s]
/proc/loadavg: 1.00 1.02 1.85 2/191 16356
/proc/meminfo: memFree=11881640/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=20241836 memory=19849224 CPUtime=1961.75 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 4754974 0 0 0 195296 879 0 0 20 0 1 0 558265655 20727640064 4962306 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 5060459 4962306 846 493 0 5053764 0
Current children cumulated CPU time: 1961.75 s
Current children cumulated vsize: 20241836 KiB
Current children cumulated memory: 19849224 KiB

[startup+2022.3 s]
/proc/loadavg: 1.03 1.04 1.80 2/191 16359
/proc/meminfo: memFree=11274692/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=20819336 memory=20455596 CPUtime=2021.73 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 4899789 0 0 0 201253 920 0 0 20 0 1 0 558265655 21319000064 5113899 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 5204834 5113899 846 493 0 5198139 0
Current children cumulated CPU time: 2021.73 s
Current children cumulated vsize: 20819336 KiB
Current children cumulated memory: 20455596 KiB

[startup+2082.3 s]
/proc/loadavg: 1.01 1.03 1.75 2/192 16372
/proc/meminfo: memFree=10668440/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=21396572 memory=21059952 CPUtime=2081.72 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5044531 0 0 0 207210 962 0 0 20 0 1 0 558265655 21910089728 5264988 33554432000 4194304 6211132 140727559063008 140727559043256 5454208 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 5349143 5264988 846 493 0 5342448 0
Current children cumulated CPU time: 2081.72 s
Current children cumulated vsize: 21396572 KiB
Current children cumulated memory: 21059952 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.02 1.70 2/192 16375
/proc/meminfo: memFree=10063604/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=21973940 memory=21664280 CPUtime=2141.7 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5189327 0 0 0 213162 1008 0 0 20 0 1 0 558265655 22501314560 5416070 33554432000 4194304 6211132 140727559063008 140727559043256 5455682 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 5493485 5416070 846 493 0 5486790 0
Current children cumulated CPU time: 2141.7 s
Current children cumulated vsize: 21973940 KiB
Current children cumulated memory: 21664280 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.02 1.66 2/192 16378
/proc/meminfo: memFree=9458332/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=22551044 memory=22268372 CPUtime=2201.67 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5334010 0 0 0 219112 1055 0 0 20 0 1 0 558265655 23092269056 5567093 33554432000 4194304 6211132 140727559063008 140727559043256 140382314236760 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 5637761 5567093 846 493 0 5631066 0
Current children cumulated CPU time: 2201.67 s
Current children cumulated vsize: 22551044 KiB
Current children cumulated memory: 22268372 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.62 2/192 16383
/proc/meminfo: memFree=8850232/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=23128412 memory=22874744 CPUtime=2261.65 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5478803 0 0 0 225067 1098 0 0 20 0 1 0 558265655 23683493888 5718686 33554432000 4194304 6211132 140727559063008 140727559043256 140382304289741 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 5782103 5718686 846 493 0 5775408 0
Current children cumulated CPU time: 2261.65 s
Current children cumulated vsize: 23128412 KiB
Current children cumulated memory: 22874744 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.58 2/192 16386
/proc/meminfo: memFree=8242976/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=23710376 memory=23481116 CPUtime=2321.64 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5623582 0 0 0 231018 1146 0 0 20 0 1 0 558265655 24279425024 5870279 33554432000 4194304 6211132 140727559063008 140727559043256 140382314236746 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 5927594 5870279 846 493 0 5920899 0
Current children cumulated CPU time: 2321.64 s
Current children cumulated vsize: 23710376 KiB
Current children cumulated memory: 23481116 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.54 2/191 16389
/proc/meminfo: memFree=7638088/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=24287348 memory=24084620 CPUtime=2381.62 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5768139 0 0 0 236967 1195 0 0 20 0 1 0 558265655 24870244352 6021155 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 6071837 6021155 846 493 0 6065142 0
Current children cumulated CPU time: 2381.62 s
Current children cumulated vsize: 24287348 KiB
Current children cumulated memory: 24084620 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.51 2/191 16394
/proc/meminfo: memFree=7033884/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=24864056 memory=24687944 CPUtime=2441.6 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 5912704 0 0 0 242914 1246 0 0 20 0 1 0 558265655 25460793344 6171986 33554432000 4194304 6211132 140727559063008 140727559043256 140382305217962 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 6216014 6171986 846 493 0 6209319 0
Current children cumulated CPU time: 2441.6 s
Current children cumulated vsize: 24864056 KiB
Current children cumulated memory: 24687944 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.48 2/191 16397
/proc/meminfo: memFree=6429480/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=25441028 memory=25293524 CPUtime=2501.58 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 6057300 0 0 0 248864 1294 0 0 20 0 1 0 558265655 26051612672 6323381 33554432000 4194304 6211132 140727559063008 140727559043256 5446650 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 6360257 6323381 846 493 0 6353562 0
Current children cumulated CPU time: 2501.58 s
Current children cumulated vsize: 25441028 KiB
Current children cumulated memory: 25293524 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.47 2/192 16400
/proc/meminfo: memFree=6247824/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=25611308 memory=25470144 CPUtime=2519.29 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 4202496 6100024 0 0 0 250621 1308 0 0 20 0 1 0 558265655 26225979392 6367536 33554432000 4194304 6211132 140727559063008 140727559043256 140382304291813 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 6402827 6367536 846 493 0 6396132 0
Current children cumulated CPU time: 2519.29 s
Current children cumulated vsize: 25611308 KiB
Current children cumulated memory: 25470144 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.19 s]
# the end of solver process 16210 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2506.51 s, system=14.2896 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.47 2/192 16400
/proc/meminfo: memFree=6247824/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=25611308 memory=25470144 CPUtime=2519.4 cores=0-7
/proc/16210/stat : 16210 (cosoco) S 16207 16210 15778 0 -1 512 6100030 0 0 0 250632 1308 0 0 20 0 1 0 558265655 26225979392 6367536 33554432000 4194304 6211132 140727559063008 140727559043256 5189582 0 16384 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 6402827 6367536 846 493 0 6396132 0
Current children cumulated CPU time: 2519.4 s
Current children cumulated vsize: 25611308 KiB
Current children cumulated memory: 25470144 KiB

[startup+2521.6 s]
/proc/loadavg: 1.00 1.01 1.47 2/193 16401
/proc/meminfo: memFree=5729612/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=25611308 memory=25470144 CPUtime=2519.54 cores=0-7
/proc/16210/stat : 16210 (cosoco) R 16207 16210 15778 0 -1 512 6100030 0 0 0 250645 1309 0 0 20 0 1 0 558265655 26225979392 6367536 33554432000 4194304 6211132 140727559063008 140727559043256 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 6402827 6367536 846 493 0 6396132 0
Current children cumulated CPU time: 2519.54 s
Current children cumulated vsize: 25611308 KiB
Current children cumulated memory: 25470144 KiB

[startup+2522 s]*
/proc/loadavg: 1.00 1.01 1.47 2/193 16401
/proc/meminfo: memFree=5189368/32770624 swapFree=66959828/67108860
[pid=16210] ppid=16207 vsize=25611308 memory=25470144 CPUtime=2519.61 cores=0-7
/proc/16210/stat : 16210 (cosoco) S 16207 16210 15778 0 -1 512 6100030 0 0 0 250651 1310 0 0 20 0 1 0 558265655 26225979392 6367536 33554432000 4194304 6211132 140727559063008 140727559043256 5189582 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 33144832 140727559070737 140727559070829 140727559070829 140727559073742 0
/proc/16210/statm: 6402827 6367536 846 493 0 6396132 0
Current children cumulated CPU time: 2519.61 s
Current children cumulated vsize: 25611308 KiB
Current children cumulated memory: 25470144 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.19
CPU time (s): 2520.8
CPU user time (s): 2506.51
CPU system time (s): 14.2896
CPU usage (%): 99.9052
Max. virtual memory (cumulated for all children) (KiB): 25611308
Max. memory (cumulated for all children) (KiB): 25470144

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2506.51
system time used= 14.2896
maximum resident set size= 25470192
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6100030
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 133091
involuntary context switches= 3151


# summary of solver processes directly reported to runsolver:
#   pid: 16210
#   total CPU time (s): 2520.8
#   total CPU user time (s): 2506.51
#   total CPU system time (s): 14.2896

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.39845 second user time and 15.8495 second system time

The end

Launcher Data 

Begin job on node127 at 2019-09-30 07:05:28
IDJOB=4409573
IDBENCH=122936
IDSOLVER=2877
FILE ID=node127/4409573-1569819969
RUNJOBID= node127-1569817234-15805
PBS_JOBID= 21705916
Free space on /tmp= 62032 MiB

SOLVER NAME= cosoco 2.0 parallel
BENCH NAME= XCSP17/Opd/Opd-sum-large/Opd-15-035-010.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4409573-1569819969/watcher-4409573-1569819969 -o /tmp/evaluation-result-4409573-1569819969/solver-4409573-1569819969 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node127-1569817234-15805 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4409573-1569819969.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= b4ffb396ba331e6a6f6e6b1bc4ae482f
RANDOM SEED=1277744086

node127.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31762104 kB
MemAvailable:   31817508 kB
Buffers:           95832 kB
Cached:           287600 kB
SwapCached:        45128 kB
Active:           269684 kB
Inactive:         166744 kB
Active(anon):       5312 kB
Inactive(anon):   129372 kB
Active(file):     264372 kB
Inactive(file):    37372 kB
Unevictable:      107512 kB
Mlocked:          107512 kB
SwapTotal:      67108860 kB
SwapFree:       66959828 kB
Dirty:              2712 kB
Writeback:             0 kB
AnonPages:        116084 kB
Mapped:            26524 kB
Shmem:             73072 kB
Slab:             168012 kB
SReclaimable:     138988 kB
SUnreclaim:        29024 kB
KernelStack:        3408 kB
PageTables:         6236 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     539044 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     69632 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      138624 kB
DirectMap2M:    33406976 kB

Free space on /tmp at the end= 61512 MiB
End job on node127 at 2019-09-30 07:47:31