Trace number 4408181

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2.0? (MO) 1786.62 1788.75

General information on the benchmark

NameBibd/Bibd-sum-open/
Bibd-sum-57-076-20-15-05.xml
MD5SUM103999ad3c646e4e3c5e6682f24aa76f
Bench CategoryCSP (decision problem)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables4332
Number of constraints1730
Number of domains1
Minimum domain size2
Maximum domain size2
Distribution of domain sizes[{"size":2,"count":4332}]
Minimum variable degree59
Maximum variable degree59
Distribution of variable degrees[{"degree":59,"count":4332}]
Minimum constraint arity57
Maximum constraint arity4332
Distribution of constraint arities[{"arity":57,"count":76},{"arity":76,"count":57},{"arity":152,"count":1596},{"arity":4332,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":1729}]
Optimization problemNO
Type of objective

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (14995 MiB free)
  memory of node 1: 16384 MiB (13954 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4408181-1569792808/watcher-4408181-1569792808 -o /tmp/evaluation-result-4408181-1569792808/solver-4408181-1569792808 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node146-1569787351-18444 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4408181-1569792808.xml 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=19366, runsolver pid=19363
Current StackSize limit: 8192 KiB


[startup+0.100193 s]*
/proc/loadavg: 2.32 2.08 2.02 3/201 19367
/proc/meminfo: memFree=29638548/32770624 swapFree=67002112/67108860
[pid=19366] ppid=19363 vsize=33544 memory=9820 CPUtime=0.1 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 2664 0 0 0 10 0 0 0 20 0 1 0 555195041 34349056 2455 33554432000 4194304 6211132 140731014385552 140731014382088 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 8386 2455 842 493 0 1691 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 33544 KiB
Current children cumulated memory: 9820 KiB

[startup+0.209957 s]*
/proc/loadavg: 2.32 2.08 2.02 3/201 19367
/proc/meminfo: memFree=29630792/32770624 swapFree=67002112/67108860
[pid=19366] ppid=19363 vsize=34488 memory=10876 CPUtime=0.21 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 2906 0 0 0 21 0 0 0 20 0 1 0 555195041 35315712 2719 33554432000 4194304 6211132 140731014385552 140731014382088 139920174958496 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 8622 2719 842 493 0 1927 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 34488 KiB
Current children cumulated memory: 10876 KiB

[startup+0.309796 s]*
/proc/loadavg: 2.32 2.08 2.02 3/201 19367
/proc/meminfo: memFree=29629676/32770624 swapFree=67002112/67108860
[pid=19366] ppid=19363 vsize=35468 memory=11668 CPUtime=0.31 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3137 0 0 0 31 0 0 0 20 0 1 0 555195041 36319232 2917 33554432000 4194304 6211132 140731014385552 140731014382088 139920174107624 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 8867 2917 842 493 0 2172 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 35468 KiB
Current children cumulated memory: 11668 KiB

[startup+0.700398 s]
/proc/loadavg: 2.32 2.08 2.02 3/201 19367
/proc/meminfo: memFree=29625168/32770624 swapFree=67002112/67108860
[pid=19366] ppid=19363 vsize=38736 memory=15100 CPUtime=0.69 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3966 0 0 0 69 0 0 0 20 0 1 0 555195041 39665664 3775 33554432000 4194304 6211132 140731014385552 140731014382088 5443128 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 9684 3775 842 493 0 2989 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 38736 KiB
Current children cumulated memory: 15100 KiB

[startup+1.5004 s]
/proc/loadavg: 2.32 2.08 2.02 3/201 19367
/proc/meminfo: memFree=29610088/32770624 swapFree=67002112/67108860
[pid=19366] ppid=19363 vsize=45808 memory=22056 CPUtime=1.48 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 5731 0 0 0 148 0 0 0 20 0 1 0 555195041 46907392 5514 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 11452 5514 842 493 0 4757 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 45808 KiB
Current children cumulated memory: 22056 KiB

[startup+3.1095 s]
/proc/loadavg: 2.32 2.08 2.02 3/201 19367
/proc/meminfo: memFree=29574536/32770624 swapFree=67002112/67108860
[pid=19366] ppid=19363 vsize=59860 memory=36228 CPUtime=3.09 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 9340 0 0 0 308 1 0 0 20 0 1 0 555195041 61296640 9057 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 14965 9057 842 493 0 8270 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 59860 KiB
Current children cumulated memory: 36228 KiB

[startup+6.30034 s]
/proc/loadavg: 2.37 2.10 2.03 3/200 19370
/proc/meminfo: memFree=30255304/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=88160 memory=64428 CPUtime=6.28 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 16388 0 0 0 626 2 0 0 20 0 1 0 555195041 90275840 16107 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 22040 16107 842 493 0 15345 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 88160 KiB
Current children cumulated memory: 64428 KiB

[startup+12.7003 s]
/proc/loadavg: 2.34 2.09 2.03 3/200 19373
/proc/meminfo: memFree=30204188/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=145068 memory=120900 CPUtime=12.68 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 30525 0 0 0 1264 4 0 0 20 0 1 0 555195041 148549632 30225 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 36267 30225 842 493 0 29572 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 145068 KiB
Current children cumulated memory: 120900 KiB

[startup+25.5003 s]
/proc/loadavg: 2.27 2.09 2.03 3/200 19373
/proc/meminfo: memFree=30089328/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=259396 memory=233832 CPUtime=25.47 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 58852 0 0 0 2540 7 0 0 20 0 1 0 555195041 265621504 58458 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 64849 58458 842 493 0 58154 0
Current children cumulated CPU time: 25.47 s
Current children cumulated vsize: 259396 KiB
Current children cumulated memory: 233832 KiB

[startup+51.1095 s]
/proc/loadavg: 2.18 2.08 2.03 3/200 19373
/proc/meminfo: memFree=29858708/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=484340 memory=460520 CPUtime=51.08 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 115004 0 0 0 5094 14 0 0 20 0 1 0 555195041 495964160 115130 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 121085 115130 842 493 0 114390 0
Current children cumulated CPU time: 51.08 s
Current children cumulated vsize: 484340 KiB
Current children cumulated memory: 460520 KiB

[startup+102.3 s]
/proc/loadavg: 2.08 2.07 2.03 3/200 19379
/proc/meminfo: memFree=29407552/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=940388 memory=913760 CPUtime=102.26 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 226818 0 0 0 10197 29 0 0 20 0 1 0 555195041 962957312 228440 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 235097 228440 842 493 0 228402 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 940388 KiB
Current children cumulated memory: 913760 KiB

[startup+162.3 s]
/proc/loadavg: 2.03 2.06 2.03 3/200 19383
/proc/meminfo: memFree=28876980/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=1470856 memory=1443336 CPUtime=162.25 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 357194 0 0 0 16179 46 0 0 20 0 1 0 555195041 1506156544 360834 33554432000 4194304 6211132 140731014385552 140731014350248 5452189 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 367714 360834 842 493 0 361019 0
Current children cumulated CPU time: 162.25 s
Current children cumulated vsize: 1470856 KiB
Current children cumulated memory: 1443336 KiB

[startup+222.3 s]
/proc/loadavg: 2.01 2.05 2.03 3/200 19387
/proc/meminfo: memFree=28346236/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=2007244 memory=1973468 CPUtime=222.23 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 487640 0 0 0 22159 64 0 0 20 0 1 0 555195041 2055417856 493367 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 501811 493367 842 493 0 495116 0
Current children cumulated CPU time: 222.23 s
Current children cumulated vsize: 2007244 KiB
Current children cumulated memory: 1973468 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.04 2.03 3/199 19393
/proc/meminfo: memFree=27816188/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=2527444 memory=2502368 CPUtime=282.22 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 617816 0 0 0 28141 81 0 0 20 0 1 0 555195041 2588102656 625592 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 631861 625592 842 493 0 625166 0
Current children cumulated CPU time: 282.22 s
Current children cumulated vsize: 2527444 KiB
Current children cumulated memory: 2502368 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.03 2.03 3/199 19397
/proc/meminfo: memFree=27283776/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=3073364 memory=3034252 CPUtime=342.22 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 748225 0 0 0 34123 99 0 0 20 0 1 0 555195041 3147124736 758563 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 768341 758563 842 493 0 761646 0
Current children cumulated CPU time: 342.22 s
Current children cumulated vsize: 3073364 KiB
Current children cumulated memory: 3034252 KiB

[startup+402.3 s]
/proc/loadavg: 2.29 2.09 2.05 3/199 19401
/proc/meminfo: memFree=26752124/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=3592388 memory=3564552 CPUtime=402.21 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 878251 0 0 0 40105 116 0 0 20 0 1 0 555195041 3678605312 891138 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 898097 891138 842 493 0 891402 0
Current children cumulated CPU time: 402.21 s
Current children cumulated vsize: 3592388 KiB
Current children cumulated memory: 3564552 KiB

[startup+462.3 s]
/proc/loadavg: 2.34 2.16 2.08 3/201 19416
/proc/meminfo: memFree=26220712/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=4147796 memory=4093960 CPUtime=462.19 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 1008609 0 0 0 46085 134 0 0 20 0 1 0 555195041 4247343104 1023490 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 1036949 1023490 842 493 0 1030254 0
Current children cumulated CPU time: 462.19 s
Current children cumulated vsize: 4147796 KiB
Current children cumulated memory: 4093960 KiB


################
# More data... #
################

Current children cumulated vsize: 10526716 KiB
Current children cumulated memory: 10451964 KiB

[startup+1242.3 s]
/proc/loadavg: 2.16 2.05 2.06 3/200 19498
/proc/meminfo: memFree=19321140/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=11044420 memory=10979976 CPUtime=1242.05 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 2697286 0 0 0 123826 379 0 0 20 0 1 0 555195041 11309486080 2744994 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 2761105 2744994 842 493 0 2754410 0
Current children cumulated CPU time: 1242.05 s
Current children cumulated vsize: 11044420 KiB
Current children cumulated memory: 10979976 KiB

[startup+1302.3 s]
/proc/loadavg: 2.35 2.14 2.09 3/200 19502
/proc/meminfo: memFree=18791360/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=11562388 memory=11508780 CPUtime=1302.03 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 2827026 0 0 0 129798 405 0 0 20 0 1 0 555195041 11839885312 2877195 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 2890597 2877195 842 493 0 2883902 0
Current children cumulated CPU time: 1302.03 s
Current children cumulated vsize: 11562388 KiB
Current children cumulated memory: 11508780 KiB

[startup+1362.3 s]
/proc/loadavg: 2.13 2.11 2.08 3/199 19509
/proc/meminfo: memFree=18259712/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=12091292 memory=12038372 CPUtime=1362.03 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 2957020 0 0 0 135770 433 0 0 20 0 1 0 555195041 12381483008 3009593 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3022823 3009593 842 493 0 3016128 0
Current children cumulated CPU time: 1362.03 s
Current children cumulated vsize: 12091292 KiB
Current children cumulated memory: 12038372 KiB

[startup+1422.3 s]
/proc/loadavg: 2.05 2.09 2.08 3/199 19513
/proc/meminfo: memFree=17730680/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=12609392 memory=12566860 CPUtime=1422.01 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3086671 0 0 0 141743 458 0 0 20 0 1 0 555195041 12912017408 3141715 33554432000 4194304 6211132 140731014385552 140731014350248 5446258 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3152348 3141715 842 493 0 3145653 0
Current children cumulated CPU time: 1422.01 s
Current children cumulated vsize: 12609392 KiB
Current children cumulated memory: 12566860 KiB

[startup+1482.3 s]
/proc/loadavg: 2.02 2.08 2.07 3/199 19517
/proc/meminfo: memFree=17199616/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=13127228 memory=13096880 CPUtime=1482 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3216245 0 0 0 147715 485 0 0 20 0 1 0 555195041 13442281472 3274220 33554432000 4194304 6211132 140731014385552 140731014350248 5451564 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3281807 3274220 842 493 0 3275112 0
Current children cumulated CPU time: 1482 s
Current children cumulated vsize: 13127228 KiB
Current children cumulated memory: 13096880 KiB

[startup+1542.3 s]
/proc/loadavg: 2.01 2.06 2.06 3/199 19524
/proc/meminfo: memFree=16670544/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=13769500 memory=13624956 CPUtime=1542 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3346306 0 0 0 153689 511 0 0 20 0 1 0 555195041 14099968000 3406239 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3442375 3406239 842 493 0 3435680 0
Current children cumulated CPU time: 1542 s
Current children cumulated vsize: 13769500 KiB
Current children cumulated memory: 13624956 KiB

[startup+1602.3 s]
/proc/loadavg: 2.00 2.05 2.06 3/199 19528
/proc/meminfo: memFree=16140892/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=14287732 memory=14153320 CPUtime=1601.98 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3475954 0 0 0 159661 537 0 0 20 0 1 0 555195041 14630637568 3538330 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3571933 3538330 842 493 0 3565238 0
Current children cumulated CPU time: 1601.98 s
Current children cumulated vsize: 14287732 KiB
Current children cumulated memory: 14153320 KiB

[startup+1662.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/200 19540
/proc/meminfo: memFree=15608400/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=14805436 memory=14683376 CPUtime=1661.98 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3605516 0 0 0 165634 564 0 0 20 0 1 0 555195041 15160766464 3670844 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3701359 3670844 842 493 0 3694664 0
Current children cumulated CPU time: 1661.98 s
Current children cumulated vsize: 14805436 KiB
Current children cumulated memory: 14683376 KiB

[startup+1722.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/201 19572
/proc/meminfo: memFree=15078364/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=15323008 memory=15211124 CPUtime=1721.96 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3735019 0 0 0 171604 592 0 0 20 0 1 0 555195041 15690760192 3802781 33554432000 4194304 6211132 140731014385552 140731014350248 5451553 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3830752 3802781 842 493 0 3824057 0
Current children cumulated CPU time: 1721.96 s
Current children cumulated vsize: 15323008 KiB
Current children cumulated memory: 15211124 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/201 19576
/proc/meminfo: memFree=14550012/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=15840712 memory=15739876 CPUtime=1781.96 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3864792 0 0 0 177567 629 0 0 20 0 1 0 555195041 16220889088 3934969 33554432000 4194304 6211132 140731014385552 140731014350248 139920174037989 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3960178 3934969 842 493 0 3953483 0
Current children cumulated CPU time: 1781.96 s
Current children cumulated vsize: 15840712 KiB
Current children cumulated memory: 15739876 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+1786 s]*
/proc/loadavg: 2.00 2.03 2.05 3/201 19576
/proc/meminfo: memFree=14514948/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=15872788 memory=15772048 CPUtime=1785.66 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 4202496 3872826 0 0 0 177937 629 0 0 20 0 1 0 555195041 16253734912 3943012 33554432000 4194304 6211132 140731014385552 140731014350248 5443128 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3968197 3943012 842 493 0 3961502 0
Current children cumulated CPU time: 1785.66 s
Current children cumulated vsize: 15872788 KiB
Current children cumulated memory: 15772048 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+1788.75 s]
# the end of solver process 19366 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1779.57 s, system=7.05028 s

Solver just ended.
Dumping a history of the last processes samples

[startup+1787.2 s]
/proc/loadavg: 2.00 2.03 2.05 3/202 19577
/proc/meminfo: memFree=14031784/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=15872788 memory=15772048 CPUtime=1785.82 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) R 19363 19366 18413 0 -1 512 3872832 0 0 0 177952 630 0 0 20 0 1 0 555195041 16253734912 3943012 33554432000 4194304 6211132 140731014385552 140731014350248 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 2 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3968197 3943012 842 493 0 3961502 0
Current children cumulated CPU time: 1785.82 s
Current children cumulated vsize: 15872788 KiB
Current children cumulated memory: 15772048 KiB

[startup+1787.6 s]
/proc/loadavg: 2.00 2.03 2.05 3/202 19577
/proc/meminfo: memFree=14031784/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=15872788 memory=15772048 CPUtime=1785.88 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) S 19363 19366 18413 0 -1 512 3872832 0 0 0 177957 631 0 0 20 0 1 0 555195041 16253734912 3943012 33554432000 4194304 6211132 140731014385552 140731014350248 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3968197 3943012 842 493 0 3961502 0
Current children cumulated CPU time: 1785.88 s
Current children cumulated vsize: 15872788 KiB
Current children cumulated memory: 15772048 KiB

[startup+1788 s]*
/proc/loadavg: 2.00 2.03 2.05 2/202 19577
/proc/meminfo: memFree=13706232/32770624 swapFree=67002480/67108860
[pid=19366] ppid=19363 vsize=15872788 memory=15772048 CPUtime=1785.88 cores=0,2,4,6
/proc/19366/stat : 19366 (cosoco) S 19363 19366 18413 0 -1 512 3872832 0 0 0 177957 631 0 0 20 0 1 0 555195041 16253734912 3943012 33554432000 4194304 6211132 140731014385552 140731014350248 5189800 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8312316 8317760 24694784 140731014390795 140731014390897 140731014390897 140731014393806 0
/proc/19366/statm: 3968197 3943012 842 493 0 3961502 0
Current children cumulated CPU time: 1785.88 s
Current children cumulated vsize: 15872788 KiB
Current children cumulated memory: 15772048 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 1788.75
CPU time (s): 1786.62
CPU user time (s): 1779.57
CPU system time (s): 7.05028
CPU usage (%): 99.8811
Max. virtual memory (cumulated for all children) (KiB): 15872788
Max. memory (cumulated for all children) (KiB): 15772048

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1779.57
system time used= 7.05028
maximum resident set size= 15772228
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3872832
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 95668
involuntary context switches= 1197


# summary of solver processes directly reported to runsolver:
#   pid: 19366
#   total CPU time (s): 1786.62
#   total CPU user time (s): 1779.57
#   total CPU system time (s): 7.05028

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 6.4483 second user time and 12.2834 second system time

The end

Launcher Data 

Begin job on node146 at 2019-09-29 23:32:50
IDJOB=4408181
IDBENCH=124102
IDSOLVER=2874
FILE ID=node146/4408181-1569792808
RUNJOBID= node146-1569787351-18444
PBS_JOBID= 21705819
Free space on /tmp= 61700 MiB

SOLVER NAME= cosoco 2.0
BENCH NAME= XCSP17/Bibd/Bibd-sum-open/Bibd-sum-57-076-20-15-05.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4408181-1569792808/watcher-4408181-1569792808 -o /tmp/evaluation-result-4408181-1569792808/solver-4408181-1569792808 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node146-1569787351-18444 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4408181-1569792808.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 103999ad3c646e4e3c5e6682f24aa76f
RANDOM SEED=1835128528

node146.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.90
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        29644268 kB
MemAvailable:   31861032 kB
Buffers:          178648 kB
Cached:          2370924 kB
SwapCached:         3452 kB
Active:          1614972 kB
Inactive:         944148 kB
Active(anon):      24256 kB
Inactive(anon):    66284 kB
Active(file):    1590716 kB
Inactive(file):   877864 kB
Unevictable:      109344 kB
Mlocked:          109344 kB
SwapTotal:      67108860 kB
SwapFree:       67002112 kB
Dirty:            241892 kB
Writeback:         72216 kB
AnonPages:        117344 kB
Mapped:            29392 kB
Shmem:             72256 kB
Slab:             159820 kB
SReclaimable:     128036 kB
SUnreclaim:        31784 kB
KernelStack:        3536 kB
PageTables:         5776 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     574396 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     67584 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      109952 kB
DirectMap2M:    33435648 kB

Free space on /tmp at the end= 61656 MiB
End job on node146 at 2019-09-30 00:02:39