Trace number 4389641

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2? (MO) 2065.37 2067.21

General information on the benchmark

NameBibd/Bibd-sum-open/
Bibd-sum-57-076-20-15-05.xml
MD5SUM103999ad3c646e4e3c5e6682f24aa76f
Bench CategoryCSP (decision problem)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables4332
Number of constraints1730
Number of domains1
Minimum domain size2
Maximum domain size2
Distribution of domain sizes[{"size":2,"count":4332}]
Minimum variable degree59
Maximum variable degree59
Distribution of variable degrees[{"degree":59,"count":4332}]
Minimum constraint arity57
Maximum constraint arity4332
Distribution of constraint arities[{"arity":57,"count":76},{"arity":76,"count":57},{"arity":152,"count":1596},{"arity":4332,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":1729}]
Optimization problemNO
Type of objective

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16375 MiB (12040 MiB free)
  memory of node 1: 16384 MiB (15492 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4389641-1568325734/watcher-4389641-1568325734 -o /tmp/evaluation-result-4389641-1568325734/solver-4389641-1568325734 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node116-1568311136-6439 --watchdog 2580 cosoco -lc -model -restarts -stick HOME/instance-4389641-1568325734.xml 

running on 4 cores: 4-7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=9504, runsolver pid=9501
Current StackSize limit: 8192 KiB


[startup+0.100165 s]*
/proc/loadavg: 2.39 2.10 2.07 3/202 9505
/proc/meminfo: memFree=28193264/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=33280 memory=9804 CPUtime=0.1 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 2624 0 0 0 10 0 0 0 20 0 1 0 408515163 34078720 2451 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 8320 2451 840 493 0 1625 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 33280 KiB
Current children cumulated memory: 9804 KiB

[startup+0.211199 s]*
/proc/loadavg: 2.39 2.10 2.07 3/202 9505
/proc/meminfo: memFree=28188832/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=34092 memory=10596 CPUtime=0.21 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 2826 0 0 0 20 1 0 0 20 0 1 0 408515163 34910208 2649 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 8523 2649 840 493 0 1828 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 34092 KiB
Current children cumulated memory: 10596 KiB

[startup+0.311003 s]*
/proc/loadavg: 2.60 2.14 2.09 3/202 9505
/proc/meminfo: memFree=28183292/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=34908 memory=11124 CPUtime=0.31 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3009 0 0 0 30 1 0 0 20 0 1 0 408515163 35745792 2781 33554432000 4194304 6210612 140735827782912 140735827747608 140453899275690 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 8727 2781 840 493 0 2032 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 34908 KiB
Current children cumulated memory: 11124 KiB

[startup+0.700328 s]
/proc/loadavg: 2.60 2.14 2.09 3/202 9505
/proc/meminfo: memFree=28187940/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=37640 memory=14028 CPUtime=0.69 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3706 0 0 0 66 3 0 0 20 0 1 0 408515163 38543360 3507 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 9410 3507 840 493 0 2715 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 37640 KiB
Current children cumulated memory: 14028 KiB

[startup+1.50038 s]
/proc/loadavg: 2.60 2.14 2.09 3/202 9505
/proc/meminfo: memFree=28187048/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=43704 memory=20196 CPUtime=1.49 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 5219 0 0 0 145 4 0 0 20 0 1 0 408515163 44752896 5049 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 10926 5049 840 493 0 4231 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 43704 KiB
Current children cumulated memory: 20196 KiB

[startup+3.1107 s]
/proc/loadavg: 2.60 2.14 2.09 3/200 9508
/proc/meminfo: memFree=29235148/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=56164 memory=32544 CPUtime=3.1 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 8368 0 0 0 306 4 0 0 20 0 1 0 408515163 57511936 8136 33554432000 4194304 6210612 140735827782912 140735827747608 140453899275694 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 14041 8136 840 493 0 7346 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 56164 KiB
Current children cumulated memory: 32544 KiB

[startup+6.30029 s]
/proc/loadavg: 2.55 2.14 2.09 3/200 9508
/proc/meminfo: memFree=29213088/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=80768 memory=56760 CPUtime=6.29 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 14551 0 0 0 624 5 0 0 20 0 1 0 408515163 82706432 14190 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 20192 14190 840 493 0 13497 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 80768 KiB
Current children cumulated memory: 56760 KiB

[startup+12.7003 s]
/proc/loadavg: 2.51 2.14 2.09 3/200 9508
/proc/meminfo: memFree=29167316/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=130284 memory=105644 CPUtime=12.69 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 26858 0 0 0 1262 7 0 0 20 0 1 0 408515163 133410816 26411 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 32571 26411 840 493 0 25876 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 130284 KiB
Current children cumulated memory: 105644 KiB

[startup+25.5003 s]
/proc/loadavg: 2.43 2.14 2.09 3/200 9508
/proc/meminfo: memFree=29068392/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=227536 memory=203480 CPUtime=25.49 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 51472 0 0 0 2539 10 0 0 20 0 1 0 408515163 232996864 50870 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 56884 50870 840 493 0 50189 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 227536 KiB
Current children cumulated memory: 203480 KiB

[startup+51.1107 s]
/proc/loadavg: 2.26 2.12 2.08 3/201 9511
/proc/meminfo: memFree=28869368/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=424544 memory=398836 CPUtime=51.1 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 100398 0 0 0 5092 18 0 0 20 0 1 0 408515163 434733056 99709 33554432000 4194304 6210612 140735827782912 140735827747608 5012846 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 106136 99709 840 493 0 99441 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 424544 KiB
Current children cumulated memory: 398836 KiB

[startup+102.3 s]
/proc/loadavg: 2.11 2.10 2.08 3/200 9514
/proc/meminfo: memFree=28479604/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=820136 memory=789460 CPUtime=102.29 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 197110 0 0 0 10197 32 0 0 20 0 1 0 408515163 839819264 197365 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 205034 197365 840 493 0 198339 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 820136 KiB
Current children cumulated memory: 789460 KiB

[startup+162.3 s]
/proc/loadavg: 2.04 2.08 2.07 3/200 9518
/proc/meminfo: memFree=28019408/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=1280512 memory=1249476 CPUtime=162.3 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 310055 0 0 0 16182 48 0 0 20 0 1 0 408515163 1311244288 312369 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 320128 312369 840 493 0 313433 0
Current children cumulated CPU time: 162.3 s
Current children cumulated vsize: 1280512 KiB
Current children cumulated memory: 1249476 KiB

[startup+222.3 s]
/proc/loadavg: 2.02 2.07 2.07 3/199 9522
/proc/meminfo: memFree=27561456/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=1746808 memory=1706492 CPUtime=222.3 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 422762 0 0 0 22166 64 0 0 20 0 1 0 408515163 1788731392 426623 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 436702 426623 840 493 0 430007 0
Current children cumulated CPU time: 222.3 s
Current children cumulated vsize: 1746808 KiB
Current children cumulated memory: 1706492 KiB

[startup+282.3 s]
/proc/loadavg: 2.01 2.06 2.06 3/199 9528
/proc/meminfo: memFree=27101568/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=2196268 memory=2167360 CPUtime=282.3 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 535453 0 0 0 28148 82 0 0 20 0 1 0 408515163 2248978432 541840 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 549067 541840 840 493 0 542372 0
Current children cumulated CPU time: 282.3 s
Current children cumulated vsize: 2196268 KiB
Current children cumulated memory: 2167360 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.05 2.05 3/199 9532
/proc/meminfo: memFree=26639872/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=2671640 memory=2626108 CPUtime=342.3 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 648580 0 0 0 34134 96 0 0 20 0 1 0 408515163 2735759360 656527 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 667910 656527 840 493 0 661215 0
Current children cumulated CPU time: 342.3 s
Current children cumulated vsize: 2671640 KiB
Current children cumulated memory: 2626108 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/199 9537
/proc/meminfo: memFree=26181144/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=3122468 memory=3084200 CPUtime=402.3 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 761150 0 0 0 40118 112 0 0 20 0 1 0 408515163 3197407232 771050 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 780617 771050 840 493 0 773922 0
Current children cumulated CPU time: 402.3 s
Current children cumulated vsize: 3122468 KiB
Current children cumulated memory: 3084200 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/199 9543
/proc/meminfo: memFree=25721808/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=3571400 memory=3542040 CPUtime=462.29 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 873684 0 0 0 46099 130 0 0 20 0 1 0 408515163 3657113600 885510 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 892850 885510 840 493 0 886155 0
Current children cumulated CPU time: 462.29 s
Current children cumulated vsize: 3571400 KiB
Current children cumulated memory: 3542040 KiB


################
# More data... #
################

Current children cumulated vsize: 10939744 KiB
Current children cumulated memory: 10872992 KiB

[startup+1482.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 9701
/proc/meminfo: memFree=18489396/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=11388148 memory=11330712 CPUtime=1482.19 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 2786439 0 0 0 147727 492 0 0 20 0 1 0 408515163 11661463552 2832678 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 2847037 2832678 840 493 0 2840342 0
Current children cumulated CPU time: 1482.19 s
Current children cumulated vsize: 11388148 KiB
Current children cumulated memory: 11330712 KiB

[startup+1542.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/199 9707
/proc/meminfo: memFree=18028452/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=11847884 memory=11790736 CPUtime=1542.19 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 2899054 0 0 0 153701 518 0 0 20 0 1 0 408515163 12132233216 2947684 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 2961971 2947684 840 493 0 2955276 0
Current children cumulated CPU time: 1542.19 s
Current children cumulated vsize: 11847884 KiB
Current children cumulated memory: 11790736 KiB

[startup+1602.3 s]
/proc/loadavg: 2.06 2.01 2.05 3/199 9712
/proc/meminfo: memFree=17571244/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=12296420 memory=12247360 CPUtime=1602.19 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3011394 0 0 0 159673 546 0 0 20 0 1 0 408515163 12591534080 3061840 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3074105 3061840 840 493 0 3067410 0
Current children cumulated CPU time: 1602.19 s
Current children cumulated vsize: 12296420 KiB
Current children cumulated memory: 12247360 KiB

[startup+1662.3 s]
/proc/loadavg: 2.02 2.01 2.05 3/199 9716
/proc/meminfo: memFree=17111836/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=12744428 memory=12705676 CPUtime=1662.17 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3123666 0 0 0 165645 572 0 0 20 0 1 0 408515163 13050294272 3176419 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3186107 3176419 840 493 0 3179412 0
Current children cumulated CPU time: 1662.17 s
Current children cumulated vsize: 12744428 KiB
Current children cumulated memory: 12705676 KiB

[startup+1722.3 s]
/proc/loadavg: 2.01 2.01 2.05 3/200 9730
/proc/meminfo: memFree=16653456/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=13192436 memory=13161760 CPUtime=1722.18 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3235898 0 0 0 171617 601 0 0 20 0 1 0 408515163 13509054464 3290440 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3298109 3290440 840 493 0 3291414 0
Current children cumulated CPU time: 1722.18 s
Current children cumulated vsize: 13192436 KiB
Current children cumulated memory: 13161760 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 9734
/proc/meminfo: memFree=16192716/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=13765144 memory=13622196 CPUtime=1782.16 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3348692 0 0 0 177589 627 0 0 20 0 1 0 408515163 14095507456 3405549 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3441286 3405549 840 493 0 3434591 0
Current children cumulated CPU time: 1782.16 s
Current children cumulated vsize: 13765144 KiB
Current children cumulated memory: 13622196 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 9738
/proc/meminfo: memFree=15734464/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=14213812 memory=14079124 CPUtime=1842.16 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3461043 0 0 0 183562 654 0 0 20 0 1 0 408515163 14554943488 3519781 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3553453 3519781 840 493 0 3546758 0
Current children cumulated CPU time: 1842.16 s
Current children cumulated vsize: 14213812 KiB
Current children cumulated memory: 14079124 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 9744
/proc/meminfo: memFree=15277120/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=14662216 memory=14535788 CPUtime=1902.15 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3573388 0 0 0 189532 683 0 0 20 0 1 0 408515163 15014109184 3633947 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3665554 3633947 840 493 0 3658859 0
Current children cumulated CPU time: 1902.15 s
Current children cumulated vsize: 14662216 KiB
Current children cumulated memory: 14535788 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 9748
/proc/meminfo: memFree=14817660/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=15110356 memory=14993968 CPUtime=1962.15 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3685611 0 0 0 195503 712 0 0 20 0 1 0 408515163 15473004544 3748492 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3777589 3748492 840 493 0 3770894 0
Current children cumulated CPU time: 1962.15 s
Current children cumulated vsize: 15110356 KiB
Current children cumulated memory: 14993968 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 9752
/proc/meminfo: memFree=14358736/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=15558100 memory=15451884 CPUtime=2022.15 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3797758 0 0 0 201473 742 0 0 20 0 1 0 408515163 15931494400 3862971 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3889525 3862971 840 493 0 3882830 0
Current children cumulated CPU time: 2022.15 s
Current children cumulated vsize: 15558100 KiB
Current children cumulated memory: 15451884 KiB



Maximum VSize exceeded: sending SIGTERM then SIGKILL

[startup+2064.4 s]
/proc/loadavg: 2.00 2.01 2.05 3/199 9755
/proc/meminfo: memFree=14038760/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=15872260 memory=15772296 CPUtime=2064.25 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 4202496 3876497 0 0 0 205663 762 0 0 20 0 1 0 408515163 16253194240 3943074 33554432000 4194304 6210612 140735827782912 140735827747608 140453898349541 0 0 4096 8405026 0 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3968065 3943074 840 493 0 3961370 0
Current children cumulated CPU time: 2064.25 s
Current children cumulated vsize: 15872260 KiB
Current children cumulated memory: 15772296 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2067.21 s]
# the end of solver process 9504 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2056.89 s, system=8.48222 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2065.6 s]
/proc/loadavg: 2.00 2.01 2.05 3/200 9756
/proc/meminfo: memFree=13745748/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=15872260 memory=15772296 CPUtime=2064.43 cores=4-7
/proc/9504/stat : 9504 (cosoco) S 9501 9504 6407 0 -1 512 3876501 0 0 0 205678 765 0 0 20 0 1 0 408515163 16253194240 3943074 33554432000 4194304 6210612 140735827782912 140735827747608 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 4 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3968065 3943074 840 493 0 3961370 0
Current children cumulated CPU time: 2064.43 s
Current children cumulated vsize: 15872260 KiB
Current children cumulated memory: 15772296 KiB

[startup+2066 s]*
/proc/loadavg: 2.00 2.01 2.05 4/200 9756
/proc/meminfo: memFree=13248120/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=15872260 memory=15772296 CPUtime=2064.49 cores=4-7
/proc/9504/stat : 9504 (cosoco) S 9501 9504 6407 0 -1 512 3876501 0 0 0 205683 766 0 0 20 0 1 0 408515163 16253194240 3943074 33554432000 4194304 6210612 140735827782912 140735827747608 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 6 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3968065 3943074 840 493 0 3961370 0
Current children cumulated CPU time: 2064.49 s
Current children cumulated vsize: 15872260 KiB
Current children cumulated memory: 15772296 KiB

[startup+2066.4 s]
/proc/loadavg: 2.00 2.01 2.05 4/200 9756
/proc/meminfo: memFree=13248120/32770688 swapFree=66960972/67108860
[pid=9504] ppid=9501 vsize=15872260 memory=15772296 CPUtime=2064.55 cores=4-7
/proc/9504/stat : 9504 (cosoco) R 9501 9504 6407 0 -1 512 3876501 0 0 0 205688 767 0 0 20 0 1 0 408515163 16253194240 3943074 33554432000 4194304 6210612 140735827782912 140735827747608 5188952 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8308220 8313664 28045312 140735827784715 140735827784817 140735827784817 140735827787726 0
/proc/9504/statm: 3968065 3943074 840 493 0 3961370 0
Current children cumulated CPU time: 2064.55 s
Current children cumulated vsize: 15872260 KiB
Current children cumulated memory: 15772296 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2067.21
CPU time (s): 2065.37
CPU user time (s): 2056.89
CPU system time (s): 8.48222
CPU usage (%): 99.9113
Max. virtual memory (cumulated for all children) (KiB): 15872260
Max. memory (cumulated for all children) (KiB): 15772296

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2056.89
system time used= 8.48222
maximum resident set size= 15772516
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 3876501
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 122201
involuntary context switches= 1820


# summary of solver processes directly reported to runsolver:
#   pid: 9504
#   total CPU time (s): 2065.37
#   total CPU user time (s): 2056.89
#   total CPU system time (s): 8.48222

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.651 second user time and 15.3556 second system time

The end

Launcher Data 

Begin job on node116 at 2019-09-13 00:01:34
IDJOB=4389641
IDBENCH=124102
IDSOLVER=2829
FILE ID=node116/4389641-1568325734
RUNJOBID= node116-1568311136-6439
PBS_JOBID= 21701125
Free space on /tmp= 56388 MiB

SOLVER NAME= cosoco 2
BENCH NAME= XCSP17/Bibd/Bibd-sum-open/Bibd-sum-57-076-20-15-05.xml
COMMAND LINE= cosoco -lc -model -restarts -stick BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4389641-1568325734/watcher-4389641-1568325734 -o /tmp/evaluation-result-4389641-1568325734/solver-4389641-1568325734 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node116-1568311136-6439 --watchdog 2580  cosoco -lc -model -restarts -stick HOME/instance-4389641-1568325734.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 103999ad3c646e4e3c5e6682f24aa76f
RANDOM SEED=1133483096

node116.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2667.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770688 kB
MemFree:        28193036 kB
MemAvailable:   31185852 kB
Buffers:           79588 kB
Cached:          3182800 kB
SwapCached:        46708 kB
Active:          1426340 kB
Inactive:        2496800 kB
Active(anon):     612460 kB
Inactive(anon):    88900 kB
Active(file):     813880 kB
Inactive(file):  2407900 kB
Unevictable:      110244 kB
Mlocked:          110244 kB
SwapTotal:      67108860 kB
SwapFree:       66960972 kB
Dirty:            149904 kB
Writeback:         77852 kB
AnonPages:        725464 kB
Mapped:            45560 kB
Shmem:             31876 kB
Slab:             246464 kB
SReclaimable:     173740 kB
SUnreclaim:        72724 kB
KernelStack:        3520 kB
PageTables:         6488 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494204 kB
Committed_AS:    1138196 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350212 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     86016 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      116160 kB
DirectMap2M:    33429504 kB

Free space on /tmp at the end= 56408 MiB
End job on node116 at 2019-09-13 00:36:01