Trace number 4398190

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
cosoco 2.O parallel? (TO) 2520.73 2523.34

General information on the benchmark

NameBibd/Bibd-sum-open/
Bibd-sum-22-033-12-08-04.xml
MD5SUMf3f766a62b1d9930afd3b37b1c8667ba
Bench CategoryCSP (decision problem)
Best result obtained on this benchmark
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark
Satisfiable
(Un)Satisfiability was proved
Number of variables726
Number of constraints287
Number of domains1
Minimum domain size2
Maximum domain size2
Distribution of domain sizes[{"size":2,"count":726}]
Minimum variable degree24
Maximum variable degree24
Distribution of variable degrees[{"degree":24,"count":726}]
Minimum constraint arity22
Maximum constraint arity726
Distribution of constraint arities[{"arity":22,"count":33},{"arity":33,"count":22},{"arity":66,"count":231},{"arity":726,"count":1}]
Number of extensional constraints0
Number of intensional constraints0
Distribution of constraint types[{"type":"lex","count":1},{"type":"sum","count":286}]
Optimization problemNO
Type of objective

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15479 MiB free)
  memory of node 1: 16384 MiB (15671 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4398190-1568601278/watcher-4398190-1568601278 -o /tmp/evaluation-result-4398190-1568601278/solver-4398190-1568601278 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node143-1568595839-29710 --watchdog 2580 cosoco -model -nbcores=8 HOME/instance-4398190-1568601278.xml 

running on 8 cores: 0,2,4,6,1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 20160 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 20190 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 31744000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 31795200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=30632, runsolver pid=30629
Current StackSize limit: 8192 KiB


[startup+0.100155 s]*
/proc/loadavg: 4.24 7.09 7.83 2/194 30633
/proc/meminfo: memFree=31894128/32770624 swapFree=67025788/67108860
[pid=30632] ppid=30629 vsize=33180 memory=9544 CPUtime=0.1 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 2573 0 0 0 10 0 0 0 20 0 1 0 436065168 33976320 2386 33554432000 4194304 6211092 140733685008416 140733685000808 139969168894949 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 8295 2386 842 493 0 1600 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 33180 KiB
Current children cumulated memory: 9544 KiB

[startup+0.209039 s]*
/proc/loadavg: 4.24 7.09 7.83 2/194 30633
/proc/meminfo: memFree=31892640/32770624 swapFree=67025788/67108860
[pid=30632] ppid=30629 vsize=34332 memory=10600 CPUtime=0.21 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 2875 0 0 0 21 0 0 0 20 0 1 0 436065168 35155968 2650 33554432000 4194304 6211092 140733685008416 140733685000808 139969168894949 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 8583 2650 842 493 0 1888 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 34332 KiB
Current children cumulated memory: 10600 KiB

[startup+0.308786 s]*
/proc/loadavg: 3.90 6.97 7.79 2/194 30633
/proc/meminfo: memFree=31891896/32770624 swapFree=67025788/67108860
[pid=30632] ppid=30629 vsize=35428 memory=11920 CPUtime=0.3 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 3153 0 0 0 30 0 0 0 20 0 1 0 436065168 36278272 2980 33554432000 4194304 6211092 140733685008416 140733685000808 5012856 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 8857 2980 842 493 0 2162 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 35428 KiB
Current children cumulated memory: 11920 KiB

[startup+0.700326 s]
/proc/loadavg: 3.90 6.97 7.79 2/194 30633
/proc/meminfo: memFree=31889664/32770624 swapFree=67025788/67108860
[pid=30632] ppid=30629 vsize=39728 memory=16204 CPUtime=0.69 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 4216 0 0 0 69 0 0 0 20 0 1 0 436065168 40681472 4051 33554432000 4194304 6211092 140733685008416 140733684989624 5446924 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 9932 4051 842 493 0 3237 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 39728 KiB
Current children cumulated memory: 16204 KiB

[startup+1.50033 s]
/proc/loadavg: 3.90 6.97 7.79 2/194 30633
/proc/meminfo: memFree=31884456/32770624 swapFree=67025788/67108860
[pid=30632] ppid=30629 vsize=48784 memory=24852 CPUtime=1.49 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 6435 0 0 0 148 1 0 0 20 0 1 0 436065168 49954816 6213 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 12196 6213 842 493 0 5501 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 48784 KiB
Current children cumulated memory: 24852 KiB

[startup+3.10879 s]
/proc/loadavg: 3.90 6.97 7.79 2/194 30633
/proc/meminfo: memFree=31861284/32770624 swapFree=67025832/67108860
[pid=30632] ppid=30629 vsize=66912 memory=42804 CPUtime=3.09 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 10999 0 0 0 307 2 0 0 20 0 1 0 436065168 68517888 10701 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 16728 10701 842 493 0 10033 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 66912 KiB
Current children cumulated memory: 42804 KiB

[startup+6.30032 s]
/proc/loadavg: 3.67 6.87 7.75 2/194 30633
/proc/meminfo: memFree=31828516/32770624 swapFree=67025848/67108860
[pid=30632] ppid=30629 vsize=103444 memory=78392 CPUtime=6.28 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 19970 0 0 0 626 2 0 0 20 0 1 0 436065168 105926656 19598 33554432000 4194304 6211092 140733685008416 140733684989624 139969178839946 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 25861 19598 842 493 0 19166 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 103444 KiB
Current children cumulated memory: 78392 KiB

[startup+12.7003 s]
/proc/loadavg: 3.45 6.78 7.71 2/194 30633
/proc/meminfo: memFree=31762096/32770624 swapFree=67025852/67108860
[pid=30632] ppid=30629 vsize=173928 memory=149284 CPUtime=12.68 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 37810 0 0 0 1264 4 0 0 20 0 1 0 436065168 178102272 37321 33554432000 4194304 6211092 140733685008416 140733684989624 5439794 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 43482 37321 842 493 0 36787 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 173928 KiB
Current children cumulated memory: 149284 KiB

[startup+25.5003 s]
/proc/loadavg: 3.08 6.59 7.64 2/194 30636
/proc/meminfo: memFree=31616304/32770624 swapFree=67025852/67108860
[pid=30632] ppid=30629 vsize=318924 memory=292816 CPUtime=25.47 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 73330 0 0 0 2537 10 0 0 20 0 1 0 436065168 326578176 73204 33554432000 4194304 6211092 140733685008416 140733684989624 5478665 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 79731 73204 842 493 0 73036 0
Current children cumulated CPU time: 25.47 s
Current children cumulated vsize: 318924 KiB
Current children cumulated memory: 292816 KiB

[startup+51.1088 s]
/proc/loadavg: 2.26 6.05 7.43 2/194 30636
/proc/meminfo: memFree=31326168/32770624 swapFree=67025852/67108860
[pid=30632] ppid=30629 vsize=611604 memory=576720 CPUtime=51.07 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 142860 0 0 0 5087 20 0 0 20 0 1 0 436065168 626282496 144180 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 152901 144180 842 493 0 146206 0
Current children cumulated CPU time: 51.07 s
Current children cumulated vsize: 611604 KiB
Current children cumulated memory: 576720 KiB

[startup+102.3 s]
/proc/loadavg: 1.55 5.27 7.09 2/192 30639
/proc/meminfo: memFree=30754728/32770624 swapFree=67025852/67108860
[pid=30632] ppid=30629 vsize=1176984 memory=1148844 CPUtime=102.23 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 280261 0 0 0 10185 38 0 0 20 0 1 0 436065168 1205231616 287211 33554432000 4194304 6211092 140733685008416 140733684989624 5478706 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 294246 287211 842 493 0 287551 0
Current children cumulated CPU time: 102.23 s
Current children cumulated vsize: 1176984 KiB
Current children cumulated memory: 1148844 KiB

[startup+162.3 s]
/proc/loadavg: 1.20 4.49 6.71 2/192 30644
/proc/meminfo: memFree=30087540/32770624 swapFree=67025852/67108860
[pid=30632] ppid=30629 vsize=1844272 memory=1815972 CPUtime=162.21 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 441016 0 0 0 16163 58 0 0 20 0 1 0 436065168 1888534528 453993 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 461068 453993 842 493 0 454373 0
Current children cumulated CPU time: 162.21 s
Current children cumulated vsize: 1844272 KiB
Current children cumulated memory: 1815972 KiB

[startup+222.3 s]
/proc/loadavg: 1.07 3.86 6.35 2/192 30647
/proc/meminfo: memFree=29418520/32770624 swapFree=67025852/67108860
[pid=30632] ppid=30629 vsize=2526036 memory=2485832 CPUtime=222.18 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 601303 0 0 0 22139 79 0 0 20 0 1 0 436065168 2586660864 621458 33554432000 4194304 6211092 140733685008416 140733684989624 5452157 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 631509 621458 842 493 0 624814 0
Current children cumulated CPU time: 222.18 s
Current children cumulated vsize: 2526036 KiB
Current children cumulated memory: 2485832 KiB

[startup+282.3 s]
/proc/loadavg: 1.03 3.34 6.02 2/192 30650
/proc/meminfo: memFree=28749728/32770624 swapFree=67025856/67108860
[pid=30632] ppid=30629 vsize=3229896 memory=3151776 CPUtime=282.15 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 761851 0 0 0 28114 101 0 0 20 0 1 0 436065168 3307413504 787944 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 807474 787944 842 493 0 800779 0
Current children cumulated CPU time: 282.15 s
Current children cumulated vsize: 3229896 KiB
Current children cumulated memory: 3151776 KiB

[startup+342.3 s]
/proc/loadavg: 1.01 2.91 5.71 2/191 30655
/proc/meminfo: memFree=28080760/32770624 swapFree=67025856/67108860
[pid=30632] ppid=30629 vsize=3866796 memory=3819288 CPUtime=342.13 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 921258 0 0 0 34091 122 0 0 20 0 1 0 436065168 3959599104 954822 33554432000 4194304 6211092 140733685008416 140733684989624 5456306 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 966699 954822 842 493 0 960004 0
Current children cumulated CPU time: 342.13 s
Current children cumulated vsize: 3866796 KiB
Current children cumulated memory: 3819288 KiB

[startup+402.3 s]
/proc/loadavg: 1.00 2.56 5.41 2/192 30665
/proc/meminfo: memFree=27403392/32770624 swapFree=67026180/67108860
[pid=30632] ppid=30629 vsize=4603540 memory=4488572 CPUtime=402.11 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 1082138 0 0 0 40063 148 0 0 20 0 1 0 436065168 4714024960 1122143 33554432000 4194304 6211092 140733685008416 140733684989624 5443112 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 1150885 1122143 842 493 0 1144190 0
Current children cumulated CPU time: 402.11 s
Current children cumulated vsize: 4603540 KiB
Current children cumulated memory: 4488572 KiB

[startup+462.3 s]
/proc/loadavg: 1.00 2.28 5.14 2/192 30668
/proc/meminfo: memFree=26735776/32770624 swapFree=67026180/67108860
[pid=30632] ppid=30629 vsize=5240968 memory=5154756 CPUtime=462.08 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 1241757 0 0 0 46037 171 0 0 20 0 1 0 436065168 5366751232 1288689 33554432000 4194304 6211092 140733685008416 140733684989624 5446238 0 0 4096 8405026 0 0 0 17 2 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 1310242 1288689 842 493 0 1303547 0
Current children cumulated CPU time: 462.08 s
Current children cumulated vsize: 5240968 KiB
Current children cumulated memory: 5154756 KiB


################
# More data... #
################

Current children cumulated vsize: 21300376 KiB
Current children cumulated memory: 21149360 KiB

[startup+1962.3 s]
/proc/loadavg: 1.00 1.01 1.82 2/192 30823
/proc/meminfo: memFree=10039768/32770624 swapFree=67026324/67108860
[pid=30632] ppid=30629 vsize=21936088 memory=21815576 CPUtime=1961.53 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 5231454 0 0 0 195124 1029 0 0 20 0 1 0 436065168 22462554112 5453894 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 5484022 5453894 842 493 0 5477327 0
Current children cumulated CPU time: 1961.53 s
Current children cumulated vsize: 21936088 KiB
Current children cumulated memory: 21815576 KiB

[startup+2022.3 s]
/proc/loadavg: 1.00 1.01 1.77 2/192 30826
/proc/meminfo: memFree=9371376/32770624 swapFree=67026324/67108860
[pid=30632] ppid=30629 vsize=22570876 memory=22481256 CPUtime=2021.52 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 5390608 0 0 0 201079 1073 0 0 20 0 1 0 436065168 23112577024 5620314 33554432000 4194304 6211092 140733685008416 140733684989624 5452146 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 5642719 5620314 842 493 0 5636024 0
Current children cumulated CPU time: 2021.52 s
Current children cumulated vsize: 22570876 KiB
Current children cumulated memory: 22481256 KiB

[startup+2082.3 s]
/proc/loadavg: 1.00 1.01 1.72 2/191 30829
/proc/meminfo: memFree=8704928/32770624 swapFree=67026324/67108860
[pid=30632] ppid=30629 vsize=23205400 memory=23146332 CPUtime=2081.5 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 5549773 0 0 0 207028 1122 0 0 20 0 1 0 436065168 23762329600 5786583 33554432000 4194304 6211092 140733685008416 140733684989624 5446648 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 5801350 5786583 842 493 0 5794655 0
Current children cumulated CPU time: 2081.5 s
Current children cumulated vsize: 23205400 KiB
Current children cumulated memory: 23146332 KiB

[startup+2142.3 s]
/proc/loadavg: 1.00 1.01 1.68 2/191 30834
/proc/meminfo: memFree=8039280/32770624 swapFree=67026324/67108860
[pid=30632] ppid=30629 vsize=24260084 memory=23810292 CPUtime=2141.48 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 5709559 0 0 0 212978 1170 0 0 20 0 1 0 436065168 24842326016 5952573 33554432000 4194304 6211092 140733685008416 140733684989624 5440650 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 6065021 5952573 842 493 0 6058326 0
Current children cumulated CPU time: 2141.48 s
Current children cumulated vsize: 24260084 KiB
Current children cumulated memory: 23810292 KiB

[startup+2202.3 s]
/proc/loadavg: 1.00 1.01 1.64 2/191 30844
/proc/meminfo: memFree=7373164/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=24894080 memory=24475192 CPUtime=2201.46 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 5868519 0 0 0 218930 1216 0 0 20 0 1 0 436065168 25491537920 6118798 33554432000 4194304 6211092 140733685008416 140733684989624 5443072 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 6223520 6118798 842 493 0 6216825 0
Current children cumulated CPU time: 2201.46 s
Current children cumulated vsize: 24894080 KiB
Current children cumulated memory: 24475192 KiB

[startup+2262.3 s]
/proc/loadavg: 1.00 1.01 1.59 2/191 30847
/proc/meminfo: memFree=6707260/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=25529000 memory=25138752 CPUtime=2261.43 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 6027775 0 0 0 224882 1261 0 0 20 0 1 0 436065168 26141696000 6284688 33554432000 4194304 6211092 140733685008416 140733684989624 5441074 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 6382250 6284688 842 493 0 6375555 0
Current children cumulated CPU time: 2261.43 s
Current children cumulated vsize: 25529000 KiB
Current children cumulated memory: 25138752 KiB

[startup+2322.3 s]
/proc/loadavg: 1.00 1.01 1.56 2/191 30852
/proc/meminfo: memFree=6040664/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=26163260 memory=25803644 CPUtime=2321.41 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 6186801 0 0 0 230832 1309 0 0 20 0 1 0 436065168 26791178240 6450911 33554432000 4194304 6211092 140733685008416 140733684989624 5446260 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 6540815 6450911 842 493 0 6534120 0
Current children cumulated CPU time: 2321.41 s
Current children cumulated vsize: 26163260 KiB
Current children cumulated memory: 25803644 KiB

[startup+2382.3 s]
/proc/loadavg: 1.00 1.01 1.52 2/191 30855
/proc/meminfo: memFree=5375020/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=26797520 memory=26468540 CPUtime=2381.4 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 6345794 0 0 0 236783 1357 0 0 20 0 1 0 436065168 27440660480 6617135 33554432000 4194304 6211092 140733685008416 140733684989624 5452157 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 6699380 6617135 842 493 0 6692685 0
Current children cumulated CPU time: 2381.4 s
Current children cumulated vsize: 26797520 KiB
Current children cumulated memory: 26468540 KiB

[startup+2442.3 s]
/proc/loadavg: 1.00 1.01 1.49 2/191 30858
/proc/meminfo: memFree=4708868/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=27556084 memory=27133416 CPUtime=2441.38 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 6505353 0 0 0 242733 1405 0 0 20 0 1 0 436065168 28217430016 6783354 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 6889021 6783354 842 493 0 6882326 0
Current children cumulated CPU time: 2441.38 s
Current children cumulated vsize: 27556084 KiB
Current children cumulated memory: 27133416 KiB

[startup+2502.3 s]
/proc/loadavg: 1.00 1.01 1.46 2/191 30863
/proc/meminfo: memFree=4045100/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=28190080 memory=27796000 CPUtime=2501.35 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 6664301 0 0 0 248680 1455 0 0 20 0 1 0 436065168 28866641920 6949000 33554432000 4194304 6211092 140733685008416 140733684989624 139969169821113 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 7047520 6949000 842 493 0 7040825 0
Current children cumulated CPU time: 2501.35 s
Current children cumulated vsize: 28190080 KiB
Current children cumulated memory: 27796000 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 1.00 1.01 1.45 2/191 30863
/proc/meminfo: memFree=3846540/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=28377256 memory=27991528 CPUtime=2519.06 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 4202496 6711184 0 0 0 250434 1472 0 0 20 0 1 0 436065168 29058310144 6997882 33554432000 4194304 6211092 140733685008416 140733684989624 139969168894949 0 0 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 7094314 6997882 842 493 0 7087619 0
Current children cumulated CPU time: 2519.06 s
Current children cumulated vsize: 28377256 KiB
Current children cumulated memory: 27991528 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Sending SIGKILL to process tree (bottom up)
??? end of watcher thread

[startup+2523.34 s]
# the end of solver process 30632 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2504.58 s, system=16.1462 s

Solver just ended.
Dumping a history of the last processes samples

[startup+2520.8 s]
/proc/loadavg: 1.00 1.01 1.45 2/191 30863
/proc/meminfo: memFree=3846540/32770624 swapFree=67026332/67108860
[pid=30632] ppid=30629 vsize=28377256 memory=27991528 CPUtime=2519.19 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 512 6711189 0 0 0 250442 1477 0 0 20 0 1 0 436065168 29058310144 6997882 33554432000 4194304 6211092 140733685008416 140733684989624 5189550 0 16384 4096 8405026 0 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 7094314 6997882 842 493 0 7087619 0
Current children cumulated CPU time: 2519.19 s
Current children cumulated vsize: 28377256 KiB
Current children cumulated memory: 27991528 KiB

[startup+2521.6 s]
/proc/loadavg: 1.08 1.03 1.46 2/192 30864
/proc/meminfo: memFree=3353168/32770624 swapFree=67026336/67108860
[pid=30632] ppid=30629 vsize=28377256 memory=27991528 CPUtime=2519.31 cores=0-7
/proc/30632/stat : 30632 (cosoco) R 30629 30632 29683 0 -1 512 6711189 0 0 0 250452 1479 0 0 20 0 1 0 436065168 29058310144 6997882 33554432000 4194304 6211092 140733685008416 140733684989624 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 7094314 6997882 842 493 0 7087619 0
Current children cumulated CPU time: 2519.31 s
Current children cumulated vsize: 28377256 KiB
Current children cumulated memory: 27991528 KiB

[startup+2522 s]*
/proc/loadavg: 1.08 1.03 1.46 1/192 30864
/proc/meminfo: memFree=2852220/32770624 swapFree=67026336/67108860
[pid=30632] ppid=30629 vsize=28377256 memory=27991528 CPUtime=2519.39 cores=0-7
/proc/30632/stat : 30632 (cosoco) S 30629 30632 29683 0 -1 512 6711189 0 0 0 250458 1481 0 0 20 0 1 0 436065168 29058310144 6997882 33554432000 4194304 6211092 140733685008416 140733684989624 5189550 0 16384 4096 8405026 18446744071580971376 0 0 17 5 0 0 0 0 0 8312316 8317760 12832768 140733685011473 140733685011565 140733685011565 140733685014478 0
/proc/30632/statm: 7094314 6997882 842 493 0 7087619 0
Current children cumulated CPU time: 2519.39 s
Current children cumulated vsize: 28377256 KiB
Current children cumulated memory: 27991528 KiB

Child ended because it received signal 9 (SIGKILL)
Real time (s): 2523.34
CPU time (s): 2520.73
CPU user time (s): 2504.58
CPU system time (s): 16.1462
CPU usage (%): 99.8967
Max. virtual memory (cumulated for all children) (KiB): 28377256
Max. memory (cumulated for all children) (KiB): 27991528

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2504.58
system time used= 16.1462
maximum resident set size= 27991560
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 6711189
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 121105
involuntary context switches= 2043


# summary of solver processes directly reported to runsolver:
#   pid: 30632
#   total CPU time (s): 2520.73
#   total CPU user time (s): 2504.58
#   total CPU system time (s): 16.1462

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.52064 second user time and 17.0481 second system time

The end

Launcher Data 

Begin job on node143 at 2019-09-16 04:33:57
IDJOB=4398190
IDBENCH=124081
IDSOLVER=2851
FILE ID=node143/4398190-1568601278
RUNJOBID= node143-1568595839-29710
PBS_JOBID= 21703394
Free space on /tmp= 59276 MiB

SOLVER NAME= cosoco 2.O parallel
BENCH NAME= XCSP17/Bibd/Bibd-sum-open/Bibd-sum-22-033-12-08-04.xml
COMMAND LINE= cosoco -model -nbcores=NBCORE BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4398190-1568601278/watcher-4398190-1568601278 -o /tmp/evaluation-result-4398190-1568601278/solver-4398190-1568601278 -C 20160 -W 2520 -M 31000 --bin-var /tmp/runsolver-exch-node143-1568595839-29710 --watchdog 2580  cosoco -model -nbcores=8 HOME/instance-4398190-1568601278.xml

TIME LIMIT= 20160 seconds
MEMORY LIMIT= 31000 MiB
NBCORE= 8

MD5SUM BENCH= f3f766a62b1d9930afd3b37b1c8667ba
RANDOM SEED=1420688607

node143.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.73
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31899164 kB
MemAvailable:   31820684 kB
Buffers:            5312 kB
Cached:           188676 kB
SwapCached:        62608 kB
Active:            30692 kB
Inactive:         251016 kB
Active(anon):       7292 kB
Inactive(anon):   178408 kB
Active(file):      23400 kB
Inactive(file):    72608 kB
Unevictable:      109672 kB
Mlocked:          109672 kB
SwapTotal:      67108860 kB
SwapFree:       67025788 kB
Dirty:              2780 kB
Writeback:             0 kB
AnonPages:        138384 kB
Mapped:            43820 kB
Shmem:             89280 kB
Slab:             182244 kB
SReclaimable:     153456 kB
SUnreclaim:        28788 kB
KernelStack:        3744 kB
PageTables:         5620 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     507288 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     75776 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      132480 kB
DirectMap2M:    33413120 kB

Free space on /tmp at the end= 58800 MiB
End job on node143 at 2019-09-16 05:16:00