Trace number 4299122

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
NACRE 1.0.4? (TO) 2519.67 2520.02

General information on the benchmark

NameQuasigroups_mini/
QuasiGroup-v7-16_mini_c18.xml
MD5SUM90b0ae5457c21a968d0990b7ba14ba70
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark1.06797
Satisfiable
(Un)Satisfiability was proved
Number of variables512
Number of constraints528
Number of domains1
Minimum domain size16
Maximum domain size16
Distribution of domain sizes[{"size":16,"count":496}]
Minimum variable degree0
Maximum variable degree34
Distribution of variable degrees[{"degree":0,"count":16},{"degree":2,"count":240},{"degree":33,"count":16},{"degree":34,"count":240}]
Minimum constraint arity1
Maximum constraint arity18
Distribution of constraint arities[{"arity":1,"count":16},{"arity":16,"count":32},{"arity":18,"count":480}]
Number of extensional constraints0
Number of intensional constraints16
Distribution of constraint types[{"type":"intension","count":16},{"type":"allDifferent","count":32},{"type":"element","count":480}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.01	c NACRE 1.0.4
2519.66/2520.01	c
2519.66/2520.01	c # time = 2519.66
2519.66/2520.01	c # Dec = 3355013
2519.66/2520.01	c # Propagate Calls = 4291691350
2519.66/2520.01	c # assignAt Calls = 5587529
2519.66/2520.01	c # removeAt Calls = 510461586
2519.66/2520.01	c # Run = 5592
2519.66/2520.01	c # Conflicts = 3143821
2519.66/2520.01	c # Memory used = 893.136 Mb
2519.66/2520.01	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (9595 MiB free)
  memory of node 1: 16384 MiB (7303 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4299122-1532782731/watcher-4299122-1532782731 -o /tmp/evaluation-result-4299122-1532782731/solver-4299122-1532782731 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node103-1532775163-27027 --watchdog 2580 nacre_mini_release HOME/instance-4299122-1532782731.xml -ca -l100 -cm 

running on 4 cores: 0,2,4,6

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=27820, runsolver pid=27817

[startup+0.100108 s]*
/proc/loadavg: 2.38 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17301476/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=29332 memory=5936 CPUtime=0.1 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1686 0 0 0 10 0 0 0 20 0 1 0 95836253 30035968 1484 33554432000 4194304 4933779 140735356962976 140735356959976 4359544 0 0 4096 24578 0 0 0 17 0 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 7333 1484 614 181 0 950 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 29332 KiB
Current children cumulated memory: 5936 KiB

[startup+0.20956 s]*
/proc/loadavg: 2.38 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17300204/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=29996 memory=6736 CPUtime=0.21 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1852 0 0 0 21 0 0 0 20 0 1 0 95836253 30715904 1684 33554432000 4194304 4933779 140735356962976 140735356960728 140106909832960 0 0 4096 24578 0 0 0 17 0 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 7499 1684 614 181 0 1116 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 29996 KiB
Current children cumulated memory: 6736 KiB

[startup+0.309016 s]*
/proc/loadavg: 2.38 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17299384/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=30628 memory=7372 CPUtime=0.31 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 2020 0 0 0 31 0 0 0 20 0 1 0 95836253 31363072 1843 33554432000 4194304 4933779 140735356962976 140735356960728 140106915976544 0 0 4096 24578 0 0 0 17 0 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 7657 1843 614 181 0 1274 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 30628 KiB
Current children cumulated memory: 7372 KiB

[startup+0.700242 s]
/proc/loadavg: 2.38 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17298268/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=33332 memory=10072 CPUtime=0.69 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 2703 0 0 0 69 0 0 0 20 0 1 0 95836253 34131968 2518 33554432000 4194304 4933779 140735356962976 140735356960728 140106915986405 0 0 4096 24578 0 0 0 17 0 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 8333 2518 614 181 0 1950 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 33332 KiB
Current children cumulated memory: 10072 KiB

[startup+1.50025 s]
/proc/loadavg: 2.38 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17294920/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=38420 memory=15072 CPUtime=1.49 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 3963 0 0 0 149 0 0 0 20 0 1 0 95836253 39342080 3768 33554432000 4194304 4933779 140735356962976 140735356961128 4358233 0 0 4096 24578 0 0 0 17 0 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 9605 3768 614 181 0 3222 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 38420 KiB
Current children cumulated memory: 15072 KiB

[startup+3.1093 s]
/proc/loadavg: 2.38 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17282148/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=48588 memory=25116 CPUtime=3.09 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 9652 0 0 0 309 0 0 0 20 0 1 0 95836253 49754112 6279 33554432000 4194304 4933779 140735356962976 140735356960744 4358240 0 0 4096 24578 0 0 0 17 0 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 12147 6279 614 181 0 5764 0
Current children cumulated CPU time: 3.09 s
Current children cumulated vsize: 48588 KiB
Current children cumulated memory: 25116 KiB

[startup+6.30035 s]
/proc/loadavg: 2.35 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17261828/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=68188 memory=44660 CPUtime=6.28 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 20382 0 0 0 626 2 0 0 20 0 1 0 95836253 69824512 11165 33554432000 4194304 4933779 140735356962976 140735356960728 4358719 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 17047 11165 614 181 0 10664 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 68188 KiB
Current children cumulated memory: 44660 KiB

[startup+12.7004 s]
/proc/loadavg: 2.32 2.13 2.02 3/187 27821
/proc/meminfo: memFree=17233956/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=90760 memory=67400 CPUtime=12.69 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 36437 0 0 0 1265 4 0 0 20 0 1 0 95836253 92938240 16850 33554432000 4194304 4933779 140735356962976 140735356960728 140106910640539 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 22690 16850 614 181 0 16307 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 90760 KiB
Current children cumulated memory: 67400 KiB

[startup+25.5004 s]
/proc/loadavg: 2.25 2.12 2.02 3/187 27821
/proc/meminfo: memFree=17189548/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=129264 memory=105572 CPUtime=25.49 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 78248 0 0 0 2542 7 0 0 20 0 1 0 95836253 132366336 26393 33554432000 4194304 4933779 140735356962976 140735356960728 4388395 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 32316 26393 614 181 0 25933 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 129264 KiB
Current children cumulated memory: 105572 KiB

[startup+51.1097 s]
/proc/loadavg: 2.16 2.11 2.02 3/187 27821
/proc/meminfo: memFree=17120556/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=185944 memory=162400 CPUtime=51.09 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 123584 0 0 0 5097 12 0 0 20 0 1 0 95836253 190406656 40600 33554432000 4194304 4933779 140735356962976 140735356960728 4437393 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 46486 40600 614 181 0 40103 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 185944 KiB
Current children cumulated memory: 162400 KiB

[startup+102.3 s]
/proc/loadavg: 2.07 2.09 2.02 3/187 27829
/proc/meminfo: memFree=17031028/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=253400 memory=229988 CPUtime=102.28 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 181985 0 0 0 10210 18 0 0 20 0 1 0 95836253 259481600 57497 33554432000 4194304 4933779 140735356962976 140735356960728 4358566 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 63350 57497 614 181 0 56967 0
Current children cumulated CPU time: 102.28 s
Current children cumulated vsize: 253400 KiB
Current children cumulated memory: 229988 KiB

[startup+162.3 s]
/proc/loadavg: 2.03 2.08 2.02 3/188 27853
/proc/meminfo: memFree=16954700/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=311196 memory=286736 CPUtime=162.28 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 249845 0 0 0 16203 25 0 0 20 0 1 0 95836253 318664704 71684 33554432000 4194304 4933779 140735356962976 140735356960680 4358228 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 77799 71684 614 181 0 71416 0
Current children cumulated CPU time: 162.28 s
Current children cumulated vsize: 311196 KiB
Current children cumulated memory: 286736 KiB

[startup+222.3 s]
/proc/loadavg: 2.01 2.06 2.02 3/188 27915
/proc/meminfo: memFree=19747676/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=350628 memory=326268 CPUtime=222.26 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 315803 0 0 0 22194 32 0 0 20 0 1 0 95836253 359043072 81567 33554432000 4194304 4933779 140735356962976 140735356960680 4437389 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 87657 81567 614 181 0 81274 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 350628 KiB
Current children cumulated memory: 326268 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.05 2.02 3/188 27916
/proc/meminfo: memFree=19628072/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=385592 memory=361304 CPUtime=282.24 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 342725 0 0 0 28188 36 0 0 20 0 1 0 95836253 394846208 90326 33554432000 4194304 4933779 140735356962976 140735356960728 4476129 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 96398 90326 614 181 0 90015 0
Current children cumulated CPU time: 282.24 s
Current children cumulated vsize: 385592 KiB
Current children cumulated memory: 361304 KiB

[startup+342.3 s]
/proc/loadavg: 2.11 2.07 2.04 3/188 27917
/proc/meminfo: memFree=19530140/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=414840 memory=390720 CPUtime=342.21 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 384118 0 0 0 34179 42 0 0 20 0 1 0 95836253 424796160 97680 33554432000 4194304 4933779 140735356962976 140735356960680 140106915986173 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 103710 97680 614 181 0 97327 0
Current children cumulated CPU time: 342.21 s
Current children cumulated vsize: 414840 KiB
Current children cumulated memory: 390720 KiB

[startup+402.3 s]
/proc/loadavg: 2.04 2.06 2.04 3/188 27918
/proc/meminfo: memFree=19429924/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=445256 memory=421324 CPUtime=402.18 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 407999 0 0 0 40174 44 0 0 20 0 1 0 95836253 455942144 105331 33554432000 4194304 4933779 140735356962976 140735356960680 4358400 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 111314 105331 614 181 0 104931 0
Current children cumulated CPU time: 402.18 s
Current children cumulated vsize: 445256 KiB
Current children cumulated memory: 421324 KiB

[startup+462.3 s]
/proc/loadavg: 2.02 2.05 2.04 3/187 27919
/proc/meminfo: memFree=19346764/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=474504 memory=450776 CPUtime=462.16 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 418674 0 0 0 46170 46 0 0 20 0 1 0 95836253 485892096 112694 33554432000 4194304 4933779 140735356962976 140735356960680 4358528 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 118626 112694 614 181 0 112243 0
Current children cumulated CPU time: 462.16 s
Current children cumulated vsize: 474504 KiB
Current children cumulated memory: 450776 KiB


################
# More data... #
################

/proc/27820/statm: 203259 197183 614 181 0 196876 0
Current children cumulated CPU time: 1722.03 s
Current children cumulated vsize: 813036 KiB
Current children cumulated memory: 788732 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27958
/proc/meminfo: memFree=18542440/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=823272 memory=798932 CPUtime=1782.04 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 869407 0 0 0 178103 101 0 0 20 0 1 0 95836253 843030528 199733 33554432000 4194304 4933779 140735356962976 140735356960680 4437572 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 205818 199733 614 181 0 199435 0
Current children cumulated CPU time: 1782.04 s
Current children cumulated vsize: 823272 KiB
Current children cumulated memory: 798932 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27959
/proc/meminfo: memFree=18521956/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=836392 memory=811884 CPUtime=1842.04 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 888085 0 0 0 184101 103 0 0 20 0 1 0 95836253 856465408 202971 33554432000 4194304 4933779 140735356962976 140735356960680 4358022 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 209098 202971 614 181 0 202715 0
Current children cumulated CPU time: 1842.04 s
Current children cumulated vsize: 836392 KiB
Current children cumulated memory: 811884 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27968
/proc/meminfo: memFree=18503372/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=845904 memory=821176 CPUtime=1902.03 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 912709 0 0 0 190098 105 0 0 20 0 1 0 95836253 866205696 205294 33554432000 4194304 4933779 140735356962976 140735356960488 4358089 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 211476 205294 614 181 0 205093 0
Current children cumulated CPU time: 1902.03 s
Current children cumulated vsize: 845904 KiB
Current children cumulated memory: 821176 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27969
/proc/meminfo: memFree=18487624/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=851024 memory=826000 CPUtime=1962.03 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 936701 0 0 0 196096 107 0 0 20 0 1 0 95836253 871448576 206500 33554432000 4194304 4933779 140735356962976 140735356960488 4437570 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 212756 206500 614 181 0 206373 0
Current children cumulated CPU time: 1962.03 s
Current children cumulated vsize: 851024 KiB
Current children cumulated memory: 826000 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27970
/proc/meminfo: memFree=18469024/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=857880 memory=832780 CPUtime=2022.02 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 958900 0 0 0 202092 110 0 0 20 0 1 0 95836253 878469120 208195 33554432000 4194304 4933779 140735356962976 140735356960680 4358028 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 214470 208195 614 181 0 208087 0
Current children cumulated CPU time: 2022.02 s
Current children cumulated vsize: 857880 KiB
Current children cumulated memory: 832780 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27971
/proc/meminfo: memFree=18449424/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=864764 memory=839548 CPUtime=2082.03 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 989597 0 0 0 208090 113 0 0 20 0 1 0 95836253 885518336 209887 33554432000 4194304 4933779 140735356962976 140735356960664 4358089 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 216191 209887 614 181 0 209808 0
Current children cumulated CPU time: 2082.03 s
Current children cumulated vsize: 864764 KiB
Current children cumulated memory: 839548 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27972
/proc/meminfo: memFree=18435048/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=869996 memory=844836 CPUtime=2142.02 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1016161 0 0 0 214087 115 0 0 20 0 1 0 95836253 890875904 211209 33554432000 4194304 4933779 140735356962976 140735356960680 4437547 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 217499 211209 614 181 0 211116 0
Current children cumulated CPU time: 2142.02 s
Current children cumulated vsize: 869996 KiB
Current children cumulated memory: 844836 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27973
/proc/meminfo: memFree=18421424/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=874388 memory=849180 CPUtime=2202.02 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1043960 0 0 0 220085 117 0 0 20 0 1 0 95836253 895373312 212295 33554432000 4194304 4933779 140735356962976 140735356960728 4358603 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 218597 212295 614 181 0 212214 0
Current children cumulated CPU time: 2202.02 s
Current children cumulated vsize: 874388 KiB
Current children cumulated memory: 849180 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27974
/proc/meminfo: memFree=18404056/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=881692 memory=856364 CPUtime=2262.01 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1071020 0 0 0 226082 119 0 0 20 0 1 0 95836253 902852608 214091 33554432000 4194304 4933779 140735356962976 140735356960728 4437393 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 220423 214091 614 181 0 214040 0
Current children cumulated CPU time: 2262.01 s
Current children cumulated vsize: 881692 KiB
Current children cumulated memory: 856364 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27975
/proc/meminfo: memFree=18387484/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=889180 memory=864488 CPUtime=2321.98 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1092119 0 0 0 232077 121 0 0 20 0 1 0 95836253 910520320 216122 33554432000 4194304 4933779 140735356962976 140735356960680 4437393 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 222295 216122 614 181 0 215912 0
Current children cumulated CPU time: 2321.98 s
Current children cumulated vsize: 889180 KiB
Current children cumulated memory: 864488 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/186 27976
/proc/meminfo: memFree=18372124/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=894184 memory=870460 CPUtime=2381.95 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1102728 0 0 0 238073 122 0 0 20 0 1 0 95836253 915644416 217615 33554432000 4194304 4933779 140735356962976 140735356960664 4358222 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 223546 217615 614 181 0 217163 0
Current children cumulated CPU time: 2381.95 s
Current children cumulated vsize: 894184 KiB
Current children cumulated memory: 870460 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27978
/proc/meminfo: memFree=18351664/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=903436 memory=879776 CPUtime=2441.95 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1106866 0 0 0 244072 123 0 0 20 0 1 0 95836253 925118464 219944 33554432000 4194304 4933779 140735356962976 140735356960680 4358024 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 225859 219944 614 181 0 219476 0
Current children cumulated CPU time: 2441.95 s
Current children cumulated vsize: 903436 KiB
Current children cumulated memory: 879776 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27987
/proc/meminfo: memFree=18330928/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=914464 memory=890576 CPUtime=2501.95 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1113490 0 0 0 250071 124 0 0 20 0 1 0 95836253 936411136 222644 33554432000 4194304 4933779 140735356962976 140735356960680 4437564 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 228616 222644 614 181 0 222233 0
Current children cumulated CPU time: 2501.95 s
Current children cumulated vsize: 914464 KiB
Current children cumulated memory: 890576 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.00 2.01 2.04 3/188 27987
/proc/meminfo: memFree=18326976/32770624 swapFree=67108860/67108860
[pid=27820] ppid=27817 vsize=916880 memory=893136 CPUtime=2519.66 cores=0,2,4,6
/proc/27820/stat : 27820 (nacre_mini_rele) R 27817 27820 26996 0 -1 4202496 1117389 0 0 0 251842 124 0 0 20 0 1 0 95836253 938885120 223284 33554432000 4194304 4933779 140735356962976 140735356960680 4437558 0 0 4096 24578 0 0 0 17 2 0 0 0 0 0 7034520 7039168 22147072 140735356970136 140735356970236 140735356970236 140735356972994 0
/proc/27820/statm: 229220 223284 614 181 0 222837 0
Current children cumulated CPU time: 2519.66 s
Current children cumulated vsize: 916880 KiB
Current children cumulated memory: 893136 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.02 s]
# the end of solver process 27820 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2518.42 s, system=1.24895 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 0
Real time (s): 2520.02
CPU time (s): 2519.67
CPU user time (s): 2518.42
CPU system time (s): 1.24895
CPU usage (%): 99.9863
Max. virtual memory (cumulated for all children) (KiB): 916880
Max. memory (cumulated for all children) (KiB): 893136

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2518.42
system time used= 1.24895
maximum resident set size= 893192
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1117432
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 399
involuntary context switches= 3160


# summary of solver processes directly reported to runsolver:
#   pid: 27820
#   total CPU time (s): 2519.67
#   total CPU user time (s): 2518.42
#   total CPU system time (s): 1.24895

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.90313 second user time and 18.7951 second system time

The end

Launcher Data 

Begin job on node103 at 2018-07-28 14:58:51
IDJOB=4299122
IDBENCH=141297
IDSOLVER=2764
FILE ID=node103/4299122-1532782731
RUNJOBID= node103-1532775163-27027
PBS_JOBID= 21022463
Free space on /tmp= 56536 MiB

SOLVER NAME= NACRE 1.0.4
BENCH NAME= XCSP18/Quasigroups_mini/QuasiGroup-v7-16_mini_c18.xml
COMMAND LINE= nacre_mini_release BENCHNAME -ca -l100 -cm
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4299122-1532782731/watcher-4299122-1532782731 -o /tmp/evaluation-result-4299122-1532782731/solver-4299122-1532782731 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node103-1532775163-27027 --watchdog 2580  nacre_mini_release HOME/instance-4299122-1532782731.xml -ca -l100 -cm

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 90b0ae5457c21a968d0990b7ba14ba70
RANDOM SEED=35666324

node103.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        17304680 kB
MemAvailable:   29002652 kB
Buffers:          302388 kB
Cached:         11579700 kB
SwapCached:            0 kB
Active:          6837128 kB
Inactive:        8021060 kB
Active(anon):    2979020 kB
Inactive(anon):    25184 kB
Active(file):    3858108 kB
Inactive(file):  7995876 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:               688 kB
Writeback:             0 kB
AnonPages:       2966684 kB
Mapped:            44728 kB
Shmem:             25228 kB
Slab:             300272 kB
SReclaimable:     272448 kB
SUnreclaim:        27824 kB
KernelStack:        3360 kB
PageTables:        10536 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    3275776 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:   2449408 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 56536 MiB
End job on node103 at 2018-07-28 15:40:51