Trace number 4299130

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
NACRE 1.0.4? (TO) 2519.83 2520.02

General information on the benchmark

NameQuasigroups_mini/
QuasiGroup-v7-20_mini_c18.xml
MD5SUM71a4fe88cf163e0e6932f74b335a06ff
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark1.28691
Satisfiable
(Un)Satisfiability was proved
Number of variables800
Number of constraints820
Number of domains1
Minimum domain size20
Maximum domain size20
Distribution of domain sizes[{"size":20,"count":780}]
Minimum variable degree0
Maximum variable degree42
Distribution of variable degrees[{"degree":0,"count":20},{"degree":2,"count":380},{"degree":41,"count":20},{"degree":42,"count":380}]
Minimum constraint arity1
Maximum constraint arity22
Distribution of constraint arities[{"arity":1,"count":20},{"arity":20,"count":40},{"arity":22,"count":760}]
Number of extensional constraints0
Number of intensional constraints20
Distribution of constraint types[{"type":"intension","count":20},{"type":"allDifferent","count":40},{"type":"element","count":760}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.02	c NACRE 1.0.4
2519.81/2520.01	c
2519.81/2520.01	c # time = 2519.81
2519.81/2520.01	c # Dec = 1695596
2519.81/2520.01	c # Propagate Calls = 3459532010
2519.81/2520.01	c # assignAt Calls = 2544953
2519.81/2520.01	c # removeAt Calls = 305904764
2519.81/2520.01	c # Run = 3068
2519.81/2520.01	c # Conflicts = 1551675
2519.81/2520.01	c # Memory used = 1039.7 Mb
2519.81/2520.01	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (9306 MiB free)
  memory of node 1: 16384 MiB (10163 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4299130-1532782913/watcher-4299130-1532782913 -o /tmp/evaluation-result-4299130-1532782913/solver-4299130-1532782913 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node103-1532775163-27028 --watchdog 2580 nacre_mini_release HOME/instance-4299130-1532782913.xml -ca -l100 -cm 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=27914, runsolver pid=27911
Current StackSize limit: 8192 KiB


[startup+0.10016 s]*
/proc/loadavg: 2.02 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19931168/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=32220 memory=8776 CPUtime=0.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 2498 0 0 0 10 0 0 0 20 0 1 0 95854514 32993280 2194 33554432000 4194304 4933779 140728054923072 140728054920072 4422718 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 8055 2194 614 181 0 1672 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 32220 KiB
Current children cumulated memory: 8776 KiB

[startup+0.209498 s]*
/proc/loadavg: 2.02 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19930424/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=32768 memory=9304 CPUtime=0.21 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 2641 0 0 0 21 0 0 0 20 0 1 0 95854514 33554432 2326 33554432000 4194304 4933779 140728054923072 140728054920072 4358086 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 8192 2326 614 181 0 1809 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 32768 KiB
Current children cumulated memory: 9304 KiB

[startup+0.309141 s]*
/proc/loadavg: 2.02 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19930052/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=33172 memory=9832 CPUtime=0.31 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 2737 0 0 0 31 0 0 0 20 0 1 0 95854514 33968128 2458 33554432000 4194304 4933779 140728054923072 140728054920072 139855771834787 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 8293 2458 614 181 0 1910 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 33172 KiB
Current children cumulated memory: 9832 KiB

[startup+0.700289 s]
/proc/loadavg: 2.02 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19928292/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=35104 memory=11680 CPUtime=0.69 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 3206 0 0 0 69 0 0 0 20 0 1 0 95854514 35946496 2920 33554432000 4194304 4933779 140728054923072 140728054920824 4422724 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 8776 2920 614 181 0 2393 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 35104 KiB
Current children cumulated memory: 11680 KiB

[startup+1.50029 s]
/proc/loadavg: 2.02 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19926144/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=38216 memory=14848 CPUtime=1.49 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 4000 0 0 0 149 0 0 0 20 0 1 0 95854514 39133184 3712 33554432000 4194304 4933779 140728054923072 140728054920824 4358297 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 9554 3712 614 181 0 3171 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 38216 KiB
Current children cumulated memory: 14848 KiB

[startup+3.10934 s]
/proc/loadavg: 2.02 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19912380/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=48752 memory=25276 CPUtime=3.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 9450 0 0 0 310 0 0 0 20 0 1 0 95854514 49922048 6319 33554432000 4194304 4933779 140728054923072 140728054920824 4358397 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 12188 6319 614 181 0 5805 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 48752 KiB
Current children cumulated memory: 25276 KiB

[startup+6.30029 s]
/proc/loadavg: 2.02 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19894928/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=65896 memory=42588 CPUtime=6.29 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 18377 0 0 0 628 1 0 0 20 0 1 0 95854514 67477504 10647 33554432000 4194304 4933779 140728054923072 140728054920840 4358949 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 16474 10647 614 181 0 10091 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 65896 KiB
Current children cumulated memory: 42588 KiB

[startup+12.7003 s]
/proc/loadavg: 2.01 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19856968/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=95896 memory=72480 CPUtime=12.69 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 34352 0 0 0 1265 4 0 0 20 0 1 0 95854514 98197504 18120 33554432000 4194304 4933779 140728054923072 140728054920824 4358185 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 23974 18120 614 181 0 17591 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 95896 KiB
Current children cumulated memory: 72480 KiB

[startup+25.5003 s]
/proc/loadavg: 2.01 2.07 2.02 3/188 27915
/proc/meminfo: memFree=19805616/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=140236 memory=116244 CPUtime=25.49 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 75050 0 0 0 2542 7 0 0 20 0 1 0 95854514 143601664 29061 33554432000 4194304 4933779 140728054923072 140728054920824 4358125 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 35059 29061 614 181 0 28676 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 140236 KiB
Current children cumulated memory: 116244 KiB

[startup+51.1095 s]
/proc/loadavg: 2.01 2.06 2.02 3/188 27915
/proc/meminfo: memFree=19717528/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=209996 memory=186212 CPUtime=51.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 122021 0 0 0 5098 12 0 0 20 0 1 0 95854514 215035904 46553 33554432000 4194304 4933779 140728054923072 140728054920776 4358222 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 52499 46553 614 181 0 46116 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 209996 KiB
Current children cumulated memory: 186212 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.05 2.02 3/188 27916
/proc/meminfo: memFree=19623236/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=275028 memory=251328 CPUtime=102.29 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 178708 0 0 0 10211 18 0 0 20 0 1 0 95854514 281628672 62832 33554432000 4194304 4933779 140728054923072 140728054920776 4437570 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 68757 62832 614 181 0 62374 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 275028 KiB
Current children cumulated memory: 251328 KiB

[startup+162.3 s]
/proc/loadavg: 2.10 2.07 2.04 3/188 27917
/proc/meminfo: memFree=19525056/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=343464 memory=319744 CPUtime=162.29 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 244096 0 0 0 16203 26 0 0 20 0 1 0 95854514 351707136 79936 33554432000 4194304 4933779 140728054923072 140728054920776 4358268 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 85866 79936 614 181 0 79483 0
Current children cumulated CPU time: 162.29 s
Current children cumulated vsize: 343464 KiB
Current children cumulated memory: 319744 KiB

[startup+222.3 s]
/proc/loadavg: 2.04 2.06 2.04 3/188 27918
/proc/meminfo: memFree=19424096/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=414296 memory=390668 CPUtime=222.28 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 281690 0 0 0 22199 29 0 0 20 0 1 0 95854514 424239104 97667 33554432000 4194304 4933779 140728054923072 140728054920824 139855777170114 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 103574 97667 614 181 0 97191 0
Current children cumulated CPU time: 222.28 s
Current children cumulated vsize: 414296 KiB
Current children cumulated memory: 390668 KiB

[startup+282.3 s]
/proc/loadavg: 2.01 2.05 2.04 3/187 27919
/proc/meminfo: memFree=19343788/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=463344 memory=439668 CPUtime=282.27 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 358532 0 0 0 28190 37 0 0 20 0 1 0 95854514 474464256 109917 33554432000 4194304 4933779 140728054923072 140728054920776 4437567 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 115836 109917 614 181 0 109453 0
Current children cumulated CPU time: 282.27 s
Current children cumulated vsize: 463344 KiB
Current children cumulated memory: 439668 KiB

[startup+342.3 s]
/proc/loadavg: 2.05 2.05 2.04 3/187 27920
/proc/meminfo: memFree=19304128/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=486136 memory=461264 CPUtime=342.25 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 425002 0 0 0 34182 43 0 0 20 0 1 0 95854514 497803264 115316 33554432000 4194304 4933779 140728054923072 140728054920824 4359669 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 121534 115316 614 181 0 115151 0
Current children cumulated CPU time: 342.25 s
Current children cumulated vsize: 486136 KiB
Current children cumulated memory: 461264 KiB

[startup+402.3 s]
/proc/loadavg: 2.02 2.04 2.04 3/187 27921
/proc/meminfo: memFree=19256408/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=507708 memory=482988 CPUtime=402.22 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 476259 0 0 0 40173 49 0 0 20 0 1 0 95854514 519892992 120747 33554432000 4194304 4933779 140728054923072 140728054920776 4358603 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 126927 120747 614 181 0 120544 0
Current children cumulated CPU time: 402.22 s
Current children cumulated vsize: 507708 KiB
Current children cumulated memory: 482988 KiB

[startup+462.3 s]
/proc/loadavg: 2.01 2.04 2.04 3/187 27922
/proc/meminfo: memFree=19197792/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=529868 memory=505344 CPUtime=462.2 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 515056 0 0 0 46166 54 0 0 20 0 1 0 95854514 542584832 126336 33554432000 4194304 4933779 140728054923072 140728054920776 4475748 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 132467 126336 614 181 0 126084 0
Current children cumulated CPU time: 462.2 s
Current children cumulated vsize: 529868 KiB
Current children cumulated memory: 505344 KiB


################
# More data... #
################

/proc/27914/statm: 233243 227371 614 181 0 226860 0
Current children cumulated CPU time: 1722.11 s
Current children cumulated vsize: 932972 KiB
Current children cumulated memory: 909484 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27969
/proc/meminfo: memFree=18486508/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=944536 memory=921036 CPUtime=1782.11 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 923617 0 0 0 178100 111 0 0 20 0 1 0 95854514 967204864 230259 33554432000 4194304 4933779 140728054923072 140728054920824 4358940 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 236134 230259 614 181 0 229751 0
Current children cumulated CPU time: 1782.11 s
Current children cumulated vsize: 944536 KiB
Current children cumulated memory: 921036 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27970
/proc/meminfo: memFree=18468032/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=956172 memory=932768 CPUtime=1842.11 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 932233 0 0 0 184099 112 0 0 20 0 1 0 95854514 979120128 233192 33554432000 4194304 4933779 140728054923072 140728054920824 4359623 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 239043 233192 614 181 0 232660 0
Current children cumulated CPU time: 1842.11 s
Current children cumulated vsize: 956172 KiB
Current children cumulated memory: 932768 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27971
/proc/meminfo: memFree=18448928/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=969044 memory=945668 CPUtime=1902.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 943088 0 0 0 190097 113 0 0 20 0 1 0 95854514 992301056 236417 33554432000 4194304 4933779 140728054923072 140728054920776 4437536 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 242261 236417 614 181 0 235878 0
Current children cumulated CPU time: 1902.1 s
Current children cumulated vsize: 969044 KiB
Current children cumulated memory: 945668 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27972
/proc/meminfo: memFree=18433932/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=978048 memory=954692 CPUtime=1962.11 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 954129 0 0 0 196096 115 0 0 20 0 1 0 95854514 1001521152 238673 33554432000 4194304 4933779 140728054923072 140728054920824 4437536 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 244512 238673 614 181 0 238129 0
Current children cumulated CPU time: 1962.11 s
Current children cumulated vsize: 978048 KiB
Current children cumulated memory: 954692 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27973
/proc/meminfo: memFree=18420308/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=987504 memory=964068 CPUtime=2022.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 965938 0 0 0 202094 116 0 0 20 0 1 0 95854514 1011204096 241017 33554432000 4194304 4933779 140728054923072 140728054920824 4358006 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 246876 241017 614 181 0 240493 0
Current children cumulated CPU time: 2022.1 s
Current children cumulated vsize: 987504 KiB
Current children cumulated memory: 964068 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27974
/proc/meminfo: memFree=18403064/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=995996 memory=972604 CPUtime=2082.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 978342 0 0 0 208092 118 0 0 20 0 1 0 95854514 1019899904 243151 33554432000 4194304 4933779 140728054923072 140728054920824 4437393 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 248999 243151 614 181 0 242616 0
Current children cumulated CPU time: 2082.1 s
Current children cumulated vsize: 995996 KiB
Current children cumulated memory: 972604 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27975
/proc/meminfo: memFree=18387236/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1004140 memory=980852 CPUtime=2142.11 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 993100 0 0 0 214091 120 0 0 20 0 1 0 95854514 1028239360 245213 33554432000 4194304 4933779 140728054923072 140728054920776 4437581 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 251035 245213 614 181 0 244652 0
Current children cumulated CPU time: 2142.11 s
Current children cumulated vsize: 1004140 KiB
Current children cumulated memory: 980852 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/186 27976
/proc/meminfo: memFree=18371132/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1015764 memory=992420 CPUtime=2202.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 1009091 0 0 0 220088 122 0 0 20 0 1 0 95854514 1040142336 248105 33554432000 4194304 4933779 140728054923072 140728054920824 4475728 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 253941 248105 614 181 0 247558 0
Current children cumulated CPU time: 2202.1 s
Current children cumulated vsize: 1015764 KiB
Current children cumulated memory: 992420 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 27978
/proc/meminfo: memFree=18351168/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1025320 memory=1002080 CPUtime=2262.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 1025782 0 0 0 226084 126 0 0 20 0 1 0 95854514 1049927680 250520 33554432000 4194304 4933779 140728054923072 140728054920760 4358211 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 256330 250520 614 181 0 249947 0
Current children cumulated CPU time: 2262.1 s
Current children cumulated vsize: 1025320 KiB
Current children cumulated memory: 1002080 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 27987
/proc/meminfo: memFree=18330556/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1035308 memory=1012060 CPUtime=2322.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 1042768 0 0 0 232082 128 0 0 20 0 1 0 95854514 1060155392 253015 33554432000 4194304 4933779 140728054923072 140728054920776 4472628 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 258827 253015 614 181 0 252444 0
Current children cumulated CPU time: 2322.1 s
Current children cumulated vsize: 1035308 KiB
Current children cumulated memory: 1012060 KiB

[startup+2382.3 s]
/proc/loadavg: 2.22 2.06 2.06 3/188 28048
/proc/meminfo: memFree=19158256/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1044164 memory=1020828 CPUtime=2382.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 1063594 0 0 0 238079 131 0 0 20 0 1 0 95854514 1069223936 255207 33554432000 4194304 4933779 140728054923072 140728054920824 4363664 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 261041 255207 614 181 0 254658 0
Current children cumulated CPU time: 2382.1 s
Current children cumulated vsize: 1044164 KiB
Current children cumulated memory: 1020828 KiB

[startup+2442.3 s]
/proc/loadavg: 2.37 2.15 2.09 3/188 28049
/proc/meminfo: memFree=19125160/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1053520 memory=1030276 CPUtime=2442.09 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 1083187 0 0 0 244076 133 0 0 20 0 1 0 95854514 1078804480 257569 33554432000 4194304 4933779 140728054923072 140728054920824 4437538 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 263380 257569 614 181 0 256997 0
Current children cumulated CPU time: 2442.09 s
Current children cumulated vsize: 1053520 KiB
Current children cumulated memory: 1030276 KiB

[startup+2502.3 s]
/proc/loadavg: 2.14 2.13 2.08 3/188 28050
/proc/meminfo: memFree=19101980/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1064548 memory=1038700 CPUtime=2502.1 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 1102586 0 0 0 250074 136 0 0 20 0 1 0 95854514 1090097152 259675 33554432000 4194304 4933779 140728054923072 140728054920776 4357887 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 266137 259675 614 181 0 259754 0
Current children cumulated CPU time: 2502.1 s
Current children cumulated vsize: 1064548 KiB
Current children cumulated memory: 1038700 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.10 2.12 2.08 3/188 28051
/proc/meminfo: memFree=19096252/32770624 swapFree=67108860/67108860
[pid=27914] ppid=27911 vsize=1065532 memory=1039700 CPUtime=2519.81 cores=1,3,5,7
/proc/27914/stat : 27914 (nacre_mini_rele) R 27911 27914 26996 0 -1 4202496 1108375 0 0 0 251844 137 0 0 20 0 1 0 95854514 1091104768 259925 33554432000 4194304 4933779 140728054923072 140728054920776 4358936 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 24141824 140728054924440 140728054924540 140728054924540 140728054927298 0
/proc/27914/statm: 266383 259925 614 181 0 260000 0
Current children cumulated CPU time: 2519.81 s
Current children cumulated vsize: 1065532 KiB
Current children cumulated memory: 1039700 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.02 s]
# the end of solver process 27914 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2518.44 s, system=1.38498 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 0
Real time (s): 2520.02
CPU time (s): 2519.83
CPU user time (s): 2518.44
CPU system time (s): 1.38498
CPU usage (%): 99.9922
Max. virtual memory (cumulated for all children) (KiB): 1065532
Max. memory (cumulated for all children) (KiB): 1039700

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2518.44
system time used= 1.38498
maximum resident set size= 1039756
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1108418
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 313
involuntary context switches= 2788


# summary of solver processes directly reported to runsolver:
#   pid: 27914
#   total CPU time (s): 2519.83
#   total CPU user time (s): 2518.44
#   total CPU system time (s): 1.38498

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.57812 second user time and 17.9457 second system time

The end

Launcher Data 

Begin job on node103 at 2018-07-28 15:01:53
IDJOB=4299130
IDBENCH=141298
IDSOLVER=2764
FILE ID=node103/4299130-1532782913
RUNJOBID= node103-1532775163-27028
PBS_JOBID= 21022463
Free space on /tmp= 56536 MiB

SOLVER NAME= NACRE 1.0.4
BENCH NAME= XCSP18/Quasigroups_mini/QuasiGroup-v7-20_mini_c18.xml
COMMAND LINE= nacre_mini_release BENCHNAME -ca -l100 -cm
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4299130-1532782913/watcher-4299130-1532782913 -o /tmp/evaluation-result-4299130-1532782913/solver-4299130-1532782913 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node103-1532775163-27028 --watchdog 2580  nacre_mini_release HOME/instance-4299130-1532782913.xml -ca -l100 -cm

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 71a4fe88cf163e0e6932f74b335a06ff
RANDOM SEED=2103150043

node103.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.48
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        19937920 kB
MemAvailable:   31635048 kB
Buffers:          302404 kB
Cached:         11578724 kB
SwapCached:            0 kB
Active:          4208992 kB
Inactive:        8021136 kB
Active(anon):     351828 kB
Inactive(anon):    25184 kB
Active(file):    3857164 kB
Inactive(file):  7995952 kB
Unevictable:        7120 kB
Mlocked:            7120 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:               796 kB
Writeback:             0 kB
AnonPages:        356180 kB
Mapped:            44360 kB
Shmem:             25228 kB
Slab:             300360 kB
SReclaimable:     272472 kB
SUnreclaim:        27888 kB
KernelStack:        3344 kB
PageTables:         5800 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     648284 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:      6144 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      124288 kB
DirectMap2M:    33421312 kB

Free space on /tmp at the end= 56536 MiB
End job on node103 at 2018-07-28 15:43:53