Trace number 4299138

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
NACRE 1.0.4? (TO) 2519.82 2520.04

General information on the benchmark

NameQuasigroups_mini/
QuasiGroup-v4-20_mini_c18.xml
MD5SUM43d4b24a9d4c7a91506dc3ee81b98d4e
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark2.05992
Satisfiable
(Un)Satisfiability was proved
Number of variables800
Number of constraints820
Number of domains2
Minimum domain size20
Maximum domain size400
Distribution of domain sizes[{"size":20,"count":400},{"size":400,"count":380}]
Minimum variable degree0
Maximum variable degree384
Distribution of variable degrees[{"degree":0,"count":20},{"degree":2,"count":380},{"degree":383,"count":20},{"degree":384,"count":380}]
Minimum constraint arity1
Maximum constraint arity401
Distribution of constraint arities[{"arity":1,"count":20},{"arity":3,"count":380},{"arity":20,"count":40},{"arity":401,"count":380}]
Number of extensional constraints380
Number of intensional constraints20
Distribution of constraint types[{"type":"extension","count":380},{"type":"intension","count":20},{"type":"allDifferent","count":40},{"type":"element","count":380}]
Optimization problemNO
Type of objective

Solver Data 

0.10/0.13	c NACRE 1.0.4
2519.78/2520.01	c
2519.78/2520.01	c # time = 2519.78
2519.78/2520.01	c # Dec = 329693
2519.78/2520.01	c # Propagate Calls = 2076459465
2519.78/2520.01	c # assignAt Calls = 3224125
2519.78/2520.01	c # removeAt Calls = 201844801
2519.78/2520.01	c # Run = 606
2519.78/2520.01	c # Conflicts = 258093
2519.78/2520.01	c # Memory used = 4352.18 Mb
2519.78/2520.01	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (7275 MiB free)
  memory of node 1: 16384 MiB (15802 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4299138-1532783665/watcher-4299138-1532783665 -o /tmp/evaluation-result-4299138-1532783665/solver-4299138-1532783665 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node102-1532776103-29944 --watchdog 2580 nacre_mini_release HOME/instance-4299138-1532783665.xml -ca -l100 -cm 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=30618, runsolver pid=30615

[startup+0.100168 s]*
/proc/loadavg: 2.00 2.03 2.05 3/188 30619
/proc/meminfo: memFree=23607348/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=57632 memory=29496 CPUtime=0.1 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 8532 0 0 0 9 1 0 0 20 0 1 0 95929643 59015168 7374 33554432000 4194304 4933779 140731153339536 140731153336536 139791144872933 0 0 4096 0 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 14408 7374 591 181 0 8025 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 57632 KiB
Current children cumulated memory: 29496 KiB

[startup+0.209757 s]*
/proc/loadavg: 2.00 2.03 2.05 3/188 30619
/proc/meminfo: memFree=23592400/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=70508 memory=42492 CPUtime=0.21 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 10247 0 0 0 20 1 0 0 20 0 1 0 95929643 72200192 10623 33554432000 4194304 4933779 140731153339536 140731153336536 4357360 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 17627 10623 600 181 0 11244 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 70508 KiB
Current children cumulated memory: 42492 KiB

[startup+0.309784 s]*
/proc/loadavg: 2.00 2.03 2.05 2/186 30619
/proc/meminfo: memFree=26758344/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=70508 memory=42492 CPUtime=0.31 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 10247 0 0 0 30 1 0 0 20 0 1 0 95929643 72200192 10623 33554432000 4194304 4933779 140731153339536 140731153338056 4475930 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 17627 10623 600 181 0 11244 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 70508 KiB
Current children cumulated memory: 42492 KiB

[startup+0.700375 s]
/proc/loadavg: 1.92 2.01 2.04 4/201 30648
/proc/meminfo: memFree=26740552/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=70772 memory=42756 CPUtime=0.69 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 10320 0 0 0 68 1 0 0 20 0 1 0 95929643 72470528 10689 33554432000 4194304 4933779 140731153339536 140731153338056 4357408 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 17693 10689 613 181 0 11310 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 70772 KiB
Current children cumulated memory: 42756 KiB

[startup+1.50033 s]
/proc/loadavg: 1.92 2.01 2.04 3/188 30680
/proc/meminfo: memFree=26740916/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=71696 memory=43812 CPUtime=1.49 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 10563 0 0 0 148 1 0 0 20 0 1 0 95929643 73416704 10953 33554432000 4194304 4933779 140731153339536 140731153337048 139791144872933 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 17924 10953 613 181 0 11541 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 71696 KiB
Current children cumulated memory: 43812 KiB

[startup+3.10954 s]
/proc/loadavg: 1.92 2.01 2.04 3/188 30680
/proc/meminfo: memFree=26721440/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=77428 memory=49880 CPUtime=3.1 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 18227 0 0 0 308 2 0 0 20 0 1 0 95929643 79286272 12470 33554432000 4194304 4933779 140731153339536 140731153337304 4357301 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 19357 12470 618 181 0 12974 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 77428 KiB
Current children cumulated memory: 49880 KiB

[startup+6.30033 s]
/proc/loadavg: 1.93 2.01 2.04 3/188 30680
/proc/meminfo: memFree=26691656/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=84724 memory=57120 CPUtime=6.29 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 23700 0 0 0 626 3 0 0 20 0 1 0 95929643 86757376 14280 33554432000 4194304 4933779 140731153339536 140731153337304 4475920 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 21181 14280 618 181 0 14798 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 84724 KiB
Current children cumulated memory: 57120 KiB

[startup+12.7003 s]
/proc/loadavg: 1.93 2.01 2.04 3/188 30680
/proc/meminfo: memFree=26642924/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=98364 memory=70904 CPUtime=12.67 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 30953 0 0 0 1264 3 0 0 20 0 1 0 95929643 100724736 17726 33554432000 4194304 4933779 140731153339536 140731153337304 4357325 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 24591 17726 618 181 0 18208 0
Current children cumulated CPU time: 12.67 s
Current children cumulated vsize: 98364 KiB
Current children cumulated memory: 70904 KiB

[startup+25.5003 s]
/proc/loadavg: 1.95 2.01 2.04 3/188 30680
/proc/meminfo: memFree=26504140/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=143140 memory=115732 CPUtime=25.47 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 45510 0 0 0 2542 5 0 0 20 0 1 0 95929643 146575360 28933 33554432000 4194304 4933779 140731153339536 140731153337304 4377584 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 35785 28933 618 181 0 29402 0
Current children cumulated CPU time: 25.47 s
Current children cumulated vsize: 143140 KiB
Current children cumulated memory: 115732 KiB

[startup+51.1096 s]
/proc/loadavg: 1.97 2.01 2.04 3/187 30680
/proc/meminfo: memFree=26238268/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=266624 memory=239144 CPUtime=51.07 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 79608 0 0 0 5098 9 0 0 20 0 1 0 95929643 273022976 59786 33554432000 4194304 4933779 140731153339536 140731153337304 4475942 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 66656 59786 618 181 0 60273 0
Current children cumulated CPU time: 51.07 s
Current children cumulated vsize: 266624 KiB
Current children cumulated memory: 239144 KiB

[startup+102.3 s]
/proc/loadavg: 1.99 2.01 2.04 3/187 30681
/proc/meminfo: memFree=25745708/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=529376 memory=501616 CPUtime=102.26 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 153995 0 0 0 10205 21 0 0 20 0 1 0 95929643 542081024 125404 33554432000 4194304 4933779 140731153339536 140731153337288 4377379 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 132344 125404 618 181 0 125961 0
Current children cumulated CPU time: 102.26 s
Current children cumulated vsize: 529376 KiB
Current children cumulated memory: 501616 KiB

[startup+162.3 s]
/proc/loadavg: 1.99 2.01 2.04 3/187 30682
/proc/meminfo: memFree=25327112/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=737672 memory=710036 CPUtime=162.26 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 212712 0 0 0 16195 31 0 0 20 0 1 0 95929643 755376128 177509 33554432000 4194304 4933779 140731153339536 140731153337240 4437564 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 184418 177509 618 181 0 178035 0
Current children cumulated CPU time: 162.26 s
Current children cumulated vsize: 737672 KiB
Current children cumulated memory: 710036 KiB

[startup+222.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 30683
/proc/meminfo: memFree=24974988/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=955444 memory=927836 CPUtime=222.26 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 293584 0 0 0 22184 42 0 0 20 0 1 0 95929643 978374656 231959 33554432000 4194304 4933779 140731153339536 140731153337240 4357365 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 238861 231959 618 181 0 232478 0
Current children cumulated CPU time: 222.26 s
Current children cumulated vsize: 955444 KiB
Current children cumulated memory: 927836 KiB

[startup+282.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/187 30684
/proc/meminfo: memFree=24700620/32770624 swapFree=67028240/67108860
[pid=30618] ppid=30615 vsize=1086452 memory=1058800 CPUtime=282.24 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 356689 0 0 0 28175 49 0 0 20 0 1 0 95929643 1112526848 264700 33554432000 4194304 4933779 140731153339536 140731153337240 4357408 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 271613 264700 618 181 0 265230 0
Current children cumulated CPU time: 282.24 s
Current children cumulated vsize: 1086452 KiB
Current children cumulated memory: 1058800 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 30693
/proc/meminfo: memFree=24431544/32770624 swapFree=67028244/67108860
[pid=30618] ppid=30615 vsize=1212180 memory=1184144 CPUtime=342.24 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 413054 0 0 0 34167 57 0 0 20 0 1 0 95929643 1241272320 296036 33554432000 4194304 4933779 140731153339536 140731153337240 4357408 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 303045 296036 618 181 0 296662 0
Current children cumulated CPU time: 342.24 s
Current children cumulated vsize: 1212180 KiB
Current children cumulated memory: 1184144 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 30694
/proc/meminfo: memFree=24154116/32770624 swapFree=67028244/67108860
[pid=30618] ppid=30615 vsize=1350364 memory=1322296 CPUtime=402.24 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 474366 0 0 0 40159 65 0 0 20 0 1 0 95929643 1382772736 330574 33554432000 4194304 4933779 140731153339536 140731153337304 4472628 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 337591 330574 618 181 0 331208 0
Current children cumulated CPU time: 402.24 s
Current children cumulated vsize: 1350364 KiB
Current children cumulated memory: 1322296 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.04 3/188 30695
/proc/meminfo: memFree=23821616/32770624 swapFree=67028244/67108860
[pid=30618] ppid=30615 vsize=1581316 memory=1553344 CPUtime=462.23 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 594014 0 0 0 46147 76 0 0 20 0 1 0 95929643 1619267584 388336 33554432000 4194304 4933779 140731153339536 140731153337240 4357567 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 395329 388336 618 181 0 388946 0
Current children cumulated CPU time: 462.23 s
Current children cumulated vsize: 1581316 KiB
Current children cumulated memory: 1553344 KiB


################
# More data... #
################

/proc/30618/statm: 850731 843624 618 181 0 844348 0
Current children cumulated CPU time: 1722.13 s
Current children cumulated vsize: 3402924 KiB
Current children cumulated memory: 3374496 KiB

[startup+1782.3 s]
/proc/loadavg: 2.08 2.03 2.05 3/187 30735
/proc/meminfo: memFree=20247704/32770624 swapFree=67028256/67108860
[pid=30618] ppid=30615 vsize=3456544 memory=3428104 CPUtime=1782.13 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 1725876 0 0 0 178018 195 0 0 20 0 1 0 95929643 3539501056 857026 33554432000 4194304 4933779 140731153339536 140731153337304 4475930 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 864136 857026 618 181 0 857753 0
Current children cumulated CPU time: 1782.13 s
Current children cumulated vsize: 3456544 KiB
Current children cumulated memory: 3428104 KiB

[startup+1842.3 s]
/proc/loadavg: 2.38 2.14 2.09 3/187 30736
/proc/meminfo: memFree=20132548/32770624 swapFree=67028256/67108860
[pid=30618] ppid=30615 vsize=3513476 memory=3485092 CPUtime=1842.12 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 1761542 0 0 0 184014 198 0 0 20 0 1 0 95929643 3597799424 871273 33554432000 4194304 4933779 140731153339536 140731153337240 4475920 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 878369 871273 618 181 0 871986 0
Current children cumulated CPU time: 1842.12 s
Current children cumulated vsize: 3513476 KiB
Current children cumulated memory: 3485092 KiB

[startup+1902.3 s]
/proc/loadavg: 2.14 2.12 2.08 3/187 30737
/proc/meminfo: memFree=19992824/32770624 swapFree=67028256/67108860
[pid=30618] ppid=30615 vsize=3589196 memory=3560796 CPUtime=1902.12 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 1801377 0 0 0 190009 203 0 0 20 0 1 0 95929643 3675336704 890199 33554432000 4194304 4933779 140731153339536 140731153337240 4357246 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 897299 890199 618 181 0 890916 0
Current children cumulated CPU time: 1902.12 s
Current children cumulated vsize: 3589196 KiB
Current children cumulated memory: 3560796 KiB

[startup+1962.3 s]
/proc/loadavg: 2.05 2.10 2.08 3/188 30760
/proc/meminfo: memFree=19831092/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=3701140 memory=3672528 CPUtime=1962.11 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 1852781 0 0 0 196002 209 0 0 20 0 1 0 95929643 3789967360 918132 33554432000 4194304 4933779 140731153339536 140731153337240 4357408 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 925285 918132 618 181 0 918902 0
Current children cumulated CPU time: 1962.11 s
Current children cumulated vsize: 3701140 KiB
Current children cumulated memory: 3672528 KiB

[startup+2022.3 s]
/proc/loadavg: 2.02 2.08 2.07 3/187 30761
/proc/meminfo: memFree=19658012/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=3815988 memory=3788584 CPUtime=2022.11 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 1906460 0 0 0 201997 214 0 0 20 0 1 0 95929643 3907571712 947146 33554432000 4194304 4933779 140731153339536 140731153337240 4475942 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 953997 947146 618 181 0 947614 0
Current children cumulated CPU time: 2022.11 s
Current children cumulated vsize: 3815988 KiB
Current children cumulated memory: 3788584 KiB

[startup+2082.3 s]
/proc/loadavg: 2.01 2.06 2.06 3/187 30762
/proc/meminfo: memFree=19515564/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=3914240 memory=3886836 CPUtime=2082.11 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 1960171 0 0 0 207991 220 0 0 20 0 1 0 95929643 4008181760 971709 33554432000 4194304 4933779 140731153339536 140731153337240 4357325 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 978560 971709 618 181 0 972177 0
Current children cumulated CPU time: 2082.11 s
Current children cumulated vsize: 3914240 KiB
Current children cumulated memory: 3886836 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.05 2.06 3/188 30771
/proc/meminfo: memFree=19382020/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4008444 memory=3980992 CPUtime=2142.1 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2010998 0 0 0 213984 226 0 0 20 0 1 0 95929643 4104646656 995248 33554432000 4194304 4933779 140731153339536 140731153337240 4388640 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1002111 995248 618 181 0 995728 0
Current children cumulated CPU time: 2142.1 s
Current children cumulated vsize: 4008444 KiB
Current children cumulated memory: 3980992 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/188 30772
/proc/meminfo: memFree=19252492/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4095116 memory=4067712 CPUtime=2202.09 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2054291 0 0 0 219978 231 0 0 20 0 1 0 95929643 4193398784 1016928 33554432000 4194304 4933779 140731153339536 140731153337240 4357365 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1023779 1016928 618 181 0 1017396 0
Current children cumulated CPU time: 2202.09 s
Current children cumulated vsize: 4095116 KiB
Current children cumulated memory: 4067712 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.04 2.05 3/188 30773
/proc/meminfo: memFree=19166596/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4136532 memory=4109128 CPUtime=2262.09 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2075999 0 0 0 225976 233 0 0 20 0 1 0 95929643 4235808768 1027282 33554432000 4194304 4933779 140731153339536 140731153337240 4377382 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1034133 1027282 618 181 0 1027750 0
Current children cumulated CPU time: 2262.09 s
Current children cumulated vsize: 4136532 KiB
Current children cumulated memory: 4109128 KiB

[startup+2322.3 s]
/proc/loadavg: 1.92 2.01 2.04 3/188 30774
/proc/meminfo: memFree=19058620/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4200168 memory=4172764 CPUtime=2322.08 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2108583 0 0 0 231971 237 0 0 20 0 1 0 95929643 4300972032 1043191 33554432000 4194304 4933779 140731153339536 140731153337240 4357325 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1050042 1043191 618 181 0 1043659 0
Current children cumulated CPU time: 2322.08 s
Current children cumulated vsize: 4200168 KiB
Current children cumulated memory: 4172764 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/188 30775
/proc/meminfo: memFree=18979288/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4242480 memory=4215076 CPUtime=2382.09 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2125494 0 0 0 237969 240 0 0 20 0 1 0 95929643 4344299520 1053769 33554432000 4194304 4933779 140731153339536 140731153337240 4357365 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1060620 1053769 618 181 0 1054237 0
Current children cumulated CPU time: 2382.09 s
Current children cumulated vsize: 4242480 KiB
Current children cumulated memory: 4215076 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/187 30777
/proc/meminfo: memFree=18866932/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4315768 memory=4288184 CPUtime=2442.08 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2156559 0 0 0 243965 243 0 0 20 0 1 0 95929643 4419346432 1072046 33554432000 4194304 4933779 140731153339536 140731153337240 4357274 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1078942 1072046 618 181 0 1072559 0
Current children cumulated CPU time: 2442.08 s
Current children cumulated vsize: 4315768 KiB
Current children cumulated memory: 4288184 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 30778
/proc/meminfo: memFree=18784376/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4358804 memory=4331148 CPUtime=2502.07 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2173671 0 0 0 249962 245 0 0 20 0 1 0 95929643 4463415296 1082787 33554432000 4194304 4933779 140731153339536 140731153337240 139791144882060 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1089701 1082787 618 181 0 1083318 0
Current children cumulated CPU time: 2502.07 s
Current children cumulated vsize: 4358804 KiB
Current children cumulated memory: 4331148 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.00 2.01 2.05 3/187 30778
/proc/meminfo: memFree=18752724/32770624 swapFree=67028268/67108860
[pid=30618] ppid=30615 vsize=4379672 memory=4352180 CPUtime=2519.78 cores=1,3,5,7
/proc/30618/stat : 30618 (nacre_mini_rele) R 30615 30618 29912 0 -1 4202496 2189538 0 0 0 251732 246 0 0 20 0 1 0 95929643 4484784128 1088045 33554432000 4194304 4933779 140731153339536 140731153337240 139791151063746 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 33153024 140731153347736 140731153347836 140731153347836 140731153350594 0
/proc/30618/statm: 1094918 1088045 618 181 0 1088535 0
Current children cumulated CPU time: 2519.78 s
Current children cumulated vsize: 4379672 KiB
Current children cumulated memory: 4352180 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.04 s]
# the end of solver process 30618 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2517.32 s, system=2.49597 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 0
Real time (s): 2520.04
CPU time (s): 2519.82
CPU user time (s): 2517.32
CPU system time (s): 2.49597
CPU usage (%): 99.9914
Max. virtual memory (cumulated for all children) (KiB): 4379672
Max. memory (cumulated for all children) (KiB): 4352180

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2517.32
system time used= 2.49597
maximum resident set size= 4352324
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 2189581
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 360
involuntary context switches= 2136


# summary of solver processes directly reported to runsolver:
#   pid: 30618
#   total CPU time (s): 2519.82
#   total CPU user time (s): 2517.32
#   total CPU system time (s): 2.49597

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 9.544 second user time and 19.1849 second system time

The end

Launcher Data 

Begin job on node102 at 2018-07-28 15:14:25
IDJOB=4299138
IDBENCH=141299
IDSOLVER=2764
FILE ID=node102/4299138-1532783665
RUNJOBID= node102-1532776103-29944
PBS_JOBID= 21022470
Free space on /tmp= 52748 MiB

SOLVER NAME= NACRE 1.0.4
BENCH NAME= XCSP18/Quasigroups_mini/QuasiGroup-v4-20_mini_c18.xml
COMMAND LINE= nacre_mini_release BENCHNAME -ca -l100 -cm
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4299138-1532783665/watcher-4299138-1532783665 -o /tmp/evaluation-result-4299138-1532783665/solver-4299138-1532783665 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node102-1532776103-29944 --watchdog 2580  nacre_mini_release HOME/instance-4299138-1532783665.xml -ca -l100 -cm

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= 43d4b24a9d4c7a91506dc3ee81b98d4e
RANDOM SEED=1759517476

node102.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        23632668 kB
MemAvailable:   28767396 kB
Buffers:          213932 kB
Cached:          5175912 kB
SwapCached:        53460 kB
Active:          4364908 kB
Inactive:        4267408 kB
Active(anon):    3168036 kB
Inactive(anon):    99020 kB
Active(file):    1196872 kB
Inactive(file):  4168388 kB
Unevictable:        6644 kB
Mlocked:            6644 kB
SwapTotal:      67108860 kB
SwapFree:       67028240 kB
Dirty:               856 kB
Writeback:             0 kB
AnonPages:       3204492 kB
Mapped:            37064 kB
Shmem:             22016 kB
Slab:             198904 kB
SReclaimable:     170604 kB
SUnreclaim:        28300 kB
KernelStack:        3584 kB
PageTables:        11308 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    3512812 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:   2985984 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 52744 MiB
End job on node102 at 2018-07-28 15:56:25