Trace number 4300338

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
NACRE 1.0.4? (TO) 2519.9 2520.01

General information on the benchmark

NameFrb/
frb-45-21-1_c18.xml
MD5SUMb87938282b595c2beb3624311f73a470
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark1880.58
Satisfiable
(Un)Satisfiability was proved
Number of variables45
Number of constraints476
Number of domains1
Minimum domain size21
Maximum domain size21
Distribution of domain sizes[{"size":21,"count":45}]
Minimum variable degree13
Maximum variable degree29
Distribution of variable degrees[{"degree":13,"count":1},{"degree":14,"count":2},{"degree":15,"count":1},{"degree":16,"count":2},{"degree":17,"count":4},{"degree":18,"count":4},{"degree":19,"count":3},{"degree":20,"count":2},{"degree":21,"count":2},{"degree":22,"count":5},{"degree":23,"count":6},{"degree":24,"count":3},{"degree":25,"count":3},{"degree":26,"count":3},{"degree":27,"count":1},{"degree":28,"count":2},{"degree":29,"count":1}]
Minimum constraint arity2
Maximum constraint arity2
Distribution of constraint arities[{"arity":2,"count":476}]
Number of extensional constraints476
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":476}]
Optimization problemNO
Type of objective

Solver Data 

0.00/0.04	c NACRE 1.0.4
2519.89/2520.01	c
2519.89/2520.01	c # time = 2519.89
2519.89/2520.01	c # Dec = 2602300
2519.89/2520.01	c # Propagate Calls = 1220176384
2519.89/2520.01	c # assignAt Calls = 4987896
2519.89/2520.01	c # removeAt Calls = 309554272
2519.89/2520.01	c # Run = 4350
2519.89/2520.01	c # Conflicts = 2560148
2519.89/2520.01	c # Memory used = 180.892 Mb
2519.89/2520.01	s UNKNOWN

Verifier Data


Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (10204 MiB free)
  memory of node 1: 16384 MiB (15799 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4300338-1532798796/watcher-4300338-1532798796 -o /tmp/evaluation-result-4300338-1532798796/solver-4300338-1532798796 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node102-1532776103-29944 --watchdog 2580 nacre_mini_release HOME/instance-4300338-1532798796.xml -ca -l100 -cm 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=32654, runsolver pid=32651
Current StackSize limit: 8192 KiB


[startup+0.100637 s]*
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26622888/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=30752 memory=7420 CPUtime=0.1 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 2041 0 0 0 10 0 0 0 20 0 1 0 97442768 31490048 1855 33554432000 4194304 4933779 140734886874848 140734886872536 140407310072805 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 7688 1855 599 181 0 1305 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 30752 KiB
Current children cumulated memory: 7420 KiB

[startup+0.209465 s]*
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26622556/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=31676 memory=8212 CPUtime=0.21 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 2282 0 0 0 21 0 0 0 20 0 1 0 97442768 32436224 2053 33554432000 4194304 4933779 140734886874848 140734886872536 4475935 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 7919 2053 599 181 0 1536 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 31676 KiB
Current children cumulated memory: 8212 KiB

[startup+0.308818 s]*
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26621812/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=32632 memory=9288 CPUtime=0.31 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 2488 0 0 0 31 0 0 0 20 0 1 0 97442768 33415168 2322 33554432000 4194304 4933779 140734886874848 140734886872536 140407310072805 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 8158 2322 599 181 0 1775 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 32632 KiB
Current children cumulated memory: 9288 KiB

[startup+0.700241 s]
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26619952/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=35464 memory=11964 CPUtime=0.69 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 3190 0 0 0 69 0 0 0 20 0 1 0 97442768 36315136 2991 33554432000 4194304 4933779 140734886874848 140734886872616 4380572 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 8866 2991 599 181 0 2483 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 35464 KiB
Current children cumulated memory: 11964 KiB

[startup+1.50025 s]
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26616604/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=40176 memory=16484 CPUtime=1.48 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 4370 0 0 0 148 0 0 0 20 0 1 0 97442768 41140224 4121 33554432000 4194304 4933779 140734886874848 140734886872600 140407310072805 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 10044 4121 599 181 0 3661 0
Current children cumulated CPU time: 1.48 s
Current children cumulated vsize: 40176 KiB
Current children cumulated memory: 16484 KiB

[startup+3.10938 s]
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26609404/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=44400 memory=20968 CPUtime=3.1 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 9153 0 0 0 309 1 0 0 20 0 1 0 97442768 45465600 5242 33554432000 4194304 4933779 140734886874848 140734886872616 140407310072805 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 11100 5242 599 181 0 4717 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 44400 KiB
Current children cumulated memory: 20968 KiB

[startup+6.30024 s]
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26603660/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=49740 memory=25732 CPUtime=6.28 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 15298 0 0 0 627 1 0 0 20 0 1 0 97442768 50933760 6433 33554432000 4194304 4933779 140734886874848 140734886872616 4437393 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 12435 6433 599 181 0 6052 0
Current children cumulated CPU time: 6.28 s
Current children cumulated vsize: 49740 KiB
Current children cumulated memory: 25732 KiB

[startup+12.7003 s]
/proc/loadavg: 2.00 2.08 2.08 3/187 32655
/proc/meminfo: memFree=26597496/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=55808 memory=32012 CPUtime=12.68 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 22753 0 0 0 1266 2 0 0 20 0 1 0 97442768 57147392 8003 33554432000 4194304 4933779 140734886874848 140734886872600 4447703 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 13952 8003 599 181 0 7569 0
Current children cumulated CPU time: 12.68 s
Current children cumulated vsize: 55808 KiB
Current children cumulated memory: 32012 KiB

[startup+25.5003 s]
/proc/loadavg: 2.00 2.07 2.08 3/187 32655
/proc/meminfo: memFree=26587328/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=64352 memory=40552 CPUtime=25.48 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 40993 0 0 0 2545 3 0 0 20 0 1 0 97442768 65896448 10138 33554432000 4194304 4933779 140734886874848 140734886872552 4437570 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 16088 10138 599 181 0 9705 0
Current children cumulated CPU time: 25.48 s
Current children cumulated vsize: 64352 KiB
Current children cumulated memory: 40552 KiB

[startup+51.1096 s]
/proc/loadavg: 2.00 2.07 2.08 3/187 32655
/proc/meminfo: memFree=26575180/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=73712 memory=50236 CPUtime=51.09 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 54021 0 0 0 5104 5 0 0 20 0 1 0 97442768 75481088 12559 33554432000 4194304 4933779 140734886874848 140734886872536 4437536 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 18428 12559 599 181 0 12045 0
Current children cumulated CPU time: 51.09 s
Current children cumulated vsize: 73712 KiB
Current children cumulated memory: 50236 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.06 2.07 3/187 32657
/proc/meminfo: memFree=26559376/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=86456 memory=61924 CPUtime=102.27 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 70107 0 0 0 10220 7 0 0 20 0 1 0 97442768 88530944 15481 33554432000 4194304 4933779 140734886874848 140734886872536 4437570 0 0 4096 24578 0 0 0 17 5 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 21614 15481 599 181 0 15231 0
Current children cumulated CPU time: 102.27 s
Current children cumulated vsize: 86456 KiB
Current children cumulated memory: 61924 KiB

[startup+162.3 s]
/proc/loadavg: 1.92 2.03 2.06 3/187 32658
/proc/meminfo: memFree=26544384/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=96320 memory=71812 CPUtime=162.26 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 87937 0 0 0 16217 9 0 0 20 0 1 0 97442768 98631680 17953 33554432000 4194304 4933779 140734886874848 140734886872552 4437570 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 24080 17953 599 181 0 17697 0
Current children cumulated CPU time: 162.26 s
Current children cumulated vsize: 96320 KiB
Current children cumulated memory: 71812 KiB

[startup+222.3 s]
/proc/loadavg: 1.97 2.02 2.06 3/187 32666
/proc/meminfo: memFree=26528640/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=105380 memory=81208 CPUtime=222.24 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 95718 0 0 0 22213 11 0 0 20 0 1 0 97442768 107909120 20302 33554432000 4194304 4933779 140734886874848 140734886872536 4437792 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 26345 20302 599 181 0 19962 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 105380 KiB
Current children cumulated memory: 81208 KiB

[startup+282.3 s]
/proc/loadavg: 1.99 2.02 2.05 3/187 32667
/proc/meminfo: memFree=26517352/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=112440 memory=88304 CPUtime=282.23 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 99599 0 0 0 28212 11 0 0 20 0 1 0 97442768 115138560 22076 33554432000 4194304 4933779 140734886874848 140734886872536 4380789 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 28110 22076 599 181 0 21727 0
Current children cumulated CPU time: 282.23 s
Current children cumulated vsize: 112440 KiB
Current children cumulated memory: 88304 KiB

[startup+342.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 32668
/proc/meminfo: memFree=26506564/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=118092 memory=94108 CPUtime=342.23 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 104790 0 0 0 34210 13 0 0 20 0 1 0 97442768 120926208 23527 33554432000 4194304 4933779 140734886874848 140734886872360 4437393 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 29523 23527 599 181 0 23140 0
Current children cumulated CPU time: 342.23 s
Current children cumulated vsize: 118092 KiB
Current children cumulated memory: 94108 KiB

[startup+402.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 32669
/proc/meminfo: memFree=26496128/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=123696 memory=99696 CPUtime=402.23 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 111397 0 0 0 40209 14 0 0 20 0 1 0 97442768 126664704 24924 33554432000 4194304 4933779 140734886874848 140734886872536 4437453 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 30924 24924 599 181 0 24541 0
Current children cumulated CPU time: 402.23 s
Current children cumulated vsize: 123696 KiB
Current children cumulated memory: 99696 KiB

[startup+462.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 32670
/proc/meminfo: memFree=26487424/32770624 swapFree=67028336/67108860
[pid=32654] ppid=32651 vsize=128300 memory=104508 CPUtime=462.21 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 119073 0 0 0 46206 15 0 0 20 0 1 0 97442768 131379200 26127 33554432000 4194304 4933779 140734886874848 140734886872600 4437570 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 32075 26127 599 181 0 25692 0
Current children cumulated CPU time: 462.21 s
Current children cumulated vsize: 128300 KiB
Current children cumulated memory: 104508 KiB


################
# More data... #
################

/proc/32654/statm: 46245 39987 599 181 0 39862 0
Current children cumulated CPU time: 1722.19 s
Current children cumulated vsize: 184980 KiB
Current children cumulated memory: 159948 KiB

[startup+1782.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 32709
/proc/meminfo: memFree=26370072/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=186824 memory=161716 CPUtime=1782.19 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 190037 0 0 0 178190 29 0 0 20 0 1 0 97442768 191307776 40429 33554432000 4194304 4933779 140734886874848 140734886872536 4437393 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 46706 40429 599 181 0 40323 0
Current children cumulated CPU time: 1782.19 s
Current children cumulated vsize: 186824 KiB
Current children cumulated memory: 161716 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 32710
/proc/meminfo: memFree=26365844/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=188284 memory=163216 CPUtime=1842.19 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 192866 0 0 0 184189 30 0 0 20 0 1 0 97442768 192802816 40804 33554432000 4194304 4933779 140734886874848 140734886872552 4437564 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 47071 40804 599 181 0 40688 0
Current children cumulated CPU time: 1842.19 s
Current children cumulated vsize: 188284 KiB
Current children cumulated memory: 163216 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/187 32712
/proc/meminfo: memFree=26361864/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=189964 memory=164884 CPUtime=1902.19 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 196168 0 0 0 190188 31 0 0 20 0 1 0 97442768 194523136 41221 33554432000 4194304 4933779 140734886874848 140734886872536 4437445 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 47491 41221 599 181 0 41108 0
Current children cumulated CPU time: 1902.19 s
Current children cumulated vsize: 189964 KiB
Current children cumulated memory: 164884 KiB

[startup+1962.3 s]
/proc/loadavg: 1.92 2.00 2.04 3/187 307
/proc/meminfo: memFree=26523684/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=191100 memory=166168 CPUtime=1962.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 197736 0 0 0 196187 31 0 0 20 0 1 0 97442768 195686400 41542 33554432000 4194304 4933779 140734886874848 140734886872600 4437393 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 47775 41542 599 181 0 41392 0
Current children cumulated CPU time: 1962.18 s
Current children cumulated vsize: 191100 KiB
Current children cumulated memory: 166168 KiB

[startup+2022.3 s]
/proc/loadavg: 1.97 2.00 2.04 3/187 315
/proc/meminfo: memFree=26364624/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=193256 memory=168212 CPUtime=2022.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 199754 0 0 0 202187 31 0 0 20 0 1 0 97442768 197894144 42053 33554432000 4194304 4933779 140734886874848 140734886872536 4437570 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 48314 42053 599 181 0 41931 0
Current children cumulated CPU time: 2022.18 s
Current children cumulated vsize: 193256 KiB
Current children cumulated memory: 168212 KiB

[startup+2082.3 s]
/proc/loadavg: 1.99 2.00 2.04 3/188 341
/proc/meminfo: memFree=26253960/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=194632 memory=169688 CPUtime=2082.19 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 202091 0 0 0 208187 32 0 0 20 0 1 0 97442768 199303168 42422 33554432000 4194304 4933779 140734886874848 140734886872536 4437517 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 48658 42422 599 181 0 42275 0
Current children cumulated CPU time: 2082.19 s
Current children cumulated vsize: 194632 KiB
Current children cumulated memory: 169688 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 342
/proc/meminfo: memFree=26174308/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=195928 memory=171020 CPUtime=2142.19 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 203860 0 0 0 214187 32 0 0 20 0 1 0 97442768 200630272 42755 33554432000 4194304 4933779 140734886874848 140734886872600 4437581 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 48982 42755 599 181 0 42599 0
Current children cumulated CPU time: 2142.19 s
Current children cumulated vsize: 195928 KiB
Current children cumulated memory: 171020 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 343
/proc/meminfo: memFree=26114596/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=198108 memory=172832 CPUtime=2202.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 206110 0 0 0 220186 32 0 0 20 0 1 0 97442768 202862592 43208 33554432000 4194304 4933779 140734886874848 140734886872408 4437393 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 49527 43208 599 181 0 43144 0
Current children cumulated CPU time: 2202.18 s
Current children cumulated vsize: 198108 KiB
Current children cumulated memory: 172832 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 344
/proc/meminfo: memFree=26063336/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=199364 memory=174320 CPUtime=2262.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 208508 0 0 0 226186 32 0 0 20 0 1 0 97442768 204148736 43580 33554432000 4194304 4933779 140734886874848 140734886872552 4437393 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 49841 43580 599 181 0 43458 0
Current children cumulated CPU time: 2262.18 s
Current children cumulated vsize: 199364 KiB
Current children cumulated memory: 174320 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.00 2.04 3/188 345
/proc/meminfo: memFree=26009412/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=200796 memory=175700 CPUtime=2322.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 211286 0 0 0 232185 33 0 0 20 0 1 0 97442768 205615104 43925 33554432000 4194304 4933779 140734886874848 140734886872360 4437389 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 50199 43925 599 181 0 43816 0
Current children cumulated CPU time: 2322.18 s
Current children cumulated vsize: 200796 KiB
Current children cumulated memory: 175700 KiB

[startup+2382.3 s]
/proc/loadavg: 2.22 2.05 2.06 3/187 346
/proc/meminfo: memFree=25958436/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=202388 memory=177100 CPUtime=2382.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 214535 0 0 0 238185 33 0 0 20 0 1 0 97442768 207245312 44275 33554432000 4194304 4933779 140734886874848 140734886872360 4437393 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 50597 44275 599 181 0 44214 0
Current children cumulated CPU time: 2382.18 s
Current children cumulated vsize: 202388 KiB
Current children cumulated memory: 177100 KiB

[startup+2442.3 s]
/proc/loadavg: 2.37 2.14 2.09 3/187 347
/proc/meminfo: memFree=25911972/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=204164 memory=178920 CPUtime=2442.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 218147 0 0 0 244184 34 0 0 20 0 1 0 97442768 209063936 44730 33554432000 4194304 4933779 140734886874848 140734886872536 4437494 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 51041 44730 599 181 0 44658 0
Current children cumulated CPU time: 2442.18 s
Current children cumulated vsize: 204164 KiB
Current children cumulated memory: 178920 KiB

[startup+2502.3 s]
/proc/loadavg: 2.14 2.11 2.08 3/187 348
/proc/meminfo: memFree=25874552/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=205680 memory=180280 CPUtime=2502.18 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 222087 0 0 0 250183 35 0 0 20 0 1 0 97442768 210616320 45070 33554432000 4194304 4933779 140734886874848 140734886872360 4437393 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 51420 45070 599 181 0 45037 0
Current children cumulated CPU time: 2502.18 s
Current children cumulated vsize: 205680 KiB
Current children cumulated memory: 180280 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+2520 s]*
/proc/loadavg: 2.10 2.11 2.08 3/187 348
/proc/meminfo: memFree=25862740/32770624 swapFree=67028340/67108860
[pid=32654] ppid=32651 vsize=206236 memory=180892 CPUtime=2519.89 cores=1,3,5,7
/proc/32654/stat : 32654 (nacre_mini_rele) R 32651 32654 29912 0 -1 4202496 226006 0 0 0 251954 35 0 0 20 0 1 0 97442768 211185664 45223 33554432000 4194304 4933779 140734886874848 140734886872360 4437389 0 0 4096 24578 0 0 0 17 1 0 0 0 0 0 7034520 7039168 17567744 140734886876312 140734886876412 140734886876412 140734886879170 0
/proc/32654/statm: 51559 45223 599 181 0 45176 0
Current children cumulated CPU time: 2519.89 s
Current children cumulated vsize: 206236 KiB
Current children cumulated memory: 180892 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2520.01 s]
# the end of solver process 32654 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2519.54 s, system=0.359994 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 0
Real time (s): 2520.01
CPU time (s): 2519.9
CPU user time (s): 2519.54
CPU system time (s): 0.359994
CPU usage (%): 99.9957
Max. virtual memory (cumulated for all children) (KiB): 206236
Max. memory (cumulated for all children) (KiB): 180892

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2519.54
system time used= 0.359994
maximum resident set size= 181016
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 226050
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 87
involuntary context switches= 2508


# summary of solver processes directly reported to runsolver:
#   pid: 32654
#   total CPU time (s): 2519.9
#   total CPU user time (s): 2519.54
#   total CPU system time (s): 0.359994

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 8.90687 second user time and 19.7302 second system time

The end

Launcher Data 

Begin job on node102 at 2018-07-28 19:26:36
IDJOB=4300338
IDBENCH=141587
IDSOLVER=2764
FILE ID=node102/4300338-1532798796
RUNJOBID= node102-1532776103-29944
PBS_JOBID= 21022470
Free space on /tmp= 52744 MiB

SOLVER NAME= NACRE 1.0.4
BENCH NAME= XCSP18/Frb/frb-45-21-1_c18.xml
COMMAND LINE= nacre_mini_release BENCHNAME -ca -l100 -cm
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4300338-1532798796/watcher-4300338-1532798796 -o /tmp/evaluation-result-4300338-1532798796/solver-4300338-1532798796 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node102-1532776103-29944 --watchdog 2580  nacre_mini_release HOME/instance-4300338-1532798796.xml -ca -l100 -cm

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b87938282b595c2beb3624311f73a470
RANDOM SEED=1147214781

node102.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.51
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        26628660 kB
MemAvailable:   31772260 kB
Buffers:          215876 kB
Cached:          5183152 kB
SwapCached:        53432 kB
Active:          1369184 kB
Inactive:        4272568 kB
Active(anon):     168580 kB
Inactive(anon):    99084 kB
Active(file):    1200604 kB
Inactive(file):  4173484 kB
Unevictable:        6644 kB
Mlocked:            6644 kB
SwapTotal:      67108860 kB
SwapFree:       67028336 kB
Dirty:              1172 kB
Writeback:             0 kB
AnonPages:        196604 kB
Mapped:            37284 kB
Shmem:             22112 kB
Slab:             198720 kB
SReclaimable:     170696 kB
SUnreclaim:        28024 kB
KernelStack:        3328 kB
PageTables:         5512 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     512712 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:    153600 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      105856 kB
DirectMap2M:    33439744 kB

Free space on /tmp at the end= 52744 MiB
End job on node102 at 2018-07-28 20:08:36