Trace number 4295985

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PicatSAT 2018-06-15? (TO) 2520.07 2519.91

General information on the benchmark

NameTravellingSalesman/
TravellingSalesman-40-30-00_c18.xml
MD5SUMcd8c3cb9db38dc1b01c6d50a781e959c
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark170
Best CPU time to get the best result obtained on this benchmark2520.12
Satisfiable
(Un)Satisfiability was proved
Number of variables80
Number of constraints41
Number of domains2
Minimum domain size33
Maximum domain size40
Distribution of domain sizes[{"size":33,"count":40},{"size":40,"count":40}]
Minimum variable degree2
Maximum variable degree3
Distribution of variable degrees[{"degree":2,"count":40},{"degree":3,"count":40}]
Minimum constraint arity3
Maximum constraint arity40
Distribution of constraint arities[{"arity":3,"count":40},{"arity":40,"count":1}]
Number of extensional constraints40
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":40},{"type":"allDifferent","count":1}]
Optimization problemYES
Type of objectivemin SUM

Solver Data


Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (3748 MiB free)
  memory of node 1: 16384 MiB (7132 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4295985-1532594753/watcher-4295985-1532594753 -o /tmp/evaluation-result-4295985-1532594753/solver-4295985-1532594753 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node104-1532594747-28433 --watchdog 2580 xcsp_picat HOME/instance-4295985-1532594753.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=28527, runsolver pid=28524

[startup+0.100112 s]*
/proc/loadavg: 1.09 1.35 1.52 3/188 28528
/proc/meminfo: memFree=11125316/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=128356 memory=19380 CPUtime=0.1 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 1929 0 0 0 10 0 0 0 20 0 1 0 77038461 131436544 4845 33554432000 4194304 5849740 140733073291616 140733073290600 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 32089 4845 707 405 0 29033 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 128356 KiB
Current children cumulated memory: 19380 KiB

[startup+0.209617 s]*
/proc/loadavg: 1.09 1.35 1.52 3/188 28528
/proc/meminfo: memFree=11124944/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=128616 memory=19588 CPUtime=0.2 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 2042 0 0 0 20 0 0 0 20 0 1 0 77038461 131702784 4897 33554432000 4194304 5849740 140733073291616 140733073290600 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 32154 4897 709 405 0 29098 0
Current children cumulated CPU time: 0.2 s
Current children cumulated vsize: 128616 KiB
Current children cumulated memory: 19588 KiB

[startup+0.308754 s]*
/proc/loadavg: 1.09 1.35 1.52 3/188 28528
/proc/meminfo: memFree=11121992/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=129128 memory=19972 CPUtime=0.3 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 2170 0 0 0 30 0 0 0 20 0 1 0 77038461 132227072 4993 33554432000 4194304 5849740 140733073291616 140733073290600 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 32282 4993 709 405 0 29226 0
Current children cumulated CPU time: 0.3 s
Current children cumulated vsize: 129128 KiB
Current children cumulated memory: 19972 KiB

[startup+0.700252 s]
/proc/loadavg: 1.09 1.35 1.52 3/188 28528
/proc/meminfo: memFree=11123852/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=131176 memory=22040 CPUtime=0.69 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 2682 0 0 0 67 2 0 0 20 0 1 0 77038461 134324224 5510 33554432000 4194304 5849740 140733073291616 140733073290600 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 32794 5510 709 405 0 29738 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 131176 KiB
Current children cumulated memory: 22040 KiB

[startup+1.50024 s]
/proc/loadavg: 1.09 1.35 1.52 3/188 28528
/proc/meminfo: memFree=11119448/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=134392 memory=25144 CPUtime=1.49 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 3454 0 0 0 144 5 0 0 20 0 1 0 77038461 137617408 6286 33554432000 4194304 5849740 140733073291616 140733073290600 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 33598 6286 709 405 0 30542 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 134392 KiB
Current children cumulated memory: 25144 KiB

[startup+3.10927 s]
/proc/loadavg: 1.09 1.35 1.52 3/188 28528
/proc/meminfo: memFree=11110832/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=140536 memory=31636 CPUtime=3.1 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 4814 0 0 0 299 11 0 0 20 0 1 0 77038461 143908864 7909 33554432000 4194304 5849740 140733073291616 140733073290856 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 35134 7909 709 405 0 32078 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 140536 KiB
Current children cumulated memory: 31636 KiB

[startup+6.30034 s]
/proc/loadavg: 1.17 1.36 1.52 3/188 28528
/proc/meminfo: memFree=11104372/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=128836 memory=20112 CPUtime=6.29 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 11897 0 0 0 616 13 0 0 20 0 1 0 77038461 131928064 5028 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 32209 5028 750 405 0 29153 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 128836 KiB
Current children cumulated memory: 20112 KiB

[startup+12.7003 s]
/proc/loadavg: 1.23 1.37 1.52 3/188 28528
/proc/meminfo: memFree=11083548/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=170704 memory=53856 CPUtime=12.69 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 16586 0 0 0 1244 25 0 0 20 0 1 0 77038461 174800896 13464 33554432000 4194304 5849740 140733073291616 140733073290776 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 42676 13464 750 405 0 39620 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 170704 KiB
Current children cumulated memory: 53856 KiB

[startup+25.5003 s]
/proc/loadavg: 1.40 1.40 1.53 3/188 28528
/proc/meminfo: memFree=11079520/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=170704 memory=53920 CPUtime=25.49 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 21472 0 0 0 2512 37 0 0 20 0 1 0 77038461 174800896 13480 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 42676 13480 750 405 0 39620 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 170704 KiB
Current children cumulated memory: 53920 KiB

[startup+51.1095 s]
/proc/loadavg: 1.61 1.45 1.54 3/188 28528
/proc/meminfo: memFree=11076976/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=170704 memory=58016 CPUtime=51.1 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 25342 0 0 0 5052 58 0 0 20 0 1 0 77038461 174800896 14504 33554432000 4194304 5849740 140733073291616 140733073290616 5317712 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 42676 14504 750 405 0 39620 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 170704 KiB
Current children cumulated memory: 58016 KiB

[startup+102.3 s]
/proc/loadavg: 1.83 1.53 1.57 3/188 28529
/proc/meminfo: memFree=11037440/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=174888 memory=71472 CPUtime=102.3 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 27371 0 0 0 10127 103 0 0 20 0 1 0 77038461 179085312 17868 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 43722 17868 750 405 0 40666 0
Current children cumulated CPU time: 102.3 s
Current children cumulated vsize: 174888 KiB
Current children cumulated memory: 71472 KiB

[startup+162.3 s]
/proc/loadavg: 1.94 1.62 1.59 3/188 28530
/proc/meminfo: memFree=10993436/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=194708 memory=93792 CPUtime=162.3 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 32857 0 0 0 16080 150 0 0 20 0 1 0 77038461 199380992 23448 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 48677 23448 750 405 0 45621 0
Current children cumulated CPU time: 162.3 s
Current children cumulated vsize: 194708 KiB
Current children cumulated memory: 93792 KiB

[startup+222.3 s]
/proc/loadavg: 1.98 1.69 1.62 3/188 28531
/proc/meminfo: memFree=10983340/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=170696 memory=61960 CPUtime=222.32 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 38335 0 0 0 22021 211 0 0 20 0 1 0 77038461 174792704 15490 33554432000 4194304 5849740 140733073291616 140733073290264 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 42674 15490 750 405 0 39618 0
Current children cumulated CPU time: 222.32 s
Current children cumulated vsize: 170696 KiB
Current children cumulated memory: 61960 KiB

[startup+282.3 s]
/proc/loadavg: 2.15 1.78 1.65 3/189 28540
/proc/meminfo: memFree=10947216/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=178936 memory=78044 CPUtime=282.32 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 40332 0 0 0 27974 258 0 0 20 0 1 0 77038461 183230464 19511 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 44734 19511 750 405 0 41678 0
Current children cumulated CPU time: 282.32 s
Current children cumulated vsize: 178936 KiB
Current children cumulated memory: 78044 KiB

[startup+342.3 s]
/proc/loadavg: 2.06 1.82 1.68 3/188 28541
/proc/meminfo: memFree=10922052/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=201012 memory=100320 CPUtime=342.33 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 46750 0 0 0 33931 302 0 0 20 0 1 0 77038461 205836288 25080 33554432000 4194304 5849740 140733073291616 140733073290344 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 50253 25080 750 405 0 47197 0
Current children cumulated CPU time: 342.33 s
Current children cumulated vsize: 201012 KiB
Current children cumulated memory: 100320 KiB

[startup+402.3 s]
/proc/loadavg: 2.02 1.85 1.69 3/188 28542
/proc/meminfo: memFree=10839880/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=228408 memory=127568 CPUtime=402.33 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 47980 0 0 0 39891 342 0 0 20 0 1 0 77038461 233889792 31892 33554432000 4194304 5849740 140733073291616 140733073290776 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 57102 31892 750 405 0 54046 0
Current children cumulated CPU time: 402.33 s
Current children cumulated vsize: 228408 KiB
Current children cumulated memory: 127568 KiB

[startup+462.3 s]
/proc/loadavg: 2.01 1.88 1.71 3/188 28543
/proc/meminfo: memFree=10814380/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=228408 memory=127736 CPUtime=462.33 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 48055 0 0 0 45854 379 0 0 20 0 1 0 77038461 233889792 31934 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 57102 31934 750 405 0 54046 0
Current children cumulated CPU time: 462.33 s
Current children cumulated vsize: 228408 KiB
Current children cumulated memory: 127736 KiB


################
# More data... #
################

/proc/28527/statm: 193540 168372 750 405 0 190484 0
Current children cumulated CPU time: 1782.43 s
Current children cumulated vsize: 774160 KiB
Current children cumulated memory: 673488 KiB

[startup+1842.3 s]
/proc/loadavg: 2.00 2.05 1.96 3/187 28606
/proc/meminfo: memFree=9984432/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=774160 memory=673488 CPUtime=1842.43 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 338479 0 0 0 183442 801 0 0 20 0 1 0 77038461 792739840 168372 33554432000 4194304 5849740 140733073291616 140733073290616 5437825 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 193540 168372 750 405 0 190484 0
Current children cumulated CPU time: 1842.43 s
Current children cumulated vsize: 774160 KiB
Current children cumulated memory: 673488 KiB

[startup+1902.3 s]
/proc/loadavg: 2.00 2.04 1.96 3/187 28607
/proc/meminfo: memFree=9983744/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=774160 memory=673488 CPUtime=1902.43 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 371011 0 0 0 189430 813 0 0 20 0 1 0 77038461 792739840 168372 33554432000 4194304 5849740 140733073291616 140733073290616 5437620 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 193540 168372 750 405 0 190484 0
Current children cumulated CPU time: 1902.43 s
Current children cumulated vsize: 774160 KiB
Current children cumulated memory: 673488 KiB

[startup+1962.3 s]
/proc/loadavg: 2.00 2.04 1.96 3/187 28608
/proc/meminfo: memFree=10270756/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=488576 memory=388032 CPUtime=1962.42 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 397603 0 0 0 195414 828 0 0 20 0 1 0 77038461 500301824 97008 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 122144 97008 750 405 0 119088 0
Current children cumulated CPU time: 1962.42 s
Current children cumulated vsize: 488576 KiB
Current children cumulated memory: 388032 KiB

[startup+2022.3 s]
/proc/loadavg: 2.00 2.03 1.96 3/187 28609
/proc/meminfo: memFree=10145600/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=513492 memory=412948 CPUtime=2022.44 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 413763 0 0 0 201404 840 0 0 20 0 1 0 77038461 525815808 103237 33554432000 4194304 5849740 140733073291616 140733073290616 5437698 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 128373 103237 750 405 0 125317 0
Current children cumulated CPU time: 2022.44 s
Current children cumulated vsize: 513492 KiB
Current children cumulated memory: 412948 KiB

[startup+2082.3 s]
/proc/loadavg: 2.00 2.02 1.96 3/187 28617
/proc/meminfo: memFree=10090156/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=567276 memory=466732 CPUtime=2082.44 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 422978 0 0 0 207389 855 0 0 20 0 1 0 77038461 580890624 116683 33554432000 4194304 5849740 140733073291616 140733073290776 5437698 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 141819 116683 750 405 0 138763 0
Current children cumulated CPU time: 2082.44 s
Current children cumulated vsize: 567276 KiB
Current children cumulated memory: 466732 KiB

[startup+2142.3 s]
/proc/loadavg: 2.00 2.02 1.96 3/187 28618
/proc/meminfo: memFree=10088196/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=567276 memory=466732 CPUtime=2142.45 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 438386 0 0 0 213380 865 0 0 20 0 1 0 77038461 580890624 116683 33554432000 4194304 5849740 140733073291616 140733073290616 5438192 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 141819 116683 750 405 0 138763 0
Current children cumulated CPU time: 2142.45 s
Current children cumulated vsize: 567276 KiB
Current children cumulated memory: 466732 KiB

[startup+2202.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28619
/proc/meminfo: memFree=10089084/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=567276 memory=466732 CPUtime=2202.45 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 449935 0 0 0 219371 874 0 0 20 0 1 0 77038461 580890624 116683 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 141819 116683 750 405 0 138763 0
Current children cumulated CPU time: 2202.45 s
Current children cumulated vsize: 567276 KiB
Current children cumulated memory: 466732 KiB

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28620
/proc/meminfo: memFree=10658236/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=567276 memory=466732 CPUtime=2262.46 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 467037 0 0 0 225361 885 0 0 20 0 1 0 77038461 580890624 116683 33554432000 4194304 5849740 140733073291616 140733073290616 5413317 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 141819 116683 750 405 0 138763 0
Current children cumulated CPU time: 2262.46 s
Current children cumulated vsize: 567276 KiB
Current children cumulated memory: 466732 KiB

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28621
/proc/meminfo: memFree=10599804/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=567276 memory=466732 CPUtime=2322.47 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 478417 0 0 0 231352 895 0 0 20 0 1 0 77038461 580890624 116683 33554432000 4194304 5849740 140733073291616 140733073290616 5339693 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 141819 116683 750 405 0 138763 0
Current children cumulated CPU time: 2322.47 s
Current children cumulated vsize: 567276 KiB
Current children cumulated memory: 466732 KiB

[startup+2382.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28623
/proc/meminfo: memFree=10481780/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=668428 memory=567884 CPUtime=2382.46 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 511317 0 0 0 237337 909 0 0 20 0 1 0 77038461 684470272 141971 33554432000 4194304 5849740 140733073291616 140733073290488 5312778 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 167107 141971 750 405 0 164051 0
Current children cumulated CPU time: 2382.46 s
Current children cumulated vsize: 668428 KiB
Current children cumulated memory: 567884 KiB

[startup+2442.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28624
/proc/meminfo: memFree=10496564/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=738500 memory=637956 CPUtime=2442.46 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 516876 0 0 0 243324 922 0 0 20 0 1 0 77038461 756224000 159489 33554432000 4194304 5849740 140733073291616 140733073290616 5437698 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 184625 159489 750 405 0 181569 0
Current children cumulated CPU time: 2442.46 s
Current children cumulated vsize: 738500 KiB
Current children cumulated memory: 637956 KiB

[startup+2502.3 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28625
/proc/meminfo: memFree=10441184/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=738500 memory=637956 CPUtime=2502.46 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 545606 0 0 0 249311 935 0 0 20 0 1 0 77038461 756224000 159489 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 184625 159489 750 405 0 181569 0
Current children cumulated CPU time: 2502.46 s
Current children cumulated vsize: 738500 KiB
Current children cumulated memory: 637956 KiB



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+2519.9 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28686
/proc/meminfo: memFree=10430160/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=804036 memory=703492 CPUtime=2520.06 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 562945 0 0 0 251066 940 0 0 20 0 1 0 77038461 823332864 175873 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 201009 175873 750 405 0 197953 0
Current children cumulated CPU time: 2520.06 s
Current children cumulated vsize: 804036 KiB
Current children cumulated memory: 703492 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+2519.91 s]
# the end of solver process 28527 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=2510.66 s, system=9.41103 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

[startup+2519.9 s]
/proc/loadavg: 2.00 2.01 1.96 3/187 28686
/proc/meminfo: memFree=10430160/32770624 swapFree=67108860/67108860
[pid=28527] ppid=28524 vsize=804036 memory=703492 CPUtime=2520.06 cores=1,3,5,7
/proc/28527/stat : 28527 (xcsp_picat) R 28524 28527 26989 0 -1 4202496 562945 0 0 0 251066 940 0 0 20 0 1 0 77038461 823332864 175873 33554432000 4194304 5849740 140733073291616 140733073290616 140104346140551 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 27033600 140733073298614 140733073298692 140733073298692 140733073301450 0
/proc/28527/statm: 201009 175873 750 405 0 197953 0
Current children cumulated CPU time: 2520.06 s
Current children cumulated vsize: 804036 KiB
Current children cumulated memory: 703492 KiB

Child ended because it received signal 15 (SIGTERM)
Real time (s): 2519.91
CPU time (s): 2520.07
CPU user time (s): 2510.66
CPU system time (s): 9.41103
CPU usage (%): 100.007
Max. virtual memory (cumulated for all children) (KiB): 839696
Max. memory (cumulated for all children) (KiB): 739024

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2510.66
system time used= 9.41103
maximum resident set size= 739024
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 562945
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 216
involuntary context switches= 825


# summary of solver processes directly reported to runsolver:
#   pid: 28527
#   total CPU time (s): 2520.07
#   total CPU user time (s): 2510.66
#   total CPU system time (s): 9.41103

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 11.4982 second user time and 17.2852 second system time

The end

Launcher Data 

Begin job on node104 at 2018-07-26 10:45:53
IDJOB=4295985
IDBENCH=141628
IDSOLVER=2754
FILE ID=node104/4295985-1532594753
RUNJOBID= node104-1532594747-28433
PBS_JOBID= 21021968
Free space on /tmp= 48344 MiB

SOLVER NAME= PicatSAT 2018-06-15
BENCH NAME= XCSP18/TravellingSalesman/TravellingSalesman-40-30-00_c18.xml
COMMAND LINE= xcsp_picat BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4295985-1532594753/watcher-4295985-1532594753 -o /tmp/evaluation-result-4295985-1532594753/solver-4295985-1532594753 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node104-1532594747-28433 --watchdog 2580  xcsp_picat HOME/instance-4295985-1532594753.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= cd8c3cb9db38dc1b01c6d50a781e959c
RANDOM SEED=9719567

node104.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.11
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.11
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.11
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.11
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        11141952 kB
MemAvailable:   31902540 kB
Buffers:          219832 kB
Cached:         20533188 kB
SwapCached:            0 kB
Active:          4171676 kB
Inactive:       16664480 kB
Active(anon):      34520 kB
Inactive(anon):    76284 kB
Active(file):    4137156 kB
Inactive(file): 16588196 kB
Unevictable:        7172 kB
Mlocked:            7172 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:              3316 kB
Writeback:           520 kB
AnonPages:         85384 kB
Mapped:            43216 kB
Shmem:             25228 kB
Slab:             492128 kB
SReclaimable:     463696 kB
SUnreclaim:        28432 kB
KernelStack:        3376 kB
PageTables:         5208 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     468832 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     24576 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      116096 kB
DirectMap2M:    33429504 kB

Free space on /tmp at the end= 48344 MiB
End job on node104 at 2018-07-26 11:27:53