Trace number 4293972

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
PicatSAT 2018-06-15? (exit code) 1341.36 1341.42

General information on the benchmark

NameCrosswordDesign/
CrosswordDesign-08-4-rom_c18.xml
MD5SUMd2036c1580b702d26a2ba71e1295e02d
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT
Best value of the objective obtained on this benchmark70
Best CPU time to get the best result obtained on this benchmark2520.2
Satisfiable
(Un)Satisfiability was proved
Number of variables256
Number of constraints64
Number of domains5
Minimum domain size2
Maximum domain size62899
Distribution of domain sizes[{"size":2,"count":16},{"size":9,"count":112},{"size":27,"count":64},{"size":62899,"count":64}]
Minimum variable degree1
Maximum variable degree8
Distribution of variable degrees[{"degree":1,"count":80},{"degree":2,"count":112},{"degree":8,"count":64}]
Minimum constraint arity11
Maximum constraint arity12
Distribution of constraint arities[{"arity":11,"count":16},{"arity":12,"count":48}]
Number of extensional constraints64
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":64}]
Optimization problemYES
Type of objectivemax SUM

Solver Data 

1341.23/1341.38	% error: OUT OF MEMORY
1341.23/1341.38	% unhandled interrupt event 
1341.23/1341.38	c xcsp_picat version 2.4#4 (June 15, 2018)

Verifier Data 

One s line expected

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (13628 MiB free)
  memory of node 1: 16384 MiB (11901 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4293972-1532404641/watcher-4293972-1532404641 -o /tmp/evaluation-result-4293972-1532404641/solver-4293972-1532404641 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node151-1532401118-22575 --watchdog 2580 xcsp_picat HOME/instance-4293972-1532404641.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2550 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2520 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
Current StackSize limit: 8192 KiB

solver pid=23051, runsolver pid=23047

[startup+0.100161 s]*
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=26129732/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=111688 memory=17480 CPUtime=0.1 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1458 0 0 0 10 0 0 0 20 0 1 0 58027350 114368512 4370 33554432000 4194304 5849740 140729478617952 140729478617952 4488923 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 27922 4370 593 405 0 24866 0
Current children cumulated CPU time: 0.1 s
Current children cumulated vsize: 111688 KiB
Current children cumulated memory: 17480 KiB

[startup+0.209605 s]*
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=26123588/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=111688 memory=23624 CPUtime=0.21 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1461 0 0 0 21 0 0 0 20 0 1 0 58027350 114368512 5906 33554432000 4194304 5849740 140729478617952 140729478617952 140719069527040 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 27922 5906 593 405 0 24866 0
Current children cumulated CPU time: 0.21 s
Current children cumulated vsize: 111688 KiB
Current children cumulated memory: 23624 KiB

[startup+0.309381 s]*
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=26117444/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=111688 memory=27720 CPUtime=0.31 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1463 0 0 0 31 0 0 0 20 0 1 0 58027350 114368512 6930 33554432000 4194304 5849740 140729478617952 140729478617952 4476406 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 27922 6930 593 405 0 24866 0
Current children cumulated CPU time: 0.31 s
Current children cumulated vsize: 111688 KiB
Current children cumulated memory: 27720 KiB

[startup+0.700397 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=26105156/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=156828 memory=66528 CPUtime=0.69 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 2473 0 0 0 68 1 0 0 20 0 1 0 58027350 160591872 16632 33554432000 4194304 5849740 140729478617952 140729478617952 4494163 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 39207 16632 597 405 0 36151 0
Current children cumulated CPU time: 0.69 s
Current children cumulated vsize: 156828 KiB
Current children cumulated memory: 66528 KiB

[startup+1.50039 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=26068296/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=222048 memory=152032 CPUtime=1.49 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 3331 0 0 0 145 4 0 0 20 0 1 0 58027350 227377152 38008 33554432000 4194304 5849740 140729478617952 140729478617952 4490517 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 55512 38008 614 405 0 52456 0
Current children cumulated CPU time: 1.49 s
Current children cumulated vsize: 222048 KiB
Current children cumulated memory: 152032 KiB

[startup+3.10976 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=25574792/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=627324 memory=328156 CPUtime=3.1 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 5850 0 0 0 299 11 0 0 20 0 1 0 58027350 642379776 82039 33554432000 4194304 5849740 140729478617952 140729478617240 4572141 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 156831 82039 614 405 0 153775 0
Current children cumulated CPU time: 3.1 s
Current children cumulated vsize: 627324 KiB
Current children cumulated memory: 328156 KiB

[startup+6.30035 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=25448336/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=1166384 memory=752660 CPUtime=6.29 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 8142 0 0 0 603 26 0 0 20 0 1 0 58027350 1194377216 188165 33554432000 4194304 5849740 140729478617952 140729478617240 4535869 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 291596 188165 614 405 0 288540 0
Current children cumulated CPU time: 6.29 s
Current children cumulated vsize: 1166384 KiB
Current children cumulated memory: 752660 KiB

[startup+12.7004 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=24670800/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=2244508 memory=1607052 CPUtime=12.69 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 12063 0 0 0 1213 56 0 0 20 0 1 0 58027350 2298376192 401763 33554432000 4194304 5849740 140729478617952 140729478617240 4539531 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 561127 401763 614 405 0 558071 0
Current children cumulated CPU time: 12.69 s
Current children cumulated vsize: 2244508 KiB
Current children cumulated memory: 1607052 KiB

[startup+25.5004 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23052
/proc/meminfo: memFree=22942972/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=4400760 memory=3294328 CPUtime=25.49 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 15104 0 0 0 2434 115 0 0 20 0 1 0 58027350 4506378240 823582 33554432000 4194304 5849740 140729478617952 140729478617240 4504018 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 1100190 823582 623 405 0 1097134 0
Current children cumulated CPU time: 25.49 s
Current children cumulated vsize: 4400760 KiB
Current children cumulated memory: 3294328 KiB

[startup+51.1095 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23053
/proc/meminfo: memFree=19675516/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=8566564 memory=6457572 CPUtime=51.1 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 19098 0 0 0 4944 166 0 0 20 0 1 0 58027350 8772161536 1614393 33554432000 4194304 5849740 140729478617952 140729478617272 4518130 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 2141641 1614393 623 405 0 2138585 0
Current children cumulated CPU time: 51.1 s
Current children cumulated vsize: 8566564 KiB
Current children cumulated memory: 6457572 KiB

[startup+102.3 s]
/proc/loadavg: 2.00 2.00 1.95 3/187 23054
/proc/meminfo: memFree=15514688/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=12751264 memory=10603592 CPUtime=102.29 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 41121 0 0 0 9570 659 0 0 20 0 1 0 58027350 13057294336 2650898 33554432000 4194304 5849740 140729478617952 140729478616952 5257261 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3187816 2650898 624 405 0 3184760 0
Current children cumulated CPU time: 102.29 s
Current children cumulated vsize: 12751264 KiB
Current children cumulated memory: 10603592 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.00 1.95 3/188 23063
/proc/meminfo: memFree=15382140/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=12884076 memory=10733976 CPUtime=162.28 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 67250 0 0 0 15001 1227 0 0 20 0 1 0 58027350 13193293824 2683494 33554432000 4194304 5849740 140729478617952 140729478616952 5260666 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3221019 2683494 624 405 0 3217963 0
Current children cumulated CPU time: 162.28 s
Current children cumulated vsize: 12884076 KiB
Current children cumulated memory: 10733976 KiB

[startup+222.3 s]
/proc/loadavg: 1.96 1.99 1.95 3/188 23148
/proc/meminfo: memFree=8972856/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=13016888 memory=10866408 CPUtime=222.29 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 95558 0 0 0 20417 1812 0 0 20 0 1 0 58027350 13329293312 2716602 33554432000 4194304 5849740 140729478617952 140729478616952 5260513 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3254222 2716602 624 405 0 3251166 0
Current children cumulated CPU time: 222.29 s
Current children cumulated vsize: 13016888 KiB
Current children cumulated memory: 10866408 KiB

[startup+282.3 s]
/proc/loadavg: 1.98 1.99 1.95 3/188 23210
/proc/meminfo: memFree=15750224/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=13133100 memory=10988384 CPUtime=282.28 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 121261 0 0 0 25887 2341 0 0 20 0 1 0 58027350 13448294400 2747096 33554432000 4194304 5849740 140729478617952 140729478616952 5269597 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3283275 2747096 624 405 0 3280219 0
Current children cumulated CPU time: 282.28 s
Current children cumulated vsize: 13133100 KiB
Current children cumulated memory: 10988384 KiB

[startup+342.3 s]
/proc/loadavg: 2.07 2.00 1.96 3/188 23211
/proc/meminfo: memFree=15633552/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=13265912 memory=11118768 CPUtime=342.27 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 151500 0 0 0 31282 2945 0 0 20 0 1 0 58027350 13584293888 2779692 33554432000 4194304 5849740 140729478617952 140729478616952 5260488 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3316478 2779692 624 405 0 3313422 0
Current children cumulated CPU time: 342.27 s
Current children cumulated vsize: 13265912 KiB
Current children cumulated memory: 11118768 KiB

[startup+402.3 s]
/proc/loadavg: 2.35 2.12 2.00 3/188 23212
/proc/meminfo: memFree=15518204/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=9039352 memory=6600764 CPUtime=402.28 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 179283 0 0 0 36742 3486 0 0 20 0 1 0 58027350 9256296448 1649679 33554432000 4194304 5849740 140729478617952 140729478617240 5257103 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 2259838 1650191 624 405 0 2256782 0
Current children cumulated CPU time: 402.28 s
Current children cumulated vsize: 9039352 KiB
Current children cumulated memory: 6600764 KiB

[startup+462.3 s]
/proc/loadavg: 2.16 2.11 2.01 3/187 23276
/proc/meminfo: memFree=9083260/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=13514940 memory=11367140 CPUtime=462.27 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 209311 0 0 0 42121 4106 0 0 20 0 1 0 58027350 13839298560 2841785 33554432000 4194304 5849740 140729478617952 140729478616952 5260674 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3378735 2841785 624 405 0 3375679 0
Current children cumulated CPU time: 462.27 s
Current children cumulated vsize: 13514940 KiB
Current children cumulated memory: 11367140 KiB


################
# More data... #
################

/proc/23051/statm: 3696766 3478369 624 405 0 3693710 0
Current children cumulated CPU time: 882.25 s
Current children cumulated vsize: 14787064 KiB
Current children cumulated memory: 13913476 KiB

[startup+942.3 s]
/proc/loadavg: 2.00 2.02 2.01 3/188 23353
/proc/meminfo: memFree=5417520/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=14886676 memory=14014568 CPUtime=942.25 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 445198 0 0 0 85991 8234 0 0 20 0 1 0 58027350 15243956224 3503642 33554432000 4194304 5849740 140729478617952 140729478616952 5269552 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3721669 3503642 624 405 0 3718613 0
Current children cumulated CPU time: 942.25 s
Current children cumulated vsize: 14886676 KiB
Current children cumulated memory: 14014568 KiB

[startup+1002.3 s]
/proc/loadavg: 2.00 2.02 2.01 3/188 23354
/proc/meminfo: memFree=1874168/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=14986288 memory=14117708 CPUtime=1002.24 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 471067 0 0 0 91459 8765 0 0 20 0 1 0 58027350 15345958912 3529427 33554432000 4194304 5849740 140729478617952 140729478616952 5269570 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3746572 3529427 624 405 0 3743516 0
Current children cumulated CPU time: 1002.24 s
Current children cumulated vsize: 14986288 KiB
Current children cumulated memory: 14117708 KiB

[startup+1062.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23355
/proc/meminfo: memFree=5412768/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=15085896 memory=14214692 CPUtime=1062.25 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 497543 0 0 0 96917 9308 0 0 20 0 1 0 58027350 15447957504 3553673 33554432000 4194304 5849740 140729478617952 140729478616952 5261040 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3771474 3553673 624 405 0 3768418 0
Current children cumulated CPU time: 1062.25 s
Current children cumulated vsize: 15085896 KiB
Current children cumulated memory: 14214692 KiB

[startup+1122.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23356
/proc/meminfo: memFree=1428620/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=15185504 memory=14315788 CPUtime=1122.24 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 524788 0 0 0 102372 9852 0 0 20 0 1 0 58027350 15549956096 3578947 33554432000 4194304 5849740 140729478617952 140729478616952 5260776 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3796376 3578947 624 405 0 3793320 0
Current children cumulated CPU time: 1122.24 s
Current children cumulated vsize: 15185504 KiB
Current children cumulated memory: 14315788 KiB

[startup+1182.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23357
/proc/meminfo: memFree=1210360/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=15285116 memory=14413652 CPUtime=1182.24 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 604874 0 0 0 107813 10411 0 0 20 0 1 0 58027350 15651958784 3603413 33554432000 4194304 5849740 140729478617952 140729478616952 5257136 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3821279 3603413 624 405 0 3818223 0
Current children cumulated CPU time: 1182.24 s
Current children cumulated vsize: 15285116 KiB
Current children cumulated memory: 14413652 KiB

[startup+1242.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23358
/proc/meminfo: memFree=1262792/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=15384724 memory=14511788 CPUtime=1242.24 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 756837 0 0 0 113252 10972 0 0 20 0 1 0 58027350 15753957376 3627947 33554432000 4194304 5849740 140729478617952 140729478616952 5260724 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3846181 3627947 624 405 0 3843125 0
Current children cumulated CPU time: 1242.24 s
Current children cumulated vsize: 15384724 KiB
Current children cumulated memory: 14511788 KiB

[startup+1302.3 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23359
/proc/meminfo: memFree=902548/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=15467728 memory=14595852 CPUtime=1302.24 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 968199 0 0 0 118764 11460 0 0 20 0 1 0 58027350 15838953472 3648963 33554432000 4194304 5849740 140729478617952 140729478616952 5269579 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3866932 3648963 624 405 0 3863876 0
Current children cumulated CPU time: 1302.24 s
Current children cumulated vsize: 15467728 KiB
Current children cumulated memory: 14595852 KiB

[startup+1341.42 s]
# the end of solver process 23051 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=1223.69 s, system=117.663 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

[startup+1305.6 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23359
/proc/meminfo: memFree=2140616/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=15484332 memory=14473100 CPUtime=1305.53 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 12593152 1065393 0 0 0 118986 11567 0 0 20 0 1 0 58027350 15855955968 3618275 33554432000 4194304 5849740 140729478617952 140729478616952 5269185 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3871083 3618275 624 405 0 3868027 0
Current children cumulated CPU time: 1305.53 s
Current children cumulated vsize: 15484332 KiB
Current children cumulated memory: 14473100 KiB

[startup+1318.4 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23360
/proc/meminfo: memFree=866180/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=15500936 memory=14628224 CPUtime=1318.33 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1108484 0 0 0 120168 11665 0 0 20 0 1 0 58027350 15872958464 3657056 33554432000 4194304 5849740 140729478617952 140729478616952 5260281 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 3875234 3657056 624 405 0 3872178 0
Current children cumulated CPU time: 1318.33 s
Current children cumulated vsize: 15500936 KiB
Current children cumulated memory: 14628224 KiB

[startup+1331.2 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23360
/proc/meminfo: memFree=7185504/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=11009472 memory=8372188 CPUtime=1331.13 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1112006 0 0 0 121396 11717 0 0 20 0 1 0 58027350 11273699328 2093047 33554432000 4194304 5849740 140729478617952 140729478616920 5250894 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 2752368 2093047 624 405 0 2749312 0
Current children cumulated CPU time: 1331.13 s
Current children cumulated vsize: 11009472 KiB
Current children cumulated memory: 8372188 KiB

[startup+1337.6 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23360
/proc/meminfo: memFree=7146764/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=11009472 memory=10201060 CPUtime=1337.53 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1114143 0 0 0 122002 11751 0 0 20 0 1 0 58027350 11273699328 2550265 33554432000 4194304 5849740 140729478617952 140729478617272 4543911 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 2752368 2550265 624 405 0 2749312 0
Current children cumulated CPU time: 1337.53 s
Current children cumulated vsize: 11009472 KiB
Current children cumulated memory: 10201060 KiB

[startup+1339.2 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23360
/proc/meminfo: memFree=5252740/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=11009472 memory=10201060 CPUtime=1339.13 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1114151 0 0 0 122162 11751 0 0 20 0 1 0 58027350 11273699328 2550265 33554432000 4194304 5849740 140729478617952 140729478617272 4496916 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 2752368 2550265 624 405 0 2749312 0
Current children cumulated CPU time: 1339.13 s
Current children cumulated vsize: 11009472 KiB
Current children cumulated memory: 10201060 KiB

[startup+1340.8 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23360
/proc/meminfo: memFree=5252864/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=11009472 memory=10201060 CPUtime=1340.73 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1114161 0 0 0 122322 11751 0 0 20 0 1 0 58027350 11273699328 2550265 33554432000 4194304 5849740 140729478617952 140729478617272 4548525 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 2752368 2550265 624 405 0 2749312 0
Current children cumulated CPU time: 1340.73 s
Current children cumulated vsize: 11009472 KiB
Current children cumulated memory: 10201060 KiB

[startup+1341.2 s]
/proc/loadavg: 2.00 2.01 2.01 3/187 23360
/proc/meminfo: memFree=5252864/32770624 swapFree=67108860/67108860
[pid=23051] ppid=23047 vsize=11009472 memory=10205156 CPUtime=1341.13 cores=1,3,5,7
/proc/23051/stat : 23051 (xcsp_picat) R 23047 23051 17273 0 -1 4202496 1114164 0 0 0 122361 11752 0 0 20 0 1 0 58027350 11273699328 2551289 33554432000 4194304 5849740 140729478617952 140729478617272 4454714 0 0 4096 8389634 0 0 0 17 1 0 0 0 0 0 7949816 9399828 34758656 140729478624438 140729478624516 140729478624516 140729478627274 0
/proc/23051/statm: 2752368 2551289 624 405 0 2749312 0
Current children cumulated CPU time: 1341.13 s
Current children cumulated vsize: 11009472 KiB
Current children cumulated memory: 10205156 KiB

[startup+1341.4 s]*
/proc/loadavg: 2.00 2.01 2.01 3/187 23360
/proc/meminfo: memFree=10729268/32770624 swapFree=67108860/67108860
Current children cumulated CPU time: 0 s
Current children cumulated vsize: 0 KiB
Current children cumulated memory: 0 KiB

Child status: 1
Real time (s): 1341.42
CPU time (s): 1341.36
CPU user time (s): 1223.69
CPU system time (s): 117.663
CPU usage (%): 99.9955
Max. virtual memory (cumulated for all children) (KiB): 15500936
Max. memory (cumulated for all children) (KiB): 14628224

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 1223.69
system time used= 117.663
maximum resident set size= 14628224
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 1114999
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 3
involuntary context switches= 935


# summary of solver processes directly reported to runsolver:
#   pid: 23051
#   total CPU time (s): 1341.36
#   total CPU user time (s): 1223.69
#   total CPU system time (s): 117.663

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 5.32875 second user time and 10.2149 second system time

The end

Launcher Data 

Begin job on node151 at 2018-07-24 05:57:22
IDJOB=4293972
IDBENCH=141303
IDSOLVER=2754
FILE ID=node151/4293972-1532404641
RUNJOBID= node151-1532401118-22575
PBS_JOBID= 21021572
Free space on /tmp= 55020 MiB

SOLVER NAME= PicatSAT 2018-06-15
BENCH NAME= XCSP18/CrosswordDesign/CrosswordDesign-08-4-rom_c18.xml
COMMAND LINE= xcsp_picat BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4293972-1532404641/watcher-4293972-1532404641 -o /tmp/evaluation-result-4293972-1532404641/solver-4293972-1532404641 -C 2520 -W 2520 -M 15500 --bin-var /tmp/runsolver-exch-node151-1532401118-22575 --watchdog 2580  xcsp_picat HOME/instance-4293972-1532404641.xml

TIME LIMIT= 2520 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= d2036c1580b702d26a2ba71e1295e02d
RANDOM SEED=1504248919

node151.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.87
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.87
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.87
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.87
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        26143440 kB
MemAvailable:   30636040 kB
Buffers:          156032 kB
Cached:          4577260 kB
SwapCached:            0 kB
Active:          2269428 kB
Inactive:        3836188 kB
Active(anon):    1322312 kB
Inactive(anon):    69372 kB
Active(file):     947116 kB
Inactive(file):  3766816 kB
Unevictable:        7784 kB
Mlocked:            7784 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:             12876 kB
Writeback:             0 kB
AnonPages:       1252940 kB
Mapped:            40448 kB
Shmem:             17032 kB
Slab:             216460 kB
SReclaimable:     189004 kB
SUnreclaim:        27456 kB
KernelStack:        3568 kB
PageTables:         7848 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:    1856640 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:   1183744 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      118144 kB
DirectMap2M:    33427456 kB

Free space on /tmp at the end= 54996 MiB
End job on node151 at 2018-07-24 06:19:43