Trace number 4266934

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerobjective functionCPU timeWall clock time
cosoco-sat 1.12SAT (TO)2329 251.96201 252.01199

General information on the benchmark

NameTravellingSalesman/TravellingSalesman-m1-n150/
TravellingSalesman-150-50-14.xml
MD5SUMb2c04361dada2322dcbf50e86c385888
Bench CategoryCOP (optimization problem)
Best result obtained on this benchmarkSAT TO
Best value of the objective obtained on this benchmark552
Best CPU time to get the best result obtained on this benchmark251.94299
Satisfiable
(Un)Satisfiability was proved
Number of variables300
Number of constraints151
Number of domains2
Minimum domain size64
Maximum domain size150
Distribution of domain sizes[{"size":64,"count":150},{"size":150,"count":150}]
Minimum variable degree2
Maximum variable degree3
Distribution of variable degrees[{"degree":2,"count":150},{"degree":3,"count":150}]
Minimum constraint arity3
Maximum constraint arity150
Distribution of constraint arities[{"arity":3,"count":150},{"arity":150,"count":1}]
Number of extensional constraints150
Number of intensional constraints0
Distribution of constraint types[{"type":"extension","count":150},{"type":"allDifferent","count":1}]
Optimization problemYES
Type of objectivemin SUM

Quality of the solution as a function of time


Download the above graph as a PDF file / as an EPS file

Solver Data 

0.25/0.32	c SAT : Variables: 32100 -- Clauses: 1978950
0.25/0.32	c best=?? lower=-9600 upper=-150 
2.24/2.33	c
2.24/2.33	c This is cosoco 1.12 --  
2.24/2.33	c
2.24/2.33	c command line: cosoco -lc -model -restarts -sat HOME/instance-4266934-1502647836.xml 
2.24/2.33	c enable optimization
2.24/2.33	c ========================================[ Problem Statistics ]===========================================
2.24/2.33	c |                                                                                                       
2.24/2.33	c |  Parse time        :         0.23 s 
2.24/2.33	c |
2.24/2.33	c |               Variables: 300
2.24/2.33	c |            Domain Sizes: 64..150
2.24/2.33	c |
2.24/2.33	c |             Constraints: 152
2.24/2.33	c |                   Arity: 3..150  -- Ternary: 150
2.24/2.33	c | 
2.24/2.33	c |                   Types: 
2.24/2.33	c |                          All Different: 1
2.24/2.33	c |                          Extension: 150  (nb tuples: 22350..22350) -- (shared: 149)
2.24/2.33	c |                          Sum: 1
2.24/2.33	c |                          
2.24/2.33	c =========================================================================================================
2.24/2.33	o -2677
2.24/2.33	c    1 calls to solver: best=-2677 lower=-2676 upper=-150 
49.24/49.37	o -2607
53.54/53.69	o -2541
53.54/53.69	c    3 calls to solver: best=-2541 lower=-2540 upper=-150 
121.23/121.35	o -2540
172.84/172.99	o -2488
172.84/172.99	c    5 calls to solver: best=-2488 lower=-2487 upper=-150 
176.44/176.53	o -2432
209.64/209.78	o -2398
209.64/209.78	c    7 calls to solver: best=-2398 lower=-2397 upper=-150 
231.24/231.31	o -2329
251.95/252.01	
251.95/252.01	c *** INTERRUPTED ***
251.95/252.01	c restarts              : 307         
251.95/252.01	c conflicts             : 2930         (12 /sec)
251.95/252.01	c decisions             : 56628        (225 /sec)
251.95/252.01	c propagations          : 5254604     (20863 /sec)
251.95/252.01	c root propagations     : 21968       
251.95/252.01	c filter calls          : 2214983        (8794 /sec)
251.95/252.01	c useless filter calls  : 2116628        (95 %)
251.95/252.01	c
251.95/252.01	c clauses               : 1981788
251.95/252.01	c SAT propagations      : 1444
251.95/252.01	c SAT conflicts         : 0 (0 learnts)
251.95/252.01	c Memory used           : 155.77 MB
251.95/252.01	c CPU time              : 251.861 s
251.95/252.01	
251.95/252.01	c nb Solutions = 8
251.95/252.01	s SATISFIABLE
251.95/252.01	c Best value found: -2329
251.95/252.01	
251.95/252.01	
251.95/252.01	v <instantiation type='solution' cost='-2329'>
251.95/252.01	v <list>c[0] c[100] c[101] c[102] c[103] c[104] c[105] c[106] c[107] c[108] c[109] c[10] c[110] c[111] c[112] c[113] c[114] c[115] c[116] c[117] c[118] c[119] c[11] c[120] c[121] c[122] c[123] c[124] c[125] c[126] c[127] c[128] c[129] c[12] c[130] c[131] c[132] c[133] c[134] c[135] c[136] c[137] c[138] c[139] c[13] c[140] c[141] c[142] c[143] c[144] c[145] c[146] c[147] c[148] c[149] c[14] c[15] c[16] c[17] c[18] c[19] c[1] c[20] c[21] c[22] c[23] c[24] c[25] c[26] c[27] c[28] c[29] c[2] c[30] c[31] c[32] c[33] c[34] c[35] c[36] c[37] c[38] c[39] c[3] c[40] c[41] c[42] c[43] c[44] c[45] c[46] c[47] c[48] c[49] c[4] c[50] c[51] c[52] c[53] c[54] c[55] c[56] c[57] c[58] c[59] c[5] c[60] c[61] c[62] c[63] c[64] c[65] c[66] c[67] c[68] c[69] c[6] c[70] c[71] c[72] c[73] c[74] c[75] c[76] c[77] c[78] c[79] c[7] c[80] c[81] c[82] c[83] c[84] c[85] c[86] c[87] c[88] c[89] c[8] c[90] c[91] c[92] c[93] c[94] c[95] c[96] c[97] c[98] c[99] c[9] d[0] d[100] d[101] d[102] d[103] d[104] d[105] d[106] d[107] d[108] d[109] d[10] d[110] d[111] d[112] d[113] d[114] d[115] d[116] d[117] d[118] d[119] d[11] d[120] d[121] d[122] d[123] d[124] d[125] d[126] d[127] d[128] d[129] d[12] d[130] d[131] d[132] d[133] d[134] d[135] d[136] d[137] d[138] d[139] d[13] d[140] d[141] d[142] d[143] d[144] d[145] d[146] d[147] d[148] d[149] d[14] d[15] d[16] d[17] d[18] d[19] d[1] d[20] d[21] d[22] d[23] d[24] d[25] d[26] d[27] d[28] d[29] d[2] d[30] d[31] d[32] d[33] d[34] d[35] d[36] d[37] d[38] d[39] d[3] d[40] d[41] d[42] d[43] d[44] d[45] d[46] d[47] d[48] d[49] d[4] d[50] d[51] d[52] d[53] d[54] d[55] d[56] d[57] d[58] d[59] d[5] d[60] d[61] d[62] d[63] d[64] d[65] d[66] d[67] d[68] d[69] d[6] d[70] d[71] d[72] d[73] d[74] d[75] d[76] d[77] d[78] d[79] d[7] d[80] d[81] d[82] d[83] d[84] d[85] d[86] d[87] d[88] d[89] d[8] d[90] d[91] d[92] d[93] d[94] d[95] d[96] d[97] d[98] d[99] d[9] </list>
251.95/252.01	v <values>38 18 22 19 15 23 17 34 78 72 52 7 70 57 48 54 50 82 109 124 136 86 16 101 120 97 140 76 81 114 103 105 131 24 113 88 122 102 126 129 21 10 138 87 30 46 123 137 94 112 143 117 5 13 28 31 35 41 42 49 55 36 62 66 67 68 71 73 58 56 59 60 27 51 43 39 11 110 75 132 145 135 107 20 89 133 134 111 98 53 26 9 25 93 8 142 125 130 80 128 100 64 119 104 69 3 61 91 0 99 118 33 45 29 40 32 12 96 106 127 84 74 47 65 141 144 149 6 95 14 77 139 92 116 83 4 37 121 1 147 148 146 108 115 85 44 90 79 63 2 1 2 1 4 2 7 6 16 25 23 22 5 4 3 3 18 11 12 19 15 24 33 2 28 30 23 24 6 15 18 11 31 6 2 26 23 41 9 22 40 12 42 18 14 1 28 14 22 33 14 12 46 36 7 4 2 2 3 3 4 2 3 2 4 3 1 4 4 1 2 2 3 3 4 1 8 43 16 20 6 6 29 9 4 19 6 7 6 18 16 47 6 22 20 1 7 3 34 19 11 13 21 32 23 12 16 21 34 36 25 53 12 4 4 29 26 2 16 7 19 4 9 13 28 16 31 22 4 31 23 24 21 14 15 35 11 42 34 3 22 2 20 2 21 23 16 33 32 18 3 </values>
251.95/252.01	v </instantiation>
251.95/252.01

Verifier Data 

OK	2329

Watcher Data 

runsolver version 3.4.0 (svn: 3012) Copyright (C) 2010-2013 Olivier ROUSSEL

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

NUMA information:
  number of nodes: 2
  memory of node 0: 16374 MiB (15365 MiB free)
  memory of node 1: 16384 MiB (15630 MiB free)
  node distances:
     10  21
     21  10

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4266934-1502647836/watcher-4266934-1502647836 -o /tmp/evaluation-result-4266934-1502647836/solver-4266934-1502647836 -C 252 -W 252 -M 15500 --bin-var /tmp/runsolver-exch-node134-1502642157-20629 --watchdog 312 cosoco -lc -model -restarts -sat HOME/instance-4266934-1502647836.xml 

running on 4 cores: 1,3,5,7

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 252 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 282 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 252 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Linux 3.10.0-514.16.1.el7.x86_64
Starting watchdog thread
solver pid=23339, runsolver pid=23336
Current StackSize limit: 8192 KiB


[startup+0.100154 s]*
/proc/loadavg: 2.06 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31739228/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=28336 memory=4356 CPUtime=0.05 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 1330 0 0 0 5 0 0 0 20 0 1 0 112290074 29016064 1089 18446744073709551615 4194304 5972588 140730986586096 140730986582744 139767913159653 0 0 1 0 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 7084 1089 720 435 0 447 0
Current children cumulated CPU time: 0.05 s
Current children cumulated vsize: 28336 KiB
Current children cumulated memory: 4356 KiB

[startup+0.209399 s]*
/proc/loadavg: 2.06 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31686092/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=86204 memory=61012 CPUtime=0.16 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 13656 0 0 0 14 2 0 0 20 0 1 0 112290074 88272896 15253 18446744073709551615 4194304 5972588 140730986586096 140730986582888 139767913159653 0 0 1 0 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 21551 15253 763 435 0 14914 0
Current children cumulated CPU time: 0.16 s
Current children cumulated vsize: 86204 KiB
Current children cumulated memory: 61012 KiB

[startup+0.309638 s]*
/proc/loadavg: 2.06 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31634528/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=137724 memory=112112 CPUtime=0.25 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 22893 0 0 0 22 3 0 0 20 0 1 0 112290074 141029376 28028 18446744073709551615 4194304 5972588 140730986586096 140730986582888 139767913159653 0 0 1 0 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 34431 28028 763 435 0 27794 0
Current children cumulated CPU time: 0.25 s
Current children cumulated vsize: 137724 KiB
Current children cumulated memory: 112112 KiB

[startup+0.700391 s]
/proc/loadavg: 2.06 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31626900/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=157748 memory=116208 CPUtime=0.64 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 23404 0 0 0 61 3 0 0 20 0 1 0 112290074 161533952 29052 18446744073709551615 4194304 5972588 140730986586096 140730986582888 4935546 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39437 29052 763 435 0 32800 0
Current children cumulated CPU time: 0.64 s
Current children cumulated vsize: 157748 KiB
Current children cumulated memory: 116208 KiB

[startup+1.50038 s]
/proc/loadavg: 2.06 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31626528/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=158036 memory=116472 CPUtime=1.44 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 23489 0 0 0 141 3 0 0 20 0 1 0 112290074 161828864 29118 18446744073709551615 4194304 5972588 140730986586096 140730986582888 4935502 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39509 29118 781 435 0 32872 0
Current children cumulated CPU time: 1.44 s
Current children cumulated vsize: 158036 KiB
Current children cumulated memory: 116472 KiB

[startup+3.10961 s]
/proc/loadavg: 2.06 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31625268/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=158908 memory=117604 CPUtime=3.04 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 32327 0 0 0 300 4 0 0 20 0 1 0 112290074 162721792 29401 18446744073709551615 4194304 5972588 140730986586096 140730986573288 4935502 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39727 29401 783 435 0 33090 0
Current children cumulated CPU time: 3.04 s
Current children cumulated vsize: 158908 KiB
Current children cumulated memory: 117604 KiB

[startup+6.30034 s]
/proc/loadavg: 2.06 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31625144/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159108 memory=117728 CPUtime=6.24 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 42295 0 0 0 619 5 0 0 20 0 1 0 112290074 162926592 29432 18446744073709551615 4194304 5972588 140730986586096 140730986582664 4935490 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39777 29432 783 435 0 33140 0
Current children cumulated CPU time: 6.24 s
Current children cumulated vsize: 159108 KiB
Current children cumulated memory: 117728 KiB

[startup+12.7004 s]
/proc/loadavg: 2.05 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31624804/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159240 memory=117884 CPUtime=12.64 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 52859 0 0 0 1259 5 0 0 20 0 1 0 112290074 163061760 29471 18446744073709551615 4194304 5972588 140730986586096 140730986582584 4935463 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39810 29471 783 435 0 33173 0
Current children cumulated CPU time: 12.64 s
Current children cumulated vsize: 159240 KiB
Current children cumulated memory: 117884 KiB

[startup+25.5005 s]
/proc/loadavg: 2.04 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31624828/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159240 memory=117952 CPUtime=25.44 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 72686 0 0 0 2538 6 0 0 20 0 1 0 112290074 163061760 29488 18446744073709551615 4194304 5972588 140730986586096 140730986582584 4935490 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39810 29488 783 435 0 33173 0
Current children cumulated CPU time: 25.44 s
Current children cumulated vsize: 159240 KiB
Current children cumulated memory: 117952 KiB

[startup+51.1096 s]
/proc/loadavg: 2.03 2.03 2.06 3/185 23340
/proc/meminfo: memFree=31623976/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159372 memory=118108 CPUtime=51.05 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 85646 0 0 0 5097 8 0 0 20 0 1 0 112290074 163196928 29527 18446744073709551615 4194304 5972588 140730986586096 140730986583544 4935478 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39843 29527 783 435 0 33206 0
Current children cumulated CPU time: 51.05 s
Current children cumulated vsize: 159372 KiB
Current children cumulated memory: 118108 KiB

[startup+102.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/184 23343
/proc/meminfo: memFree=31623272/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159504 memory=118148 CPUtime=102.23 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 100700 0 0 0 10214 9 0 0 20 0 1 0 112290074 163332096 29537 18446744073709551615 4194304 5972588 140730986586096 140730986582952 4935446 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39876 29537 783 435 0 33239 0
Current children cumulated CPU time: 102.23 s
Current children cumulated vsize: 159504 KiB
Current children cumulated memory: 118148 KiB

[startup+162.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/185 23346
/proc/meminfo: memFree=31623000/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159504 memory=118324 CPUtime=162.23 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 103092 0 0 0 16214 9 0 0 20 0 1 0 112290074 163332096 29581 18446744073709551615 4194304 5972588 140730986586096 140730986583544 4935490 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39876 29581 783 435 0 33239 0
Current children cumulated CPU time: 162.23 s
Current children cumulated vsize: 159504 KiB
Current children cumulated memory: 118324 KiB

[startup+222.3 s]
/proc/loadavg: 2.08 2.03 2.05 3/189 23363
/proc/meminfo: memFree=31619596/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159504 memory=118336 CPUtime=222.24 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 104069 0 0 0 22215 9 0 0 20 0 1 0 112290074 163332096 29584 18446744073709551615 4194304 5972588 140730986586096 140730986573288 4935502 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39876 29584 783 435 0 33239 0
Current children cumulated CPU time: 222.24 s
Current children cumulated vsize: 159504 KiB
Current children cumulated memory: 118336 KiB



Maximum wall clock time exceeded: sending SIGTERM then SIGKILL

[startup+252 s]*
/proc/loadavg: 2.36 2.11 2.07 3/189 23429
/proc/meminfo: memFree=31449712/32770624 swapFree=67108764/67108860
[pid=23339] ppid=23336 vsize=159504 memory=118376 CPUtime=251.95 cores=1,3,5,7
/proc/23339/stat : 23339 (cosoco) R 23336 23339 4674 0 -1 4202496 105016 0 0 0 25185 10 0 0 20 0 1 0 112290074 163332096 29594 18446744073709551615 4194304 5972588 140730986586096 140730986583544 4935454 0 0 1 8405026 0 0 0 17 3 0 0 5 0 0 8070896 8075904 14155776 140730986594761 140730986594861 140730986594861 140730986598350 0
/proc/23339/statm: 39876 29594 783 435 0 33239 0
Current children cumulated CPU time: 251.95 s
Current children cumulated vsize: 159504 KiB
Current children cumulated memory: 118376 KiB

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

[startup+252.012 s]
# the end of solver process 23339 was just reported to runsolver
# cumulated CPU time of all completed processes:  user=251.861 s, system=0.101001 s

Solver just ended.
??? end of watcher thread
Dumping a history of the last processes samples

Child status: 1
Real time (s): 252.012
CPU time (s): 251.962
CPU user time (s): 251.861
CPU system time (s): 0.101001
CPU usage (%): 99.9803
Max. virtual memory (cumulated for all children) (KiB): 159504
Max. memory (cumulated for all children) (KiB): 118376

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 251.861
system time used= 0.101001
maximum resident set size= 118456
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 105065
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 10
involuntary context switches= 141


# summary of solver processes directly reported to runsolver:
#   pid: 23339
#   total CPU time (s): 251.962
#   total CPU user time (s): 251.861
#   total CPU system time (s): 0.101001

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 0.989193 second user time and 1.91889 second system time

The end

Launcher Data 

Begin job on node134 at 2017-08-13 19:56:42
IDJOB=4266934
IDBENCH=135805
IDSOLVER=2656
FILE ID=node134/4266934-1502647836
RUNJOBID= node134-1502642157-20629
PBS_JOBID= 
Free space on /tmp= 62028 MiB

SOLVER NAME= cosoco-sat 1.12
BENCH NAME= XCSP17/TravellingSalesman/TravellingSalesman-m1-n150/TravellingSalesman-150-50-14.xml
COMMAND LINE= cosoco -lc -model -restarts -sat BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4266934-1502647836/watcher-4266934-1502647836 -o /tmp/evaluation-result-4266934-1502647836/solver-4266934-1502647836 -C 252 -W 252 -M 15500 --bin-var /tmp/runsolver-exch-node134-1502642157-20629 --watchdog 312  cosoco -lc -model -restarts -sat HOME/instance-4266934-1502647836.xml

TIME LIMIT= 252 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 4

MD5SUM BENCH= b2c04361dada2322dcbf50e86c385888
RANDOM SEED=1061441103

node134.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor:
performance

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.43
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 1
siblings	: 4
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        31740232 kB
MemAvailable:   31862568 kB
Buffers:          217312 kB
Cached:           208708 kB
SwapCached:            4 kB
Active:           474168 kB
Inactive:         124592 kB
Active(anon):     117352 kB
Inactive(anon):    97332 kB
Active(file):     356816 kB
Inactive(file):    27260 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:      67108860 kB
SwapFree:       67108764 kB
Dirty:              2456 kB
Writeback:             0 kB
AnonPages:        172904 kB
Mapped:            58048 kB
Shmem:             41564 kB
Slab:             135104 kB
SReclaimable:     108192 kB
SUnreclaim:        26912 kB
KernelStack:        3264 kB
PageTables:         5248 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     474432 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      354332 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     14336 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      101760 kB
DirectMap2M:    33443840 kB

Free space on /tmp at the end= 62024 MiB
End job on node134 at 2017-08-13 20:00:54