Trace number 4242508

Some explanations

A solver is run under the control of another program named runsolver. runsolver is in charge of imposing the CPU time limit and the memory limit to the solver. It also monitors some information about the process. The trace of the execution of a solver is divided into four (or five) parts:
  1. SOLVER DATA
    This is the output of the solver (stdout and stderr).
    Note that some very long lines in this section may be truncated by your web browser ! In such a case, you may want to use the "Download as text" link to get the trace as a text file.

    When the --timestamp option is passed to the runsolver program, each line output by the solver is prepended with a timestamp which indicates at what time the line was output by the solver. Times are relative to the start of the program, given in seconds. The first timestamp (if present) is estimated CPU time. The last timestamp is wall clock time.

    As some 'v lines' may be very long (sometimes several megabytes), the 'v line' output by your solver may be split on several lines to help limit the size of the trace recorded in the database. In any case, the exact output of your solver is preserved in a trace file.
  2. VERIFIER DATA
    The output of the solver is piped to a verifier program which will search a value line "v " and, if found, will check that the given interpretation satisfies all constraints.
  3. CONVERSION SCRIPT DATA (Optionnal)
    When a conversion script is used, this section shows the messages that were output by the conversion script.
  4. WATCHER DATA
    This is the informations gathered by the runsolver program. It first prints the different limits. There's a first limit on CPU time set to X seconds (see the parameters in the trace). After this time has ellapsed, runsolver sends a SIGTERM and 2 seconds later a SIGKILL to the solver. For safety, there's also another limit set to X+30 seconds which will send a SIGXPU to the solver. The last limit is on the virtual memory used by the process (see the parameters in the trace).
    Every ten seconds, the runsolver process fetches the content of /proc/loadavg, /proc/pid/stat and /proc/pid/statm (see man proc) and prints it as raw data. This is only recorded in case we need to investigate the behaviour of a solver. The memory used by the solver (vsize) is also given every ten seconds.
    When the solver exits, runsolver prints some informations such as status and time. CPU usage is the ratio CPU Time/Real Time.
  5. LAUNCHER DATA
    These informations are related to the script which will launch the solver. The most important informations are the command line given to the solver, the md5sum of the different files and the dump of the /proc/cpuinfo and /proc/meminfo which provides some useful information on the computer.

Solver answer on this benchmark

Solver NameAnswerCPU timeWall clock time
R1: cosoco 1.1? (TO) 2400.04 2400.5

General information on the benchmark

NameKnightTour/KnightTour-int-s1/
KnightTour-13-int.xml
MD5SUM5aa475b3f1257aceb66c8baaa9eb8eb1
Bench CategoryCSP (decision problem)
Best result obtained on this benchmarkUNSAT
Best value of the objective obtained on this benchmark
Best CPU time to get the best result obtained on this benchmark2.7322199
Satisfiable
(Un)Satisfiability was proved
Number of variables169
Number of constraints171
Number of domains1
Minimum domain size169
Maximum domain size169
Distribution of domain sizes[{"size":169,"count":169}]
Minimum variable degree2
Maximum variable degree3
Distribution of variable degrees[{"degree":2,"count":167},{"degree":3,"count":2}]
Minimum constraint arity2
Maximum constraint arity169
Distribution of constraint arities[{"arity":2,"count":170},{"arity":169,"count":1}]
Number of extensional constraints0
Number of intensional constraints169
Distribution of constraint types[{"type":"intension","count":169},{"type":"allDifferent","count":1},{"type":"instantiation","count":1}]
Optimization problemNO
Type of objective

Solver Data 

0.19/0.21	c root propagations     : 1827        
0.19/0.21	c restarts enabled, print stats at every restarts
0.19/0.21	c
0.19/0.21	c
0.19/0.21	c
0.19/0.21	c This is cosoco 1.0 --  
0.19/0.21	c
0.19/0.21	c ========================================[ Problem Statistics ]===========================================
0.19/0.21	c |                                                                                                       
0.19/0.21	c |  Parse time        :         0.21 s 
0.19/0.21	c |
0.19/0.21	c |               Variables: 169
0.19/0.21	c |            Domain Sizes: 169..169
0.19/0.21	c |
0.19/0.21	c |             Constraints: 172
0.19/0.21	c |                   Arity: 1..169  -- Unary: 2  -- Binary: 169
0.19/0.21	c | 
0.19/0.21	c |                   Types: 
0.19/0.21	c |                          All Different: 1
0.19/0.21	c |                          Intension: 169
0.19/0.21	c |                          Unary: 2
0.19/0.21	c |                          
0.19/0.21	c =========================================================================================================
0.19/0.21	c conflicts      decisions      filterCalls    propagations   Root Prop.     Nogoods        min/max/avg depth
0.69/0.73	c 101            393            8564           114044         169            0              52   151   140 
1.10/1.18	c 211            817            18153          200176         169            0              48   152   137 
1.59/1.66	c 332            1289           27995          309571         169            0              43   154   143 
2.20/2.26	c 465            1660           36325          414111         169            0              63   116   81  
2.89/2.90	c 611            2122           47021          533655         169            0              63   141   114 
3.49/3.50	c 771            2485           55392          627158         169            0              43   108   98  
3.99/4.01	c 947            3096           66375          738166         169            0              44   152   143 
4.39/4.48	c 1140           3724           78901          855157         169            0              44   147   137 
4.99/5.06	c 1352           4277           90151          973586         169            0              42   145   119 
5.89/5.93	c 1585           5000           106253         1195558        169            0              46   148   139 
6.59/6.61	c 1841           5714           119988         1360031        169            0              45   147   130 
7.49/7.52	c 2122           6570           139287         1553722        169            0              43   152   137 
8.39/8.44	c 2431           7123           151624         1699314        169            0              43   68   64   
9.49/9.51	c 2770           8216           175404         1959790        169            0              55   153   145 
10.19/10.22	c 3142           9262           197940         2142928        169            0              43   149   141 
11.09/11.18	c 3551           10446          222285         2409211        169            0              43   154   145 
11.89/11.99	c 4000           11748          245749         2648402        169            0              65   151   144 
13.19/13.28	c 4493           13196          277255         3007731        169            0              44   151   140 
15.19/15.20	c 5035           15211          320531         3625985        169            0              63   151   140 
16.38/16.41	c 5631           16786          353242         3963289        169            0              63   151   143 
18.38/18.47	c 6286           18458          388373         4358003        169            0              43   147   131 
20.28/20.39	c 7006           20286          424098         4732510        169            0              46   148   130 
21.78/21.84	c 7798           22653          470203         5083302        169            0              43   149   142 
23.48/23.54	c 8669           24886          510463         5514633        169            0              65   150   130 
25.48/25.53	c 9627           27514          556040         6003165        169            0              63   147   136 
28.59/28.61	c 10680          30358          628712         6907669        169            0              65   144   137 
29.48/29.53	c 11838          33530          674536         7247631        169            0              65   153   146 
32.78/32.80	c 13111          37095          745591         8080539        169            0              65   148   133 
35.88/35.91	c 14511          40987          831629         8764477        169            0              46   149   143 
38.88/38.95	c 16051          45380          923178         9591487        169            0              63   147   140 
43.58/43.65	c 17745          50694          1030126        10649901       169            0              43   154   138 
47.68/47.76	c 19608          56304          1125358        11697461       169            0              63   150   135 
51.78/51.88	c 21657          61418          1207995        12585024       169            0              35   138   122 
54.48/54.53	c 23910          67527          1306099        13252909       169            0              63   154   146 
59.58/59.61	c 26388          75274          1465212        14787372       169            0              43   150   142 
64.88/64.91	c 29113          84142          1633902        15909229       169            0              43   149   141 
68.77/68.87	c 32110          92895          1780717        16969534       169            0              90   156   145 
75.77/75.85	c 35406          101162         1936341        18248086       169            0              44   150   139 
83.27/83.33	c 39031          111184         2139819        20521109       169            0              56   150   141 
95.07/95.12	c 43018          123836         2412951        23701462       169            0              43   150   140 
103.97/104.04	c 47403          136709         2662364        26323105       169            0              61   152   143 
112.86/112.92	c 52226          150211         2944235        28690865       169            0              63   150   144 
121.07/121.13	c 57531          166146         3232015        31165877       169            0              63   154   144 
136.47/136.53	c 63366          181237         3546847        34217224       169            0              43   147   133 
150.86/150.95	c 69784          199368         3893478        37654689       169            0              62   151   136 
164.55/164.66	c 76843          221075         4312493        40752440       169            0              44   150   141 
175.96/176.08	c 84607          244575         4696643        44521727       169            0              65   152   141 
190.65/190.78	c 93147          270585         5166608        48259244       169            0              43   151   137 
214.14/214.20	c 102541         297160         5696769        52938315       169            0              43   147   135 
225.45/225.58	c 112874         327562         6168723        55898421       169            0              65   152   143 
243.34/243.42	c 124240         361953         6779765        61413252       169            0              63   156   146 
265.04/265.17	c 136742         394269         7328431        66001883       169            0              46   148   136 
288.83/288.90	c 150494         439179         8126803        70887970       169            0              44   154   142 
312.43/312.51	c 165621         485334         8953337        76096152       169            0              43   151   141 
340.82/340.95	c 182260         542448         9941484        81939156       169            0              44   154   142 
368.21/368.36	c 200562         593688         10806857       87665776       169            0              46   155   146 
402.41/402.55	c 220694         651415         11844346       97896114       169            0              61   154   143 
421.70/421.82	c 242839         710617         12868472       104534545      169            0              63   152   144 
460.80/460.91	c 267198         776928         14142136       115586467      169            0              43   152   142 
493.59/493.70	c 293992         854864         15457804       123814827      169            0              44   153   143 
524.78/524.96	c 323465         937544         16798495       131889503      169            0              43   150   141 
550.18/550.30	c 355885         1033895        18165982       141300591      169            0              65   154   145 
619.17/619.33	c 391547         1129469        20147772       160600393      169            0              43   154   143 
737.04/737.24	c 430775         1239400        22366549       184483129      169            0              63   150   135 
781.33/781.53	c 473925         1363257        24387087       200512592      169            0              63   154   144 
879.92/880.11	c 521390         1498831        27066810       226934082      169            0              44   153   144 
961.40/961.68	c 573601         1658214        29904984       251142884      169            0              63   152   142 
1007.89/1008.14	c 631033         1834867        32757713       267623035      169            0              63   155   146 
1148.37/1148.61	c 694208         2013755        35712998       293205028      169            0              44   148   137 
1259.75/1260.07	c 763700         2228509        39581190       320683916      169            0              43   156   145 
1382.63/1382.93	c 840141         2443799        43696175       361351832      169            0              80   154   146 
1518.00/1518.30	c 924226         2684744        48142457       398680871      169            0              63   155   145 
1693.57/1694.00	c 1016719        2945556        53281492       438912907      169            0              43   152   143 
1830.94/1831.35	c 1118461        3277948        58897706       468527595      169            0              44   155   143 
2081.29/2081.72	c 1230377        3591374        65213266       524819065      169            0              43   154   139 
2316.74/2317.25	c 1353484        3947163        72013700       582909213      169            0              43   153   143

Verifier Data 

One s line expected

Watcher Data 

runsolver Copyright (C) 2010-2013 Olivier ROUSSEL

This is runsolver version 3.3.7 (svn: 2553)

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

command line: BIN/runsolver --timestamp -w /tmp/evaluation-result-4242508-1500663783/watcher-4242508-1500663783 -o /tmp/evaluation-result-4242508-1500663783/solver-4242508-1500663783 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1500648178-9851 --watchdog 2560 HOME/cosoco -lc -model -restarts HOME/instance-4242508-1500663783.xml 

pid=14343
running on 8 cores: 1,3,5,7,9,11,13,15

Enforcing CPUTime limit (soft limit, will send SIGTERM then SIGKILL): 2400 seconds
Enforcing CPUTime limit (hard limit, will send SIGXCPU): 2430 seconds
Enforcing wall clock limit (soft limit, will send SIGTERM then SIGKILL): 2500 seconds
Enforcing VSIZE limit (soft limit, will send SIGTERM then SIGKILL): 15872000 KiB
Enforcing VSIZE limit (hard limit, stack expansion will fail with SIGSEGV, brk() and mmap() will return ENOMEM): 15923200 KiB
Current StackSize limit: 8192 KiB


[startup+0 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 14346
/proc/meminfo: memFree=32002224/32770624 swapFree=5992/197356
[pid=14346] ppid=14343 vsize=27384 CPUtime=0 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 1118 0 0 0 0 0 0 0 20 0 1 0 1648267 28041216 919 33554432000 4194304 5970956 140732588652352 140732588649000 140159358446565 0 0 4096 0 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 6846 919 748 434 0 210 0

[startup+0.100146 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 14346
/proc/meminfo: memFree=32002224/32770624 swapFree=5992/197356
[pid=14346] ppid=14343 vsize=27996 CPUtime=0.09 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 1283 0 0 0 9 0 0 0 20 0 1 0 1648267 28667904 1110 33554432000 4194304 5970956 140732588652352 140732588646056 140159358446565 0 0 4096 0 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 6999 1110 787 434 0 363 0
Current children cumulated CPU time (s) 0.09
Current children cumulated vsize (KiB) 27996

[startup+0.200254 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 14346
/proc/meminfo: memFree=32002224/32770624 swapFree=5992/197356
[pid=14346] ppid=14343 vsize=28884 CPUtime=0.19 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 1510 0 0 0 19 0 0 0 20 0 1 0 1648267 29577216 1300 33554432000 4194304 5970956 140732588652352 140732588646056 140159358446565 0 0 4096 0 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7221 1300 788 434 0 585 0
Current children cumulated CPU time (s) 0.19
Current children cumulated vsize (KiB) 28884

[startup+0.300248 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 14346
/proc/meminfo: memFree=32002224/32770624 swapFree=5992/197356
[pid=14346] ppid=14343 vsize=29400 CPUtime=0.29 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 1684 0 0 0 29 0 0 0 20 0 1 0 1648267 30105600 1494 33554432000 4194304 5970956 140732588652352 140732588649416 140159358446565 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7350 1494 831 434 0 714 0
Current children cumulated CPU time (s) 0.29
Current children cumulated vsize (KiB) 29400

[startup+0.700207 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 14346
/proc/meminfo: memFree=32002224/32770624 swapFree=5992/197356
[pid=14346] ppid=14343 vsize=29928 CPUtime=0.69 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 1801 0 0 0 69 0 0 0 20 0 1 0 1648267 30646272 1626 33554432000 4194304 5970956 140732588652352 140732588649160 4974816 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7482 1626 831 434 0 846 0
Current children cumulated CPU time (s) 0.69
Current children cumulated vsize (KiB) 29928

[startup+1.50023 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14347
/proc/meminfo: memFree=31998304/32770624 swapFree=5992/197144
[pid=14346] ppid=14343 vsize=30060 CPUtime=1.49 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 1838 0 0 0 149 0 0 0 20 0 1 0 1648267 30781440 1626 33554432000 4194304 5970956 140732588652352 140732588649688 5069968 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7515 1626 831 434 0 879 0
Current children cumulated CPU time (s) 1.49
Current children cumulated vsize (KiB) 30060

[startup+3.10021 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14347
/proc/meminfo: memFree=31997688/32770624 swapFree=5992/197144
[pid=14346] ppid=14343 vsize=30324 CPUtime=3.09 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 2734 0 0 0 309 0 0 0 20 0 1 0 1648267 31051776 1757 33554432000 4194304 5970956 140732588652352 140732588649688 5071026 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7581 1757 832 434 0 945 0
Current children cumulated CPU time (s) 3.09
Current children cumulated vsize (KiB) 30324

[startup+6.30021 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14347
/proc/meminfo: memFree=31997284/32770624 swapFree=5992/197156
[pid=14346] ppid=14343 vsize=30456 CPUtime=6.29 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 3631 0 0 0 629 0 0 0 20 0 1 0 1648267 31186944 1773 33554432000 4194304 5970956 140732588652352 140732588649688 4970607 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7614 1773 832 434 0 978 0
Current children cumulated CPU time (s) 6.29
Current children cumulated vsize (KiB) 30456

[startup+12.7002 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14347
/proc/meminfo: memFree=31996664/32770624 swapFree=5992/197164
[pid=14346] ppid=14343 vsize=30588 CPUtime=12.69 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 4555 0 0 0 1269 0 0 0 20 0 1 0 1648267 31322112 1790 33554432000 4194304 5970956 140732588652352 140732588649688 4970936 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1790 832 434 0 1011 0
Current children cumulated CPU time (s) 12.69
Current children cumulated vsize (KiB) 30588

[startup+25.5003 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14347
/proc/meminfo: memFree=31996228/32770624 swapFree=5992/197180
[pid=14346] ppid=14343 vsize=30588 CPUtime=25.48 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 5918 0 0 0 2548 0 0 0 20 0 1 0 1648267 31322112 1834 33554432000 4194304 5970956 140732588652352 140732588649688 5070008 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1834 832 434 0 1011 0
Current children cumulated CPU time (s) 25.48
Current children cumulated vsize (KiB) 30588

[startup+51.1003 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14347
/proc/meminfo: memFree=31996364/32770624 swapFree=5992/197212
[pid=14346] ppid=14343 vsize=30588 CPUtime=51.08 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 6368 0 0 0 5108 0 0 0 20 0 1 0 1648267 31322112 1838 33554432000 4194304 5970956 140732588652352 140732588649688 4974128 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1838 832 434 0 1011 0
Current children cumulated CPU time (s) 51.08
Current children cumulated vsize (KiB) 30588

[startup+102.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14348
/proc/meminfo: memFree=31996712/32770624 swapFree=5992/197244
[pid=14346] ppid=14343 vsize=30588 CPUtime=102.27 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 6814 0 0 0 10227 0 0 0 20 0 1 0 1648267 31322112 1846 33554432000 4194304 5970956 140732588652352 140732588649688 5070930 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1846 832 434 0 1011 0
Current children cumulated CPU time (s) 102.27
Current children cumulated vsize (KiB) 30588

[startup+162.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14349
/proc/meminfo: memFree=31996400/32770624 swapFree=5992/197308
[pid=14346] ppid=14343 vsize=30588 CPUtime=162.26 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 7261 0 0 0 16226 0 0 0 20 0 1 0 1648267 31322112 1846 33554432000 4194304 5970956 140732588652352 140732588649688 4967778 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1846 832 434 0 1011 0
Current children cumulated CPU time (s) 162.26
Current children cumulated vsize (KiB) 30588

[startup+222.308 s]
/proc/loadavg: 2.07 2.03 2.05 3/243 14350
/proc/meminfo: memFree=31996624/32770624 swapFree=5992/197368
[pid=14346] ppid=14343 vsize=30588 CPUtime=222.25 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 7708 0 0 0 22225 0 0 0 20 0 1 0 1648267 31322112 1847 33554432000 4194304 5970956 140732588652352 140732588649688 5070008 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1847 832 434 0 1011 0
Current children cumulated CPU time (s) 222.25
Current children cumulated vsize (KiB) 30588

[startup+282.3 s]
/proc/loadavg: 2.06 2.04 2.05 3/243 14351
/proc/meminfo: memFree=31996772/32770624 swapFree=5992/197408
[pid=14346] ppid=14343 vsize=30588 CPUtime=282.23 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 8153 0 0 0 28223 0 0 0 20 0 1 0 1648267 31322112 1847 33554432000 4194304 5970956 140732588652352 140732588649688 5070008 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1847 832 434 0 1011 0
Current children cumulated CPU time (s) 282.23
Current children cumulated vsize (KiB) 30588

[startup+342.3 s]
/proc/loadavg: 2.02 2.03 2.05 3/243 14352
/proc/meminfo: memFree=31996856/32770624 swapFree=5992/197464
[pid=14346] ppid=14343 vsize=30588 CPUtime=342.22 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 8600 0 0 0 34222 0 0 0 20 0 1 0 1648267 31322112 1847 33554432000 4194304 5970956 140732588652352 140732588649688 4961411 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1847 832 434 0 1011 0
Current children cumulated CPU time (s) 342.22
Current children cumulated vsize (KiB) 30588

[startup+402.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/243 14353
/proc/meminfo: memFree=31996752/32770624 swapFree=5992/197512
[pid=14346] ppid=14343 vsize=30588 CPUtime=402.21 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 9047 0 0 0 40221 0 0 0 20 0 1 0 1648267 31322112 1847 33554432000 4194304 5970956 140732588652352 140732588649688 5071028 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1847 832 434 0 1011 0
Current children cumulated CPU time (s) 402.21
Current children cumulated vsize (KiB) 30588

[startup+462.3 s]
/proc/loadavg: 2.05 2.04 2.05 3/243 14361
/proc/meminfo: memFree=31995784/32770624 swapFree=5992/197548
[pid=14346] ppid=14343 vsize=30588 CPUtime=462.2 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 9494 0 0 0 46220 0 0 0 20 0 1 0 1648267 31322112 1847 33554432000 4194304 5970956 140732588652352 140732588649688 4969696 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1847 832 434 0 1011 0
Current children cumulated CPU time (s) 462.2
Current children cumulated vsize (KiB) 30588

[startup+522.3 s]
/proc/loadavg: 2.02 2.03 2.05 3/243 14362
/proc/meminfo: memFree=31996652/32770624 swapFree=5992/197584
[pid=14346] ppid=14343 vsize=30588 CPUtime=522.18 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 9935 0 0 0 52218 0 0 0 20 0 1 0 1648267 31322112 1847 33554432000 4194304 5970956 140732588652352 140732588649688 4977769 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1847 832 434 0 1011 0
Current children cumulated CPU time (s) 522.18
Current children cumulated vsize (KiB) 30588

[startup+582.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/243 14363
/proc/meminfo: memFree=31997016/32770624 swapFree=5992/197632
[pid=14346] ppid=14343 vsize=30588 CPUtime=582.17 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 10383 0 0 0 58217 0 0 0 20 0 1 0 1648267 31322112 1847 33554432000 4194304 5970956 140732588652352 140732588649688 4908971 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1847 832 434 0 1011 0
Current children cumulated CPU time (s) 582.17
Current children cumulated vsize (KiB) 30588


################
# More data... #
################


[startup+1542.31 s]
/proc/loadavg: 2.03 2.04 2.05 3/242 14454
/proc/meminfo: memFree=31466848/32770624 swapFree=5992/198612
[pid=14346] ppid=14343 vsize=30588 CPUtime=1542 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 15582 0 0 0 154199 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4909007 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1542
Current children cumulated vsize (KiB) 30588

[startup+1602.3 s]
/proc/loadavg: 2.01 2.03 2.05 3/243 14456
/proc/meminfo: memFree=31466140/32770624 swapFree=5992/198660
[pid=14346] ppid=14343 vsize=30588 CPUtime=1601.98 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 15744 0 0 0 160197 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4964496 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1601.98
Current children cumulated vsize (KiB) 30588

[startup+1662.3 s]
/proc/loadavg: 2.00 2.03 2.05 3/243 14464
/proc/meminfo: memFree=31463992/32770624 swapFree=5992/198700
[pid=14346] ppid=14343 vsize=30588 CPUtime=1661.97 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 15904 0 0 0 166196 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4974624 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1661.97
Current children cumulated vsize (KiB) 30588

[startup+1722.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/243 14465
/proc/meminfo: memFree=31464136/32770624 swapFree=5992/198744
[pid=14346] ppid=14343 vsize=30588 CPUtime=1721.96 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 16348 0 0 0 172195 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070898 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1721.96
Current children cumulated vsize (KiB) 30588

[startup+1782.3 s]
/proc/loadavg: 2.00 2.02 2.05 3/243 14466
/proc/meminfo: memFree=31463776/32770624 swapFree=5992/198776
[pid=14346] ppid=14343 vsize=30588 CPUtime=1781.95 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 16530 0 0 0 178194 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4969776 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1781.95
Current children cumulated vsize (KiB) 30588

[startup+1842.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 14467
/proc/meminfo: memFree=31464768/32770624 swapFree=5992/198820
[pid=14346] ppid=14343 vsize=30588 CPUtime=1841.94 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 16977 0 0 0 184193 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070016 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1841.94
Current children cumulated vsize (KiB) 30588

[startup+1902.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14469
/proc/meminfo: memFree=31465112/32770624 swapFree=5992/198868
[pid=14346] ppid=14343 vsize=30588 CPUtime=1901.93 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 17181 0 0 0 190192 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070025 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1901.93
Current children cumulated vsize (KiB) 30588

[startup+1962.3 s]
/proc/loadavg: 2.05 2.03 2.05 3/244 14470
/proc/meminfo: memFree=31465112/32770624 swapFree=5992/198908
[pid=14346] ppid=14343 vsize=30588 CPUtime=1961.91 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 17358 0 0 0 196190 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4970523 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 1961.91
Current children cumulated vsize (KiB) 30588

[startup+2022.3 s]
/proc/loadavg: 2.02 2.02 2.05 3/244 14471
/proc/meminfo: memFree=31464988/32770624 swapFree=5992/198952
[pid=14346] ppid=14343 vsize=30588 CPUtime=2021.9 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 17535 0 0 0 202189 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4967825 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2021.9
Current children cumulated vsize (KiB) 30588

[startup+2082.3 s]
/proc/loadavg: 2.01 2.02 2.05 3/244 14472
/proc/meminfo: memFree=31465284/32770624 swapFree=5992/198984
[pid=14346] ppid=14343 vsize=30588 CPUtime=2081.9 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 17979 0 0 0 208189 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070791 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2081.9
Current children cumulated vsize (KiB) 30588

[startup+2142.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14473
/proc/meminfo: memFree=31463856/32770624 swapFree=5992/199020
[pid=14346] ppid=14343 vsize=30588 CPUtime=2141.88 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 18143 0 0 0 214187 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070006 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2141.88
Current children cumulated vsize (KiB) 30588

[startup+2202.31 s]
/proc/loadavg: 2.00 2.01 2.05 3/243 14474
/proc/meminfo: memFree=31464984/32770624 swapFree=5992/199060
[pid=14346] ppid=14343 vsize=30588 CPUtime=2201.87 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 18318 0 0 0 220186 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070892 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2201.87
Current children cumulated vsize (KiB) 30588

[startup+2262.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14483
/proc/meminfo: memFree=31463724/32770624 swapFree=5992/199100
[pid=14346] ppid=14343 vsize=30588 CPUtime=2261.86 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 18481 0 0 0 226185 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070892 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2261.86
Current children cumulated vsize (KiB) 30588

[startup+2322.3 s]
/proc/loadavg: 2.00 2.01 2.05 3/244 14484
/proc/meminfo: memFree=31464468/32770624 swapFree=5992/199136
[pid=14346] ppid=14343 vsize=30588 CPUtime=2321.84 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 18927 0 0 0 232183 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5070008 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2321.84
Current children cumulated vsize (KiB) 30588

[startup+2382.3 s]
/proc/loadavg: 2.10 2.04 2.05 3/244 14485
/proc/meminfo: memFree=31465588/32770624 swapFree=5992/199172
[pid=14346] ppid=14343 vsize=30588 CPUtime=2381.83 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 19083 0 0 0 238182 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 5323218 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2381.83
Current children cumulated vsize (KiB) 30588



Maximum CPU time exceeded: sending SIGTERM then SIGKILL

[startup+2400.5 s]
/proc/loadavg: 2.08 2.04 2.05 3/244 14485
/proc/meminfo: memFree=31465588/32770624 swapFree=5992/199176
[pid=14346] ppid=14343 vsize=30588 CPUtime=2400.03 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 19197 0 0 0 240002 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4976570 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2400.03
Current children cumulated vsize (KiB) 30588

Sending SIGTERM to process tree (bottom up)
Sleeping 2 seconds

Solver just ended.
??? exit watcher main loop
??? sigkill at end of watcher main loop
??? kill 14346 sig 9
??? end of watcher thread
Dumping a history of the last processes samples

[startup+2400.5 s]
/proc/loadavg: 2.08 2.04 2.05 3/244 14485
/proc/meminfo: memFree=31465588/32770624 swapFree=5992/199176
[pid=14346] ppid=14343 vsize=30588 CPUtime=2400.03 cores=1,3,5,7,9,11,13,15
/proc/14346/stat : 14346 (cosoco) R 14343 14346 9819 0 -1 4202496 19197 0 0 0 240002 1 0 0 20 0 1 0 1648267 31322112 1848 33554432000 4194304 5970956 140732588652352 140732588649688 4976570 0 0 4096 8405026 0 0 0 17 9 0 0 0 0 0 8070896 8075904 8372224 140732588655809 140732588655939 140732588655939 140732588658638 0
/proc/14346/statm: 7647 1848 832 434 0 1011 0
Current children cumulated CPU time (s) 2400.03
Current children cumulated vsize (KiB) 30588

Child status: 1
Real time (s): 2400.5
CPU time (s): 2400.04
CPU user time (s): 2400.02
CPU system time (s): 0.015996
CPU usage (%): 99.9808
Max. virtual memory (cumulated for all children) (KiB): 30588

getrusage(RUSAGE_CHILDREN,...) data:
user time used= 2400.02
system time used= 0.015996
maximum resident set size= 7436
integral shared memory size= 0
integral unshared data size= 0
integral unshared stack size= 0
page reclaims= 19225
page faults= 0
swaps= 0
block input operations= 0
block output operations= 0
messages sent= 0
messages received= 0
signals received= 0
voluntary context switches= 7
involuntary context switches= 122023

??? join timestamper begins
??? join timestamper ends
??? end of timestamper thread
runsolver used 7.14276 second user time and 10.6085 second system time

The end

Launcher Data 

Begin job on node103 at 2017-07-21 21:03:03
IDJOB=4242508
IDBENCH=134602
IDSOLVER=2620
FILE ID=node103/4242508-1500663783
RUNJOBID= node103-1500648178-9851
PBS_JOBID= 20623311
Free space on /tmp= 62012 MiB

SOLVER NAME= cosoco 1.1
BENCH NAME= XCSP17/KnightTour/KnightTour-int-s1/KnightTour-13-int.xml
COMMAND LINE= DIR/cosoco -lc -model -restarts BENCHNAME
RUNSOLVER COMMAND LINE= BIN/runsolver --timestamp -w /tmp/evaluation-result-4242508-1500663783/watcher-4242508-1500663783 -o /tmp/evaluation-result-4242508-1500663783/solver-4242508-1500663783 -C 2400 -W 2500 -M 15500 --bin-var /tmp/runsolver-exch-node103-1500648178-9851 --watchdog 2560  HOME/cosoco -lc -model -restarts HOME/instance-4242508-1500663783.xml

TIME LIMIT= 2400 seconds
MEMORY LIMIT= 15500 MiB
NBCORE= 8

MD5SUM BENCH= 5aa475b3f1257aceb66c8baaa9eb8eb1
RANDOM SEED=1256405569

node103.alineos.net Linux 3.10.0-514.16.1.el7.x86_64 #1 SMP Wed Apr 12 15:04:24 UTC 2017

/proc/cpuinfo:
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 16
initial apicid	: 16
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 2
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 2
initial apicid	: 2
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 3
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 18
initial apicid	: 18
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 4
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 4
initial apicid	: 4
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 5
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 20
initial apicid	: 20
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 6
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 6
initial apicid	: 6
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 7
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 22
initial apicid	: 22
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 8
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 1
initial apicid	: 1
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 9
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2400.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 0
cpu cores	: 4
apicid		: 17
initial apicid	: 17
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 10
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1733.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 3
initial apicid	: 3
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 11
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1733.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 1
cpu cores	: 4
apicid		: 19
initial apicid	: 19
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 12
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 2668.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 5
initial apicid	: 5
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 13
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 2
cpu cores	: 4
apicid		: 21
initial apicid	: 21
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 14
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 0
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 7
initial apicid	: 7
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5333.12
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:

processor	: 15
vendor_id	: GenuineIntel
cpu family	: 6
model		: 26
model name	: Intel(R) Xeon(R) CPU           X5550  @ 2.67GHz
stepping	: 5
microcode	: 0x19
cpu MHz		: 1600.000
cache size	: 8192 KB
physical id	: 1
siblings	: 8
core id		: 3
cpu cores	: 4
apicid		: 23
initial apicid	: 23
fpu		: yes
fpu_exception	: yes
cpuid level	: 11
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 popcnt lahf_lm ida dtherm tpr_shadow vnmi flexpriority ept vpid
bogomips	: 5332.55
clflush size	: 64
cache_alignment	: 64
address sizes	: 40 bits physical, 48 bits virtual
power management:


/proc/meminfo:
MemTotal:       32770624 kB
MemFree:        32002060 kB
MemAvailable:   32001076 kB
Buffers:           38064 kB
Cached:           270876 kB
SwapCached:            0 kB
Active:           184848 kB
Inactive:         205732 kB
Active(anon):      84500 kB
Inactive(anon):     8648 kB
Active(file):     100348 kB
Inactive(file):   197084 kB
Unevictable:        5992 kB
Mlocked:            5996 kB
SwapTotal:      67108860 kB
SwapFree:       67108860 kB
Dirty:              2504 kB
Writeback:             0 kB
AnonPages:         91428 kB
Mapped:            31592 kB
Shmem:              8852 kB
Slab:              66660 kB
SReclaimable:      34836 kB
SUnreclaim:        31824 kB
KernelStack:        4464 kB
PageTables:         4980 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    83494172 kB
Committed_AS:     354060 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      350232 kB
VmallocChunk:   34341910524 kB
HardwareCorrupted:     0 kB
AnonHugePages:     36864 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      112000 kB
DirectMap2M:    33433600 kB

Free space on /tmp at the end= 62008 MiB
End job on node103 at 2017-07-21 21:43:03