blob: e49c72a2e4cd70315bd9118f81090e60d2ed6e60 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
|
.. This work is licensed under a Creative Commons Attribution 4.0
.. International License.
.. http://creativecommons.org/licenses/by/4.0
.. Copyright 2020, Samsung Electronics
.. Links
.. _HELM Best Practices Guide: https://docs.helm.sh/chart_best_practices/#requirements
.. _kubectl Cheat Sheet: https://kubernetes.io/docs/reference/kubectl/cheatsheet/
.. _Kubernetes documentation for emptyDir: https://kubernetes.io/docs/concepts/storage/volumes/#emptydir
.. _metallb Metal Load Balancer installation: https://metallb.universe.tf/installation/
.. _http://cd.onap.info:30223/mso/logging/debug: http://cd.onap.info:30223/mso/logging/debug
.. _Onboarding and Distributing a Vendor Software Product: https://wiki.onap.org/pages/viewpage.action?pageId=1018474
.. _README.md: https://gerrit.onap.org/r/gitweb?p=oom.git;a=blob;f=kubernetes/README.md
.. figure:: oomLogoV2-medium.png
:align: right
.. _oom_setup_ingress_controller:
Ingress controller setup on HA Kubernetes Cluster
#################################################
This guide provides instruction how to setup experimental ingress controller
feature. For this, we are hosting our cluster on OpenStack VMs and using the
Rancher Kubernetes Engine (RKE) to deploy and manage our Kubernetes Cluster and
ingress controller
.. contents::
:depth: 1
:local:
..
The result at the end of this tutorial will be:
#. Customization of the cluster.yaml file for ingress controller support
#. Installation and configuration test DNS server for ingress host resolution
on testing machines
#. Installation and configuration MLB (Metal Load Balancer) required for
exposing ingress service
#. Installation and configuration NGINX ingress controller
#. Additional info how to deploy ONAP with services exposed via Ingress
controller
Customize cluster.yml file
==========================
Before setup cluster for ingress purposes DNS cluster IP and ingress provider
should be configured and following:
.. code-block:: yaml
---
<...>
restore:
restore: false
snapshot_name: ""
ingress:
provider: none
dns:
provider: coredns
upstreamnameservers:
- <custer_dns_ip>:31555
Where the <cluster_dns_ip> should be set to the same IP as the CONTROLPANE
node.
For external load balancer purposes, minimum one of the worker node should be
configured with external IP address accessible outside the cluster. It can be
done using the following example node configuration:
.. code-block:: yaml
---
<...>
- address: <external_ip>
internal_address: <internal_ip>
port: "22"
role:
- worker
hostname_override: "onap-worker-0"
user: ubuntu
ssh_key_path: "~/.ssh/id_rsa"
<...>
Where the <external_ip> is external worker node IP address, and <internal_ip>
is internal node IP address if it is required.
DNS server configuration and installation
=========================================
DNS server deployed on the Kubernetes cluster makes it easy to use services
exposed through ingress controller because it resolves all subdomain related to
the ONAP cluster to the load balancer IP. Testing ONAP cluster requires a lot
of entries on the target machines in the /etc/hosts. Adding many entries into
the configuration files on testing machines is quite problematic and error
prone. The better wait is to create central DNS server with entries for all
virtual host pointed to simpledemo.onap.org and add custom DNS server as a
target DNS server for testing machines and/or as external DNS for Kubernetes
cluster.
DNS server has automatic installation and configuration script, so installation
is quite easy::
> cd kubernetes/contrib/dns-server-for-vhost-ingress-testing
> ./deploy\_dns.sh
After DNS deploy you need to setup DNS entry on the target testing machine.
Because DNS listen on non standard port configuration require iptables rules
on the target machine. Please follow the configuration proposed by the deploy
scripts.
Example output depends on the IP address and example output looks like bellow::
DNS server already deployed:
1. You can add the DNS server to the target machine using following commands:
sudo iptables -t nat -A OUTPUT -p tcp -d 192.168.211.211 --dport 53 -j DNAT --to-destination 10.10.13.14:31555
sudo iptables -t nat -A OUTPUT -p udp -d 192.168.211.211 --dport 53 -j DNAT --to-destination 10.10.13.14:31555
sudo sysctl -w net.ipv4.conf.all.route_localnet=1
sudo sysctl -w net.ipv4.ip_forward=1
2. Update /etc/resolv.conf file with nameserver 192.168.211.211 entry on your target machine
MetalLB Load Balancer installation and configuration
====================================================
By default pure Kubernetes cluster requires external load balancer if we want
to expose external port using LoadBalancer settings. For this purpose MetalLB
can be used. Before installing the MetalLB you need to ensure that at least one
worker has assigned IP accessible outside the cluster.
MetalLB Load balancer can be easily installed using automatic install script::
> cd kubernetes/contrib/metallb-loadbalancer-inst
> ./install-metallb-on-cluster.sh
Configuration Ngninx ingress controller
=======================================
After installation DNS server and ingress controller we can install and
configure ingress controller.
It can be done using the following commands::
> cd kubernetes/contrib/ingress-nginx-post-inst
> kubectl apply -f nginx_ingress_cluster_config.yaml
> kubectl apply -f nginx_ingress_enable_optional_load_balacer_service.yaml
After deploy NGINX ingress controller you can ensure that the ingress port is
exposed as load balancer service with external IP address::
> kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default-http-backend ClusterIP 10.10.10.10 <none> 80/TCP 25h
ingress-nginx LoadBalancer 10.10.10.11 10.12.13.14 80:31308/TCP,443:30314/TCP 24h
ONAP with ingress exposed services
==================================
If you want to deploy onap with services exposed through ingress controller you
can use full onap deploy script::
> onap/resources/overrides/onap-all-ingress-nginx-vhost.yaml
Ingress also can be enabled on any onap setup override using following code:
.. code-block:: yaml
---
<...>
global:
<...>
ingress:
enabled: true
|
