diff options
author | 2022-11-21 14:13:23 +0100 | |
---|---|---|
committer | 2022-11-28 14:34:32 +0100 | |
commit | 9eb2c11d39161671070085b21f73847b61ea8e5f (patch) | |
tree | 6fe047a945b43678ad86f372553bd69b9b476d10 /docs/sections/guides | |
parent | 693e816b299d2c2c77be62510808256836bf926a (diff) |
[DOC] ServiceMesh documentation
Add instructions for SM Installation (addon guide), ONAP configuration
(customize_override) and access guide
Issue-ID: OOM-3006
Signed-off-by: Andreas Geissler <andreas-geissler@telekom.de>
Change-Id: I31acad2c8c84ed3382e1c43897333f9ff565ff6d
Diffstat (limited to 'docs/sections/guides')
6 files changed, 383 insertions, 116 deletions
diff --git a/docs/sections/guides/access_guides/oom_access_info.rst b/docs/sections/guides/access_guides/oom_access_info.rst index 2e779105f2..4e9866725e 100644 --- a/docs/sections/guides/access_guides/oom_access_info.rst +++ b/docs/sections/guides/access_guides/oom_access_info.rst @@ -3,19 +3,174 @@ .. http://creativecommons.org/licenses/by/4.0 .. Copyright (C) 2022 Nordix Foundation +.. Links +.. _Kubernetes LoadBalancer: https://kubernetes.io/docs/concepts/services-networking/service/#loadbalancer +.. _Kubernetes NodePort: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport + .. _oom_access_info_guide: OOM Access Info ---------------- +############### .. figure:: ../../resources/images/oom_logo/oomLogoV2-medium.png :align: right +Access via NodePort/Loadbalancer +******************************** + +The ONAP deployment created by OOM operates in a private IP network that isn't +publicly accessible (i.e. OpenStack VMs with private internal network) which +blocks access to the ONAP User Interfaces. +To enable direct access to a service from a user's own environment (a laptop etc.) +the application's internal port is exposed through a `Kubernetes NodePort`_ or +`Kubernetes LoadBalancer`_ object. + +Typically, to be able to access the Kubernetes nodes publicly a public address +is assigned. In OpenStack this is a floating IP address. + +Most ONAP applications use the `NodePort` as predefined `service:type`, +which opens allows access to the service through the the IP address of each +Kubernetes node. +When using the `Loadbalancer` as `service:type` `Kubernetes LoadBalancer`_ object +which gets a separate IP address. + +.. note:: + The following example uses the `ONAP Portal`, which is not actively maintained + in Kohn and will be replaced in the future + +When e.g. the `portal-app` chart is deployed a Kubernetes service is created that +instantiates a load balancer. The LB chooses the private interface of one of +the nodes as in the example below (10.0.0.4 is private to the K8s cluster only). +Then to be able to access the portal on port 8989 from outside the K8s & +OpenStack environment, the user needs to assign/get the floating IP address that +corresponds to the private IP as follows:: + + > kubectl -n onap get services|grep "portal-app" + portal-app LoadBalancer 10.43.142.201 10.0.0.4 8989:30215/TCP,8006:30213/TCP,8010:30214/TCP 1d app=portal-app,release=dev + + +In this example, use the 11.0.0.4 private address as a key find the +corresponding public address which in this example is 10.12.6.155. If you're +using OpenStack you'll do the lookup with the horizon GUI or the OpenStack CLI +for your tenant (openstack server list). That IP is then used in your +`/etc/hosts` to map the fixed DNS aliases required by the ONAP Portal as shown +below:: + + 10.12.6.155 portal.api.simpledemo.onap.org + 10.12.6.155 vid.api.simpledemo.onap.org + 10.12.6.155 sdc.api.fe.simpledemo.onap.org + 10.12.6.155 sdc.workflow.plugin.simpledemo.onap.org + 10.12.6.155 sdc.dcae.plugin.simpledemo.onap.org + 10.12.6.155 portal-sdk.simpledemo.onap.org + 10.12.6.155 policy.api.simpledemo.onap.org + 10.12.6.155 aai.api.sparky.simpledemo.onap.org + 10.12.6.155 cli.api.simpledemo.onap.org + 10.12.6.155 msb.api.discovery.simpledemo.onap.org + 10.12.6.155 msb.api.simpledemo.onap.org + 10.12.6.155 clamp.api.simpledemo.onap.org + 10.12.6.155 so.api.simpledemo.onap.org + 10.12.6.155 sdc.workflow.plugin.simpledemo.onap.org + +Ensure you've disabled any proxy settings the browser you are using to access +the portal and then simply access now the new ssl-encrypted URL: +``https://portal.api.simpledemo.onap.org:30225/ONAPPORTAL/login.htm`` + +.. note:: + Using the HTTPS based Portal URL the Browser needs to be configured to accept + unsecure credentials. + Additionally when opening an Application inside the Portal, the Browser + might block the content, which requires to disable the blocking and reloading + of the page + +.. note:: + Besides the ONAP Portal the Components can deliver additional user interfaces, + please check the Component specific documentation. + +.. note:: + + | Alternatives Considered: + + - Kubernetes port forwarding was considered but discarded as it would + require the end user to run a script that opens up port forwarding tunnels + to each of the pods that provides a portal application widget. + + - Reverting to a VNC server similar to what was deployed in the Amsterdam + release was also considered but there were many issues with resolution, + lack of volume mount, /etc/hosts dynamic update, file upload that were + a tall order to solve in time for the Beijing release. + + Observations: + + - If you are not using floating IPs in your Kubernetes deployment and + directly attaching a public IP address (i.e. by using your public provider + network) to your K8S Node VMs' network interface, then the output of + 'kubectl -n onap get services | grep "portal-app"' + will show your public IP instead of the private network's IP. Therefore, + you can grab this public IP directly (as compared to trying to find the + floating IP first) and map this IP in /etc/hosts. + Some relevant information regarding accessing OOM from outside the cluster etc +ONAP Nodeports +============== + +NodePorts are used to allow client applications, that run outside of +Kubernetes, access to ONAP components deployed by OOM. +A NodePort maps an externally reachable port to an internal port of an ONAP +microservice. +It should be noted that the use of NodePorts is temporary. +An alternative solution based on Ingress Controller, which initial support is +already in place. It is planned to become a default deployment option in the +London release. + +More information from official Kubernetes documentation about +`Kubernetes NodePort`_. + +The following table lists all the NodePorts used by ONAP. + +.. csv-table:: NodePorts table + :file: ../../resources/csv/nodeports.csv + :widths: 20,20,20,20,20 + :header-rows: 1 + + +This table retrieves information from the ONAP deployment using the following +Kubernetes command: + +.. code-block:: bash + + kubectl get svc -n onap -o go-template='{{range .items}}{{range.spec.ports}}{{if .nodePort}}{{.nodePort}}{{.}}{{"\n"}}{{end}}{{end}}{{end}}' + + +(Optional) Access via Ingress +***************************** + +Using Ingress as access method requires the installation of an Ingress +controller and the configuration of the ONAP deployment to use it. + +For "ONAP on ServiceMesh" you can find the instructions in: + +- :ref:`oom_base_optional_addons` +- :ref:`oom_customize_overrides` + +In the ServiceMesh deployment the Istio IngressGateway is the only access point +for ONAP component interfaces. +Usually the Ingress is accessed via a LoadBalancer IP (<ingress-IP>), +which is used as central address. +All APIs/UIs are provided via separate URLs which are routed to the component service. +To use these URLs they need to be resolvable via DNS or via /etc/hosts. + +The domain name is usually defined in the `global` section of the ONAP helm-charts, +`virtualhost.baseurl` (here "simpledemo.onap.org") whereas the hostname of +the service (e.g. "sdc-fe-ui") is defined in the component's chart. + +.. code-block:: none -.. toctree:: - :maxdepth: 1 + <ingress-IP> kiali.simpledemo.onap.org + <ingress-IP> cds-ui.simpledemo.onap.org + <ingress-IP> sdc-fe-ui.simpledemo.onap.org + ... - oom_ingress_access.rst +To access e.g. the SDC UI now the new ssl-encrypted URL: +``https://sdc-fe-ui.simpledemo.onap.org/sdc1`` diff --git a/docs/sections/guides/access_guides/oom_ingress_access.rst b/docs/sections/guides/access_guides/oom_ingress_access.rst deleted file mode 100644 index 0c64375098..0000000000 --- a/docs/sections/guides/access_guides/oom_ingress_access.rst +++ /dev/null @@ -1,18 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 -.. International License. -.. http://creativecommons.org/licenses/by/4.0 -.. Copyright (C) 2022 Nordix Foundation - -.. Links - - -.. figure:: ../../resources/images/oom_logo/oomLogoV2-medium.png - :align: right - -.. _oom_ingress_access: - - -Ingress access to OOM -##################### - -TBD diff --git a/docs/sections/guides/deployment_guides/oom_customize_overrides.rst b/docs/sections/guides/deployment_guides/oom_customize_overrides.rst index 3acb8b6ee6..a49543cc0c 100644 --- a/docs/sections/guides/deployment_guides/oom_customize_overrides.rst +++ b/docs/sections/guides/deployment_guides/oom_customize_overrides.rst @@ -32,7 +32,7 @@ Users can customize the override files to suit their required deployment. Enabling/Disabling Components ------------------------------ +***************************** Here is an example of the nominal entries that need to be provided. Different values files are available for different contexts. @@ -43,6 +43,68 @@ Different values files are available for different contexts. | -Some other heading ------------------- -adva
\ No newline at end of file +(Optional) "ONAP on Service Mesh" +********************************* + +To enable "ONAP on Service Mesh" both "ServiceMesh" and "Ingress" +configuration entries need to be configured before deployment. + +Global settings relevant for ServiceMesh: + +.. code-block:: yaml + + global: + ingress: + # generally enable ingress for ONAP components + enabled: false + # enable all component's Ingress interfaces + enable_all: false + # default Ingress base URL + # can be overwritten in component by setting ingress.baseurlOverride + virtualhost: + baseurl: "simpledemo.onap.org" + # All http requests via ingress will be redirected on Ingress controller + # only valid for Istio Gateway (ServiceMesh enabled) + config: + ssl: "redirect" + # you can set an own Secret containing a certificate + # only valid for Istio Gateway (ServiceMesh enabled) + # tls: + # secret: 'my-ingress-cert' + # optional: Namespace of the Istio IngressGateway + # only valid for Istio Gateway (ServiceMesh enabled) + namespace: istio-ingress + ... + serviceMesh: + enabled: true + tls: true + # be aware that linkerd is not well tested + engine: "istio" # valid value: istio or linkerd + aafEnabled: false + cmpv2Enabled: false + tlsEnabled: false + msbEnabled: false + +ServiceMesh settings: + +- enabled: true → enables ServiceMesh functionality in the ONAP Namespace (Istio: enables Sidecar deployment) +- tls: true → enables mTLS encryption in Sidecar communication +- engine: istio → sets the SM engine (currently only Istio is supported) +- aafEnabled: false → disables AAF usage for TLS interfaces +- tlsEnabled: false → disables creation of TLS in component services +- cmpv2Enabled: false → disable cmpv2 feature +- msbEnabled: false → MSB is not used in Istio setup (Open, if all components are MSB independend) + +Ingress settings: + +- enabled: true → enables Ingress using: Nginx (when SM disabled), Istio IngressGateway (when SM enabled) +- enable_all: true → enables Ingress configuration in each component +- virtualhost.baseurl: "simpledemo.onap.org" → sets globally the URL for all Interfaces set by the components, + resulting in e.g. "aai-api.simpledemo.onap.org", can be overwritten in the component via: ingress.baseurlOverride +- config.ssl: redirect → sets in the Ingress globally the redirection of all Interfaces from http (port 80) to https (port 443) +- config.tls.secret: "..." → (optional) overrides the default selfsigned SSL certificate with a certificate stored in the specified secret +- namespace: istio-ingress → (optional) overrides the namespace of the ingress gateway which is used for the created SSL certificate + +.. note:: + For "ONAP on Istio" an example override file (`onap-all-ingress-istio.yaml`) + can be found in the `oom/kubernetes/onap/resources/overrides/` directory. diff --git a/docs/sections/guides/infra_guides/oom_base_optional_addons.rst b/docs/sections/guides/infra_guides/oom_base_optional_addons.rst index 4b4fbf7883..5f81a363e9 100644 --- a/docs/sections/guides/infra_guides/oom_base_optional_addons.rst +++ b/docs/sections/guides/infra_guides/oom_base_optional_addons.rst @@ -5,6 +5,10 @@ .. Links .. _Prometheus stack README: https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack#readme +.. _ONAP Next Generation Security & Logging Structure: https://wiki.onap.org/pages/viewpage.action?pageId=103417456 +.. _Istio best practices: https://docs.solo.io/gloo-mesh-enterprise/latest/setup/prod/namespaces/ +.. _Istio setup guide: https://istio.io/latest/docs/setup/install/helm/ +.. _Kiali setup guide: https://kiali.io/docs/installation/installation-guide/example-install/ .. _oom_base_optional_addons: @@ -39,3 +43,140 @@ To install the prometheus stack, execute the following: - To install prometheus, execute the following, replacing the <recommended-pm-version> with the version defined in the :ref:`versions_table` table:: > helm install prometheus prometheus-community/kube-prometheus-stack --namespace=prometheus --create-namespace --version=<recommended-pm-version> + +ONAP on Service Mesh +******************** + +.. warning:: + "ONAP on Service Mesh" is not fully supported in "Kohn". Full support is + planned for London release to support the + `ONAP Next Generation Security & Logging Structure`_ + +.. figure:: ../../resources/images/servicemesh/ServiceMesh.png + :align: center + +ONAP is currenty planned to support Istio as default ServiceMesh platform. +Therefor the following instructions describe the setup of Istio and required tools. +Used `Istio best practices`_ and `Istio setup guide`_ + +Istio Platform Installation +=========================== + +Install Istio Basic Platform +---------------------------- + +- Configure the Helm repository:: + + > helm repo add istio https://istio-release.storage.googleapis.com/charts + + > helm repo update + +- Create a namespace for "mesh-level" configurations:: + + > kubectl create namespace istio-config + +- Create a namespace istio-system for Istio components:: + + > kubectl create namespace istio-system + +- Install the Istio Base chart which contains cluster-wide resources used by the + Istio control plane, replacing the <recommended-istio-version> with the version + defined in the :ref:`versions_table` table:: + + > helm upgrade -i istio-base istio/base -n istio-system --version <recommended-istio-version> + +- Install the Istio Base Istio Discovery chart which deploys the istiod service, replacing the + <recommended-istio-version> with the version defined in the :ref:`versions_table` table + (enable the variable to enforce the (sidecar) proxy startup before the container start):: + + > helm upgrade -i istiod istio/istiod -n istio-system --version <recommended-istio-version> + --wait --set global.proxy.holdApplicationUntilProxyStarts=true --set meshConfig.rootNamespace=istio-config + +Add an EnvoyFilter for HTTP header case +--------------------------------------- + +When handling HTTP/1.1, Envoy will normalize the header keys to be all lowercase. +While this is compliant with the HTTP/1.1 spec, in practice this can result in issues +when migrating existing systems that might rely on specific header casing. +In our case a problem was detected in the SDC client implementation, which relies on +uppercase header values. To solve this problem in general we add a EnvoyFilter to keep +the uppercase header in the istio-config namespace to apply for all namespaces, but +set the context to SIDECAR_INBOUND to avoid problems in the connection between Istio-Gateway and Services + +- Create a EnvoyFilter file (e.g. envoyfilter-case.yaml) + + .. collapse:: envoyfilter-case.yaml + + .. include:: ../../resources/yaml/envoyfilter-case.yaml + :code: yaml + +- Apply the change to Istio:: + + > kubectl apply -f envoyfilter-case.yaml + +Install Istio Gateway +--------------------- + +- Create a namespace istio-ingress for the Istio Ingress gateway + and enable istio-injection:: + + > kubectl create namespace istio-ingress + + > kubectl label namespace istio-ingress istio-injection=enabled + +- Install the Istio Gateway chart,replacing the + <recommended-istio-version> with the version defined in + the :ref:`versions_table` table:: + + > helm upgrade -i istio-ingressgateway istio/gateway -n istio-ingress + --version <recommended-istio-version> --wait + +Kiali Installation +================== + +Kiali is used to visualize the Network traffic in a ServiceMesh enabled cluster +For setup the kiali operator is used, see `Kiali setup guide`_ + +- Install kiali-operator namespace:: + + > kubectl create namespace kiali-operator + + > kubectl label namespace kiali-operator istio-injection=enabled + +- Install the kiali-operator:: + + > helm repo add kiali https://kiali.org/helm-charts + + > helm repo update kiali + + > helm install --namespace kiali-operator kiali/kiali-operator + +- Create Kiali CR file (e.g. kiali.yaml) + + .. collapse:: kiali.yaml + + .. include:: ../../resources/yaml/kiali.yaml + :code: yaml + +- Install kiali:: + + > kubectl apply -f kiali.yaml + +- Create Ingress gateway entry for the kiali web interface + using the configured Ingress <base-url> (here "simpledemo.onap.org") + as described in :ref:`oom_customize_overrides` + + .. collapse:: kiali-ingress.yaml + + .. include:: ../../resources/yaml/kiali-ingress.yaml + :code: yaml + +- Add the Ingress entry for Kiali:: + + > kubectl -n istio-system apply -f kiali-ingress.yaml + + +Jaeger Installation +=================== + +To be done...
\ No newline at end of file diff --git a/docs/sections/guides/infra_guides/oom_infra_setup.rst b/docs/sections/guides/infra_guides/oom_infra_setup.rst index f9668de458..ed7b05a103 100644 --- a/docs/sections/guides/infra_guides/oom_infra_setup.rst +++ b/docs/sections/guides/infra_guides/oom_infra_setup.rst @@ -52,14 +52,23 @@ The versions of software that are supported by OOM are as follows: .. _versions_table: -.. table:: OOM Software Requirements - - ============== =========== ======= ======== ======== ============ ================= ======= - Release Kubernetes Helm kubectl Docker Cert-Manager Prometheus Stack Strimzi - ============== =========== ======= ======== ======== ============ ================= ======= - Jakarta 1.22.4 3.6.3 1.22.4 20.10.x 1.8.0 35.x 0.28.0 - Kohn 1.23.8 3.8.2 1.23.8 20.10.x 1.8.0 35.x 0.32.0 - ============== =========== ======= ======== ======== ============ ================= ======= +.. table:: OOM Software Requirements (base) + + ============== =========== ======= ======== ======== ============ ======= + Release Kubernetes Helm kubectl Docker Cert-Manager Strimzi + ============== =========== ======= ======== ======== ============ ======= + Jakarta 1.22.4 3.6.3 1.22.4 20.10.x 1.8.0 0.28.0 + Kohn 1.23.8 3.8.2 1.23.8 20.10.x 1.8.0 0.32.0 + ============== =========== ======= ======== ======== ============ ======= + +.. table:: OOM Software Requirements (optional) + + ============== ================= ====== + Release Prometheus Stack Istio + ============== ================= ====== + Jakarta 35.x --- + Kohn 35.x 1.15.1 + ============== ================= ====== .. toctree:: diff --git a/docs/sections/guides/user_guides/oom_user_guide.rst b/docs/sections/guides/user_guides/oom_user_guide.rst index c0f4f6ef73..449d5de3fa 100644 --- a/docs/sections/guides/user_guides/oom_user_guide.rst +++ b/docs/sections/guides/user_guides/oom_user_guide.rst @@ -234,88 +234,6 @@ can be modified, for example the `so`'s `liveness` probe could be disabled <...> -Accessing the ONAP Portal using OOM and a Kubernetes Cluster ------------------------------------------------------------- - -The ONAP deployment created by OOM operates in a private IP network that isn't -publicly accessible (i.e. OpenStack VMs with private internal network) which -blocks access to the ONAP Portal. To enable direct access to this Portal from a -user's own environment (a laptop etc.) the portal application's port 8989 is -exposed through a `Kubernetes LoadBalancer`_ object. - -Typically, to be able to access the Kubernetes nodes publicly a public address -is assigned. In OpenStack this is a floating IP address. - -When the `portal-app` chart is deployed a Kubernetes service is created that -instantiates a load balancer. The LB chooses the private interface of one of -the nodes as in the example below (10.0.0.4 is private to the K8s cluster only). -Then to be able to access the portal on port 8989 from outside the K8s & -OpenStack environment, the user needs to assign/get the floating IP address that -corresponds to the private IP as follows:: - - > kubectl -n onap get services|grep "portal-app" - portal-app LoadBalancer 10.43.142.201 10.0.0.4 8989:30215/TCP,8006:30213/TCP,8010:30214/TCP 1d app=portal-app,release=dev - - -In this example, use the 11.0.0.4 private address as a key find the -corresponding public address which in this example is 10.12.6.155. If you're -using OpenStack you'll do the lookup with the horizon GUI or the OpenStack CLI -for your tenant (openstack server list). That IP is then used in your -`/etc/hosts` to map the fixed DNS aliases required by the ONAP Portal as shown -below:: - - 10.12.6.155 portal.api.simpledemo.onap.org - 10.12.6.155 vid.api.simpledemo.onap.org - 10.12.6.155 sdc.api.fe.simpledemo.onap.org - 10.12.6.155 sdc.workflow.plugin.simpledemo.onap.org - 10.12.6.155 sdc.dcae.plugin.simpledemo.onap.org - 10.12.6.155 portal-sdk.simpledemo.onap.org - 10.12.6.155 policy.api.simpledemo.onap.org - 10.12.6.155 aai.api.sparky.simpledemo.onap.org - 10.12.6.155 cli.api.simpledemo.onap.org - 10.12.6.155 msb.api.discovery.simpledemo.onap.org - 10.12.6.155 msb.api.simpledemo.onap.org - 10.12.6.155 clamp.api.simpledemo.onap.org - 10.12.6.155 so.api.simpledemo.onap.org - 10.12.6.155 sdc.workflow.plugin.simpledemo.onap.org - -Ensure you've disabled any proxy settings the browser you are using to access -the portal and then simply access now the new ssl-encrypted URL: -``https://portal.api.simpledemo.onap.org:30225/ONAPPORTAL/login.htm`` - -.. note:: - Using the HTTPS based Portal URL the Browser needs to be configured to accept - unsecure credentials. - Additionally when opening an Application inside the Portal, the Browser - might block the content, which requires to disable the blocking and reloading - of the page - -.. note:: - Besides the ONAP Portal the Components can deliver additional user interfaces, - please check the Component specific documentation. - -.. note:: - - | Alternatives Considered: - - - Kubernetes port forwarding was considered but discarded as it would - require the end user to run a script that opens up port forwarding tunnels - to each of the pods that provides a portal application widget. - - - Reverting to a VNC server similar to what was deployed in the Amsterdam - release was also considered but there were many issues with resolution, - lack of volume mount, /etc/hosts dynamic update, file upload that were - a tall order to solve in time for the Beijing release. - - Observations: - - - If you are not using floating IPs in your Kubernetes deployment and - directly attaching a public IP address (i.e. by using your public provider - network) to your K8S Node VMs' network interface, then the output of - 'kubectl -n onap get services | grep "portal-app"' - will show your public IP instead of the private network's IP. Therefore, - you can grab this public IP directly (as compared to trying to find the - floating IP first) and map this IP in /etc/hosts. .. figure:: ../../resources/images/oom_logo/oomLogoV2-Monitor.png :align: right |