diff options
| -rw-r--r-- | ansible/README.md | 280 |
1 files changed, 280 insertions, 0 deletions
diff --git a/ansible/README.md b/ansible/README.md new file mode 100644 index 00000000..8d6e41ee --- /dev/null +++ b/ansible/README.md @@ -0,0 +1,280 @@ +# Offline Kubernetes deployment with Ansible + +This solution is generic offline deployment for Kubernetes which can host any +Kubernetes application in completely offline manner without internet connection. + +It's developed for the purpose of hosting [ONAP](https://www.onap.org/) and proven +to work with that. ONAP is a very big Kubernetes application deployed by using +[Helm](https://helm.sh/) charts. + +By default this solution supports deploying Kubernetes application with Helm charts. + +## Ansible execution/bootstrap + +There are two ways how to easily run our ansible playbooks. + +If you already have or can install a docker then you can build the provided `Dockerfile` for the ansible and run playbooks in the docker container. + +Another way to deploy ansible is via chroot environment which is bundled together within this directory. + +### (Re)build docker image and/or chroot archive + +Inside the docker directory is the `Dockerfile` and `build_ansible_image.sh` script. You can run `build_ansible_image.sh` script somewhere with internet connectivity and it will download all required packages needed for building the ansible docker image and for exporting it into flat chroot environment. + +Note: git must be installed before this script is called + +Built image is exported into `ansible_chroot.tgz` archive in the same (./docker) directory. + +This script has two optional arguments: + +1. ansible version +1. docker image name + +Note: if optional arguments are not used, docker image name will be set to `ansible` by default. + +### Launching ansible playbook using chroot environment + +This is our default and preferred way of running ansible playbooks in an offline environment as there is no dependency on docker to be installed on the system. Chroot environment is already provided by included archive `ansible_chroot.tgz`. + +It should be available in ./docker directory as the end-result of our packaging script or after manual running of `build_ansible_image.sh` referenced above. + +All playbooks can be executed via `./run_playbook.sh` wrapper script. + +To get more info about how the `./run_playbook.sh` wrapper script should be used, run: +```bash +./run_playbook.sh +``` + +The main purpose of this wrapper script is to provide ansible to machine where it was bootstrapped w/o need of installing additional packages, for displaying ansible-playbook command help one can run this following help command: +```bash +./run_playbook.sh --help +``` + +#### Developers notes from chroot approach + +* There are two scripts which work in tandem for creating and running chroot +* First can convert docker image into chroot directory +* Second will automate chrooting (necessary steps for chroot to work and cleanup) +* Both of them have help - just run: + +```bash +$ cd docker +$ ./create_docker_chroot.sh help +$ ./run_chroot.sh help +``` + +#### Example usage: + +``` +$ sudo su +$ docker/create_docker_chroot.sh convert some_docker_image ./new_name_for_chroot +$ cat ./new_name_for_chroot/README.md +$ docker/run_chroot.sh execute ./new_name_for_chroot cat /etc/os-release 2>/dev/null +``` + +### Launching ansible playbook using docker container (ALTERNATIVE APPROACH) + +This option is here just to keep support for the older method which rely on a running docker service. For the offline deployment use the chroot option as indicated above. + +You will not need `ansible_chroot.tgz` archive anymore, but the new requirement is a prebuilt docker image of ansible (based on our `Dockerfile`). It should be available in your local docker repository (otherwise the default name `ansible` may fetch unwanted image from default registry). + +To trigger this functionality and run `ansible-playbook` inside a docker container instead of the chroot environment, you must first set the `ANSIBLE_DOCKER_IMAGE` variable. The value must be a name of the built ansible docker image - built using our provided `Dockerfile`. + +Image name can be obtained again from `docker images` output. + +``` +E.g. +REPOSITORY TAG IMAGE ID CREATED SIZE +ansible latest c44633617bfb 2 hours ago 127 MB +``` + +Usage is basically the same as with the default chroot way - the only difference is the existence of the env variable: + +```bash +export ANSIBLE_DOCKER_IMAGE=ansible +./run_playbook.sh <args> +``` + +or in a single command + +```bash +ANSIBLE_DOCKER_IMAGE=ansible ./run_playbook.sh <args> +``` + +--- + +## Playbooks + +This chapter introduces all playbooks that each can be run independently, however +the order of running playbooks is implicitly fixed. + +The complete offline deployment is handled by `site.yml` Ansible playbook. +This playbook contains imports for all the other playbooks needed to deploy +wanted Kubernetes application: + - `upload_resources.yml` + - `infrastructure.yml` + - `rancher_kubernetes.yml` + - `application.yml` - this is an application related playbook + +### Resource upload + +Resource upload(nexus data, docker images, packages, etc.) is handled by playbook `upload_resources.yml`. + +**Preconditions:** + - setup nfs server for folder with resources (`resources.tar`) so it can be mounted on infrastructure server. + +### Infrastructure setup + +Infrastructure deployment consists of: + - docker installed on all nodes + - dnsmasq docker container setup and start + - vncserver docker container setup and start + - nexus3 docker container - consists of docker images for application, Rancher and other resources + - nginx docker container - serves mainly as proxy for nexus, and also for providing other resources via HTTP/HTTPS + +Infrastructure setup is handled by `infrastructure.yml` playbook. + +**Preconditions**: + - resources uploaded to infrastructure server (usually handled by `upload_resources.yml` playbook). All folders from resources should be located in directory defined in `{{ app_data_path }}` variable (e.g. `/opt/onap`). + +### Kubernetes cluster deployment + +Kubernetes cluster is deployment is handled by `rancher_kubernetes.yml` playbook. + +**Preconditions**: + - infrastructure deployed by running `infrastructure.yml` playbook + +### Kubernetes application installation + +Application installation is handled by `application.yml` playbook. + +Application is dependent on resources: + - images, files, etc. included in resources.tar handled by upload_resources.yml playbook. + - Helm charts pointing to those images and files. + - Application specific configuration + +Binary resources must be handled to be in place already during packaging and that +is instructed in **HERE/TODO**. + +Application Helm charts and configuration can also be packaged into the package itself and is then available on ./application folder. However it can also be +copied there after installer package is deployed and before installing the application. + +Application Helm charts and configuration is better described in [application/README.md](./application/README.md) + +**Preconditions**: + - Kubernetes cluster must be up and running i.e. `rancher_kubernetes.yml` playbook has been run. + +## Running playbooks +To run ansible playbook call `run_playbook.sh` with same arguments as you would +call ansible-playbook. + +Script `run_playbook.sh` must be executed from ansible directory to work correctly. + +Example: +```bash +./run_playbook.sh -i application/hosts.yml playbook.yml +``` + +## Deployment + +This chapter describes how to start whole deployment of offline solution from scratch +after offline software package is installed (untarred) to target machine from which +Ansible will be run towards the target servers. Target servers are already in place +and running. + +After offline software package is untarred, locate the ansible directory in there +having this README file you are reading. + +### Preparations + +**SSH keys**: + +In our playbooks we expects to have passwordless login to all ansible administrated nodes. +There is a helping playbook created to arrange that. Before setup playbook is executed +please insert IPs to inventory file as described below/ + +**Inventory**: + +File **./inventory/hosts.yml** is an example inventory file that can be used as a +baseline. Copy inventory/hosts.yml file to ./application folder and modify needed +parts. + + - `./application/hosts.yml`: + ``` + # This group contains hosts with all resources (binaries, packages, etc.) + # in tarball. + all: + vars: + # this key is supposed to be generated during setup.yml playbook execution + # change it just when you have better one working for all nodes + ansible_ssh_private_key_file: /root/.ssh/offline_ssh_key + ansible_ssh_common_args: '-o StrictHostKeyChecking=no' + + children: + resources: + hosts: + resource-host: + ansible_host: 10.8.8.5 + + # This is group of hosts where nexus, nginx, dns and all other required + # services are running. + infrastructure: + hosts: + infrastructure-server: + ansible_host: 10.8.8.9 + + # This is group of hosts which are/will be part of Kubernetes cluster. + kubernetes: + hosts: + kubernetes-node-1: + ansible_host: 10.8.8.13 + + nfs-server: + hosts: + kubernetes-node-1 + ``` + + + +after setting up inventory file helping setup playbook can be executed to arrange passwordless +login to all nodes. It make sense to launch it just when root user can't access all nodes in +passwordless way using ssh key. If root user has access already with ssh key, please override +just ansible_ssh_private_key_file in inventory file accordingly and there is no need to launch +setup playbook. + +If there is non-root user with ssh key access configured launch setup.yml in this way: + +```bash +./run_playbook.sh -i application/hosts.yml setup.yml -u <user> -e ansible_ssh_private_key_file=<key> +``` +e.g. +./run_playbook.sh -i application/hosts.yml setup.yml -u cloud-user -e ansible_ssh_private_key_file=/root/.ssh/id_rsa + +in case of password login to nodes, run playbook with --ask-pass param + +```bash +./run_playbook.sh -i application/hosts.yml setup.yml -u <user> --ask-pass +``` + +After running that playbook, passwordless login should be arranged for root user +using key ~/.ssh/offline_ssh_key which is preconfigured in inventory file. + + +**Deployment related configuration**: + +Please check and review all options in `group_vars/all.yml` and `group_vars/infrastructure.yml`. + +After inventory is setup and configuration is fine, all other playbooks (except application.yml) could be already be run and empty Kubernetes cluster (without application) could be setup. + +**Application configuration**: + +To deploy also application into Kubernetes, configure application configuration +needed by `application.yml` playbook. +See instructions on separate README.md file on [application/README.md](./application/README.md). + +### Run Deployment + +To deploy run: +```bash +./run_playbook.sh -i application/hosts.yml site.yml -e @application/application_configuration.yml +``` |