added initial files

Issue-ID: INT-1698

Change-Id: I3405a7815bc51dbdf2cb61cd2003d46f428f059c
Signed-off-by: yoonsoonjahng <yoonsoon.jahng@yoppworks.com>

22 files changed, 411 insertions, 0 deletions

diff --git a/README.md b/README.md
new file mode 100644
index 0000000..f3ba03b
--- /dev/null
+++ b/README.md
@@ -0,0 +1,97 @@
+# Infrastructure as code for OpenStack deployment of ONAP
+
+## Build your infrastructure with Terragrunt(Terraform) for ONAP
+
+Preparing ONAP for deployment requires Openstack VMs with Kubernetes and helm installed. 
+Building underlying infrastructure for ONAP with Openstack GUI or command-line interfaces is not only time-consuming but also prone to mistakes.
+By providing Infrastructure as Code for deploying ONAP, building and managing the underlying infrastructure become simpler and easier.
+This [link](https://docs.onap.org/en/casablanca/submodules/oom.git/docs/oom_setup_kubernetes_rancher.html#onap-on-kubernetes-with-rancher) shows how to set up the underlying infrastructure with Openstack GUI and Command line tool.
+
+This Terragrunt(Terraform) code provides the same infrastructure as you would create through the process outlined in the link above.
+
+
+## Directory structure
+```
+openlab             # Terragrunt scripts to feed configuration into the Terraform modules 
+ └ RegionOne        # For multi regions. e.g, us-east-1
+   └ stage          # Environment specific configuration. e.g, QA/Stage/Prod
+     └ resource
+```
+
+Infrastrucuture is organized hierarchically in folders.
+The root level folder represents an account for clouds such as Openstack or AWS.
+The second and third levels represent the region in a cloud and environment under the region respectively.
+
+### Preparation
+1. You need a cloud storage bucket to store an intermediate state for your infrastructure. The remote state enables your team to work together as well.  We tested this code using a Google Storage bucket. You can choose AWS S3 as well as others.
+
+
+2. Openstack is the primary target for this code. We tested this code onto Openstack v3.8 (Ocata) 
+We deployed VMs and K8s with the scripts and after that we deployed ONAP Frankfurt version with OOM.
+
+### Usage
+#### Set up environment variables for your target cloud.
+
+1.a You need to export cloud storage credential.
+For instance, if you use Google Storage bucket, you can download the credentials from Google UI or the command-line tool.
+Go to Google Cloud project's `IAM & Admin` menu and choose the service account associated with the storage.
+You can export the credential as a JSON formatted file.  Then
+`export GOOGLE_APPLICATION_CREDENTIALS=/path/to/credential-file`.
+Please, refer to the following [link](https://cloud.google.com/iam/docs/creating-managing-service-account-keys).
+
+Second, you need to export Openstack credentials. You can use the openstack.rc file downloaded from your Openstack project.
+Please, refer to the following [link](https://docs.openstack.org/ocata/user-guide/common/cli-set-environment-variables-using-openstack-rc.html) for details.
+
+#### Fill in files
+ - `account.hcl`: Top-level configuration for a cloud account.
+ - `region.hcl`: The second level configuration for a region within the cloud
+ - `env.hcl`: The third level configuration for an environment within the region
+ - `terragrunt.hcl`: files under the compute directory. Since Kubernetes deployment needs 2 types of nodes (control and worker) plus 1 NFS cluster, these files under the compute directory contain the configuration for Kubernentes and NFS nodes 
+
+####  Building all modules for an environment
+Move to an environmental level folder, e.g stage.
+Then run `terragrunt apply-all` followed by `terraform init`
+If you run the command above for the first time and your version of Terraform is <= 0.13-beta, You may end up the error below
+`Provider "rke" not available for installation.`
+You have to manually download the `rke` provider and copy it under a Terraform's cached plug-in directory.
+After copying the rke provider, move to the `rancher` directory and run `terragrunt init`.
+Please, refer to the following [link](https://github.com/rancher/terraform-provider-rke#installing-the-provider) for details.
+
+https://github.com/gruntwork-io/terragrunt-infrastructure-live-example#deploying-all-modules-in-a-region
+
+####  Updating infrastructure version
+Infrastructure may evolve. You can use existing infrastructure as it is or updating the infrastructure to meet a new requirement.
+To deploy a different version of infrastructure, you can change a tag of `source` module version.
+Please, refer to the below document.
+If you like to test a new module (Terraform code) with Terragrunt, you just need to change the source attribute within Terrafrom block in each terragurnt.hcl file.
+[link](https://www.terraform.io/docs/modules/sources.html#generic-git-repository)
+
+####  Using Kubernetes and helm
+
+Please, refer to [link](https://github.com/gruntwork-io/terragrunt-infrastructure-live-example#example-infrastructure-live-for-terragrunt)
+
+#### Obtaining your KUBECONFIG
+Finally, You need to export Kubernenetes credentials.
+This credential is used when Helm service account is created. 
+For example, `export KUBECONFIG=/path/to/kube_config_cluster.yaml`
+In default, `kube_config_cluster.yaml` will be created under `path/to/openlab/RegionOne/stage` directory once you run `terragrunt apply-all`
+
+## Google Cloud Backend for Terraform
+To use the Google Cloud Storage backend for Terraform -- it stores state and manages locking -- you'll need to install the Google Cloud SDK.  Follow the instructions here https://cloud.google.com/sdk.
+You can do this task with Google Cloud's Web.
+
+1. Create a service account
+gcloud iam service-accounts create `service-account-name` 
+
+2. Binding the service account with a role 
+gcloud projects add-iam-policy-binding `project id` --member "serviceAccount:service-account-name-above@project-id.iam.gserviceaccount.com" --role "roles/proper-role-such-as-storage-user"
+
+3. To create a key for the service account created above
+gcloud iam service-accounts keys create account.json --iam-account `service-account-name-above@project-id.iam.gserviceaccount.com
+
+4. Create a storage bucket
+gsutil mb -p project-id gs://storage-bucket-name
+
+## Secrets
+How to hide your secret and provide it via a key management tool. Please, refer to the link below.
+Refer to https://nderraugh.github.io/
diff --git a/openlab/README.md b/openlab/README.md
new file mode 100644
index 0000000..1df82fb
--- /dev/null
+++ b/openlab/README.md
@@ -0,0 +1,3 @@
+# openlab
+
+Terraform scripts for Openlab(Windriver Lab)
diff --git a/openlab/modules/kubernetes/kubernetes/main.tf b/openlab/modules/kubernetes/kubernetes/main.tf
new file mode 100644
index 0000000..eccbcd4
--- /dev/null
+++ b/openlab/modules/kubernetes/kubernetes/main.tf
@@ -0,0 +1,48 @@
+provider "helm" {
+  version = "~> 0.10.0"
+  init_helm_home = true
+  install_tiller = true
+  service_account = var.service_account
+  namespace    = var.namespace
+  tiller_image = "gcr.io/kubernetes-helm/tiller:v2.16.6"
+
+  kubernetes {
+    config_path = "${var.kubernetes_home}/kube_config_cluster.yaml"
+  }
+}
+
+provider "kubernetes" {
+  version = ">= 1.10"
+  load_config_file = true
+}
+
+resource "kubernetes_service_account" "tiller" {
+  automount_service_account_token = true
+
+  metadata {
+    name = var.service_account
+    namespace = var.namespace
+  }
+}
+
+resource "kubernetes_cluster_role_binding" "tiller" {
+  metadata {
+    name = "tiller"
+  }
+
+  role_ref {
+    api_group = "rbac.authorization.k8s.io"
+    kind      = "ClusterRole"
+    name      = "cluster-admin"
+  }
+
+  subject {
+    kind      = "ServiceAccount"
+    name      = kubernetes_service_account.tiller.metadata.0.name
+    namespace = kubernetes_service_account.tiller.metadata.0.namespace
+  }
+
+  provisioner "local-exec" {
+    command = "helm init --service-account ${var.service_account};kubectl -n ${var.namespace} rollout status deployment/tiller-deploy"
+  }
+}
diff --git a/openlab/modules/kubernetes/kubernetes/variables.tf b/openlab/modules/kubernetes/kubernetes/variables.tf
new file mode 100644
index 0000000..6e26b85
--- /dev/null
+++ b/openlab/modules/kubernetes/kubernetes/variables.tf
@@ -0,0 +1,5 @@
+variable "kubernetes_home" {}
+
+variable "service_account" {}
+
+variable "namespace" {}
\ No newline at end of file
diff --git a/openlab/modules/kubernetes/rancher/data.tf b/openlab/modules/kubernetes/rancher/data.tf
new file mode 100644
index 0000000..fa38a55
--- /dev/null
+++ b/openlab/modules/kubernetes/rancher/data.tf
@@ -0,0 +1,15 @@
+data "terraform_remote_state" "control_plane_floating_ips" {
+  backend = var.backend
+  config = {
+    bucket      = var.backend_state
+    prefix      = "${var.region}/${var.environment}/compute/control/terraform.tfstate"
+  }
+}
+
+data "terraform_remote_state" "worker_node_floating_ips" {
+  backend = var.backend
+  config = {
+    bucket      = var.backend_state
+    prefix      = "${var.region}/${var.environment}/compute/worker/terraform.tfstate"
+  }
+}
diff --git a/openlab/modules/kubernetes/rancher/main.tf b/openlab/modules/kubernetes/rancher/main.tf
new file mode 100644
index 0000000..b787f0b
--- /dev/null
+++ b/openlab/modules/kubernetes/rancher/main.tf
@@ -0,0 +1,55 @@
+# If your terraform version is < 0.13-beta, manual installation is needed.
+# https://github.com/rancher/terraform-provider-rke
+provider "rke" {}
+
+locals {
+  control_node = [for ip in (flatten(data.terraform_remote_state.control_plane_floating_ips.outputs.floating_ips)): {
+      address = ip
+      role = "control"
+   }]
+  worker_node = [for ip in (flatten(data.terraform_remote_state.worker_node_floating_ips.outputs.floating_ips)): {
+      address = ip
+      role = "worker"
+  }]
+
+  all_node = concat(local.control_node, local.worker_node)
+}
+
+resource "rke_cluster" "cluster" {
+  kubernetes_version = var.kubernetes_version
+  cluster_name = var.kubernetes_cluster_name
+
+  dynamic nodes {
+
+    for_each = local.all_node
+
+    content {
+      address = nodes.value.address
+      user = var.kubernetes_user
+      role = (nodes.value.role == "control") ? [ "controlplane", "etcd" ] : ["worker"]
+    }
+  }
+
+  # You have to have private key on your machine excuting terraform
+  # An Openstack keypair is generated and stored within the remote state at
+  # "${var.region}/${var.environment}/keypair/terraform.tfstate"
+  ssh_key_path = var.ssh_private_key_path
+
+  disable_port_check = false
+
+  depends_on = [null_resource.wait-for-docker]
+}
+
+resource "local_file" "kube_cluster_yaml" {
+  filename = "${var.kubernetes_home}/kube_config_cluster.yaml"
+  sensitive_content = rke_cluster.cluster.kube_config_yaml
+}
+
+resource "null_resource" "wait-for-docker" {
+  provisioner "local-exec" {
+    # wait untill VM's bootstrapping's done
+    # If your VMs for Computing node have finished bootstrapping already,
+    # you may not need this waiting time
+    command = "sleep 120"
+  }
+}
diff --git a/openlab/modules/kubernetes/rancher/output.tf b/openlab/modules/kubernetes/rancher/output.tf
new file mode 100644
index 0000000..aa25e43
--- /dev/null
+++ b/openlab/modules/kubernetes/rancher/output.tf
@@ -0,0 +1,7 @@
+output "ips" {
+  value = local.all_node
+}
+
+output "kube_config_file" {
+  value = local_file.kube_cluster_yaml.filename
+}
\ No newline at end of file
diff --git a/openlab/modules/kubernetes/rancher/variables.tf b/openlab/modules/kubernetes/rancher/variables.tf
new file mode 100644
index 0000000..8d63243
--- /dev/null
+++ b/openlab/modules/kubernetes/rancher/variables.tf
@@ -0,0 +1,17 @@
+variable "backend" {}
+
+variable "backend_state" {}
+
+variable "region" {}
+
+variable "environment" {}
+
+variable "ssh_private_key_path" {}
+
+variable "kubernetes_user" {}
+
+variable "kubernetes_version" {}
+
+variable "kubernetes_cluster_name" {}
+
+variable "kubernetes_home" {}
diff --git a/openlab/modules/openstack/compute/data.tf b/openlab/modules/openstack/compute/data.tf
new file mode 100644
index 0000000..24c6c62
--- /dev/null
+++ b/openlab/modules/openstack/compute/data.tf
@@ -0,0 +1,36 @@
+data "terraform_remote_state" "keypair" {
+  backend = var.backend
+  config = {
+    bucket      = var.backend_state
+    prefix      = "${var.region}/${var.environment}/keypair/terraform.tfstate"
+  }
+}
+
+data "terraform_remote_state" "network" {
+  backend = var.backend
+  config = {
+    bucket      = var.backend_state
+    prefix      = "${var.region}/${var.environment}/network/terraform.tfstate"
+  }
+}
+
+data "terraform_remote_state" "securitygroup" {
+  backend = var.backend
+  config = {
+    bucket      = var.backend_state
+    prefix      = "${var.region}/${var.environment}/securitygroup/terraform.tfstate"
+  }
+}
+
+data "openstack_images_image_v2" "vm_image" {
+  name = var.image
+}
+
+data "openstack_compute_flavor_v2" "flavor" {
+  name = var.flavor
+}
+
+data "openstack_networking_network_v2" "egress_network" {
+  name = var.network
+}
+
diff --git a/openlab/modules/openstack/compute/main.tf b/openlab/modules/openstack/compute/main.tf
new file mode 100644
index 0000000..1e08737
--- /dev/null
+++ b/openlab/modules/openstack/compute/main.tf
@@ -0,0 +1,27 @@
+resource "openstack_compute_instance_v2" "nodes" {
+  name      = "${var.cluster_name}-${var.node_name}-${count.index}"
+  image_id  = data.openstack_images_image_v2.vm_image.id
+  flavor_id = data.openstack_compute_flavor_v2.flavor.id
+  key_pair  = data.terraform_remote_state.keypair.outputs.name
+  network {
+    name = data.terraform_remote_state.network.outputs.name
+  }
+  security_groups = [ data.terraform_remote_state.securitygroup.outputs.name ]
+
+  user_data = var.user_data
+
+  count = var.node_count
+}
+
+resource "openstack_compute_floatingip_v2" "floatingip" {
+  pool = var.floating_ip_pool
+
+  count = var.node_count
+}
+
+resource "openstack_compute_floatingip_associate_v2" "floatipassociation" {
+  floating_ip = openstack_compute_floatingip_v2.floatingip[count.index].address
+  instance_id = openstack_compute_instance_v2.nodes[count.index].id
+
+  count = var.node_count
+}
diff --git a/openlab/modules/openstack/compute/output.tf b/openlab/modules/openstack/compute/output.tf
new file mode 100644
index 0000000..28ae6d0
--- /dev/null
+++ b/openlab/modules/openstack/compute/output.tf
@@ -0,0 +1,11 @@
+output "floating_ips" {
+  value = [openstack_compute_floatingip_associate_v2.floatipassociation.*.floating_ip]
+}
+
+output "fixed_ips" {
+  value = [openstack_compute_instance_v2.nodes.*.access_ip_v4]
+}
+
+output "hostnames" {
+  value = [openstack_compute_instance_v2.nodes.*.name]
+}
\ No newline at end of file
diff --git a/openlab/modules/openstack/compute/variables.tf b/openlab/modules/openstack/compute/variables.tf
new file mode 100644
index 0000000..871719d
--- /dev/null
+++ b/openlab/modules/openstack/compute/variables.tf
@@ -0,0 +1,18 @@
+variable "cluster_name" {
+  description = "A name for the cluster"
+  type = string
+}
+
+variable "backend" {}
+variable "backend_state" {}
+variable "region" {}
+variable "environment" {}
+
+variable "node_name" {}
+variable "image" {}
+variable "flavor" {}
+variable "network" {}
+variable "floating_ip_pool" {}
+variable "user_data" {}
+
+variable "node_count" {}
diff --git a/openlab/modules/openstack/keypair/main.tf b/openlab/modules/openstack/keypair/main.tf
new file mode 100644
index 0000000..5b1384a
--- /dev/null
+++ b/openlab/modules/openstack/keypair/main.tf
@@ -0,0 +1,6 @@
+resource "openstack_compute_keypair_v2" "key" {
+  # You cna find a public/private key pair from your remote state storage.
+  name = "${var.cluster_name}-key"
+  # In order to generate a new keypair via existing public key
+  public_key = var.ssh_public_key
+}
diff --git a/openlab/modules/openstack/keypair/output.tf b/openlab/modules/openstack/keypair/output.tf
new file mode 100644
index 0000000..9c042e6
--- /dev/null
+++ b/openlab/modules/openstack/keypair/output.tf
@@ -0,0 +1,3 @@
+output "name" {
+  value = openstack_compute_keypair_v2.key.name
+}
\ No newline at end of file
diff --git a/openlab/modules/openstack/keypair/variables.tf b/openlab/modules/openstack/keypair/variables.tf
new file mode 100644
index 0000000..b3c6a92
--- /dev/null
+++ b/openlab/modules/openstack/keypair/variables.tf
@@ -0,0 +1,6 @@
+variable "cluster_name" {
+  description = "A name for the cluster"
+  type = string
+}
+
+variable "ssh_public_key" { }
diff --git a/openlab/modules/openstack/network/data.tf b/openlab/modules/openstack/network/data.tf
new file mode 100644
index 0000000..fad533f
--- /dev/null
+++ b/openlab/modules/openstack/network/data.tf
@@ -0,0 +1,3 @@
+data "openstack_networking_network_v2" "egress_network" {
+  name = var.network
+}
\ No newline at end of file
diff --git a/openlab/modules/openstack/network/main.tf b/openlab/modules/openstack/network/main.tf
new file mode 100644
index 0000000..80dd70e
--- /dev/null
+++ b/openlab/modules/openstack/network/main.tf
@@ -0,0 +1,25 @@
+resource "openstack_networking_network_v2" "network" {
+  name              = "${var.cluster_name}-network"
+  admin_state_up    = "true"
+}
+
+resource "openstack_networking_subnet_v2" "subnet" {
+  name        = "${var.cluster_name}-subnet"
+  network_id  = openstack_networking_network_v2.network.id
+  cidr        = "192.168.64.0/24"
+  ip_version  = 4
+  gateway_ip  = "192.168.64.1"
+  enable_dhcp = "true"
+  dns_nameservers = [ "8.8.8.8", "8.8.4.4" ]
+}
+
+resource "openstack_networking_router_v2" "router" {
+  name                = "${var.cluster_name}-router"
+  admin_state_up      = true
+  external_network_id = data.openstack_networking_network_v2.egress_network.id
+}
+
+resource "openstack_networking_router_interface_v2" "router_interface" {
+  router_id = openstack_networking_router_v2.router.id
+  subnet_id = openstack_networking_subnet_v2.subnet.id
+}
\ No newline at end of file
diff --git a/openlab/modules/openstack/network/output.tf b/openlab/modules/openstack/network/output.tf
new file mode 100644
index 0000000..1ca1b0a
--- /dev/null
+++ b/openlab/modules/openstack/network/output.tf
@@ -0,0 +1,3 @@
+output "name" {
+  value = openstack_networking_network_v2.network.name
+}
\ No newline at end of file
diff --git a/openlab/modules/openstack/network/variables.tf b/openlab/modules/openstack/network/variables.tf
new file mode 100644
index 0000000..86de2f0
--- /dev/null
+++ b/openlab/modules/openstack/network/variables.tf
@@ -0,0 +1,9 @@
+variable "cluster_name" {
+  description = "A name for the cluster"
+  type = string
+}
+
+variable "network" {
+  description = "A network to the Internet"
+  type = string
+}
diff --git a/openlab/modules/openstack/securitygroup/main.tf b/openlab/modules/openstack/securitygroup/main.tf
new file mode 100644
index 0000000..af7cbb4
--- /dev/null
+++ b/openlab/modules/openstack/securitygroup/main.tf
@@ -0,0 +1,10 @@
+resource "openstack_networking_secgroup_v2" "securitygroup" {
+  name        = "${var.cluster_name}-securitygroup"
+  description = "RKE security group"
+}
+
+resource "openstack_networking_secgroup_rule_v2" "securitygroup_rule" {
+  direction         = "ingress"
+  ethertype         = "IPv4"
+  security_group_id = openstack_networking_secgroup_v2.securitygroup.id
+}
diff --git a/openlab/modules/openstack/securitygroup/output.tf b/openlab/modules/openstack/securitygroup/output.tf
new file mode 100644
index 0000000..825f4bf
--- /dev/null
+++ b/openlab/modules/openstack/securitygroup/output.tf
@@ -0,0 +1,3 @@
+output "name" {
+  value = openstack_networking_secgroup_v2.securitygroup.name
+}
\ No newline at end of file
diff --git a/openlab/modules/openstack/securitygroup/variables.tf b/openlab/modules/openstack/securitygroup/variables.tf
new file mode 100644
index 0000000..826fcfb
--- /dev/null
+++ b/openlab/modules/openstack/securitygroup/variables.tf
@@ -0,0 +1,4 @@
+variable "cluster_name" {
+  description = "A name for the cluster"
+  type = string
+}