[INT] various updates to AKS deployment scripts

- Adding support for pluggable post-install scripts. See README.
- Adding pre_install check to validate the correct software is installed before attempting deployment.
- Updating AKS deploy command line to support azure cli 2.0.75.

Issue-ID: INT-1398
Signed-off-by: stark, steven <steven.stark@att.com>
Change-Id: I1ff819e96a3f34c88580e90e8d6f2b79979a7a61

21 files changed, 247 insertions, 36 deletions

diff --git a/deployment/aks/README.md b/deployment/aks/README.md
index 4eb37cbc3..a5be9a904 100644
--- a/deployment/aks/README.md
+++ b/deployment/aks/README.md
@@ -54,6 +54,8 @@ options:
 -f, --no-prompt           executes with no prompt for confirmation
 -n, --no-install          don't install ONAP
 -o, --override            create integration override for robot configuration
+-d, --no-validate         dont validate pre-reqs before executing deployment
+-p, --post-install        execute post-install scripts
 -h, --help                provide brief overview of script
  
 This script deploys a cloud environment in Azure.
@@ -173,6 +175,45 @@ The template used to create the override file is ``./util/integration-override.t
 In ``cloud.conf``, there's a parameter ``OOM_OVERRIDES`` available that's used to provide command line overrides to ``helm deploy``. This uses the standard helm syntax, so if you're using it the value should look like ``OOM_OVERRIDES="--set vid.enabled=false,so.image=abc"``. If you don't want to override anything, just set this value to an empty string.
 
 
+### Pre Install
+
+When you run ``cloud.sh`` it will execute ``pre_install.sh`` first, which checks a few things:
+
+- It checks you have the correct pre-reqs installed. So, it'll make sure you have kubectl, azure cli, helm, etc...
+- It checks that the version of kubernetes in ``cloud.conf`` is available in Azure.
+- It checks the version of azure cli is >= to the baselined version (you can check this version by looking at the top of ``pre_install.sh``). The Azure cli is introduced changes in minor versions that aren't backwards compatible.
+- It checks that the version of kubectl installed is at **MOST** 1 minor version different than the version of kubernetes in ``cloud.conf``.
+
+If you would like to skip ``pre_install.sh`` and run the deployment anyways, pass the flag ``--no-validate`` to ``cloud.sh``, like this:
+
+```
+$ ./cloud.sh --no-validate
+
+```
+
+
+### Post Install
+
+After ONAP is deployed, you have the option of executing an arbitrary set of post-install scripts. This is enabled by passing the ``--post-install`` flag to ``cloud.sh``, like this:
+
+```
+$ ./cloud.sh --post-install
+
+```
+
+These post-install scripts need to be executable from the command line, and will be provided two parameters that they can use to perform their function:
+
+- /path/to/onap.conf : This is created during the deployment, and has various ONAP and OpenStack parameters.
+- /path/to/cloud.conf : this is the same ``cloud.conf`` that's used during the original deployment.
+
+
+Your post-install scripts can disregard these parameters, or source them and use the parameters as-needed.
+
+Included with this repo is one post-install script (``000_bootstrap_onap.sh``)that bootstraps AAI, VID, and SO with cloud and customer details so that ONAP is ready to model and instantiate a VNF.
+
+In order to include other custom post-install scripts, simply put them in the ``post-install`` directory, and make sure to set its mode to executable. They are executed in alphabetical order. 
+
+
 ## Post Deployment
 
 After ONAP and DevStack are deployed, there will be a ``deployment.notes`` file with instructions on how to access the various components. The ``BUILD_DIR`` specified in ``cloud.conf`` will contain a new ssh key, kubeconfig, and other deployment artifacts as well. 
diff --git a/deployment/aks/cloud.sh b/deployment/aks/cloud.sh
index ac471a91b..555ba12cc 100755
--- a/deployment/aks/cloud.sh
+++ b/deployment/aks/cloud.sh
@@ -17,6 +17,8 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
 
 NO_PROMPT=0
 NO_INSTALL=0
+NO_VALIDATE=0
+POST_INSTALL=0
 OVERRIDE=0
 
 OPENSTACK_CLI_POD="os-cli-0"
@@ -38,6 +40,8 @@ while test $# -gt 0; do
       echo "-f, --no-prompt           executes with no prompt for confirmation"
       echo "-n, --no-install          don't install ONAP"
       echo "-o, --override            create integration override for robot configuration"
+      echo "-d, --no-validate         dont validate pre-reqs before executing deployment"
+      echo "-p, --post-install        execute post-install scripts"
       echo "-h, --help                provide brief overview of script"
       echo " "
       echo "This script deploys a cloud environment in Azure."
@@ -62,6 +66,14 @@ while test $# -gt 0; do
       shift
       OVERRIDE=1
       ;;
+    -d|--no-validate)
+      shift
+      NO_VALIDATE=1
+      ;;
+    -p|--post-install)
+      shift
+      POST_INSTALL=1
+      ;;
     *)
       echo "Unknown Argument. Try running with --help ."
       exit 0
@@ -69,6 +81,13 @@ while test $# -gt 0; do
   esac
 done
 
+if [ $NO_VALIDATE = 0 ]; then
+  $DIR/pre_install.sh "$AKS_K8_VERSION" "$LOCATION"
+  if [ $? -ne 0 ]; then
+    exit 1
+  fi
+fi
+
 cat <<EOF
 
 Here are the parameters to be used in this build:
@@ -317,6 +336,7 @@ $DIR/util/create_openstack_cli.sh "$BUILD_DIR/kubeconfig" \
                                   "$BUILD_DIR/openstack_rc" \
                                   "$OPENSTACK_CLI_POD"
 
+
 if [ $OVERRIDE = 1 ]; then
 
 $DIR/util/create_integration_override.sh "$BUILD_DIR" \
@@ -329,48 +349,21 @@ $DIR/util/create_integration_override.sh "$BUILD_DIR" \
 
 fi
 
+
 if [ $NO_INSTALL = 0 ]; then
 
 ### Starting OOM install ###
 echo "Installing ONAP..."
 $DIR/create_onap.sh "$BUILD" \
                     "$BUILD_DIR/kubeconfig" \
-                    "$NFS_PRIVATE_IP" \
                     "$OOM_BRANCH" \
                     "$BUILD_DIR" \
                     "$CHART_VERSION" \
                     "$OOM_OVERRIDES"
 
-### Starting OOM install ###
-echo "Configuring ONAP..."
-
-cat > "$BUILD_DIR/onap.conf" <<EOF
-export CLLI=$CLLI
-export CLOUD_OWNER=$CLOUD_OWNER
-export CLOUD_REGION=$CLOUD_REGION
-export OPENSTACK_IP=$DEVSTACK_PRIVATE_IP
-export OPENSTACK_USER=$OPENSTACK_USER
-export OPENSTACK_PASS=$OPENSTACK_PASS
-export OPENSTACK_TENANT=$OPENSTACK_TENANT
-export OPENSTACK_REGION=$OPENSTACK_REGION
-export CUSTOMER=$CUSTOMER
-export SUBSCRIBER=$SUBSCRIBER
-export SERVICE_TYPE=$SERVICE_TYPE
-export AZ=$AZ
-export OE=$OE
-export LOB=$LOB
-export PLATFORM=$PLATFORM
-export PROJECT=$PROJECT
-export OS_ID=$OS_ID
-export OS_TENANT_ROLE=$OS_TENANT_ROLE
-export OS_KEYSTONE=$OS_KEYSTONE
-export KUBECONFIG=$BUILD_DIR/kubeconfig
-EOF
-
-$DIR/bootstrap_onap.sh "$BUILD_DIR/onap.conf"
-
 fi
 
+
 set +x
 
 cat > "$BUILD_DIR/deployment.notes" <<EOF
@@ -422,3 +415,40 @@ $AKS_PUBLIC_IP_ADDRESS policy.api.simpledemo.onap.org
 EOF
 
 cat "$BUILD_DIR/deployment.notes"
+
+
+if [ $POST_INSTALL = 1 ]; then
+
+echo "Executing post installation scripts..."
+sleep 3
+
+cat > "$BUILD_DIR/onap.conf" <<EOF
+export CLLI=$CLLI
+export CLOUD_OWNER=$CLOUD_OWNER
+export CLOUD_REGION=$CLOUD_REGION
+export OPENSTACK_IP=$DEVSTACK_PRIVATE_IP
+export OPENSTACK_USER=$OPENSTACK_USER
+export OPENSTACK_PASS=$OPENSTACK_PASS
+export OPENSTACK_TENANT=$OPENSTACK_TENANT
+export OPENSTACK_REGION=$OPENSTACK_REGION
+export CUSTOMER=$CUSTOMER
+export SUBSCRIBER=$SUBSCRIBER
+export SERVICE_TYPE=$SERVICE_TYPE
+export AZ=$AZ
+export OE=$OE
+export LOB=$LOB
+export PLATFORM=$PLATFORM
+export PROJECT=$PROJECT
+export OS_ID=$OS_ID
+export OS_TENANT_ROLE=$OS_TENANT_ROLE
+export OS_KEYSTONE=$OS_KEYSTONE
+export KUBECONFIG=$BUILD_DIR/kubeconfig
+export NFS_PRIVATE_IP=$NFS_PRIVATE_IP
+export DEVSTACK_PRIVATE_IP=$DEVSTACK_PRIVATE_IP
+export PRIVATE_KEY=$PRIVATE_KEY
+EOF
+
+$DIR/post_install.sh "$BUILD_DIR/onap.conf" "$DIR/cloud.conf"
+
+fi
+
diff --git a/deployment/aks/create_aks.sh b/deployment/aks/create_aks.sh
index 23f6ae341..cd4680dff 100755
--- a/deployment/aks/create_aks.sh
+++ b/deployment/aks/create_aks.sh
@@ -216,7 +216,9 @@ az aks create --name "$AKS_NAME" \
               --dns-service-ip "$AKS_DNS_IP" \
               --admin-username "$AKS_ADMIN_USER" \
               --ssh-key-value "$PUBLIC_KEY" \
-              --vnet-subnet-id "$AKS_SUBNET_ID"
+              --vnet-subnet-id "$AKS_SUBNET_ID" \
+              --vm-set-type "AvailabilitySet" \
+              --load-balancer-sku "basic"
 echo ""
 
 AKS_MANAGEMENT_RESOURCE_GROUP_NAME=`az group list --query "[?starts_with(name, 'MC_${AKS_RESOURCE_GROUP_NAME}')].name | [0]" --output tsv`
diff --git a/deployment/aks/create_onap.sh b/deployment/aks/create_onap.sh
index 4abddd64f..4f3a673bb 100755
--- a/deployment/aks/create_onap.sh
+++ b/deployment/aks/create_onap.sh
@@ -17,11 +17,10 @@ set -x
 
 BUILD_NAME=$1
 KUBECONFIG=$2
-NFS_SERVER_IP=$3
-OOM_BRANCH=$4
-BUILD_DIR=$5
-CHART_VERSION=$6
-OOM_OVERRIDES=$7
+OOM_BRANCH=$3
+BUILD_DIR=$4
+CHART_VERSION=$5
+OOM_OVERRIDES=$6
 
 pushd .
 
diff --git a/deployment/aks/bootstrap_onap.sh b/deployment/aks/post-install/000_bootstrap_onap.sh
index 99661a9b6..9904a54d2 100755
--- a/deployment/aks/bootstrap_onap.sh
+++ b/deployment/aks/post-install/000_bootstrap_onap.sh
@@ -24,7 +24,7 @@ fi
 
 . $CONF
 
-kubectl create configmap onap-bootstrap --from-file=$DIR/post-install/ --from-file=kubeconfig=$KUBECONFIG --from-file=onap.conf=$CONF
+kubectl create configmap onap-bootstrap --from-file=$DIR/bootstrap/ --from-file=kubeconfig=$KUBECONFIG --from-file=onap.conf=$CONF
 
 cat  <<EOF | kubectl apply -f -
 apiVersion: v1
diff --git a/deployment/aks/post-install/bootstrap.sh b/deployment/aks/post-install/bootstrap/bootstrap.sh
index 646a502e2..646a502e2 100755
--- a/deployment/aks/post-install/bootstrap.sh
+++ b/deployment/aks/post-install/bootstrap/bootstrap.sh
diff --git a/deployment/aks/post-install/create_az.sh b/deployment/aks/post-install/bootstrap/create_az.sh
index 1e0407c12..1e0407c12 100755
--- a/deployment/aks/post-install/create_az.sh
+++ b/deployment/aks/post-install/bootstrap/create_az.sh
diff --git a/deployment/aks/post-install/create_clli.sh b/deployment/aks/post-install/bootstrap/create_clli.sh
index dec8b8b06..dec8b8b06 100755
--- a/deployment/aks/post-install/create_clli.sh
+++ b/deployment/aks/post-install/bootstrap/create_clli.sh
diff --git a/deployment/aks/post-install/create_cloud_region.sh b/deployment/aks/post-install/bootstrap/create_cloud_region.sh
index dd83cb387..dd83cb387 100755
--- a/deployment/aks/post-install/create_cloud_region.sh
+++ b/deployment/aks/post-install/bootstrap/create_cloud_region.sh
diff --git a/deployment/aks/post-install/create_cloud_region_relationship.sh b/deployment/aks/post-install/bootstrap/create_cloud_region_relationship.sh
index 7acdea9d2..7acdea9d2 100755
--- a/deployment/aks/post-install/create_cloud_region_relationship.sh
+++ b/deployment/aks/post-install/bootstrap/create_cloud_region_relationship.sh
diff --git a/deployment/aks/post-install/create_cloud_region_subscriber_relationship.sh b/deployment/aks/post-install/bootstrap/create_cloud_region_subscriber_relationship.sh
index d37b5653f..d37b5653f 100755
--- a/deployment/aks/post-install/create_cloud_region_subscriber_relationship.sh
+++ b/deployment/aks/post-install/bootstrap/create_cloud_region_subscriber_relationship.sh
diff --git a/deployment/aks/post-install/create_cloud_site.sh b/deployment/aks/post-install/bootstrap/create_cloud_site.sh
index 0253b0e16..0253b0e16 100755
--- a/deployment/aks/post-install/create_cloud_site.sh
+++ b/deployment/aks/post-install/bootstrap/create_cloud_site.sh
diff --git a/deployment/aks/post-install/create_customer.sh b/deployment/aks/post-install/bootstrap/create_customer.sh
index 0beebe97a..0beebe97a 100755
--- a/deployment/aks/post-install/create_customer.sh
+++ b/deployment/aks/post-install/bootstrap/create_customer.sh
diff --git a/deployment/aks/post-install/create_lob.sh b/deployment/aks/post-install/bootstrap/create_lob.sh
index a14367ddf..a14367ddf 100755
--- a/deployment/aks/post-install/create_lob.sh
+++ b/deployment/aks/post-install/bootstrap/create_lob.sh
diff --git a/deployment/aks/post-install/create_owning_entity.sh b/deployment/aks/post-install/bootstrap/create_owning_entity.sh
index eba395591..eba395591 100755
--- a/deployment/aks/post-install/create_owning_entity.sh
+++ b/deployment/aks/post-install/bootstrap/create_owning_entity.sh
diff --git a/deployment/aks/post-install/create_platform.sh b/deployment/aks/post-install/bootstrap/create_platform.sh
index 46dc1b4e6..46dc1b4e6 100755
--- a/deployment/aks/post-install/create_platform.sh
+++ b/deployment/aks/post-install/bootstrap/create_platform.sh
diff --git a/deployment/aks/post-install/create_project.sh b/deployment/aks/post-install/bootstrap/create_project.sh
index 7b455e3b0..7b455e3b0 100755
--- a/deployment/aks/post-install/create_project.sh
+++ b/deployment/aks/post-install/bootstrap/create_project.sh
diff --git a/deployment/aks/post-install/create_service_type.sh b/deployment/aks/post-install/bootstrap/create_service_type.sh
index c712c53f6..c712c53f6 100755
--- a/deployment/aks/post-install/create_service_type.sh
+++ b/deployment/aks/post-install/bootstrap/create_service_type.sh
diff --git a/deployment/aks/post-install/create_subscription.sh b/deployment/aks/post-install/bootstrap/create_subscription.sh
index 4bb2e6412..4bb2e6412 100755
--- a/deployment/aks/post-install/create_subscription.sh
+++ b/deployment/aks/post-install/bootstrap/create_subscription.sh
diff --git a/deployment/aks/post_install.sh b/deployment/aks/post_install.sh
new file mode 100755
index 000000000..c48f9d5c7
--- /dev/null
+++ b/deployment/aks/post_install.sh
@@ -0,0 +1,31 @@
+#!/bin/bash
+# Copyright 2019 AT&T Intellectual Property. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
+
+ONAP_CONF=$1
+CLOUD_CONF=$2
+
+pushd .
+
+cd $DIR/post-install
+
+for filename in *; do
+  if [ -f $filename ]; then
+    ./$filename "$ONAP_CONF" "$CLOUD_CONF"
+  fi
+done
+
+popd
\ No newline at end of file
diff --git a/deployment/aks/pre_install.sh b/deployment/aks/pre_install.sh
new file mode 100755
index 000000000..6bbbab9b6
--- /dev/null
+++ b/deployment/aks/pre_install.sh
@@ -0,0 +1,108 @@
+#!/bin/bash
+# Copyright 2019 AT&T Intellectual Property. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+KUBE_VERSION=$1
+LOCATION=$2
+
+COMMANDS="kubectl helm make java az"
+
+CLI_MAJOR="2"
+CLI_MINOR="0"
+CLI_INC="75"
+
+function check_requirement() {
+  req=$1
+
+  command -v $1
+  if [ $? -ne 0 ]; then
+    echo "$1 was not found on machine. Please install it before proceeding."
+    exit 1
+  fi
+}
+
+echo "Checking requirements are installed..."
+
+for req in $COMMANDS; do
+  check_requirement $req
+done
+
+echo "Checking K8 version is available in Azure..."
+if [ -z "$KUBE_VERSION" ]; then
+  echo "K8 version not provided in cloud.conf."
+  echo "Update cloud.conf with the desired version."
+  exit 1
+fi
+
+if [ -z "$LOCATION" ]; then
+  echo "Location not provided in cloud.conf."
+  echo "Update cloud.conf with the desired location."
+  exit 1
+fi
+
+supported_k8_versions=`az aks get-versions --location $LOCATION --output json --query 'orchestrators[].orchestratorVersion'`
+echo $supported_k8_versions | grep -q $KUBE_VERSION
+if [ $? -ne 0 ]; then
+  echo "K8 version $KUBE_VERSION is not supported in location $LOCATION"
+  echo "The supported versions are $supported_k8_versions."
+  echo "Update cloud.conf with a supported version."
+  exit 1
+fi
+
+echo "Checking Azure CLI version..."
+installed_cli_version=`az --version | grep -e "^azure-cli"  | awk '{print $2}'`
+installed_major=`echo $installed_cli_version | cut -d "." -f 1`
+installed_minor=`echo $installed_cli_version | cut -d "." -f 2`
+installed_inc=`echo $installed_cli_version | cut -d "." -f 3`
+
+if [ $installed_major -lt $CLI_MAJOR ]; then
+  echo "Azure cli version is out of date."
+  echo "Major version required is $CLI_MAJOR but $installed_major is installed."
+  exit 1
+fi
+
+if [ $installed_minor -lt $CLI_MINOR ]; then
+  echo "Azure cli version is out of date."
+  echo "Minor version required is $CLI_INC but $installed_inc is installed."
+  exit 1
+fi
+
+if [ $installed_inc -lt $CLI_INC ]; then
+  echo "Azure cli version is out of date."
+  echo "Incremental version required is $CLI_INC but $installed_inc is installed."
+  exit 1
+fi
+
+echo "Checking kubectl version is compatible with the K8 version..."
+kubectl_version=`kubectl version --client --short | awk '{print $3}'`
+kubectl_major=`echo $kubectl_version | cut -d "." -f 1 | sed 's/v//'`
+kubectl_minor=`echo $kubectl_version | cut -d "." -f 2`
+k8_major=`echo $KUBE_VERSION | cut -d "." -f 1`
+k8_minor=`echo $KUBE_VERSION | cut -d "." -f 2`
+
+if [ $kubectl_major -ne $k8_major ]; then
+  echo "kubectl major version $kubectl_major doesn't equal kubernetes server version $k8_major"
+  exit 1
+fi
+
+minor_difference=`echo "$(($kubectl_minor-$k8_minor))"`
+minor_abs_diff=`echo $minor_difference | tr -d -`
+if [ $minor_abs_diff -gt 1 ]; then
+  echo "The difference between k8 minor version $KUBE_VERSION and kubectl minor version $kubectl_version is greater than 1"
+  echo "Kubernetes supports kubectl within 1 minor version."
+  exit 1
+fi
+
+echo "All requirements satisfied..."
+sleep 1