diff options
65 files changed, 1395 insertions, 251 deletions
@@ -35,3 +35,6 @@ Steps: * Run the plugin: * `cd k8s/deployments && ./start.sh` +# Troubleshooting + +* Logs can be set as "warn" or "info" in the config.json file of the orchestrator directory. By default the log level is set as "warn". diff --git a/build/Dockerfile b/build/Dockerfile index ea9b0014..d7ba3c35 100644 --- a/build/Dockerfile +++ b/build/Dockerfile @@ -30,7 +30,13 @@ COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/rsync/rsyn COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/ovnaction/ovnaction ./ COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/clm/clm ./ COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/build/entrypoint ./ -COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/orchestrator/api/json-schemas ./json-schemas +COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/orchestrator/json-schemas ./json-schemas +COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/ncm/json-schemas ./json-schemas +COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/clm/json-schemas ./json-schemas +COPY --chown=emco --from=0 /go/src/github.com/onap/multicloud-k8s/src/ovnaction/json-schemas ./json-schemas + + + USER emco diff --git a/deployments/kubernetes/Readme.md b/deployments/kubernetes/Readme.md index b5a99a65..ba319837 100644 --- a/deployments/kubernetes/Readme.md +++ b/deployments/kubernetes/Readme.md @@ -1,23 +1,41 @@ -# Steps for running v2 API microservices +# Steps for running EMCO API microservices ### Steps to install packages -**1. Create namespace for ONAP4K8s v2 Microservices** +**1. Create namespace for EMCO Microservices** `$ kubectl create namespace onap4k8s` -**2. Create Databases used by ONAP4K8s v2 Microservices for Etcd and Mongo** +**2. Create Databases used by EMCO Microservices for Etcd and Mongo** `$ kubectl apply -f onap4k8sdb.yaml -n onap4k8s` -**3. create ONAP4K8s v2 Microservices** +**3. create EMCO Microservices** `$ kubectl apply -f onap4k8s.yaml -n onap4k8s` ### Steps to cleanup packages -**1. Cleanup ONAP4K8s v2 Microservies** +**1. Cleanup EMCO Microservies** `$ kubectl delete -f onap4k8s.yaml -n onap4k8s` -**2. Cleanup v2 Microservices for Etcd and Mongo** +**2. Cleanup EMCO Microservices for Etcd and Mongo** `$ kubectl delete -f onap4k8sdb.yaml -n onap4k8s` + +# Steps for running the monitor microservice in clusters + +The EMCO microservices utilize the monitor microservice to collect +status information from clusters to which EMCO deploys applications. +It must be installed in each cluster to which EMCO deploys applications. + +### Steps to install monitor in a cluster + +**1. Instantiate the monitor resources + + $ kubectl apply -f monitor-deploy.yaml + +### Steps to cleanup monitor in a cluster + +**1. Cleanup the monitor resources + + $ kubectl delete -f monitor-deploy.yaml diff --git a/deployments/kubernetes/monitor-deploy.yaml b/deployments/kubernetes/monitor-deploy.yaml new file mode 100644 index 00000000..29bb1eb6 --- /dev/null +++ b/deployments/kubernetes/monitor-deploy.yaml @@ -0,0 +1,314 @@ +apiVersion: apiextensions.k8s.io/v1beta1 +kind: CustomResourceDefinition +metadata: + name: resourcebundlestates.k8splugin.io +spec: + group: k8splugin.io + names: + kind: ResourceBundleState + listKind: ResourceBundleStateList + plural: resourcebundlestates + singular: resourcebundlestate + scope: Namespaced + subresources: + status: {} + validation: + openAPIV3Schema: + properties: + apiVersion: + description: 'APIVersion defines the versioned schema of this representation + of an object. Servers should convert recognized schemas to the latest + internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#resources' + type: string + kind: + description: 'Kind is a string value representing the REST resource this + object represents. Servers may infer this from the endpoint the client + submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/api-conventions.md#types-kinds' + type: string + metadata: + type: object + spec: + properties: + selector: + type: object + required: + - selector + type: object + status: + properties: + podStatuses: + items: + type: object + type: array + ready: + type: boolean + resourceCount: + format: int32 + type: integer + serviceStatuses: + items: + type: object + type: array + configMapStatuses: + items: + type: object + type: array + deploymentStatuses: + items: + type: object + type: array + secretStatuses: + items: + type: object + type: array + daemonSetStatuses: + items: + type: object + type: array + ingressStatuses: + items: + type: object + type: array + jobStatuses: + items: + type: object + type: array + statefulSetStatuses: + items: + type: object + type: array + required: + - ready + - resourceCount + - podStatuses + - serviceStatuses + - configMapStatuses + - deploymentStatuses + - secretStatuses + - daemonSetStatuses + - ingressStatuses + - jobStatuses + - statefulSetStatuses + type: object + version: v1alpha1 + versions: + - name: v1alpha1 + served: true + storage: true + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: Role +metadata: + creationTimestamp: null + name: monitor +rules: +- apiGroups: + - "" + resources: + - pods + - services + - endpoints + - persistentvolumeclaims + - events + - configmaps + - secrets + verbs: + - '*' +- apiGroups: + - apps + resources: + - deployments + - daemonsets + - replicasets + - statefulsets + verbs: + - '*' +- apiGroups: + - monitoring.coreos.com + resources: + - servicemonitors + verbs: + - get + - create +- apiGroups: + - apps + resourceNames: + - monitor + resources: + - deployments/finalizers + verbs: + - update +- apiGroups: + - "" + resources: + - pods + verbs: + - get +- apiGroups: + - apps + resources: + - replicasets + verbs: + - get +- apiGroups: + - k8splugin.io + resources: + - '*' + verbs: + - '*' +- apiGroups: + - batch + resources: + - '*' + verbs: + - '*' +- apiGroups: + - extensions + resources: + - '*' + verbs: + - '*' + +--- +apiVersion: rbac.authorization.k8s.io/v1 +kind: ClusterRole +metadata: + creationTimestamp: null + name: monitor +rules: +- apiGroups: + - "" + resources: + - pods + - services + - endpoints + - persistentvolumeclaims + - events + - configmaps + - secrets + verbs: + - '*' +- apiGroups: + - apps + resources: + - deployments + - daemonsets + - replicasets + - statefulsets + verbs: + - '*' +- apiGroups: + - monitoring.coreos.com + resources: + - servicemonitors + verbs: + - get + - create +- apiGroups: + - apps + resourceNames: + - monitor + resources: + - deployments/finalizers + verbs: + - update +- apiGroups: + - "" + resources: + - pods + verbs: + - get +- apiGroups: + - apps + resources: + - replicasets + verbs: + - get +- apiGroups: + - k8splugin.io + resources: + - '*' + verbs: + - '*' +- apiGroups: + - batch + resources: + - '*' + verbs: + - '*' +- apiGroups: + - extensions + resources: + - '*' + verbs: + - '*' + +--- +kind: RoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: monitor +subjects: +- kind: ServiceAccount + name: monitor +roleRef: + kind: Role + name: monitor + apiGroup: rbac.authorization.k8s.io + +--- +kind: ClusterRoleBinding +apiVersion: rbac.authorization.k8s.io/v1 +metadata: + name: monitor +subjects: +- kind: ServiceAccount + name: monitor + namespace: default +roleRef: + kind: ClusterRole + name: monitor + apiGroup: rbac.authorization.k8s.io + +--- +apiVersion: v1 +kind: ServiceAccount +metadata: + name: monitor + +--- +apiVersion: apps/v1 +kind: Deployment +metadata: + name: monitor + labels: + "emco/deployment-id": "monitor" +spec: + replicas: 1 + selector: + matchLabels: + "emco/deployment-id": "monitor" + template: + metadata: + labels: + "emco/deployment-id": "monitor" + spec: + serviceAccountName: monitor + containers: + - name: monitor + # Replace this with the built image name + image: emcov2/monitor:latest + command: + - /opt/monitor/monitor + imagePullPolicy: IfNotPresent + env: + - name: WATCH_NAMESPACE + value: "" + - name: POD_NAME + valueFrom: + fieldRef: + fieldPath: metadata.name + - name: OPERATOR_NAME + value: "monitor" + diff --git a/docs/emco_apis.yaml b/docs/emco_apis.yaml index 163e3d69..5f9c5510 100644 --- a/docs/emco_apis.yaml +++ b/docs/emco_apis.yaml @@ -3210,10 +3210,10 @@ components: maxLength: 128 example: "provider-1" defaultGateway: - type: boolean + type: "string" description: Is this interface default gateway maxLength: 128 - example: false + example: "false" ipAddress: type: string description: Name of the network diff --git a/kud/tests/plugin_fw.sh b/kud/tests/plugin_fw.sh index 9c981773..28df372c 100755 --- a/kud/tests/plugin_fw.sh +++ b/kud/tests/plugin_fw.sh @@ -63,7 +63,7 @@ payload="$(cat <<EOF "rb-name": "${rb_name}", "rb-version": "${rb_version}", "profile-name": "${profile_name}", - "release-name": "${release_name}", + "release-name": "dummy", "namespace": "${namespace}" } EOF @@ -92,6 +92,7 @@ payload="$(cat <<EOF "rb-name": "${rb_name}", "rb-version": "${rb_version}", "profile-name": "${profile_name}", + "release-name": "${release_name}", "cloud-region": "${cloud_region_id}", "labels": {"testCaseName": "plugin_fw.sh"}, "override-values": {"global.onapPrivateNetworkName": "onap-private-net-test"} @@ -120,6 +121,8 @@ response="$(call_api "${base_url}/instance/${vnf_id}")" echo "$response" print_msg "Assert additional label has been assigned to rb instance" test "$(jq -r .request.labels.testCaseName <<< "${response}")" == plugin_fw.sh +print_msg "Assert ReleaseName has been correctly overriden" +test "$(jq -r .request.release-name <<< "${response}")" == "${release_name}" #Teardown print_msg "Deleting VNF Instance" diff --git a/src/clm/api/clusterhandler.go b/src/clm/api/clusterhandler.go index 84dd3230..75fcc561 100644 --- a/src/clm/api/clusterhandler.go +++ b/src/clm/api/clusterhandler.go @@ -28,10 +28,16 @@ import ( "net/textproto" clusterPkg "github.com/onap/multicloud-k8s/src/clm/pkg/cluster" + "github.com/onap/multicloud-k8s/src/orchestrator/pkg/infra/validation" "github.com/gorilla/mux" ) +var cpJSONFile string = "json-schemas/metadata.json" +var ckvJSONFile string = "json-schemas/cluster-kv.json" +var clJSONFile string = "json-schemas/cluster-label.json" + + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type clusterHandler struct { @@ -55,6 +61,12 @@ func (h clusterHandler) createClusterProviderHandler(w http.ResponseWriter, r *h return } + err, httpError := validation.ValidateJsonSchemaData(cpJSONFile, p) + if err != nil { + http.Error(w, err.Error(), httpError) + return + } + // Name is required. if p.Metadata.Name == "" { http.Error(w, "Missing name in POST request", http.StatusBadRequest) @@ -148,6 +160,12 @@ func (h clusterHandler) createClusterHandler(w http.ResponseWriter, r *http.Requ return } + err, httpError := validation.ValidateJsonSchemaData(cpJSONFile, p) + if err != nil { + http.Error(w, err.Error(), httpError) + return + } + //Read the file section and ignore the header file, _, err := r.FormFile("file") if err != nil { @@ -333,6 +351,12 @@ func (h clusterHandler) createClusterLabelHandler(w http.ResponseWriter, r *http err := json.NewDecoder(r.Body).Decode(&p) + err, httpError := validation.ValidateJsonSchemaData(clJSONFile, p) + if err != nil { + http.Error(w, err.Error(), httpError) + return + } + // LabelName is required. if p.LabelName == "" { http.Error(w, "Missing label name in POST request", http.StatusBadRequest) @@ -413,6 +437,13 @@ func (h clusterHandler) createClusterKvPairsHandler(w http.ResponseWriter, r *ht err := json.NewDecoder(r.Body).Decode(&p) + // Verify JSON Body + err, httpError := validation.ValidateJsonSchemaData(ckvJSONFile, p) + if err != nil { + http.Error(w, err.Error(), httpError) + return + } + // KvPairsName is required. if p.Metadata.Name == "" { http.Error(w, "Missing Key Value pair name in POST request", http.StatusBadRequest) diff --git a/src/clm/api/clusterhandler_test.go b/src/clm/api/clusterhandler_test.go index 076718df..a866835b 100644 --- a/src/clm/api/clusterhandler_test.go +++ b/src/clm/api/clusterhandler_test.go @@ -186,6 +186,12 @@ func (m *mockClusterManager) DeleteClusterKvPairs(provider, clusterName, kvpair return m.Err } +func init() { + cpJSONFile = "../json-schemas/metadata.json" + ckvJSONFile = "../json-schemas/cluster-kv.json" + clJSONFile = "../json-schemas/cluster-label.json" +} + func TestClusterProviderCreateHandler(t *testing.T) { testCases := []struct { label string diff --git a/src/orchestrator/api/json-schemas/cluster-kv.json b/src/clm/json-schemas/cluster-kv.json index c7013bab..c7013bab 100644 --- a/src/orchestrator/api/json-schemas/cluster-kv.json +++ b/src/clm/json-schemas/cluster-kv.json diff --git a/src/orchestrator/api/json-schemas/cluster-label.json b/src/clm/json-schemas/cluster-label.json index 22267b3d..22267b3d 100644 --- a/src/orchestrator/api/json-schemas/cluster-label.json +++ b/src/clm/json-schemas/cluster-label.json diff --git a/src/orchestrator/api/json-schemas/metadata.json b/src/clm/json-schemas/metadata.json index 960545ee..960545ee 100644 --- a/src/orchestrator/api/json-schemas/metadata.json +++ b/src/clm/json-schemas/metadata.json diff --git a/src/clm/pkg/cluster/cluster.go b/src/clm/pkg/cluster/cluster.go index 26a9d6df..fb8768d6 100644 --- a/src/clm/pkg/cluster/cluster.go +++ b/src/clm/pkg/cluster/cluster.go @@ -19,6 +19,7 @@ package cluster import ( "time" + "github.com/onap/multicloud-k8s/src/orchestrator/pkg/appcontext" "github.com/onap/multicloud-k8s/src/orchestrator/pkg/infra/db" mtypes "github.com/onap/multicloud-k8s/src/orchestrator/pkg/module/types" "github.com/onap/multicloud-k8s/src/orchestrator/pkg/state" @@ -423,6 +424,14 @@ func (v *ClusterClient) DeleteCluster(provider, name string) error { // remove the app contexts associated with this cluster if stateVal == state.StateEnum.Terminated { + // Verify that the appcontext has completed terminating + ctxid := state.GetLastContextIdFromStateInfo(s) + acStatus, err := state.GetAppContextStatus(ctxid) + if err == nil && + !(acStatus.Status == appcontext.AppContextStatusEnum.Terminated || acStatus.Status == appcontext.AppContextStatusEnum.TerminateFailed) { + return pkgerrors.Errorf("Network intents for cluster have not completed terminating " + name) + } + for _, id := range state.GetContextIdsFromStateInfo(s) { context, err := state.GetAppContextFromId(id) if err != nil { diff --git a/src/k8splugin/Makefile b/src/k8splugin/Makefile index 77196afa..a1cda8dd 100644 --- a/src/k8splugin/Makefile +++ b/src/k8splugin/Makefile @@ -8,13 +8,14 @@ ############################################################################## export GO111MODULE=on +GO ?= go .PHONY: plugins all: clean plugins CGO_ENABLED=1 GOOS=linux GOARCH=amd64 - @go build -tags netgo -o ./k8plugin ./cmd/main.go + @$(GO) build -tags netgo -o ./k8plugin ./cmd/main.go # The following is done this way as each patch on CI runs build and each merge runs deploy. So for build we don't need to build binary and hence # no need to create a static binary with additional flags. However, for generating binary, additional build flags are necessary. This if used with @@ -25,14 +26,14 @@ deploy: build .PHONY: test test: clean - @go build -race -buildmode=plugin -o ./mock_files/mock_plugins/mockplugin.so ./mock_files/mock_plugins/mockplugin.go - @go test -race ./... + @$(GO) build -race -buildmode=plugin -o ./mock_files/mock_plugins/mockplugin.so ./mock_files/mock_plugins/mockplugin.go + @$(GO) test -race ./... format: - @go fmt ./... + @$(GO) fmt ./... plugins: - @find plugins -maxdepth 1 -type d -not -path plugins -exec sh -c "ls {}/plugin.go | xargs go build -buildmode=plugin -tags netgo -o $(basename {}).so" \; + @find plugins -maxdepth 1 -type d -not -path plugins -exec sh -c "ls {}/plugin.go | xargs $(GO) build -buildmode=plugin -tags netgo -o $(basename {}).so" \; clean: @find . -name "*so" -delete @@ -40,5 +41,5 @@ clean: .PHONY: cover cover: - @go test -race ./... -coverprofile=coverage.out - @go tool cover -html=coverage.out -o coverage.html + @$(GO) test -race ./... -coverprofile=coverage.out + @$(GO) tool cover -html=coverage.out -o coverage.html diff --git a/src/k8splugin/api/brokerhandler.go b/src/k8splugin/api/brokerhandler.go index c98e1c48..05f94b7d 100644 --- a/src/k8splugin/api/brokerhandler.go +++ b/src/k8splugin/api/brokerhandler.go @@ -153,12 +153,20 @@ func (b brokerInstanceHandler) createHandler(w http.ResponseWriter, r *http.Requ return } + releaseName, ok := directives["k8s-rb-instance-release-name"] + if !ok { + //Release name is not mandatory argument, but we're not using profile's default + //as it could conflict if someone wanted to instantiate single profile multiple times + releaseName = req.VFModuleID + } + // Setup the resource parameters for making the request var instReq app.InstanceRequest instReq.RBName = req.VFModuleModelInvariantID instReq.RBVersion = req.VFModuleModelVersionID instReq.ProfileName = profileName instReq.CloudRegion = cloudRegion + instReq.ReleaseName = releaseName instReq.Labels = map[string]string{ "stack-name": req.TemplateData.StackName, } diff --git a/src/k8splugin/internal/app/instance.go b/src/k8splugin/internal/app/instance.go index ad677644..220c82da 100644 --- a/src/k8splugin/internal/app/instance.go +++ b/src/k8splugin/internal/app/instance.go @@ -35,6 +35,7 @@ type InstanceRequest struct { RBName string `json:"rb-name"` RBVersion string `json:"rb-version"` ProfileName string `json:"profile-name"` + ReleaseName string `json:"release-name"` CloudRegion string `json:"cloud-region"` Labels map[string]string `json:"labels"` OverrideValues map[string]string `json:"override-values"` @@ -42,19 +43,21 @@ type InstanceRequest struct { // InstanceResponse contains the response from instantiation type InstanceResponse struct { - ID string `json:"id"` - Request InstanceRequest `json:"request"` - Namespace string `json:"namespace"` - Resources []helm.KubernetesResource `json:"resources"` + ID string `json:"id"` + Request InstanceRequest `json:"request"` + Namespace string `json:"namespace"` + ReleaseName string `json:"release-name"` + Resources []helm.KubernetesResource `json:"resources"` } // InstanceMiniResponse contains the response from instantiation // It does NOT include the created resources. // Use the regular GET to get the created resources for a particular instance type InstanceMiniResponse struct { - ID string `json:"id"` - Request InstanceRequest `json:"request"` - Namespace string `json:"namespace"` + ID string `json:"id"` + Request InstanceRequest `json:"request"` + ReleaseName string `json:"release-name"` + Namespace string `json:"namespace"` } // PodStatus defines the observed state of ResourceBundleState @@ -144,7 +147,7 @@ func (v *InstanceClient) Create(i InstanceRequest) (InstanceResponse, error) { } //Execute the kubernetes create command - sortedTemplates, err := rb.NewProfileClient().Resolve(i.RBName, i.RBVersion, i.ProfileName, overrideValues) + sortedTemplates, releaseName, err := rb.NewProfileClient().Resolve(i.RBName, i.RBVersion, i.ProfileName, overrideValues, i.ReleaseName) if err != nil { return InstanceResponse{}, pkgerrors.Wrap(err, "Error resolving helm charts") } @@ -165,10 +168,11 @@ func (v *InstanceClient) Create(i InstanceRequest) (InstanceResponse, error) { //Compose the return response resp := InstanceResponse{ - ID: id, - Request: i, - Namespace: profile.Namespace, - Resources: createdResources, + ID: id, + Request: i, + Namespace: profile.Namespace, + ReleaseName: releaseName, + Resources: createdResources, } key := InstanceKey{ @@ -252,9 +256,10 @@ func (v *InstanceClient) List(rbname, rbversion, profilename string) ([]Instance } miniresp := InstanceMiniResponse{ - ID: resp.ID, - Request: resp.Request, - Namespace: resp.Namespace, + ID: resp.ID, + Request: resp.Request, + Namespace: resp.Namespace, + ReleaseName: resp.ReleaseName, } //Filter based on the accepted keys diff --git a/src/k8splugin/internal/rb/profile.go b/src/k8splugin/internal/rb/profile.go index 6efa23b8..f8b07abf 100644 --- a/src/k8splugin/internal/rb/profile.go +++ b/src/k8splugin/internal/rb/profile.go @@ -268,50 +268,51 @@ func (v *ProfileClient) Download(rbName, rbVersion, prName string) ([]byte, erro } //Resolve returns the path where the helm chart merged with -//configuration overrides resides. +//configuration overrides resides and final ReleaseName picked for instantiation func (v *ProfileClient) Resolve(rbName string, rbVersion string, - profileName string, values []string) ([]helm.KubernetesResourceTemplate, error) { + profileName string, values []string, overrideReleaseName string) ([]helm.KubernetesResourceTemplate, string, error) { var sortedTemplates []helm.KubernetesResourceTemplate + var finalReleaseName string //Download and process the profile first //If everything seems okay, then download the definition prData, err := v.Download(rbName, rbVersion, profileName) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Downloading Profile") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Downloading Profile") } prPath, err := ExtractTarBall(bytes.NewBuffer(prData)) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Extracting Profile Content") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Extracting Profile Content") } prYamlClient, err := ProcessProfileYaml(prPath, v.manifestName) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Processing Profile Manifest") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Processing Profile Manifest") } definitionClient := NewDefinitionClient() definition, err := definitionClient.Get(rbName, rbVersion) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Getting Definition Metadata") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Getting Definition Metadata") } defData, err := definitionClient.Download(rbName, rbVersion) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Downloading Definition") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Downloading Definition") } chartBasePath, err := ExtractTarBall(bytes.NewBuffer(defData)) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Extracting Definition Charts") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Extracting Definition Charts") } //Get the definition ID and download its contents profile, err := v.Get(rbName, rbVersion, profileName) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Getting Profile") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Getting Profile") } //Copy the profile configresources to the chart locations @@ -321,22 +322,28 @@ func (v *ProfileClient) Resolve(rbName string, rbVersion string, // chartpath: chart/config/resources/config.yaml err = prYamlClient.CopyConfigurationOverrides(chartBasePath) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Copying configresources to chart") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Copying configresources to chart") + } + + if overrideReleaseName == "" { + finalReleaseName = profile.ReleaseName + } else { + finalReleaseName = overrideReleaseName } helmClient := helm.NewTemplateClient(profile.KubernetesVersion, profile.Namespace, - profile.ReleaseName) + finalReleaseName) chartPath := filepath.Join(chartBasePath, definition.ChartName) sortedTemplates, err = helmClient.GenerateKubernetesArtifacts(chartPath, []string{prYamlClient.GetValues()}, values) if err != nil { - return sortedTemplates, pkgerrors.Wrap(err, "Generate final k8s yaml") + return sortedTemplates, finalReleaseName, pkgerrors.Wrap(err, "Generate final k8s yaml") } - return sortedTemplates, nil + return sortedTemplates, finalReleaseName, nil } // Returns an empty profile with the following contents diff --git a/src/k8splugin/internal/rb/profile_test.go b/src/k8splugin/internal/rb/profile_test.go index 26b0161d..a434e5a1 100644 --- a/src/k8splugin/internal/rb/profile_test.go +++ b/src/k8splugin/internal/rb/profile_test.go @@ -18,12 +18,14 @@ package rb import ( "bytes" + "os" "reflect" "sort" "strings" "testing" "github.com/onap/multicloud-k8s/src/k8splugin/internal/db" + "github.com/onap/multicloud-k8s/src/k8splugin/internal/helm" pkgerrors "github.com/pkg/errors" ) @@ -597,19 +599,142 @@ func TestDownloadProfile(t *testing.T) { } func TestResolveProfile(t *testing.T) { + profileContent := []byte("H4sICLmjT1wAA3Byb2ZpbGUudGFyAO1Y32/bNhD2s/6Kg/KyYZZsy" + + "78K78lLMsxY5gRxmqIYhoKWaJsYJWokZdfo+r/vSFmunCZNBtQJ1vF7sXX36e54vDN5T" + + "knGFlTpcEtS3jgO2ohBr2c/EXc/29Gg1+h0e1F32Ol1B1Gj3Ymifr8B7SPFc4BCaSIBG" + + "lII/SXeY/r/KIIg8NZUKiayEaw7nt7mdOQBrAkvqBqBL1ArWULflRJbJz4SYpEt2FJSJ" + + "QoZ21cAAlgwTnOiVyPQWFQLwVuqmCdMthKac7FNaVZWmqWjkRWRuuSvScF1gFZVwYOEr" + + "luapjknaOazd186Z98S7tver+3j0f5v1/q/18f+7w56bdf/zwFF5ZqV/WtbH6YioVdCa" + + "hRkJEVBVSFBvUNRmyNpesgwors0lmkqM8KNzRG8iqLIWN45GUGv57l+fkFUP9PH9GF6f" + + "IgH+kP9b76b/o+GUb9r5J1O1I0a0D9mUBX+5/1/55g+io9/sf+DnuF1sA4Gbv+fA1++p" + + "n0dH4+c/92oPaztv+n/fn84dOf/c+AETkW+lWy50hC1O69gguc1R6HEw5xoHAuaKIq9E" + + "+8ELvCikCmaQJElVIJeURjnJMaPnaYJt+UoAVHYhu8Mwd+p/O9/RAtbUUBKtnj+aygUR" + + "RNM2ZkB6PuY5hpvCzhY4L2fkSymsGF6Zd3sjIRo4u3OhJhrgmyC/ByfFnUeEG0DLrHSO" + + "h+1WpvNJiQ23FDIZYuXVNW6mJyeT2fnAYZsX3qdcaoUSPpXwSQudr4FkmNEMZljnJxsQ" + + "EggOPmgTgsT8UYyzbJlE5RY6A2RFK0kTGnJ5oU+SFcVH666TsCEkQz88QwmMx9+Gs8ms" + + "ybaeDO5+eXy9Q28GV9fj6c3k/MZXF7D6eX0bHIzuZzi088wnr6FXyfTsyZQTBa6oe9za" + + "eLHIJlJJE1M1maUHgSwEGVAKqcxW7AY15UtC7KksDS3uQyXAzmVKVNmOxWGl6AVzlKmb" + + "VGozxcVeh7J2W01S2LOVAsHyj9ZlozgbP+74qVUk4RoMtrfMD98wCzGvEiwXHD3U5GFi" + + "4Jzo/QhhI8fd0yFu3c/fa/d8zmZU67KsRRDefCt/Qu7YdQSw1PzNTS3W1QGnyRVef+N5" + + "YHDKZao/4MP/ju/siEpp0SVQYbX5UNlxxJwizCFyzuMWXkLNySzIyZs4wBrTpXE23I62" + + "wlPRZHp0qJCC7EWslxpSnS8uqgt/YmLr2btnZXaDhnwA4NPzueT8lEt126AyExPY44rS" + + "YA1bJPl15JgRaEdM9CKv/f1YDHdE5e1cYVFdiUwoduDJC+5mBMe5nstbndCF9Zfxakpa" + + "1aNP2LK/Xffhuc3fTNfUYlfzH8a/h97qhmVaikNPi2+nItq8exGtLA+SdW9rgUvUvqbq" + + "YkDi6mRXNk/V1pUxy0uYsI1S+meU+XsPo2kJLnMOKZGy4J6Xt3XgZuHTayEKv3XZLjy+" + + "yJ66WPQwcHBwcHBwcHBwcHBwcHBwcHhm8Q/mTHqWgAoAAA=") + defContent := []byte("H4sICEetS1wAA3ZhdWx0LWNvbnN1bC1kZXYudGFyAO0c7XLbNjK/+R" + + "QYujdJehatb+V4czPnOmnPk9bO2Gk7nbaTgUhIxpgiGAK0o3P9QPca92S3C5AU9GXZiax" + + "c7rA/LJEAFovdxX4AK1/RIlGNSKSySBoxuzp4sn1oAgx6Pf0JsPipv7c63XZ70O61W4Mn" + + "zVZ7MGg9Ib1HoGUJCqloTsiTXAh1V79N7V8oXC3K/+iC5iqY0kmytTlQwP1ud538W51Wf" + + "0H+3QF8kObWKLgD/s/lv0eORDbN+fhCkXaz9YIcp4ol8DLPRE4VF+k+vIq8PW+PfM8jlk" + + "oWkyKNWU7UBSOHGY3go2zZJz+xXMIY0g6a5Bl28Msm//lfAcNUFGRCpyQVihSSAQouyYg" + + "njLAPEcsU4SmJxCRLOE0jRq65utDTlEgCQPFLiUIMFYXeFPpn8DSy+xGqNMEGLpTKwoOD" + + "6+vrgGpyA5GPDxLTVR58f3z06uT8VQNI1oN+TBMmJcnZ+4LnsNjhlNAMKIroEOhM6DURO" + + "aHjnEGbEkjxdc4VT8f7RIqRuqY5Aywxlyrnw0LNsauiD1ZtdwCG0ZT4h+fk+Nwn3xyeH5" + + "/vA46fj9/+4/THt+Tnw7Ozw5O3x6/OyekZOTo9eXn89vj0BJ6+JYcnv5DXxycv9wkDZsE" + + "07EOWI/1AJEdGshi5ds7YHAEjYQiSGYv4iEewrnRc0DEjY3HF8hSWQzKWT7hEcUogLwYs" + + "CZ9wpZVCLi8q8Dya8VIBQnLV8mImo5xnSj9ru4IMS2iRRhfkJzQ8iJcY44OMBPtDJiJmX" + + "konDFAs2CbAn9X4m8Ffgp53VT2C9EB+n3s3fXmwZP+vaFIwuVUHsMH+d1vd3oL977X6TW" + + "f/dwHO/jv7vzX7v/epAHN8l4ghTdApjPi4MCoIjmGEdkoGW5hirCcIPQJaGLM3Ildvcjb" + + "iH0LSabbhbYYqLBUDBQzJzS2sqpK/JoVPgEue/os4jOUMq88WuKE+vNZmtfRgYTNooXPK" + + "iiR5IwDRNCSHyTWdSsQ9SugY9YilWr9iNizGY2R/Y25aWWSwIVWtlp7u+EoPikMyoolk2" + + "xHAoTXr40nBYLY46OFWlSwH7QuJygumXyRi/C5hVww4fHzy7enqTjFV9F3M4dXTA4PtAF" + + "891Y3INWmwl6aAvOg1m9YLGZJGy6uFZuZQYP2MhBFsGhFoHOMmC4G+iCYXQqrQQgqTUnV" + + "RSt8sQysUEF32UFG2AtnTX8Pw9/BFu9l8WjeqRMLSJIrZXrF5824C81+W79HoGAGRtJgM" + + "YXOCUeQpuDfQZOnlTIv1SBQpKCasF7X/nCUsgqUaRaejEU+5mlZqn+ViyBZ0IKM5xGYK9" + + "oiX8CtYk9TMxXGcJi9ZQqfnDIbEsJ5W02wnLuL5d3skZUCTpPkUVb9cDakQlhNfXzDQe6" + + "bQtpJhzuhlJniqpEago0XcKrBOKcjrF2BRBZPpU9wi6NLBwaTwLQPJAVpcBfoLlsNoVu0" + + "awzfAHPOPWYhnm4olvKBPIikm7IxFCeWTauefMaQDWmmELPgBpIAvafwzeBF2CqigTfJ/" + + "wtv2dxy+T1Bib7RCHcQgbpajcjfSkawaz4uhaZcTaW8Az8Otwg1xapoBypPS5KH1W4qxP" + + "bNbTlY1AOPBLdAEB8MOamtlrwxoSLpdzwMx5SUX2bxd+txBjoO1sBT/KwZRA1UQGG1tjo" + + "ef/3UH/YE7/9sF3CH/GDyGmE5Y+qnHgZvyv2Z7MC9/sC6dvsv/dgF7Lv9z+d9jnP8Bz+T" + + "BVcu75CnEAS9rW+JB9EgxOgnrGOTmBrgYJUUM6gLSn4g0GEGuhI0+CcjtbdlTgvRWd69b" + + "6/4JHbKkjPuBlLWj6gEQ5OMJpe4YmEsQDISgsTF7U6n3HwTDaZiP+H/2if/Or3DkEFBTa" + + "YgMzsxDhUd3ABEBC8cLPc5NnIadUCJIdhmvS9PxJ3MqZwfxBqOsIniNfUJVdPG9tfR7Lr" + + "4y+iUWS0I6e5lDeG9+3osf1XLLLMvE6PVcDZNuh8S3mKBfBdpxARa/nmutMq2gS+N4YyX" + + "kFn5zQBDM0nUQd5VZVX2sRgsrzkdR3X/1NXn+vm+SVfiCztX/fZYh2mkpLrRevAmoLXrK" + + "ID6wQ3B7VpNm/IA6MYfRThyYig50rqr4hNV9Kp6tasGs6DRNplWWtFEg5TH+AyXSGFJIa" + + "cC67Ewyhk6QCMyTqntIxqwCvYjFngVxzWX/OxGIPdUKcldhwHMKPb31rjqrWCDoc4clDn" + + "YEd8T/ld355KugDfF/u99avP8ZdNz9/27Axf8u/n+s+38T+pex7f3i/tLmPHrov5Rf/Le" + + "F/+a4dkUUiA0GWx2oNGb8XOxdnedW89/c8BFh71dj9avTYZ80yv7ZQ4LR2XHwcsw2f9dm" + + "xW1+p9lG/q2YoxozI75BQLJsM3XswzJ1YObHTD0outYTpnE1Wy6UiEQSkrdHb5ZSr3smR" + + "XdqyGew/0v+X2+DLR7+Pvmo8982dHfnvzuAdfI32rsdNXi4/Hu9rpP/TmCD/LdSDbwh/m" + + "+1+93F+L876Ln4fxdgx////hemAANyOIlFJPfJNyyBTICmELa5+N/F/59Y/6sNSn3SLDU" + + "JOljSCgNsFJp+Y3/KCmBjhVyV7+PBBvu/lWrgjec/gyX7P+i2nP3fBTj77+z/F1P/S4w5" + + "glmpIhGwbAisTPWZihYUluqCyspiaKzYdsuF9/A3LCmwCKQOcxdpgXtBV+Vm5lQjr5rh+" + + "YqlyjTiUkB9ysJFrdPG1dXFmSQvUs1ybASF0pLBM4HLF5Kgh1S6bnFVvbIphsQ7MzyTEp" + + "IrkXMmzQWyeZyGJGUfCtkJREozVP6whWG3GVtXP4LnZdGlR2ZvziwMQkyAGLv12FwE1s8" + + "NPT40LlqjToSpZNYXbR6pnm20pqAxYAmVikdBJGbdSvxDRsEdoY3Ab2Ev6FXozarxvg/4" + + "jBd+eCa2osYa+1YKpK/g9JUXQYMOuzDXZzhTWMeI5VjJGesBsOvr6k5VXbPpnysBedpky" + + "YVacXN1vr5YU6P92GpvQubrvfUV4Dbs/wb/v5VqwIfn/4Net+Py/13AveX/rj5oD1T2sG" + + "BwU/7f73cW6v/anb7L/3cCNzcHX3suCHRB4LaCwK8Pbm89T6sVIWdMiuTKzFrbDx0/ATP" + + "1bz+oSfgD8vaCzX6/UneVxQhCHfz9gayRVHKuB0JbGQwi2TmPY5YSPrJ+ZPKMjQO93Do0" + + "fA44C4krRFQjkSTiGp90hBl6+latuiJKZXlrRcJqBns5JvgzC8cbI1gFBESrLijNvVXZx" + + "1Qt2VdABt3SrI0SL4Pgo7HtW6L72/9ZPPlQB7DB/nc6ve6i/e93Xf3HTsDZf2f/d2f/a9" + + "NtDoMX8tZpAEPQD2gjrMmzCp/LPsg2nXiDSEoruo+23AisXH9tpScM7FnK5aQaFsyb9rI" + + "6wUJv2/jKSi/SqUnDkwbdIOcwznqdVmgsjGY+nUeuRY6KgHwvW4YUUsy13mU2buZewPXd" + + "QY1V25DlPFUj4v9J+neNqPBi7YU1erHy1lrCevbWuHRZhe3WVirNEnMki3KG/0fkkqXr1" + + "WVp3iPcxKUKhHOHI9hicndoy0P915R7UCmvRQ7JdvWtLLHnSUgYfpBnQl9u0OT5PeQTGN" + + "LtKOArbCXh35aKRmyplqUjun+Ey4D+d69z1l9TCf3rYpu/+wZJoFtmHWkBRhY6zjQiRKU" + + "wfZEl5deKFeQPMux3WRrNcFRDb36D0b/5IXziQNz28GRe7v/mVxjsd5qb9gskp36+vfVL" + + "Tq0nx6zULKMm7VEDp/8RuH/8V5eKPTD733z/01zO/6G/i/92AS7+c/HfbuO/MuN/KkllU" + + "bzSj1de6pqDyg3ZLMk3Y59ZDh5f1PEJxDuSqecYDhyCqcdhqFditFxRqmkox0kM4Rbiwb" + + "mOq0LBsgN5xllgiHuuqasCAL3sVx8yWhJS9dcIddhYnlusjRjmSqCtWEFjsHy5XaW8ki3" + + "Lpw0Gx8q1/oFXCuAz+x39lU/O9ckL8Rv+oh/93CbLwRbhYef/H+H8n2z2/612e8H/w5P7" + + "/287Aef/nf9/PP9vOcIF97/e/y06vnv7uwe4sJpAyJfBugFR1Sz4w6ApeV/QBDgCUrFv5" + + "bUFxFgFp6EoM6pwNlyQhIAloqjOUgCBr4shMJBhnaPx/JwlMXAwZ4Z/Rm205j8D3UIGvQ" + + "RZQl9kOgrk+XoOzX68tJ3wYJb0N/RJ0NzPUr5y4YEDBw4cOHDgwIEDBw4cOHDgwIEDBw4" + + "cOHDgwIEDB18K/AcxEDJDAHgAAA==") testCases := []struct { label string rbname, rbversion, prname string + releaseName string expected map[string][]string + expectedReleaseName string expectedError string mockdb *db.MockDB }{ { - label: "Resolve Resource Bundle Profile", - rbname: "testresourcebundle", - rbversion: "v1", - prname: "profile1", - expected: map[string][]string{}, + label: "Resolve Resource Bundle Profile with override release name", + rbname: "testresourcebundle", + rbversion: "v1", + prname: "profile1", + expectedReleaseName: "testprofilereleasename", + expected: map[string][]string{}, + mockdb: &db.MockDB{ + Items: map[string]map[string][]byte{ + ProfileKey{RBName: "testresourcebundle", RBVersion: "v1", + ProfileName: "profile1"}.String(): { + "profilemetadata": []byte( + "{\"profile-name\":\"profile1\"," + + "\"release-name\":\"testprofilereleasename\"," + + "\"namespace\":\"testnamespace\"," + + "\"rb-name\":\"testresourcebundle\"," + + "\"rb-version\":\"v1\"," + + "\"kubernetesversion\":\"1.12.3\"}"), + // base64 encoding of vagrant/tests/vnfs/testrb/helm/profile + "profilecontent": profileContent, + }, + DefinitionKey{RBName: "testresourcebundle", RBVersion: "v1"}.String(): { + "defmetadata": []byte( + "{\"rb-name\":\"testresourcebundle\"," + + "\"rb-version\":\"v1\"," + + "\"chart-name\":\"vault-consul-dev\"," + + "\"description\":\"testresourcebundle\"}"), + // base64 encoding of vagrant/tests/vnfs/testrb/helm/vault-consul-dev + "defcontent": defContent, + }, + }, + }, + }, + { + label: "Resolve Resource Bundle Profile", + rbname: "testresourcebundle", + rbversion: "v1", + prname: "profile1", + releaseName: "overwritereleasename", + expectedReleaseName: "overwritereleasename", + expected: map[string][]string{}, mockdb: &db.MockDB{ Items: map[string]map[string][]byte{ ProfileKey{RBName: "testresourcebundle", RBVersion: "v1", @@ -622,29 +747,7 @@ func TestResolveProfile(t *testing.T) { "\"rb-version\":\"v1\"," + "\"kubernetesversion\":\"1.12.3\"}"), // base64 encoding of vagrant/tests/vnfs/testrb/helm/profile - "profilecontent": []byte("H4sICLmjT1wAA3Byb2ZpbGUudGFyAO1Y32/bNhD2s/6Kg/KyYZZsy" + - "78K78lLMsxY5gRxmqIYhoKWaJsYJWokZdfo+r/vSFmunCZNBtQJ1vF7sXX36e54vDN5T" + - "knGFlTpcEtS3jgO2ohBr2c/EXc/29Gg1+h0e1F32Ol1B1Gj3Ymifr8B7SPFc4BCaSIBG" + - "lII/SXeY/r/KIIg8NZUKiayEaw7nt7mdOQBrAkvqBqBL1ArWULflRJbJz4SYpEt2FJSJ" + - "QoZ21cAAlgwTnOiVyPQWFQLwVuqmCdMthKac7FNaVZWmqWjkRWRuuSvScF1gFZVwYOEr" + - "luapjknaOazd186Z98S7tver+3j0f5v1/q/18f+7w56bdf/zwFF5ZqV/WtbH6YioVdCa" + - "hRkJEVBVSFBvUNRmyNpesgwors0lmkqM8KNzRG8iqLIWN45GUGv57l+fkFUP9PH9GF6f" + - "IgH+kP9b76b/o+GUb9r5J1O1I0a0D9mUBX+5/1/55g+io9/sf+DnuF1sA4Gbv+fA1++p" + - "n0dH4+c/92oPaztv+n/fn84dOf/c+AETkW+lWy50hC1O69gguc1R6HEw5xoHAuaKIq9E" + - "+8ELvCikCmaQJElVIJeURjnJMaPnaYJt+UoAVHYhu8Mwd+p/O9/RAtbUUBKtnj+aygUR" + - "RNM2ZkB6PuY5hpvCzhY4L2fkSymsGF6Zd3sjIRo4u3OhJhrgmyC/ByfFnUeEG0DLrHSO" + - "h+1WpvNJiQ23FDIZYuXVNW6mJyeT2fnAYZsX3qdcaoUSPpXwSQudr4FkmNEMZljnJxsQ" + - "EggOPmgTgsT8UYyzbJlE5RY6A2RFK0kTGnJ5oU+SFcVH666TsCEkQz88QwmMx9+Gs8ms" + - "ybaeDO5+eXy9Q28GV9fj6c3k/MZXF7D6eX0bHIzuZzi088wnr6FXyfTsyZQTBa6oe9za" + - "eLHIJlJJE1M1maUHgSwEGVAKqcxW7AY15UtC7KksDS3uQyXAzmVKVNmOxWGl6AVzlKmb" + - "VGozxcVeh7J2W01S2LOVAsHyj9ZlozgbP+74qVUk4RoMtrfMD98wCzGvEiwXHD3U5GFi" + - "4Jzo/QhhI8fd0yFu3c/fa/d8zmZU67KsRRDefCt/Qu7YdQSw1PzNTS3W1QGnyRVef+N5" + - "YHDKZao/4MP/ju/siEpp0SVQYbX5UNlxxJwizCFyzuMWXkLNySzIyZs4wBrTpXE23I62" + - "wlPRZHp0qJCC7EWslxpSnS8uqgt/YmLr2btnZXaDhnwA4NPzueT8lEt126AyExPY44rS" + - "YA1bJPl15JgRaEdM9CKv/f1YDHdE5e1cYVFdiUwoduDJC+5mBMe5nstbndCF9Zfxakpa" + - "1aNP2LK/Xffhuc3fTNfUYlfzH8a/h97qhmVaikNPi2+nItq8exGtLA+SdW9rgUvUvqbq" + - "YkDi6mRXNk/V1pUxy0uYsI1S+meU+XsPo2kJLnMOKZGy4J6Xt3XgZuHTayEKv3XZLjy+" + - "yJ66WPQwcHBwcHBwcHBwcHBwcHBwcHhm8Q/mTHqWgAoAAA="), + "profilecontent": profileContent, }, DefinitionKey{RBName: "testresourcebundle", RBVersion: "v1"}.String(): { "defmetadata": []byte( @@ -653,81 +756,26 @@ func TestResolveProfile(t *testing.T) { "\"chart-name\":\"vault-consul-dev\"," + "\"description\":\"testresourcebundle\"}"), // base64 encoding of vagrant/tests/vnfs/testrb/helm/vault-consul-dev - "defcontent": []byte("H4sICEetS1wAA3ZhdWx0LWNvbnN1bC1kZXYudGFyAO0c7XLbNjK/+R" + - "QYujdJehatb+V4czPnOmnPk9bO2Gk7nbaTgUhIxpgiGAK0o3P9QPca92S3C5AU9GXZiax" + - "c7rA/LJEAFovdxX4AK1/RIlGNSKSySBoxuzp4sn1oAgx6Pf0JsPipv7c63XZ70O61W4Mn" + - "zVZ7MGg9Ib1HoGUJCqloTsiTXAh1V79N7V8oXC3K/+iC5iqY0kmytTlQwP1ud538W51Wf" + - "0H+3QF8kObWKLgD/s/lv0eORDbN+fhCkXaz9YIcp4ol8DLPRE4VF+k+vIq8PW+PfM8jlk" + - "oWkyKNWU7UBSOHGY3go2zZJz+xXMIY0g6a5Bl28Msm//lfAcNUFGRCpyQVihSSAQouyYg" + - "njLAPEcsU4SmJxCRLOE0jRq65utDTlEgCQPFLiUIMFYXeFPpn8DSy+xGqNMEGLpTKwoOD" + - "6+vrgGpyA5GPDxLTVR58f3z06uT8VQNI1oN+TBMmJcnZ+4LnsNjhlNAMKIroEOhM6DURO" + - "aHjnEGbEkjxdc4VT8f7RIqRuqY5Aywxlyrnw0LNsauiD1ZtdwCG0ZT4h+fk+Nwn3xyeH5" + - "/vA46fj9/+4/THt+Tnw7Ozw5O3x6/OyekZOTo9eXn89vj0BJ6+JYcnv5DXxycv9wkDZsE" + - "07EOWI/1AJEdGshi5ds7YHAEjYQiSGYv4iEewrnRc0DEjY3HF8hSWQzKWT7hEcUogLwYs" + - "CZ9wpZVCLi8q8Dya8VIBQnLV8mImo5xnSj9ru4IMS2iRRhfkJzQ8iJcY44OMBPtDJiJmX" + - "konDFAs2CbAn9X4m8Ffgp53VT2C9EB+n3s3fXmwZP+vaFIwuVUHsMH+d1vd3oL977X6TW" + - "f/dwHO/jv7vzX7v/epAHN8l4ghTdApjPi4MCoIjmGEdkoGW5hirCcIPQJaGLM3Ildvcjb" + - "iH0LSabbhbYYqLBUDBQzJzS2sqpK/JoVPgEue/os4jOUMq88WuKE+vNZmtfRgYTNooXPK" + - "iiR5IwDRNCSHyTWdSsQ9SugY9YilWr9iNizGY2R/Y25aWWSwIVWtlp7u+EoPikMyoolk2" + - "xHAoTXr40nBYLY46OFWlSwH7QuJygumXyRi/C5hVww4fHzy7enqTjFV9F3M4dXTA4PtAF" + - "891Y3INWmwl6aAvOg1m9YLGZJGy6uFZuZQYP2MhBFsGhFoHOMmC4G+iCYXQqrQQgqTUnV" + - "RSt8sQysUEF32UFG2AtnTX8Pw9/BFu9l8WjeqRMLSJIrZXrF5824C81+W79HoGAGRtJgM" + - "YXOCUeQpuDfQZOnlTIv1SBQpKCasF7X/nCUsgqUaRaejEU+5mlZqn+ViyBZ0IKM5xGYK9" + - "oiX8CtYk9TMxXGcJi9ZQqfnDIbEsJ5W02wnLuL5d3skZUCTpPkUVb9cDakQlhNfXzDQe6" + - "bQtpJhzuhlJniqpEago0XcKrBOKcjrF2BRBZPpU9wi6NLBwaTwLQPJAVpcBfoLlsNoVu0" + - "awzfAHPOPWYhnm4olvKBPIikm7IxFCeWTauefMaQDWmmELPgBpIAvafwzeBF2CqigTfJ/" + - "wtv2dxy+T1Bib7RCHcQgbpajcjfSkawaz4uhaZcTaW8Az8Otwg1xapoBypPS5KH1W4qxP" + - "bNbTlY1AOPBLdAEB8MOamtlrwxoSLpdzwMx5SUX2bxd+txBjoO1sBT/KwZRA1UQGG1tjo" + - "ef/3UH/YE7/9sF3CH/GDyGmE5Y+qnHgZvyv2Z7MC9/sC6dvsv/dgF7Lv9z+d9jnP8Bz+T" + - "BVcu75CnEAS9rW+JB9EgxOgnrGOTmBrgYJUUM6gLSn4g0GEGuhI0+CcjtbdlTgvRWd69b" + - "6/4JHbKkjPuBlLWj6gEQ5OMJpe4YmEsQDISgsTF7U6n3HwTDaZiP+H/2if/Or3DkEFBTa" + - "YgMzsxDhUd3ABEBC8cLPc5NnIadUCJIdhmvS9PxJ3MqZwfxBqOsIniNfUJVdPG9tfR7Lr" + - "4y+iUWS0I6e5lDeG9+3osf1XLLLMvE6PVcDZNuh8S3mKBfBdpxARa/nmutMq2gS+N4YyX" + - "kFn5zQBDM0nUQd5VZVX2sRgsrzkdR3X/1NXn+vm+SVfiCztX/fZYh2mkpLrRevAmoLXrK" + - "ID6wQ3B7VpNm/IA6MYfRThyYig50rqr4hNV9Kp6tasGs6DRNplWWtFEg5TH+AyXSGFJIa" + - "cC67Ewyhk6QCMyTqntIxqwCvYjFngVxzWX/OxGIPdUKcldhwHMKPb31rjqrWCDoc4clDn" + - "YEd8T/ld355KugDfF/u99avP8ZdNz9/27Axf8u/n+s+38T+pex7f3i/tLmPHrov5Rf/Le" + - "F/+a4dkUUiA0GWx2oNGb8XOxdnedW89/c8BFh71dj9avTYZ80yv7ZQ4LR2XHwcsw2f9dm" + - "xW1+p9lG/q2YoxozI75BQLJsM3XswzJ1YObHTD0outYTpnE1Wy6UiEQSkrdHb5ZSr3smR" + - "XdqyGew/0v+X2+DLR7+Pvmo8982dHfnvzuAdfI32rsdNXi4/Hu9rpP/TmCD/LdSDbwh/m" + - "+1+93F+L876Ln4fxdgx////hemAANyOIlFJPfJNyyBTICmELa5+N/F/59Y/6sNSn3SLDU" + - "JOljSCgNsFJp+Y3/KCmBjhVyV7+PBBvu/lWrgjec/gyX7P+i2nP3fBTj77+z/F1P/S4w5" + - "glmpIhGwbAisTPWZihYUluqCyspiaKzYdsuF9/A3LCmwCKQOcxdpgXtBV+Vm5lQjr5rh+" + - "YqlyjTiUkB9ysJFrdPG1dXFmSQvUs1ybASF0pLBM4HLF5Kgh1S6bnFVvbIphsQ7MzyTEp" + - "IrkXMmzQWyeZyGJGUfCtkJREozVP6whWG3GVtXP4LnZdGlR2ZvziwMQkyAGLv12FwE1s8" + - "NPT40LlqjToSpZNYXbR6pnm20pqAxYAmVikdBJGbdSvxDRsEdoY3Ab2Ev6FXozarxvg/4" + - "jBd+eCa2osYa+1YKpK/g9JUXQYMOuzDXZzhTWMeI5VjJGesBsOvr6k5VXbPpnysBedpky" + - "YVacXN1vr5YU6P92GpvQubrvfUV4Dbs/wb/v5VqwIfn/4Net+Py/13AveX/rj5oD1T2sG" + - "BwU/7f73cW6v/anb7L/3cCNzcHX3suCHRB4LaCwK8Pbm89T6sVIWdMiuTKzFrbDx0/ATP" + - "1bz+oSfgD8vaCzX6/UneVxQhCHfz9gayRVHKuB0JbGQwi2TmPY5YSPrJ+ZPKMjQO93Do0" + - "fA44C4krRFQjkSTiGp90hBl6+latuiJKZXlrRcJqBns5JvgzC8cbI1gFBESrLijNvVXZx" + - "1Qt2VdABt3SrI0SL4Pgo7HtW6L72/9ZPPlQB7DB/nc6ve6i/e93Xf3HTsDZf2f/d2f/a9" + - "NtDoMX8tZpAEPQD2gjrMmzCp/LPsg2nXiDSEoruo+23AisXH9tpScM7FnK5aQaFsyb9rI" + - "6wUJv2/jKSi/SqUnDkwbdIOcwznqdVmgsjGY+nUeuRY6KgHwvW4YUUsy13mU2buZewPXd" + - "QY1V25DlPFUj4v9J+neNqPBi7YU1erHy1lrCevbWuHRZhe3WVirNEnMki3KG/0fkkqXr1" + - "WVp3iPcxKUKhHOHI9hicndoy0P915R7UCmvRQ7JdvWtLLHnSUgYfpBnQl9u0OT5PeQTGN" + - "LtKOArbCXh35aKRmyplqUjun+Ey4D+d69z1l9TCf3rYpu/+wZJoFtmHWkBRhY6zjQiRKU" + - "wfZEl5deKFeQPMux3WRrNcFRDb36D0b/5IXziQNz28GRe7v/mVxjsd5qb9gskp36+vfVL" + - "Tq0nx6zULKMm7VEDp/8RuH/8V5eKPTD733z/01zO/6G/i/92AS7+c/HfbuO/MuN/KkllU" + - "bzSj1de6pqDyg3ZLMk3Y59ZDh5f1PEJxDuSqecYDhyCqcdhqFditFxRqmkox0kM4Rbiwb" + - "mOq0LBsgN5xllgiHuuqasCAL3sVx8yWhJS9dcIddhYnlusjRjmSqCtWEFjsHy5XaW8ki3" + - "Lpw0Gx8q1/oFXCuAz+x39lU/O9ckL8Rv+oh/93CbLwRbhYef/H+H8n2z2/612e8H/w5P7" + - "/287Aef/nf9/PP9vOcIF97/e/y06vnv7uwe4sJpAyJfBugFR1Sz4w6ApeV/QBDgCUrFv5" + - "bUFxFgFp6EoM6pwNlyQhIAloqjOUgCBr4shMJBhnaPx/JwlMXAwZ4Z/Rm205j8D3UIGvQ" + - "RZQl9kOgrk+XoOzX68tJ3wYJb0N/RJ0NzPUr5y4YEDBw4cOHDgwIEDBw4cOHDgwIEDBw4" + - "cOHDgwIEDB18K/AcxEDJDAHgAAA=="), + "defcontent": defContent, }, }, }, }, } + cleanup := func(krts []helm.KubernetesResourceTemplate) { + for _, krt := range krts { + os.RemoveAll(krt.FilePath) + } + } + for _, testCase := range testCases { t.Run(testCase.label, func(t *testing.T) { db.DBconn = testCase.mockdb impl := NewProfileClient() - data, err := impl.Resolve(testCase.rbname, testCase.rbversion, testCase.prname, - []string{}) + data, releaseName, err := impl.Resolve(testCase.rbname, + testCase.rbversion, testCase.prname, []string{}, testCase.releaseName) + defer cleanup(data) if err != nil { if testCase.expectedError == "" { t.Errorf("Resolve returned an unexpected error %s", err) @@ -736,7 +784,10 @@ func TestResolveProfile(t *testing.T) { t.Errorf("Resolve returned an unexpected error %s", err) } } - t.Log(data) + if testCase.expectedReleaseName != releaseName { + t.Errorf("Resolve returned unexpected release name '%s', expected '%s'", + releaseName, testCase.expectedReleaseName) + } }) } } diff --git a/src/ncm/api/networkhandler.go b/src/ncm/api/networkhandler.go index ed266697..6f305be1 100644 --- a/src/ncm/api/networkhandler.go +++ b/src/ncm/api/networkhandler.go @@ -30,6 +30,9 @@ import ( "github.com/gorilla/mux" ) +var vnJSONFile string = "json-schemas/virtual-network.json" + + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type networkHandler struct { @@ -86,6 +89,12 @@ func (h networkHandler) createNetworkHandler(w http.ResponseWriter, r *http.Requ return } + err, httpError := validation.ValidateJsonSchemaData(vnJSONFile, p) +if err != nil { + http.Error(w, err.Error(), httpError) + return +} + // Name is required. if p.Metadata.Name == "" { http.Error(w, "Missing name in POST request", http.StatusBadRequest) diff --git a/src/ncm/api/providernethandler.go b/src/ncm/api/providernethandler.go index 66a41a4d..48d5d7ea 100644 --- a/src/ncm/api/providernethandler.go +++ b/src/ncm/api/providernethandler.go @@ -31,6 +31,8 @@ import ( "github.com/gorilla/mux" ) +var pnetJSONFile string = "json-schemas/provider-network.json" + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type providernetHandler struct { @@ -139,6 +141,12 @@ func (h providernetHandler) createProviderNetHandler(w http.ResponseWriter, r *h return } + err, httpError := validation.ValidateJsonSchemaData(pnetJSONFile, p) +if err != nil { + http.Error(w, err.Error(), httpError) + return +} + // Name is required. if p.Metadata.Name == "" { http.Error(w, "Missing name in POST request", http.StatusBadRequest) diff --git a/src/orchestrator/api/json-schemas/provider-network.json b/src/ncm/json-schemas/provider-network.json index 0aef0304..0aef0304 100644 --- a/src/orchestrator/api/json-schemas/provider-network.json +++ b/src/ncm/json-schemas/provider-network.json diff --git a/src/orchestrator/api/json-schemas/virtual-network.json b/src/ncm/json-schemas/virtual-network.json index f2bc9d3d..f2bc9d3d 100644 --- a/src/orchestrator/api/json-schemas/virtual-network.json +++ b/src/ncm/json-schemas/virtual-network.json diff --git a/src/orchestrator/api/add_intents_handler.go b/src/orchestrator/api/add_intents_handler.go index 21a33739..66f3839a 100644 --- a/src/orchestrator/api/add_intents_handler.go +++ b/src/orchestrator/api/add_intents_handler.go @@ -27,6 +27,8 @@ import ( "github.com/gorilla/mux" ) +var addIntentJSONFile string = "json-schemas/deployment-intent.json" + type intentHandler struct { client moduleLib.IntentManager } @@ -46,9 +48,8 @@ func (h intentHandler) addIntentHandler(w http.ResponseWriter, r *http.Request) return } - jsonFile := "json-schemas/deployment-intent.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, i) + err, httpError := validation.ValidateJsonSchemaData(addIntentJSONFile, i) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/app_intent_handler.go b/src/orchestrator/api/app_intent_handler.go index a51b9b92..1d48f8a6 100644 --- a/src/orchestrator/api/app_intent_handler.go +++ b/src/orchestrator/api/app_intent_handler.go @@ -26,6 +26,8 @@ import ( moduleLib "github.com/onap/multicloud-k8s/src/orchestrator/pkg/module" ) +var appIntentJSONFile string = "json-schemas/generic-placement-intent-app.json" + /* Used to store backend implementation objects Also simplifies mocking for unit testing purposes */ @@ -48,9 +50,8 @@ func (h appIntentHandler) createAppIntentHandler(w http.ResponseWriter, r *http. return } - jsonFile := "json-schemas/generic-placement-intent-app.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, a) + err, httpError := validation.ValidateJsonSchemaData(appIntentJSONFile, a) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/app_profilehandler.go b/src/orchestrator/api/app_profilehandler.go index 2fa0f26d..58cd2145 100644 --- a/src/orchestrator/api/app_profilehandler.go +++ b/src/orchestrator/api/app_profilehandler.go @@ -34,6 +34,8 @@ import ( pkgerrors "github.com/pkg/errors" ) +var appProfileJSONFile string = "json-schemas/metadata.json" + /* Used to store backend implementation objects Also simplifies mocking for unit testing purposes */ @@ -73,9 +75,8 @@ func (h appProfileHandler) createAppProfileHandler(w http.ResponseWriter, r *htt return } - jsonFile := "json-schemas/metadata.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, ap) + err, httpError := validation.ValidateJsonSchemaData(appProfileJSONFile, ap) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/apphandler.go b/src/orchestrator/api/apphandler.go index 7d901a8f..7429f5c2 100644 --- a/src/orchestrator/api/apphandler.go +++ b/src/orchestrator/api/apphandler.go @@ -33,6 +33,8 @@ import ( "github.com/gorilla/mux" ) +var appJSONFile string = "json-schemas/metadata.json" + // appHandler to store backend implementations objects // Also simplifies mocking for unit testing purposes type appHandler struct { @@ -70,9 +72,8 @@ func (h appHandler) createAppHandler(w http.ResponseWriter, r *http.Request) { return } - jsonFile := "json-schemas/metadata.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, a) + err, httpError := validation.ValidateJsonSchemaData(appJSONFile, a) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/composite_app_handler.go b/src/orchestrator/api/composite_app_handler.go index 1e69c353..d531b28d 100644 --- a/src/orchestrator/api/composite_app_handler.go +++ b/src/orchestrator/api/composite_app_handler.go @@ -26,6 +26,8 @@ import ( moduleLib "github.com/onap/multicloud-k8s/src/orchestrator/pkg/module" ) +var caJSONFile string = "json-schemas/composite-app.json" + // compositeAppHandler to store backend implementations objects // Also simplifies mocking for unit testing purposes type compositeAppHandler struct { @@ -47,9 +49,8 @@ func (h compositeAppHandler) createHandler(w http.ResponseWriter, r *http.Reques http.Error(w, err.Error(), http.StatusUnprocessableEntity) return } - jsonFile := "json-schemas/composite-app.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, c) + err, httpError := validation.ValidateJsonSchemaData(caJSONFile, c) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/composite_profilehandler.go b/src/orchestrator/api/composite_profilehandler.go index 5a3fb19c..de1c8f2d 100644 --- a/src/orchestrator/api/composite_profilehandler.go +++ b/src/orchestrator/api/composite_profilehandler.go @@ -27,6 +27,8 @@ import ( "github.com/gorilla/mux" ) +var caprofileJSONFile string = "json-schemas/metadata.json" + /* Used to store backend implementation objects Also simplifies mocking for unit testing purposes */ @@ -49,9 +51,8 @@ func (h compositeProfileHandler) createHandler(w http.ResponseWriter, r *http.Re return } - jsonFile := "json-schemas/metadata.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, cpf) + err, httpError := validation.ValidateJsonSchemaData(caprofileJSONFile, cpf) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/composite_profilehandler_test.go b/src/orchestrator/api/composite_profilehandler_test.go index ec3ec24b..d1e34230 100644 --- a/src/orchestrator/api/composite_profilehandler_test.go +++ b/src/orchestrator/api/composite_profilehandler_test.go @@ -70,6 +70,10 @@ func (m *mockCompositeProfileManager) DeleteCompositeProfile(name string, projec return m.Err } +func init() { + caprofileJSONFile = "../json-schemas/metadata.json" +} + func Test_compositeProfileHandler_createHandler(t *testing.T) { testCases := []struct { label string diff --git a/src/orchestrator/api/controllerhandler.go b/src/orchestrator/api/controllerhandler.go index be300d89..418aa70f 100644 --- a/src/orchestrator/api/controllerhandler.go +++ b/src/orchestrator/api/controllerhandler.go @@ -28,6 +28,8 @@ import ( pkgerrors "github.com/pkg/errors" ) +var controllerJSONFile string = "json-schemas/controller.json" + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type controllerHandler struct { @@ -87,9 +89,8 @@ func (h controllerHandler) createHandler(w http.ResponseWriter, r *http.Request) return } - jsonFile := "json-schemas/controller.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, m) + err, httpError := validation.ValidateJsonSchemaData(controllerJSONFile, m) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/controllerhandler_test.go b/src/orchestrator/api/controllerhandler_test.go index 6cbb2a79..ecc2d1f7 100644 --- a/src/orchestrator/api/controllerhandler_test.go +++ b/src/orchestrator/api/controllerhandler_test.go @@ -73,6 +73,10 @@ func (m *mockControllerManager) InitControllers() { return } +func init() { + controllerJSONFile = "../json-schemas/controller.json" +} + func TestControllerCreateHandler(t *testing.T) { testCases := []struct { label string diff --git a/src/orchestrator/api/deployment_intent_groups_handler.go b/src/orchestrator/api/deployment_intent_groups_handler.go index 76dea14c..b9466a2c 100644 --- a/src/orchestrator/api/deployment_intent_groups_handler.go +++ b/src/orchestrator/api/deployment_intent_groups_handler.go @@ -27,6 +27,8 @@ import ( "github.com/gorilla/mux" ) +var dpiJSONFile string = "json-schemas/deployment-group-intent.json" + /* Used to store backend implementation objects Also simplifies mocking for unit testing purposes */ @@ -49,9 +51,8 @@ func (h deploymentIntentGroupHandler) createDeploymentIntentGroupHandler(w http. return } - jsonFile := "json-schemas/deployment-group-intent.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, d) + err, httpError := validation.ValidateJsonSchemaData(dpiJSONFile, d) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/generic_placement_intent_handler.go b/src/orchestrator/api/generic_placement_intent_handler.go index 16f1f234..2415ae2c 100644 --- a/src/orchestrator/api/generic_placement_intent_handler.go +++ b/src/orchestrator/api/generic_placement_intent_handler.go @@ -27,6 +27,8 @@ import ( "github.com/gorilla/mux" ) +var gpiJSONFile string = "json-schemas/generic-placement-intent.json" + /* Used to store backend implementation objects Also simplifies mocking for unit testing purposes */ @@ -49,9 +51,8 @@ func (h genericPlacementIntentHandler) createGenericPlacementIntentHandler(w htt return } - jsonFile := "json-schemas/generic-placement-intent.json" // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, g) + err, httpError := validation.ValidateJsonSchemaData(gpiJSONFile, g) if err != nil { http.Error(w, err.Error(), httpError) return diff --git a/src/orchestrator/api/projecthandler.go b/src/orchestrator/api/projecthandler.go index 6b512804..07cd79ce 100644 --- a/src/orchestrator/api/projecthandler.go +++ b/src/orchestrator/api/projecthandler.go @@ -26,6 +26,8 @@ import ( moduleLib "github.com/onap/multicloud-k8s/src/orchestrator/pkg/module" ) +var projectJSONFile string = "json-schemas/metadata.json" + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type projectHandler struct { @@ -47,9 +49,9 @@ func (h projectHandler) createHandler(w http.ResponseWriter, r *http.Request) { http.Error(w, err.Error(), http.StatusUnprocessableEntity) return } - jsonFile := "json-schemas/metadata.json" + // Verify JSON Body - err, httpError := validation.ValidateJsonSchemaData(jsonFile, p) + err, httpError := validation.ValidateJsonSchemaData(projectJSONFile, p) if err != nil { http.Error(w, err.Error(), httpError) return @@ -126,7 +128,7 @@ func (h projectHandler) getHandler(w http.ResponseWriter, r *http.Request) { projects, err := h.client.GetAllProjects() if err != nil { - http.Error(w, err.Error(), http.StatusInternalServerError) + http.Error(w, err.Error(), http.StatusNotFound) return } @@ -147,7 +149,7 @@ func (h projectHandler) getHandler(w http.ResponseWriter, r *http.Request) { ret, err := h.client.GetProject(name) if err != nil { - http.Error(w, err.Error(), http.StatusInternalServerError) + http.Error(w, err.Error(), http.StatusNotFound) return } diff --git a/src/orchestrator/api/projecthandler_test.go b/src/orchestrator/api/projecthandler_test.go index 6810099f..5e88bab9 100644 --- a/src/orchestrator/api/projecthandler_test.go +++ b/src/orchestrator/api/projecthandler_test.go @@ -64,6 +64,10 @@ func (m *mockProjectManager) GetAllProjects() ([]moduleLib.Project, error) { return []moduleLib.Project{}, m.Err } +func init() { + projectJSONFile = "../json-schemas/metadata.json" +} + func TestProjectCreateHandler(t *testing.T) { testCases := []struct { label string @@ -273,7 +277,7 @@ func TestProjectGetHandler(t *testing.T) { }, { label: "Get Non-Exiting Project", - expectedCode: http.StatusInternalServerError, + expectedCode: http.StatusNotFound, name: "nonexistingproject", projectClient: &mockProjectManager{ Items: []moduleLib.Project{}, diff --git a/src/orchestrator/config.json b/src/orchestrator/config.json new file mode 100644 index 00000000..1acd835d --- /dev/null +++ b/src/orchestrator/config.json @@ -0,0 +1,16 @@ +{ + "ca-file": "ca.cert", + "server-cert": "server.cert", + "server-key": "server.key", + "password": "", + "database-ip": "172.31.0.2", + "database-type": "mongo", + "plugin-dir": "plugins", + "etcd-ip": "127.0.0.1", + "etcd-cert": "", + "etcd-key": "", + "etcd-ca-file": "", + "service-port": "9015", + "log-level": "warn" + +} diff --git a/src/orchestrator/api/json-schemas/composite-app.json b/src/orchestrator/json-schemas/composite-app.json index 3f976831..3f976831 100644 --- a/src/orchestrator/api/json-schemas/composite-app.json +++ b/src/orchestrator/json-schemas/composite-app.json diff --git a/src/orchestrator/api/json-schemas/composite-profile.json b/src/orchestrator/json-schemas/composite-profile.json index e404a64c..e404a64c 100644 --- a/src/orchestrator/api/json-schemas/composite-profile.json +++ b/src/orchestrator/json-schemas/composite-profile.json diff --git a/src/orchestrator/api/json-schemas/controller.json b/src/orchestrator/json-schemas/controller.json index 3263ff21..3263ff21 100644 --- a/src/orchestrator/api/json-schemas/controller.json +++ b/src/orchestrator/json-schemas/controller.json diff --git a/src/orchestrator/api/json-schemas/deployment-group-intent.json b/src/orchestrator/json-schemas/deployment-group-intent.json index 2740747b..2740747b 100644 --- a/src/orchestrator/api/json-schemas/deployment-group-intent.json +++ b/src/orchestrator/json-schemas/deployment-group-intent.json diff --git a/src/orchestrator/api/json-schemas/deployment-intent.json b/src/orchestrator/json-schemas/deployment-intent.json index 6bdc0b43..6bdc0b43 100644 --- a/src/orchestrator/api/json-schemas/deployment-intent.json +++ b/src/orchestrator/json-schemas/deployment-intent.json diff --git a/src/orchestrator/api/json-schemas/generic-placement-intent-app.json b/src/orchestrator/json-schemas/generic-placement-intent-app.json index 0b6447c2..0b6447c2 100644 --- a/src/orchestrator/api/json-schemas/generic-placement-intent-app.json +++ b/src/orchestrator/json-schemas/generic-placement-intent-app.json diff --git a/src/orchestrator/api/json-schemas/generic-placement-intent.json b/src/orchestrator/json-schemas/generic-placement-intent.json index 44df9087..44df9087 100644 --- a/src/orchestrator/api/json-schemas/generic-placement-intent.json +++ b/src/orchestrator/json-schemas/generic-placement-intent.json diff --git a/src/orchestrator/json-schemas/metadata.json b/src/orchestrator/json-schemas/metadata.json new file mode 100644 index 00000000..960545ee --- /dev/null +++ b/src/orchestrator/json-schemas/metadata.json @@ -0,0 +1,37 @@ + +{ + "$schema": "http://json-schema.org/schema#", + "type": "object", + "properties": { + "metadata": { + "required": ["name"], + "properties": { + "userData2": { + "description": "User relevant data for the resource", + "type": "string", + "example": "Some more data", + "maxLength": 512 + }, + "userData1": { + "description": "User relevant data for the resource", + "type": "string", + "example": "Some data", + "maxLength": 512 + }, + "name": { + "description": "Name of the resource", + "type": "string", + "example": "ResName", + "maxLength": 128, + "pattern": "[-_0-9a-zA-Z]+$" + }, + "description": { + "description": "Description for the resource", + "type": "string", + "example": "Resource description", + "maxLength": 1024 + } + } + } + } + }
\ No newline at end of file diff --git a/src/orchestrator/pkg/appcontext/appcontext.go b/src/orchestrator/pkg/appcontext/appcontext.go index db2ba432..5d757940 100644 --- a/src/orchestrator/pkg/appcontext/appcontext.go +++ b/src/orchestrator/pkg/appcontext/appcontext.go @@ -37,30 +37,30 @@ type AppContext struct { // AppContextStatus represents the current status of the appcontext // Instantiating - instantiate has been invoked and is still in progress // Instantiated - instantiate has completed -// PreTerminate - terminate has been invoked when in Instantiating status - need to clean up first // Terminating - terminate has been invoked and is still in progress // Terminated - terminate has completed -// Failed - the instantiate or terminate action has failed +// InstantiateFailed - the instantiate action has failed +// TerminateFailed - the terminate action has failed type AppContextStatus struct { Status StatusValue } type StatusValue string type statuses struct { - Instantiating StatusValue - Instantiated StatusValue - PreTerminate StatusValue - Terminating StatusValue - Terminated StatusValue - Failed StatusValue + Instantiating StatusValue + Instantiated StatusValue + Terminating StatusValue + Terminated StatusValue + InstantiateFailed StatusValue + TerminateFailed StatusValue } var AppContextStatusEnum = &statuses{ - Instantiating: "Instantiating", - Instantiated: "Instantiated", - PreTerminate: "PreTerminate", - Terminating: "Terminating", - Terminated: "Terminated", - Failed: "Failed", + Instantiating: "Instantiating", + Instantiated: "Instantiated", + Terminating: "Terminating", + Terminated: "Terminated", + InstantiateFailed: "InstantiateFailed", + TerminateFailed: "TerminateFailed", } // CompositeAppMeta consists of projectName, CompositeAppName, diff --git a/src/orchestrator/pkg/infra/config/config.go b/src/orchestrator/pkg/infra/config/config.go index fca8bfbd..43191489 100644 --- a/src/orchestrator/pkg/infra/config/config.go +++ b/src/orchestrator/pkg/infra/config/config.go @@ -44,6 +44,7 @@ type Configuration struct { GrpcServerNameOverride string `json:"grpc-server-name-override"` ServicePort string `json:"service-port"` KubernetesLabelName string `json:"kubernetes-label-name"` + LogLevel string `json:"log-level"` } // Config is the structure that stores the configuration @@ -98,6 +99,8 @@ func defaultConfiguration() *Configuration { GrpcServerNameOverride: "", ServicePort: "9015", KubernetesLabelName: "orchestrator.io/rb-instance-id", + LogLevel: "warn", + } } diff --git a/src/orchestrator/pkg/infra/logutils/logger.go b/src/orchestrator/pkg/infra/logutils/logger.go index 2e8f9969..209114a3 100644 --- a/src/orchestrator/pkg/infra/logutils/logger.go +++ b/src/orchestrator/pkg/infra/logutils/logger.go @@ -2,6 +2,9 @@ package logutils import ( log "github.com/sirupsen/logrus" + "github.com/onap/multicloud-k8s/src/orchestrator/pkg/infra/config" + "strings" + ) //Fields is type that will be used by the calling function @@ -10,6 +13,13 @@ type Fields map[string]interface{} func init() { // Log as JSON instead of the default ASCII formatter. log.SetFormatter(&log.JSONFormatter{}) + if strings.EqualFold(config.GetConfiguration().LogLevel, "warn") { + log.SetLevel(log.WarnLevel) + + } + if strings.EqualFold(config.GetConfiguration().LogLevel, "info") { + log.SetLevel(log.InfoLevel) + } } // Error uses the fields provided and logs @@ -26,3 +36,4 @@ func Warn(msg string, fields Fields) { func Info(msg string, fields Fields) { log.WithFields(log.Fields(fields)).Info(msg) } + diff --git a/src/orchestrator/pkg/module/deployment_intent_groups.go b/src/orchestrator/pkg/module/deployment_intent_groups.go index f9829853..dec6391f 100644 --- a/src/orchestrator/pkg/module/deployment_intent_groups.go +++ b/src/orchestrator/pkg/module/deployment_intent_groups.go @@ -21,6 +21,7 @@ import ( "reflect" "time" + "github.com/onap/multicloud-k8s/src/orchestrator/pkg/appcontext" "github.com/onap/multicloud-k8s/src/orchestrator/pkg/infra/db" "github.com/onap/multicloud-k8s/src/orchestrator/pkg/state" @@ -271,6 +272,14 @@ func (c *DeploymentIntentGroupClient) DeleteDeploymentIntentGroup(di string, p s // remove the app contexts associated with thie Deployment Intent Group if stateVal == state.StateEnum.Terminated { + // Verify that the appcontext has completed terminating + ctxid := state.GetLastContextIdFromStateInfo(s) + acStatus, err := state.GetAppContextStatus(ctxid) + if err == nil && + !(acStatus.Status == appcontext.AppContextStatusEnum.Terminated || acStatus.Status == appcontext.AppContextStatusEnum.TerminateFailed) { + return pkgerrors.Errorf("DeploymentIntentGroup has not completed terminating: " + di) + } + for _, id := range state.GetContextIdsFromStateInfo(s) { context, err := state.GetAppContextFromId(id) if err != nil { diff --git a/src/orchestrator/pkg/state/state_helper.go b/src/orchestrator/pkg/state/state_helper.go index 9d59fb75..1f926f8f 100644 --- a/src/orchestrator/pkg/state/state_helper.go +++ b/src/orchestrator/pkg/state/state_helper.go @@ -17,6 +17,8 @@ package state import ( + "encoding/json" + "github.com/onap/multicloud-k8s/src/orchestrator/pkg/appcontext" pkgerrors "github.com/pkg/errors" ) @@ -69,3 +71,30 @@ func GetContextIdsFromStateInfo(s StateInfo) []string { return ids } + +func GetAppContextStatus(ctxid string) (appcontext.AppContextStatus, error) { + + ac, err := GetAppContextFromId(ctxid) + if err != nil { + return appcontext.AppContextStatus{}, err + } + + h, err := ac.GetCompositeAppHandle() + if err != nil { + return appcontext.AppContextStatus{}, err + } + sh, err := ac.GetLevelHandle(h, "status") + if err != nil { + return appcontext.AppContextStatus{}, err + } + s, err := ac.GetValue(sh) + if err != nil { + return appcontext.AppContextStatus{}, err + } + acStatus := appcontext.AppContextStatus{} + js, _ := json.Marshal(s) + json.Unmarshal(js, &acStatus) + + return acStatus, nil + +} diff --git a/src/orchestrator/scripts/start-dev.sh b/src/orchestrator/scripts/start-dev.sh index 003773be..ad21dc47 100755 --- a/src/orchestrator/scripts/start-dev.sh +++ b/src/orchestrator/scripts/start-dev.sh @@ -28,6 +28,6 @@ echo "Compiling source code" pushd $opath generate_config make all -cp -r $k8s_path/src/orchestrator/api/json-schemas $k8s_path/src/orchestrator +cp -r $k8s_path/src/orchestrator/json-schemas $k8s_path/src/orchestrator ./orchestrator popd diff --git a/src/orchestrator/utils/helm/helm.go b/src/orchestrator/utils/helm/helm.go index 80cdfe5a..3f8b6e9e 100644 --- a/src/orchestrator/utils/helm/helm.go +++ b/src/orchestrator/utils/helm/helm.go @@ -21,7 +21,7 @@ import ( utils "github.com/onap/multicloud-k8s/src/orchestrator/utils" pkgerrors "github.com/pkg/errors" - "log" + "fmt" "io/ioutil" @@ -43,6 +43,8 @@ import ( "k8s.io/helm/pkg/renderutil" "k8s.io/helm/pkg/tiller" "k8s.io/helm/pkg/timeconv" + logger "github.com/onap/multicloud-k8s/src/orchestrator/pkg/infra/logutils" + ) //KubernetesResourceTemplate - Represents the template that is used to create a particular @@ -174,7 +176,8 @@ func (h *TemplateClient) GenerateKubernetesArtifacts(inputPath string, valueFile if err != nil { return retData, pkgerrors.Wrap(err, "Got error creating temp dir") } - log.Printf("The o/p dir:: %s ", outputDir) + logger.Info(":: The o/p dir:: ", logger.Fields{"OutPutDirectory ":outputDir}) + if namespace == "" { namespace = "default" @@ -302,20 +305,20 @@ func (h *TemplateClient) Resolve(appContent []byte, appProfileContent []byte, ov if err != nil { return sortedTemplates, pkgerrors.Wrap(err, "Extracting appContent") } - log.Printf("The chartBasePath :: %s", chartBasePath) + logger.Info("The chartBasePath ::", logger.Fields{"chartBasePath":chartBasePath}) //prPath is the tmp path where the appProfileContent is extracted. prPath, err := utils.ExtractTarBall(bytes.NewBuffer(appProfileContent)) if err != nil { return sortedTemplates, pkgerrors.Wrap(err, "Extracting Profile Content") } - log.Printf("The profile path:: %s", prPath) + logger.Info("The profile path:: ", logger.Fields{"Profile Path":prPath}) prYamlClient, err := ProcessProfileYaml(prPath, h.manifestName) if err != nil { return sortedTemplates, pkgerrors.Wrap(err, "Processing Profile Manifest") } - log.Println("Got the profileYamlClient..") + logger.Info("Got the profileYamlClient..", logger.Fields{}) err = prYamlClient.CopyConfigurationOverrides(chartBasePath) if err != nil { diff --git a/src/orchestrator/utils/utils.go b/src/orchestrator/utils/utils.go index 22ce903b..b591ccb5 100644 --- a/src/orchestrator/utils/utils.go +++ b/src/orchestrator/utils/utils.go @@ -23,13 +23,14 @@ import ( "io" "io/ioutil" - "log" "os" "path" "path/filepath" pkgerrors "github.com/pkg/errors" yaml "gopkg.in/yaml.v3" + log "github.com/onap/multicloud-k8s/src/orchestrator/pkg/infra/logutils" + ) // ListYamlStruct is applied when the kind is list @@ -69,15 +70,15 @@ type YamlStruct struct { func (y YamlStruct) isValid() bool { if y.APIVersion == "" { - log.Printf("apiVersion is missing in manifest file") + log.Info("apiVersion is missing in manifest file", log.Fields{}) return false } if y.Kind == "" { - log.Printf("kind is missing in manifest file") + log.Info("kind is missing in manifest file", log.Fields{}) return false } if y.Metadata.Name == "" { - log.Printf("metadata.name is missing in manifest file") + log.Info("metadata.name is missing in manifest file", log.Fields{}) return false } return true @@ -107,13 +108,14 @@ func ExtractYamlParameters(f string) (YamlStruct, error) { li := strings.LastIndex(filename, "/") fn := string(filename[li+1:]) yamlStruct.Metadata.Name = fn - log.Printf("Setting the metadata name as :: %s", fn) + log.Info("Setting the metadata", log.Fields{"MetaDataName":fn}) } if yamlStruct.isValid() { - log.Printf("YAML parameters for file ::%s \n %v", f, yamlStruct) + log.Info(":: YAML parameters ::", log.Fields{"fileName": f, "yamlStruct":yamlStruct}) + return yamlStruct, nil } - log.Printf("YAML file ::%s has errors", f) + log.Info("YAML file has errors", log.Fields{"fileName": f}) return YamlStruct{}, pkgerrors.Errorf("Cant extract parameters from yaml file :: %s", filename) } @@ -204,13 +206,3 @@ func EnsureDirectory(f string) error { } return os.MkdirAll(base, 0755) } - -// func main() { -// filename := "./test.yaml" -// yamlStruct, err := ExtractYamlParameters(filename) -// if err!=nil { -// log.Print(err) -// } -// fmt.Printf("%s+%s", yamlStruct.Metadata.Name, yamlStruct.Kind) -// fmt.Printf("%v", yamlStruct) -// } diff --git a/src/ovnaction/api/netcontrolintenthandler.go b/src/ovnaction/api/netcontrolintenthandler.go index fe2109b6..631f13c4 100644 --- a/src/ovnaction/api/netcontrolintenthandler.go +++ b/src/ovnaction/api/netcontrolintenthandler.go @@ -24,10 +24,16 @@ import ( moduleLib "github.com/onap/multicloud-k8s/src/ovnaction/pkg/module" pkgerrors "github.com/pkg/errors" + "github.com/onap/multicloud-k8s/src/orchestrator/pkg/infra/validation" + "github.com/gorilla/mux" ) +var netCntIntJSONFile string = "json-schemas/metadata.json" + + + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type netcontrolintentHandler struct { @@ -65,6 +71,13 @@ func (h netcontrolintentHandler) createHandler(w http.ResponseWriter, r *http.Re return } + err, httpError := validation.ValidateJsonSchemaData(netCntIntJSONFile, nci) +if err != nil { + http.Error(w, err.Error(), httpError) + return +} + + // Name is required. if nci.Metadata.Name == "" { http.Error(w, "Missing name in POST request", http.StatusBadRequest) diff --git a/src/ovnaction/api/workloadifintenthandler.go b/src/ovnaction/api/workloadifintenthandler.go index cf8f45bf..e7be6317 100644 --- a/src/ovnaction/api/workloadifintenthandler.go +++ b/src/ovnaction/api/workloadifintenthandler.go @@ -29,6 +29,8 @@ import ( "github.com/gorilla/mux" ) +var netIfJSONFile string = "json-schemas/network-load-interface.json" + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type workloadifintentHandler struct { @@ -102,6 +104,12 @@ func (h workloadifintentHandler) createHandler(w http.ResponseWriter, r *http.Re return } + err, httpError := validation.ValidateJsonSchemaData(netIfJSONFile, wif) +if err != nil { + http.Error(w, err.Error(), httpError) + return +} + // Name is required. if wif.Metadata.Name == "" { http.Error(w, "Missing name in POST request", http.StatusBadRequest) diff --git a/src/ovnaction/api/workloadintenthandler.go b/src/ovnaction/api/workloadintenthandler.go index cf7ecebc..acf4edbb 100644 --- a/src/ovnaction/api/workloadintenthandler.go +++ b/src/ovnaction/api/workloadintenthandler.go @@ -29,6 +29,8 @@ import ( "github.com/gorilla/mux" ) +var workloadIntJSONFile string = "json-schemas/network-workload.json" + // Used to store backend implementations objects // Also simplifies mocking for unit testing purposes type workloadintentHandler struct { @@ -82,6 +84,12 @@ func (h workloadintentHandler) createHandler(w http.ResponseWriter, r *http.Requ return } + err, httpError := validation.ValidateJsonSchemaData(workloadIntJSONFile, wi) +if err != nil { + http.Error(w, err.Error(), httpError) + return +} + // Name is required. if wi.Metadata.Name == "" { http.Error(w, "Missing name in POST request", http.StatusBadRequest) diff --git a/src/ovnaction/json-schemas/metadata.json b/src/ovnaction/json-schemas/metadata.json new file mode 100644 index 00000000..960545ee --- /dev/null +++ b/src/ovnaction/json-schemas/metadata.json @@ -0,0 +1,37 @@ + +{ + "$schema": "http://json-schema.org/schema#", + "type": "object", + "properties": { + "metadata": { + "required": ["name"], + "properties": { + "userData2": { + "description": "User relevant data for the resource", + "type": "string", + "example": "Some more data", + "maxLength": 512 + }, + "userData1": { + "description": "User relevant data for the resource", + "type": "string", + "example": "Some data", + "maxLength": 512 + }, + "name": { + "description": "Name of the resource", + "type": "string", + "example": "ResName", + "maxLength": 128, + "pattern": "[-_0-9a-zA-Z]+$" + }, + "description": { + "description": "Description for the resource", + "type": "string", + "example": "Resource description", + "maxLength": 1024 + } + } + } + } + }
\ No newline at end of file diff --git a/src/orchestrator/api/json-schemas/network-load-interface.json b/src/ovnaction/json-schemas/network-load-interface.json index dd5b40d1..896f4f2b 100644 --- a/src/orchestrator/api/json-schemas/network-load-interface.json +++ b/src/ovnaction/json-schemas/network-load-interface.json @@ -37,8 +37,8 @@ }, "defaultGateway": { "description": "Is this interface default gateway", - "type": "boolean", - "example": false, + "type": "string", + "example": "false", "maxLength": 128 } } diff --git a/src/orchestrator/api/json-schemas/network-workload.json b/src/ovnaction/json-schemas/network-workload.json index c5dc14cb..c5dc14cb 100644 --- a/src/orchestrator/api/json-schemas/network-workload.json +++ b/src/ovnaction/json-schemas/network-workload.json diff --git a/src/rsync/go.mod b/src/rsync/go.mod index 0fd2c787..b5f5c93e 100644 --- a/src/rsync/go.mod +++ b/src/rsync/go.mod @@ -27,6 +27,7 @@ require ( replace ( github.com/onap/multicloud-k8s/src/clm => ../clm github.com/onap/multicloud-k8s/src/monitor => ../monitor + github.com/onap/multicloud-k8s/src/orchestrator => ../orchestrator k8s.io/api => k8s.io/api v0.17.3 k8s.io/apiextensions-apiserver => k8s.io/apiextensions-apiserver v0.17.3 k8s.io/apimachinery => k8s.io/apimachinery v0.17.3 diff --git a/src/rsync/go.sum b/src/rsync/go.sum index 00637891..95895d07 100644 --- a/src/rsync/go.sum +++ b/src/rsync/go.sum @@ -1216,10 +1216,14 @@ github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c h1:u40Z8hqBAAQyv+vATcGgV github.com/xdg/scram v0.0.0-20180814205039-7eeb5667e42c/go.mod h1:lB8K/P019DLNhemzwFU4jHLhdvlE6uDZjXFejJXr49I= github.com/xdg/stringprep v1.0.0 h1:d9X0esnoa3dFsV0FG35rAT0RIhYFlPq7MiP+DW89La0= github.com/xdg/stringprep v1.0.0/go.mod h1:Jhud4/sHMO4oL310DaZAKk9ZaJ08SJfe+sJh0HrGL1Y= +github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f h1:J9EGpcZtP0E/raorCMxlFGSTBrsSlaDGf3jU/qvAE2c= github.com/xeipuuv/gojsonpointer v0.0.0-20180127040702-4e3ac2762d5f/go.mod h1:N2zxlSyiKSe5eX1tZViRH5QA0qijqEDrYZiPEAiq3wU= +github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415 h1:EzJWgHovont7NscjpAxXsDA8S8BMYve8Y5+7cuRE7R0= github.com/xeipuuv/gojsonreference v0.0.0-20180127040603-bd5ef7bd5415/go.mod h1:GwrjFmJcFw6At/Gs6z4yjiIwzuJ1/+UwLxMQDVQXShQ= github.com/xeipuuv/gojsonschema v0.0.0-20180618132009-1d523034197f/go.mod h1:5yf86TLmAcydyeJq5YvxkGPE2fm/u4myDekKRoLuqhs= github.com/xeipuuv/gojsonschema v1.1.0/go.mod h1:5yf86TLmAcydyeJq5YvxkGPE2fm/u4myDekKRoLuqhs= +github.com/xeipuuv/gojsonschema v1.2.0 h1:LhYJRs+L4fBtjZUfuSZIKGeVu0QRy8e5Xi7D17UxZ74= +github.com/xeipuuv/gojsonschema v1.2.0/go.mod h1:anYRn/JVcOK2ZgGU+IjEV4nwlhoK5sQluxsYJ78Id3Y= github.com/xiang90/probing v0.0.0-20160813154853-07dd2e8dfe18/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU= github.com/xlab/handysort v0.0.0-20150421192137-fb3537ed64a1/go.mod h1:QcJo0QPSfTONNIgpN5RA8prR7fF8nkF6cTWTcNerRO8= diff --git a/src/rsync/pkg/client/client.go b/src/rsync/pkg/client/client.go index a489b951..5920dea5 100644 --- a/src/rsync/pkg/client/client.go +++ b/src/rsync/pkg/client/client.go @@ -25,7 +25,7 @@ import ( // DefaultValidation default action to validate. If `true` all resources by // default will be validated. -const DefaultValidation = true +const DefaultValidation = false // Client is a kubernetes client, like `kubectl` type Client struct { diff --git a/src/rsync/pkg/context/context.go b/src/rsync/pkg/context/context.go index f77482e6..4b886ec7 100644 --- a/src/rsync/pkg/context/context.go +++ b/src/rsync/pkg/context/context.go @@ -21,6 +21,7 @@ import ( "encoding/json" "fmt" "strings" + "sync" "time" "github.com/onap/multicloud-k8s/src/orchestrator/pkg/appcontext" @@ -36,7 +37,9 @@ import ( ) type CompositeAppContext struct { - cid interface{} + cid interface{} + chans []chan bool + mutex sync.Mutex } func getRes(ac appcontext.AppContext, name string, app string, cluster string) ([]byte, interface{}, error) { @@ -144,26 +147,150 @@ func instantiateResource(ac appcontext.AppContext, c *kubeclient.Client, name st return nil } +func updateResourceStatus(ac appcontext.AppContext, resState resourcestatus.ResourceStatus, app string, cluster string, aov map[string][]string) error { + + for _, res := range aov["resorder"] { + + rh, err := ac.GetResourceHandle(app, cluster, res) + if err != nil { + return err + } + sh, err := ac.GetLevelHandle(rh, "status") + if err != nil { + return err + } + + s, err := ac.GetValue(sh) + if err != nil { + return err + } + rStatus := resourcestatus.ResourceStatus{} + js, err := json.Marshal(s) + if err != nil { + return err + } + err = json.Unmarshal(js, &rStatus) + if err != nil { + return err + } + // no need to update a status that has reached a 'done' status + if rStatus.Status == resourcestatus.RsyncStatusEnum.Deleted || + rStatus.Status == resourcestatus.RsyncStatusEnum.Applied || + rStatus.Status == resourcestatus.RsyncStatusEnum.Failed { + continue + } + + err = ac.UpdateStatusValue(sh, resState) + if err != nil { + return err + } + } + + return nil + +} + +// return true if all resources have reached a 'done' status - e.g. Applied, Deleted or Failed +func allResourcesDone(ac appcontext.AppContext, app string, cluster string, aov map[string][]string) bool { + + for _, res := range aov["resorder"] { + + rh, err := ac.GetResourceHandle(app, cluster, res) + if err != nil { + return false + } + sh, err := ac.GetLevelHandle(rh, "status") + if err != nil { + return false + } + + s, err := ac.GetValue(sh) + if err != nil { + return false + } + rStatus := resourcestatus.ResourceStatus{} + js, err := json.Marshal(s) + if err != nil { + return false + } + err = json.Unmarshal(js, &rStatus) + if err != nil { + return false + } + if rStatus.Status != resourcestatus.RsyncStatusEnum.Deleted && + rStatus.Status != resourcestatus.RsyncStatusEnum.Applied && + rStatus.Status != resourcestatus.RsyncStatusEnum.Failed { + return false + } + } + + return true + +} + // Wait for 2 secs const waitTime = 2 -func waitForClusterReady(c *kubeclient.Client, cluster string) error { +func waitForClusterReady(instca *CompositeAppContext, ac appcontext.AppContext, c *kubeclient.Client, appname string, cluster string, aov map[string][]string) error { + + forceDone := false + resStateUpdated := false + ch := addChan(instca) + + rch := make(chan error, 1) + checkReachable := func() { + err := c.IsReachable() + rch <- err + } + + go checkReachable() +Loop: for { - if err := c.IsReachable(); err != nil { - // TODO: Add more realistic error checking - // TODO: Add Incremental wait logic here - time.Sleep(waitTime * time.Second) - } else { + select { + case rerr := <-rch: + if rerr == nil { + break Loop + } else { + logutils.Info("Cluster is not reachable - keep trying::", logutils.Fields{"cluster": cluster}) + go checkReachable() + } + case <-ch: + statusFailed := resourcestatus.ResourceStatus{ + Status: resourcestatus.RsyncStatusEnum.Failed, + } + err := updateResourceStatus(ac, statusFailed, appname, cluster, aov) + if err != nil { + deleteChan(instca, ch) + return err + } + forceDone = true + break Loop + case <-time.After(waitTime * time.Second): + // on first timeout - cluster is apparently not reachable, update resources in + // this group to 'Retrying' + if !resStateUpdated { + statusRetrying := resourcestatus.ResourceStatus{ + Status: resourcestatus.RsyncStatusEnum.Retrying, + } + err := updateResourceStatus(ac, statusRetrying, appname, cluster, aov) + if err != nil { + deleteChan(instca, ch) + return err + } + resStateUpdated = true + } break } } - logutils.Info("Cluster is reachable::", logutils.Fields{ - "cluster": cluster, - }) + + deleteChan(instca, ch) + if forceDone { + return pkgerrors.Errorf("Termination of rsync cluster retry: " + cluster) + } return nil } -// initializeResourceStatus sets the initial status of every resource appropriately based on the state of the AppContext +// initializeAppContextStatus sets the initial status of every resource appropriately based on the state of the AppContext func initializeAppContextStatus(ac appcontext.AppContext, acStatus appcontext.AppContextStatus) error { h, err := ac.GetCompositeAppHandle() if err != nil { @@ -320,12 +447,18 @@ func updateEndingAppContextStatus(ac appcontext.AppContext, handle interface{}, js, _ := json.Marshal(s) json.Unmarshal(js, &acStatus) - if failure { - acStatus.Status = appcontext.AppContextStatusEnum.Failed - } else if acStatus.Status == appcontext.AppContextStatusEnum.Instantiating { - acStatus.Status = appcontext.AppContextStatusEnum.Instantiated + if acStatus.Status == appcontext.AppContextStatusEnum.Instantiating { + if failure { + acStatus.Status = appcontext.AppContextStatusEnum.InstantiateFailed + } else { + acStatus.Status = appcontext.AppContextStatusEnum.Instantiated + } } else if acStatus.Status == appcontext.AppContextStatusEnum.Terminating { - acStatus.Status = appcontext.AppContextStatusEnum.Terminated + if failure { + acStatus.Status = appcontext.AppContextStatusEnum.TerminateFailed + } else { + acStatus.Status = appcontext.AppContextStatusEnum.Terminated + } } else { return pkgerrors.Errorf("Invalid AppContextStatus %v", acStatus) } @@ -337,20 +470,197 @@ func updateEndingAppContextStatus(ac appcontext.AppContext, handle interface{}, return nil } +func getAppContextStatus(ac appcontext.AppContext) (*appcontext.AppContextStatus, error) { + + h, err := ac.GetCompositeAppHandle() + if err != nil { + return nil, err + } + sh, err := ac.GetLevelHandle(h, "status") + if err != nil { + return nil, err + } + s, err := ac.GetValue(sh) + if err != nil { + return nil, err + } + acStatus := appcontext.AppContextStatus{} + js, _ := json.Marshal(s) + json.Unmarshal(js, &acStatus) + + return &acStatus, nil + +} + type fn func(ac appcontext.AppContext, client *kubeclient.Client, res string, app string, cluster string, label string) error type statusfn func(client *kubeclient.Client, app string, cluster string, label string) error -func applyFnComApp(cid interface{}, acStatus appcontext.AppContextStatus, f fn, sfn statusfn, breakonError bool) error { - con := connector.Init(cid) +func addChan(instca *CompositeAppContext) chan bool { + + instca.mutex.Lock() + c := make(chan bool) + instca.chans = append(instca.chans, c) + instca.mutex.Unlock() + + return c +} + +func deleteChan(instca *CompositeAppContext, c chan bool) error { + + var i int + instca.mutex.Lock() + for i = 0; i < len(instca.chans); i++ { + if instca.chans[i] == c { + break + } + } + + if i == len(instca.chans) { + instca.mutex.Unlock() + return pkgerrors.Errorf("Given channel was not found:") + } + instca.chans[i] = instca.chans[len(instca.chans)-1] + instca.chans = instca.chans[:len(instca.chans)-1] + instca.mutex.Unlock() + + return nil +} + +func waitForDone(ac appcontext.AppContext) { + count := 0 + for { + time.Sleep(1 * time.Second) + count++ + if count == 60*60 { + logutils.Info("Wait for done watcher running..", logutils.Fields{}) + count = 0 + } + acStatus, _ := getAppContextStatus(ac) + if acStatus.Status == appcontext.AppContextStatusEnum.Instantiated || + acStatus.Status == appcontext.AppContextStatusEnum.InstantiateFailed { + return + } + } + return +} + +func kickoffRetryWatcher(instca *CompositeAppContext, ac appcontext.AppContext, acStatus appcontext.AppContextStatus, wg *errgroup.Group) { + + wg.Go(func() error { + + var count int + + count = 0 + for { + time.Sleep(1 * time.Second) + count++ + if count == 60*60 { + logutils.Info("Retry watcher running..", logutils.Fields{}) + count = 0 + } + + cStatus, err := getAppContextStatus(ac) + if err != nil { + logutils.Error("Failed to get the app context status", logutils.Fields{ + "error": err, + }) + return err + } + flag, err := getAppContextFlag(ac) + if err != nil { + logutils.Error("Failed to get the stop flag", logutils.Fields{ + "error": err, + }) + return err + } else { + if flag == true { + instca.mutex.Lock() + for i := 0; i < len(instca.chans); i++ { + instca.chans[i] <- true + logutils.Info("kickoffRetryWatcher - send an exit message", logutils.Fields{}) + } + instca.mutex.Unlock() + break + } + } + if acStatus.Status == appcontext.AppContextStatusEnum.Instantiating { + if cStatus.Status == appcontext.AppContextStatusEnum.Instantiated || + cStatus.Status == appcontext.AppContextStatusEnum.InstantiateFailed { + break + } + } else { + if cStatus.Status == appcontext.AppContextStatusEnum.Terminated || + cStatus.Status == appcontext.AppContextStatusEnum.TerminateFailed { + break + } + } + + } + return nil + }) + +} + +func getAppContextFlag(ac appcontext.AppContext) (bool, error) { + h, err := ac.GetCompositeAppHandle() + if err != nil { + return false, err + } + sh, err := ac.GetLevelHandle(h, "stopflag") + if sh == nil { + return false, err + } else { + v, err := ac.GetValue(sh) + if err != nil { + return false, err + } else { + return v.(bool), nil + } + } +} + +func updateAppContextFlag(cid interface{}, sf bool) error { + ac := appcontext.AppContext{} + _, err := ac.LoadAppContext(cid) + if err != nil { + return err + } + hc, err := ac.GetCompositeAppHandle() + if err != nil { + return err + } + sh, err := ac.GetLevelHandle(hc, "stopflag") + if sh == nil { + _, err = ac.AddLevelValue(hc, "stopflag", sf) + } else { + err = ac.UpdateValue(sh, sf) + } + if err != nil { + return err + } + return nil +} + +func applyFnComApp(instca *CompositeAppContext, acStatus appcontext.AppContextStatus, f fn, sfn statusfn, breakonError bool) error { + con := connector.Init(instca.cid) //Cleanup defer con.RemoveClient() ac := appcontext.AppContext{} - h, err := ac.LoadAppContext(cid) + h, err := ac.LoadAppContext(instca.cid) if err != nil { return err } + // if terminating, wait for all retrying instantiate threads to exit + if acStatus.Status == appcontext.AppContextStatusEnum.Terminating { + waitForDone(ac) + err := updateAppContextFlag(instca.cid, false) + if err != nil { + return err + } + } + // initialize appcontext status err = initializeAppContextStatus(ac, acStatus) if err != nil { @@ -375,6 +685,8 @@ func applyFnComApp(cid interface{}, acStatus appcontext.AppContextStatus, f fn, }) id, _ := ac.GetCompositeAppHandle() g, _ := errgroup.WithContext(context.Background()) + wg, _ := errgroup.WithContext(context.Background()) + kickoffRetryWatcher(instca, ac, acStatus, wg) // Iterate over all the subapps for _, app := range appList["apporder"] { appName := app @@ -414,8 +726,13 @@ func applyFnComApp(cid interface{}, acStatus appcontext.AppContextStatus, f fn, json.Unmarshal([]byte(resorder.(string)), &aov) // Keep retrying for reachability for { + done := allResourcesDone(ac, appName, cluster, aov) + if done { + break + } + // Wait for cluster to be reachable - err = waitForClusterReady(c, cluster) + err := waitForClusterReady(instca, ac, c, appName, cluster, aov) if err != nil { // TODO: Add error handling return err @@ -479,19 +796,39 @@ func applyFnComApp(cid interface{}, acStatus appcontext.AppContextStatus, f fn, logutils.Error("Encountered error updating AppContext status", logutils.Fields{"error": err}) return err } + if err := wg.Wait(); err != nil { + logutils.Error("Encountered error in watcher thread", logutils.Fields{"error": err}) + return err + } return nil } // InstantiateComApp Instantiate Apps in Composite App func (instca *CompositeAppContext) InstantiateComApp(cid interface{}) error { - go applyFnComApp(cid, appcontext.AppContextStatus{Status: appcontext.AppContextStatusEnum.Instantiating}, + instca.cid = cid + instca.chans = []chan bool{} + instca.mutex = sync.Mutex{} + err := updateAppContextFlag(cid, false) + if err != nil { + logutils.Error("Encountered error updating AppContext flag", logutils.Fields{"error": err}) + return err + } + go applyFnComApp(instca, appcontext.AppContextStatus{Status: appcontext.AppContextStatusEnum.Instantiating}, instantiateResource, addStatusTracker, true) return nil } // TerminateComApp Terminates Apps in Composite App func (instca *CompositeAppContext) TerminateComApp(cid interface{}) error { - go applyFnComApp(cid, appcontext.AppContextStatus{Status: appcontext.AppContextStatusEnum.Terminating}, + instca.cid = cid + instca.chans = []chan bool{} + instca.mutex = sync.Mutex{} + err := updateAppContextFlag(cid, true) + if err != nil { + logutils.Error("Encountered error updating AppContext flag", logutils.Fields{"error": err}) + return err + } + go applyFnComApp(instca, appcontext.AppContextStatus{Status: appcontext.AppContextStatusEnum.Terminating}, terminateResource, deleteStatusTracker, false) return nil } diff --git a/src/tools/emcoctl/Readme.md b/src/tools/emcoctl/Readme.md index ecbcf4de..105398c1 100644 --- a/src/tools/emcoctl/Readme.md +++ b/src/tools/emcoctl/Readme.md @@ -84,3 +84,134 @@ For deleting one resource anchor can be provided as an arguement `$ emcoctl delete <anchor>` `$ emcoctl delete projects/testvfw/composite-apps/compositevfw/v1/deployment-intent-groups/vfw_deployment_intent_group` + + +## Using helm charts through emcoctl + +When you need to use emcoctl for deploying helm +charts the following steps are required. + +1. Make sure that the composite app which you are planning to deploy, the tree structure is as below + +``` + +$ tree collection/app1/ +collection/app1/ +├── helm +│ └── collectd +│ ├── Chart.yaml +│ ├── resources +│ │ └── collectd.conf +│ ├── templates +│ │ ├── configmap.yaml +│ │ ├── daemonset.yaml +│ │ ├── _helpers.tpl +│ │ ├── NOTES.txt +│ │ └── service.yaml +│ └── values.yaml +└── profile + ├── manifest.yaml + └── override_values.yaml + +5 directories, 10 files + +$ tree collection/m3db/ +collection/m3db/ +├── helm +│ └── m3db +│ ├── Chart.yaml +│ ├── del.yaml +│ ├── templates +│ │ └── m3dbcluster.yaml +│ └── values.yaml +└── profile + ├── manifest.yaml + └── override_values.yaml + +4 directories, 6 files + +``` + +### NOTE +``` +* In the above example, we have a composite app : collection +The collection composite-app shown has two apps : app1(collectd) +and m3db +* Each app has two dirs : a. HELM and b. PROFILE. +* Helm dir shall have the real helm charts of the app. +* profile shall have the two files - manifest.yaml and override_values.yaml for creating the customized profile. +``` + +### Commands for making the tar files from helm. + +``` + tar -czf collectd.tar.gz -C $test_folder/vnfs/comp-app/collection/app1/helm . + tar -czf collectd_profile.tar.gz -C $test_folder/vnfs/comp-app/collection/app1/profile . + ---------------------------------------- + tar -czf m3db.tar.gz -C $test_folder/vnfs/comp-app/collection/m3db/helm . + tar -czf m3db_profile.tar.gz -C $test_folder/vnfs/comp-app/collection/m3db/profile . +``` + +Once you have generated the tar files, you need to give the path in file which you are applying using the emcoctl. For eg: + +``` +#adding collectd app to the composite app +version: emco/v2 +resourceContext: + anchor: projects/proj1/composite-apps/collection-composite-app/v1/apps +metadata : + name: collectd + description: "description for app" + userData1: test1 + userData2: test2 +file: + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/collectd.tar.gz + +``` + +``` +#adding collectd app profiles to the composite profile +version: emco/v2 +resourceContext: + anchor: projects/proj1/composite-apps/collection-composite-app/v1/composite-profiles/collection-composite-profile/profiles +metadata : + name: collectd-profile + description: test + userData1: test1 + userData2: test2 +spec: + app-name: collectd +file: + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/collectd_profile.tar.gz + +``` + +### Running the emcoctl + +``` +* Make sure that the emcoctl is build.You can build it by issuing the 'make' command. +Dir : $MULTICLOUD-K8s_HOME/src/tools/emcoctl +``` +* Then run the emcoctl by command: +``` +./emcoctl --config ./examples/emco-cfg.yaml apply -f ./examples/test.yaml + +``` + +Here, emco-cfg.yaml contains the config/port details of each of the microservices you are using. +A sample configuration is : + +``` + orchestrator: + host: localhost + port: 9015 + clm: + host: localhost + port: 9019 + ncm: + host: localhost + port: 9016 + ovnaction: + host: localhost + port: 9051 +``` diff --git a/src/tools/emcoctl/examples/emco-cfg.yaml b/src/tools/emcoctl/examples/emco-cfg.yaml index c1703def..a7e284ab 100644 --- a/src/tools/emcoctl/examples/emco-cfg.yaml +++ b/src/tools/emcoctl/examples/emco-cfg.yaml @@ -3,10 +3,10 @@ port: 9015 clm: host: localhost - port: 9061 + port: 9019 ncm: host: localhost - port: 9031 + port: 9016 ovnaction: host: localhost - port: 9051
\ No newline at end of file + port: 9018 diff --git a/src/tools/emcoctl/examples/test.yaml b/src/tools/emcoctl/examples/test.yaml index 924f7e55..e54ff36e 100644 --- a/src/tools/emcoctl/examples/test.yaml +++ b/src/tools/emcoctl/examples/test.yaml @@ -33,7 +33,7 @@ metadata : userData1: test1 userData2: test2 file: - kubeconfig + /home/otc/.kube/config --- #Add label cluster @@ -77,7 +77,7 @@ metadata : userData1: test1 userData2: test2 file: - prometheus-operator.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/prometheus-operator.tar.gz --- #adding collectd app to the composite app @@ -90,7 +90,7 @@ metadata : userData1: test1 userData2: test2 file: - collectd.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/collectd.tar.gz --- #creating collection composite profile entry @@ -116,7 +116,9 @@ metadata : spec: app-name: prometheus-operator file: - prometheus-operator_profile.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/prometheus-operator_profile.tar.gz + + --- #adding collectd app profiles to the composite profile @@ -131,7 +133,8 @@ metadata : spec: app-name: collectd file: - collectd_profile.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/collectd_profile.tar.gz + --- #create the generic placement intent @@ -160,7 +163,7 @@ spec: app-name: prometheus-operator intent: allOf: - - provider-name: cluster-provider1 + - provider-name: provider1 cluster-label-name: edge-cluster --- #add the prometheus app placement intent to the generic placement intent @@ -176,7 +179,7 @@ spec: app-name: collectd intent: allOf: - - provider-name: cluster-provider1 + - provider-name: provider1 cluster-label-name: edge-cluster --- diff --git a/src/tools/emcoctl/examples/vfw.yaml b/src/tools/emcoctl/examples/vfw.yaml index b5df4834..c84a1bab 100644 --- a/src/tools/emcoctl/examples/vfw.yaml +++ b/src/tools/emcoctl/examples/vfw.yaml @@ -38,7 +38,7 @@ resourceContext: metadata : name: edge01 file: - kubeconfig + /home/otc/.kube/config --- #Add label cluster @@ -89,7 +89,7 @@ spec: --- version: emco/v2 resourceContext: - anchor: cluster-providers/vfw-cluster-provider/clusters/edge01/apply + anchor: cluster-providers/vfw-cluster-provider/clusters/edge01/apply --- #create project @@ -100,7 +100,7 @@ metadata : name: testvfw --- -#creating collection composite app entry +#creating vfw composite app entry version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps @@ -110,37 +110,37 @@ spec: version: v1 --- -#adding prometheus app to the composite app +#adding packetgen app to the composite app version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/apps metadata : name: packetgen file: - /home/vagrant/multicloud-k8s/kud/demo/composite-firewall/packetgen.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/packetgen.tar.gz --- -#adding prometheus app to the composite app +#adding firewall app to the composite app version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/apps metadata : name: firewall file: - /home/vagrant/multicloud-k8s/kud/demo/composite-firewall/firewall.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/firewall.tar.gz --- -#adding collectd app to the composite app +#adding sink app to the composite app version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/apps metadata : name: sink file: - /home/vagrant/multicloud-k8s/kud/demo/composite-firewall/sink.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/sink.tar.gz --- -#creating collection composite profile entry +#creating vfw composite profile entry version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/composite-profiles @@ -148,7 +148,7 @@ metadata : name: vfw_composite-profile --- -#adding prometheus app profiles to the composite profile +#adding packetgen app profiles to the composite profile version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/composite-profiles/vfw_composite-profile/profiles @@ -157,7 +157,7 @@ metadata : spec: app-name: packetgen file: - /home/vagrant/multicloud-k8s/kud/demo/composite-firewall/profile.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/profile.tar.gz --- #adding firewall app profiles to the composite profile @@ -169,7 +169,7 @@ metadata : spec: app-name: firewall file: - /home/vagrant/multicloud-k8s/kud/demo/composite-firewall/profile.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/profile.tar.gz --- #adding firewall app profiles to the composite profile @@ -181,7 +181,7 @@ metadata : spec: app-name: sink file: - /home/vagrant/multicloud-k8s/kud/demo/composite-firewall/profile.tar.gz + /opt/csar/cb009bfe-bbee-11e8-9766-525400435678/profile.tar.gz --- #create the generic placement intent @@ -194,7 +194,7 @@ spec: logical-cloud: NA --- -#add the prometheus app placement intent to the generic placement intent +#add the packetgen app placement intent to the generic placement intent version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/generic-placement-intents/fw-placement-intent/app-intents @@ -207,7 +207,7 @@ spec: - provider-name: vfw-cluster-provider cluster-label-name: LabelA --- -#add the prometheus app placement intent to the generic placement intent +#add the firewall app placement intent to the generic placement intent version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/generic-placement-intents/fw-placement-intent/app-intents @@ -221,7 +221,7 @@ spec: cluster-label-name: LabelA --- -#add the prometheus app placement intent to the generic placement intent +#add the sink app placement intent to the generic placement intent version: emco/v2 resourceContext: anchor: projects/testvfw/composite-apps/compositevfw/v1/generic-placement-intents/fw-placement-intent/app-intents |