diff options
Diffstat (limited to 'external-schema-repo-generator/README.md')
| -rw-r--r-- | external-schema-repo-generator/README.md | 175 |
1 files changed, 175 insertions, 0 deletions
diff --git a/external-schema-repo-generator/README.md b/external-schema-repo-generator/README.md new file mode 100644 index 0000000..1ac06b2 --- /dev/null +++ b/external-schema-repo-generator/README.md @@ -0,0 +1,175 @@ +StndDefined schemas ConfigMap generator +======================================= + +## Description +StndDefined schemas Kubernetes ConfigMap generator is a Makefile with two targets running shell scripts: 'generate' and +'install'. This Makefile may be used by VES Collector users to generate and install ConfigMap containing schemas +and mapping file for stndDefined validation in VES pod. Additionally script creates file with snippets containing +auto-generated configuration of volumes finally mounted in VES deployment. Process of generation of ConfigMap spec file +is configurable via environment.config file. + +Generator files are available in oom/utils ONAP Gerrit repository under external-schema-repo-generator folder. + +## Requirements and limitations + +### Environment +Target *generate* from Makefile requires stable internet connection to properly clone Git repositories. +Target *install* should be ran on the RKE node, so it can create ConfigMap in the same Kubernetes environment as VES Pod +is installed. It is possible to generate a spec in one environment and move it together with all ConfigMap generator +files to the RKE environment. + +### Repository limitations +When running the script branches from selected repository are being downloaded. Time of script execution depends mostly +on repository size and number of schemas. All .yaml files from selected directory in Git repository will be considered +as schemas and attached to ConfigMap spec. + +### Generator tool files integration +It is recommended to consider files of this tool as a unity and not split them when moving generator through +environments. Generator tool files that are required are: +- Makefile +- install.sh +- generate.sh +- environment.config + +## Instruction + +### Parameters description +Before running any target from Makefile, configuration in *environment.config* must be properly prepared. Description of +the configurable properties is below. + +- **SPEC_CONFIGMAP_FILENAME** - Filename name of ConfigMap spec that will be generated. +- **K8S_CONFIGMAP_NAME** - Kubernetes name of ConfigMap that will be generated and installed. +- **SNIPPET_FILENAME** - Filename of snippet with autogenerated content that should be added to VES deployment. + +- **REPOSITORY_URL_HTTPS** - URL to remote Git repository which lets cloning repository via HTTPS. +- **REPOSITORY_BRANCH** - Valid branch from selected Git repository which contains schemas desired to mount. Script +accepts an array of branches from which schemas will be collected. To pass an array split branch names with space and +cover list in quotation marks. +- **SCHEMAS_LOCATION** - Path to schemas directory on selected Git repo. All YAML files from this repository will be +considered as schema and added to ConfigMap. + +- **VENDOR_NAME** - Name of organisation delivering schemas, used only for naming of schemas destination directory in +VES. + +### Running commands + +To run ConfigMap spec generation as well as snippet file used for mounting ConfigMap to specific Deployment use: + +``` +make generate +``` + +To run ConfigMap installation in Kubernetes use: + +**NOTE**: Remember about running this command on RKE node. + +``` +make install +``` + +**NOTE**: It is possible that ConfigMap with selected K8S_CONFIGMAP_NAME already exists in Kubernetes. In such situation +either regenerate spec with new K8S_CONFIGMAP_NAME or remove existing ConfigMap from Kubernetes and install spec again. +To remove ConfigMap from Kubernetes use: +``` +kubectl -n onap delete configmap <CONFIGMAP_NAME> +``` + +## ConfigMap validation +After running the script ConfigMap spec file is generated in current working directory. +Spec file can be manually validated via any text editor. The last file included in spec is schema-map.json file with +mappings of external URLs to prepared local URLs. + +To check whether it has been created use command: + +``` +kubectl -n onap get configmap | grep <CONFIGMAP_NAME> +``` + +A ConfigMap with configured name should be visible on the list. + +## Mounting ConfigMap into VES Collector + +To mount created ConfigMap in VES, its deployment must be edited. It can be done with: +``` +kubectl -n onap edit deployment dep-dcae-ves-collector +``` + +Snippets with content that should be added to VES deployment are generated in file with name setup in configuration file +under SNIPPET_FILENAME property. No extra configuration in VES is needed when using them. + +**NOTE**: When using Vi text editor for deployment edition, correct input mode must be set to keep proper indentation +when pasting snippets. Use ``:set paste `` to turn paste mode on. To close paste mode use ``:set nopaste`` + +1. Add volumeMounts element + + In spec.template.spec.containers[0].volumeMounts add new list element: + + **NOTE**: spec.template.spec.containers[0] should be a container with the image: + *nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector::x.x.x*. + It should be the first container, but make sure that the correct container is being edited. + + ``` + volumeMounts: + - ... + - mountPath: /opt/app/VESCollector/etc/externalRepoCustom + name: custom-3gpp-schemas + ``` + + - mountPath - Directory context for schemas. **NOTE**: must be the same as configuration of VES Collector property + *collector.externalSchema.schemasLocation* in *collector.properties*. This property might be modified via Consul UI, + later after changes in deployment. + + - name - Name of ConfigMap volume. Must be the same as set in the 2. step in *name* field. + +2. Add volumes element + + In spec.template.spec.volumes add a new list element with all desired to mount schemas from ConfigMap in + *items* list. *key* are file names from generated previously spec and *path* is relative path from directory set up + in step 1 as *mountPath*. + + Sample spec.template.spec.volumes content: + + ``` + volumes: + - configMap: + defaultMode: 420 + items: + - key: schema-map.json + path: schema-map.json + - key: SA88-Rel16-faultMnS.yaml + path: 3gpp/rep/sa5/data-models/SA88-Rel16/OpenAPI/faultMnS.yaml + - ... + name: stnd-defined-configmap + name: custom-3gpp-schemas + - ... + ``` + Fields description: + - name - Name of ConfigMap volume. Must be the same as set in the 1. step in *name* field. + - configMap.name - name of installed Kubernetes ConfigMap described in K8S_CONFIGMAP_NAME configuration setting + - configMap.items[].key - name of mounted file from installed previously ConfigMap. + **NOTE**: Not every schema from ConfigMap must be listed in *items* (warning message will be logged on VES startup if + so), but mapping file named as described in MAPPING_FILE_NAME setting is required for correct stndDefined validation + in VES. + - configMap.items[].path: Relative path from *mountPath* from step 1 which describes location of schema location in + VES container. + **NOTE 1**: For correct schemas detection in VES Collector *path* of each schema must be the same as its localURL in + mapping file. Mapping file is included as the last file in generated ConfigMap spec. + **NOTE 2**: For correct mapping file detection in VES Collector its *path* must be the same as in property + *collector.externalSchema.mappingFileLocation* in *collector.properties*. This property might be modified via Consul + UI, later after changes in deployment. + +3. Save and close an editor, K8S will automatically detect changes, terminate old VES Pod and deploy new one with +mounted ConfigMap. Correctness of new VES Pod initialization and mounting ConfigMap can be tracked using +`kubectl -n onap describe pod <VES_POD_NAME>`. + +To check if mounted schemas and VES properties configuration are correctly aligned see logs of VES. Each schema which is +present in mapping file, but is not detected by VES will be logged on application startup e.g.: +``` +2020-10-01 11:46:55.872 WARN 24 [ Thread-5] o.o.d.s.s.s.e.s.m.s.UrlMapperFactory : Local schema resource missing. Schema file with path /opt/app/VESCollector/etc/externalRepoCustom/3gpp/rep/sa5/MnS/tree/master/OpenAPI/5gcNrm.yaml has not been found. +2020-10-01 11:46:55.872 WARN 24 [ Thread-5] o.o.d.s.s.s.e.s.m.s.UrlMapperFactory : Mapping for publicURL ("https://forge.3gpp.org/rep/sa5/MnS/tree/master/OpenAPI/5gcNrm.yaml") will not be added to validator. +``` + +To see logs of VES use: +``` +kubectl -n onap logs <VES_POD_NAME> +``` |