diff options
15 files changed, 404 insertions, 10 deletions
diff --git a/docs/sections/installation_oom.rst b/docs/sections/installation_oom.rst index 443b970a..671919a2 100644 --- a/docs/sections/installation_oom.rst +++ b/docs/sections/installation_oom.rst @@ -52,6 +52,7 @@ The dcaegen2 chart has the following sub-charts: * ``dcae-redis``: deploys the DCAE Redis cluster. * ``dcae-dashboard``: deploys the DCAE Dashboard for managing DCAE microservices deployments * ``dcae-servicechange-handler``: deploys the DCAE service change handler service. A subchart (``dcae-inventory-api``) deploys the DCAE inventory API service. +* ``dcae-ves-openapi-manager``: deploys the DCAE service validator of VES_EVENT type artifacts from distributed services. DCAE Deployment --------------- @@ -73,6 +74,7 @@ These include: * Inventory postgres database service (launched as a dependency of the inventory API service) * DCAE postgres database service (launched as a dependency of the bootstrap service) * DCAE Redis cluster + * VES OpenAPI Manager Some of the DCAE subcharts include an initContainer that checks to see if other services that they need in order to run have become ready. The installation diff --git a/docs/sections/services/serviceindex.rst b/docs/sections/services/serviceindex.rst index 10c2bd1b..9a585e5d 100644 --- a/docs/sections/services/serviceindex.rst +++ b/docs/sections/services/serviceindex.rst @@ -10,19 +10,20 @@ DCAE Service components .. toctree::
:maxdepth: 1
- ./snmptrap/index.rst
- ./ves-http/index.rst
- ./ves-hv/index.rst
- ./mapper/index.rst
- ./prh/index.rst
+ ./bbs-event-processor/index.rst
+ ./datalake-handler/index.rst
./dfc/index.rst
./heartbeat-ms/index.rst
+ ./kpi-computation-ms/index.rst
+ ./mapper/index.rst
./pm-mapper/index.rst
./pm-subscription-handler/index.rst
- ./bbs-event-processor/index.rst
- ./son-handler/index.rst
+ ./prh/index.rst
./restconf/index.rst
- ./tcagen2-docker/index.rst
- ./datalake-handler/index.rst
./slice-analysis-ms/index.rst
- ./kpi-computation-ms/index.rst
\ No newline at end of file + ./snmptrap/index.rst
+ ./son-handler/index.rst
+ ./tcagen2-docker/index.rst
+ ./ves-hv/index.rst
+ ./ves-http/index.rst
+ ./ves-openapi-manager/index.rst
diff --git a/docs/sections/services/ves-openapi-manager/architecture.rst b/docs/sections/services/ves-openapi-manager/architecture.rst new file mode 100644 index 00000000..2e4c4276 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/architecture.rst @@ -0,0 +1,30 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-architecture: + +VES OpenAPI Manager architecture +================================ +Functionalities of VES OpenAPI Manager require communication with other ONAP components. Because of that, SDC +Distribution Client has been used as a library to achieve such a communication. There are two components required by +application to work: SDC BE and Message Router. SDC Distribution Client provides communication with both of them when +it's properly configured (for application configuration instruction refer to: :ref:`ves-openapi-manager-deployment` ). + +.. image:: resources/architecture.png + + +.. _ves-openapi-manager-flow: + +VES OpenAPI Manager workflow +---------------------------- +VES OpenAPI Manager workflow can be split into phases: + +1) Listening for Service Model distribution events +2) Optional downloading of artifacts depending on Service Model contents. At least one Service Model resource must contain *VES_EVENT* type artifacts. +3) Optional validation of artifacts depending on content of downloaded artifacts. Artifact must contain stndDefined events declaration. + + +VES OpenAPI Manager workflow is presented on the diagram below. + +.. image:: resources/workflow.png + diff --git a/docs/sections/services/ves-openapi-manager/artifacts.rst b/docs/sections/services/ves-openapi-manager/artifacts.rst new file mode 100644 index 00000000..f42a470a --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/artifacts.rst @@ -0,0 +1,26 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-artifacts: + +VES OpenAPI Manager artifacts and delivery +========================================== +VES OpenAPI Manager is delivered as a docker container and published in ONAP Nexus repository following image naming +convention. + +Image +----- +Full image name is *onap/org.onap.dcaegen2.platform.ves-openapi-manager*. + +Versioning +---------- +VES OpenAPI Manager keeps its Changelog in the repository. It's available here: +`Changelog <https://github.com/onap/dcaegen2-platform-ves-openapi-manager/blob/master/Changelog.md>`_ + +Use *latest* image tag to get the most recent version of VES OpenAPI Manager. + +Repository +---------- +Repository with the code of VES OpenAPI Manager is available on ONAP Gerrit: +`Gerrit <https://gerrit.onap.org/r/admin/repos/dcaegen2/platform/ves-openapi-manager>`_ + diff --git a/docs/sections/services/ves-openapi-manager/deployment.rst b/docs/sections/services/ves-openapi-manager/deployment.rst new file mode 100644 index 00000000..fdec791a --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/deployment.rst @@ -0,0 +1,69 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-deployment: + +VES OpenAPI Manager deployment +============================== +VES OpenAPI Manager is a simple Java application which can be started by using only Java 11+, yet it has some +prerequisites to work correctly: + +1) File with OpenAPI schemas mappings. +2) Access to two ONAP services: SDC BE and Message Router. + +These prerequisites are met by default when using Helm charts created for VES OpenAPI Manager in OOM. It's described in +more detail in *Helm chart* section. + +There is also available a simple configuration via environment variables which are optional. It's described in more +detail in *Environment variables* section. + +File with OpenAPI schemas mappings +---------------------------------- +VES OpenAPI checks whether schemaReferences of distributed service align with stndDefined schemas from VES Collector. +To achieve that application should receive a file with mappings used by VES. Because there are few ways to run the +application, it contains its own default file which assure that application will work. Default file may be overwritten +or edited any time, even during application runtime. + +Helm charts which are used to deploy application in ONAP cluster are configured to overwrite default mapping file with +file from predefined ConfigMap (named *dcae-external-repo-configmap-schema-map*), which is also used by VES Collector. +Using ConfigMap ensures that both: VES OpenAPI Manager and VES Collector use the exact same file. + +.. warning:: + VES OpenAPI Manager does not check if the used mapping file is the same file that VES uses. Within ONAP, the working + assumption is, that both: VES openAPI Manager and VES Collector leverage the same Kubernetes ConfigMaps, which + contain the schema-mapping file and respective openAPI descriptions. + +VES OpenAPI Manager has a configurable property which contains path to the mapping file. It has to be set before the +application startup. It can be done by setting environment variable *SCHEMA_MAP_PATH*. Helm charts are preconfigured to +set this variable. + +Environment variables +--------------------- +There are environment variables which must be used for configuration. Helm chart contain predefined values which are +valid when running VES OpenAPI Manager from its released image in the ONAP cluster. + ++-----------------+---------------------------+----------------------+ +| Variable name | Description | Helm chart values | ++=================+===========================+======================+ +| SCHEMA_MAP_PATH | Path to the mapping file. | /app/schema-map.json | ++-----------------+---------------------------+----------------------+ +| ASDC_ADDRESS | URL to SDC BE. | sdc-be:8443 | ++-----------------+---------------------------+----------------------+ + + +Helm chart +---------- +By default VES OpenAPI Manger is deployed via Helm as the DCAE subcomponent in the ONAP cluster. Helm chart is +configured to deploy application with all prerequisites met. It achieves that by: + +1) Mounting ConfigMap with mapping file under */app/schema-map.json* path. +2) Proper setting environment variables to values described in section *Environment variables*. Mapping file path is set to point to mounted file and SDC BE URL is set to internal port available only from Kubernetes cluster. +3) Setting Readiness check. It waits for other ONAP components to start: SDC BE, Message Router. VES OpenAPI Manager Pod will not start until they are not ready. + +Local deployment +---------------- +It's possible to run VES OpenAPI Manager in local environment which connects to external lab with ONAP. This way +requires exposing ports of some services on lab, creating local port tunneling and running VES OpenAPI Manager (using +docker-compose or IDE e.g. IntelliJ). + +It's described in more detail in the README in project repository (`README <https://github.com/onap/dcaegen2-platform-ves-openapi-manager/blob/master/README.md>`_).
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/index.rst b/docs/sections/services/ves-openapi-manager/index.rst new file mode 100644 index 00000000..e6cc15c3 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/index.rst @@ -0,0 +1,27 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +VES OpenAPI Manager +=================== + +VES OpenAPI Manager has been created to validate the presence of OpenAPI schemas declared in *VES_EVENT* type artifacts, +within the DCAE run-time environment during Service Model distribution in SDC. When deployed, it automatically listens +to events of Service Models distributions by using SDC Distribution Client in order to read the declared OpenAPI +descriptions. Purpose of this component is to partially validate artifacts of type *VES_EVENT* from Resources of +distributed services. During validation phase it checks whether *stndDefined* events defined in VES_EVENT type artifact, +contain only *schemaReferences* that local copies are accessible by DCAE VES Collector. If any of schemaReference is +absent in local externalSchema repository, the VES OpenAPI Manager informs ONAP user which schemas need to be uploaded +to the DCAE run-time environment. + + +VES OpenAPI Manager overview and functions +------------------------------------------ + +.. toctree:: + :maxdepth: 1 + + ./architecture.rst + ./artifacts.rst + ./deployment.rst + ./use-cases.rst diff --git a/docs/sections/services/ves-openapi-manager/resources/architecture.png b/docs/sections/services/ves-openapi-manager/resources/architecture.png Binary files differnew file mode 100644 index 00000000..5df1198f --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/architecture.png diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml new file mode 100644 index 00000000..73c6919e --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml @@ -0,0 +1,37 @@ +--- +event: + presence: required + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: notification} + eventName: {presence: required, value: Noti_MyPnf-Acme_FileReady} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.0.1} + vesEventListenerVersion: {presence: required, value: 7.0.1} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + notificationFields: + presence: required + structure: + changeIdentifier: {presence: required, value: PM_MEAS_FILES} + changeType: {presence: required, value: fileReady} + notificationFieldsVersion: {presence: required, value: 2.0} + arrayOfNamedHashMap: + presence: required + array: + - name: {presence: required} + hashMap: {presence: required, structure: { + keyValuePair: {presence: required, structure: {key: {presence: required, value: location}, value: {presence: required}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: compression}, value: {presence: required, value: gzip}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatType}, value: {presence: required, value: org.3GPP.32.435}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatVersion}, value: {presence: required, value: V10}}}} + } +...
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml new file mode 100644 index 00000000..6bba640e --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml @@ -0,0 +1,29 @@ +--- +event: + presence: required + comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification" + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: stndDefined} + eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.1} + vesEventListenerVersion: {presence: required, value: 7.2} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"} + stndDefinedFields: + presence: required + structure: + data: {presence: required} + stndDefinedFieldsVersion: {presence: required, value: "1.0"} + +...
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml new file mode 100644 index 00000000..ad3ca469 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml @@ -0,0 +1,30 @@ +--- +event: + presence: required + comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification" + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: stndDefined} + eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.1} + vesEventListenerVersion: {presence: required, value: 7.2} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"} + stndDefinedFields: + presence: required + structure: + schemaReference: { presence: required, value: "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm" } + data: {presence: required} + stndDefinedFieldsVersion: {presence: required, value: "1.0"} + +...
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json new file mode 100644 index 00000000..e11851c0 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json @@ -0,0 +1,14 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml" + }, + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml" + }, + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml" + } +]
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json new file mode 100644 index 00000000..fb34f5ae --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json @@ -0,0 +1,6 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml" + } +]
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map.json b/docs/sections/services/ves-openapi-manager/resources/schema-map.json new file mode 100644 index 00000000..97883fbc --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map.json @@ -0,0 +1,6 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml" + } +]
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/workflow.png b/docs/sections/services/ves-openapi-manager/resources/workflow.png Binary files differnew file mode 100644 index 00000000..423792b6 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/workflow.png diff --git a/docs/sections/services/ves-openapi-manager/use-cases.rst b/docs/sections/services/ves-openapi-manager/use-cases.rst new file mode 100644 index 00000000..aa3a4bf8 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/use-cases.rst @@ -0,0 +1,117 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-use-cases: + +VES OpenAPI Manager validation use-cases +======================================== +The main VES OpenAPI Manager use case is to verify if the schemaReferences declared in *VES_EVENT* type artifacts are +present in the local DCAE run-time externalSchemaRepo and show validation results to user in SDC UI. + +The general flow of VES OpenAPI Manager is available here :ref:`ves-openapi-manager-flow`. + +Based on the referenced flow, there are few possible behaviours of VES OpenAPI Manager. In this section two main flows: +successful and unsuccessful validation will be described step by step. + +Validation prerequisites +------------------------ +Validation phase takes place only when specific conditions are met. + +1) VES OpenAPI Manager is properly configured: client is connected to SDC and mapping file is present and pointed in configuration. Configuration is described in detail here: :ref:`ves-openapi-manager-deployment`. +2) Distribution of a Service Model takes place in SDC. +3) Service contains an *VES_EVENT* type artifact. +4) Artifact content is correctly downloaded. + +Validation description +---------------------- +When *schemaReference* field from artifact is being validated, only the part of the URI that indicates public openAPI +description file location is taken into consideration. + +For example when *schemaReference* with value +*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm* +is found in artifact, then only the part before # sign (public openAPI description file location URI part) is being +validated. This way part which would be validated is +*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml*. + +Mapping file must have a predefined JSON format of list of objects (mappings) with publicURL and localURL fields. +Example with 3 mappings: + +.. literalinclude:: resources/schema-map-example.json + :language: json + +When *schemaReference* is split, it's compared to each publicURL from mapping file. If there is no publicURL in mapping +file which matches schemaReference, then schemaReference is marked as invalid. This process is executed for all +stndDefined events defined in *VES_EVENT* artifact, which declare a schemaReference. All invalid references are returned +to user via SDC UI when validation of a complete artifact ends. + +Based on returned information with invalid references user can take action and e.g. add mappings and schemas to DCAE +run-time environment by editing ConfigMaps which store them. + ++-----------------------------------------+-------------------------+ +| ConfigMap name | Content | ++=========================================+=========================+ +| dcae-external-repo-configmap-schema-map | Mapping file | ++-----------------------------------------+-------------------------+ +| dcae-external-repo-configmap-sa88-rel16 | OpenAPI schemas content,| +| | example stores 3GPP | +| | sa88-rel16 schemas | ++-----------------------------------------+-------------------------+ + + +Successful validation case +-------------------------- +There are few ways to get a successful validation status - *DEPLOY_OK*. + +1) When artifact *VES_EVENT* does not contain *stndDefined* events definitions. Only *stndDefined* event are validated. +2) When artifact *VES_EVENT* contains *stndDefined* events definitions but *schemaReference* fields are not present. +3) When artifact *VES_EVENT* contains *stndDefined* events definitions and each *schemaReference* of the event is present in the mapping file. + + +*VES_EVENT* artifact may contain more than one event definition. Examples of valid artifacts with single events are +below. + +Example of valid artifact without *stndDefined* event definition (case 1): + +.. literalinclude:: resources/artifact-no-stndDefined.yaml + :language: yaml + +Example of valid artifact with *stndDefined* event definition, but without schemaReference field (case 2): + +.. literalinclude:: resources/artifact-stndDefined-no-schemaReference.yaml + :language: yaml + +Example of artifact with *stndDefined* event definition (case 3): + +.. literalinclude:: resources/artifact-stndDefined.yaml + :language: yaml + +which is valid when mapping file contains a mapping of schemaReference field. +Example of mapping file content which makes example artifact valid: + +.. literalinclude:: resources/schema-map.json + :language: json + +Unsuccessful validation case +---------------------------- +Another case is an unsuccessful validation case which sends status *DEPLOY_ERROR* with error message containing listed +*schemaReference* that are missing from mapping file. Fail case might occur: + +1) When artifact *VES_EVENT* contains *stndDefined* events definitions and any of *schemaReference* is not present in mapping file. + +Example of artifact with *stndDefined* event definition: + +.. literalinclude:: resources/artifact-stndDefined.yaml + :language: yaml + +which is invalid when mapping file does not contain a mapping of schemaReference field. +Example of mapping file which makes example artifact invalid: + +.. literalinclude:: resources/schema-map-invalid.json + :language: json + +Validation results +------------------ +There are two ways to receive validation results. + +1) Via SDC UI. Results are available in *Service->Distributions* view. To see results in SDC UI user has to wait up to few minutes. +2) In VES OpenAPI Manager logs. They are printed right after validation.
\ No newline at end of file |
