From 5fb2f1d247be63de9e567c2fa0da3d3f0f1b4601 Mon Sep 17 00:00:00 2001 From: Michal Banka Date: Wed, 10 Mar 2021 14:30:09 +0100 Subject: Add ves-openapi-manager documentation Signed-off-by: Michal Banka Change-Id: I84294800070ddbec7bee1bc3ea4ac63df076cc53 Issue-ID: DCAEGEN2-2571 --- docs/sections/services/serviceindex.rst | 21 ++-- .../services/ves-openapi-manager/architecture.rst | 30 ++++++ .../services/ves-openapi-manager/artifacts.rst | 26 +++++ .../services/ves-openapi-manager/deployment.rst | 69 ++++++++++++ .../services/ves-openapi-manager/index.rst | 27 +++++ .../ves-openapi-manager/resources/architecture.png | Bin 0 -> 23024 bytes .../resources/artifact-no-stndDefined.yaml | 37 +++++++ .../artifact-stndDefined-no-schemaReference.yaml | 29 +++++ .../resources/artifact-stndDefined.yaml | 30 ++++++ .../resources/schema-map-example.json | 14 +++ .../resources/schema-map-invalid.json | 6 ++ .../ves-openapi-manager/resources/schema-map.json | 6 ++ .../ves-openapi-manager/resources/workflow.png | Bin 0 -> 110243 bytes .../services/ves-openapi-manager/use-cases.rst | 117 +++++++++++++++++++++ 14 files changed, 402 insertions(+), 10 deletions(-) create mode 100644 docs/sections/services/ves-openapi-manager/architecture.rst create mode 100644 docs/sections/services/ves-openapi-manager/artifacts.rst create mode 100644 docs/sections/services/ves-openapi-manager/deployment.rst create mode 100644 docs/sections/services/ves-openapi-manager/index.rst create mode 100644 docs/sections/services/ves-openapi-manager/resources/architecture.png create mode 100644 docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml create mode 100644 docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml create mode 100644 docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml create mode 100644 docs/sections/services/ves-openapi-manager/resources/schema-map-example.json create mode 100644 docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json create mode 100644 docs/sections/services/ves-openapi-manager/resources/schema-map.json create mode 100644 docs/sections/services/ves-openapi-manager/resources/workflow.png create mode 100644 docs/sections/services/ves-openapi-manager/use-cases.rst (limited to 'docs/sections/services') diff --git a/docs/sections/services/serviceindex.rst b/docs/sections/services/serviceindex.rst index 10c2bd1b..9a585e5d 100644 --- a/docs/sections/services/serviceindex.rst +++ b/docs/sections/services/serviceindex.rst @@ -10,19 +10,20 @@ DCAE Service components .. toctree:: :maxdepth: 1 - ./snmptrap/index.rst - ./ves-http/index.rst - ./ves-hv/index.rst - ./mapper/index.rst - ./prh/index.rst + ./bbs-event-processor/index.rst + ./datalake-handler/index.rst ./dfc/index.rst ./heartbeat-ms/index.rst + ./kpi-computation-ms/index.rst + ./mapper/index.rst ./pm-mapper/index.rst ./pm-subscription-handler/index.rst - ./bbs-event-processor/index.rst - ./son-handler/index.rst + ./prh/index.rst ./restconf/index.rst - ./tcagen2-docker/index.rst - ./datalake-handler/index.rst ./slice-analysis-ms/index.rst - ./kpi-computation-ms/index.rst \ No newline at end of file + ./snmptrap/index.rst + ./son-handler/index.rst + ./tcagen2-docker/index.rst + ./ves-hv/index.rst + ./ves-http/index.rst + ./ves-openapi-manager/index.rst diff --git a/docs/sections/services/ves-openapi-manager/architecture.rst b/docs/sections/services/ves-openapi-manager/architecture.rst new file mode 100644 index 00000000..2e4c4276 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/architecture.rst @@ -0,0 +1,30 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-architecture: + +VES OpenAPI Manager architecture +================================ +Functionalities of VES OpenAPI Manager require communication with other ONAP components. Because of that, SDC +Distribution Client has been used as a library to achieve such a communication. There are two components required by +application to work: SDC BE and Message Router. SDC Distribution Client provides communication with both of them when +it's properly configured (for application configuration instruction refer to: :ref:`ves-openapi-manager-deployment` ). + +.. image:: resources/architecture.png + + +.. _ves-openapi-manager-flow: + +VES OpenAPI Manager workflow +---------------------------- +VES OpenAPI Manager workflow can be split into phases: + +1) Listening for Service Model distribution events +2) Optional downloading of artifacts depending on Service Model contents. At least one Service Model resource must contain *VES_EVENT* type artifacts. +3) Optional validation of artifacts depending on content of downloaded artifacts. Artifact must contain stndDefined events declaration. + + +VES OpenAPI Manager workflow is presented on the diagram below. + +.. image:: resources/workflow.png + diff --git a/docs/sections/services/ves-openapi-manager/artifacts.rst b/docs/sections/services/ves-openapi-manager/artifacts.rst new file mode 100644 index 00000000..f42a470a --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/artifacts.rst @@ -0,0 +1,26 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-artifacts: + +VES OpenAPI Manager artifacts and delivery +========================================== +VES OpenAPI Manager is delivered as a docker container and published in ONAP Nexus repository following image naming +convention. + +Image +----- +Full image name is *onap/org.onap.dcaegen2.platform.ves-openapi-manager*. + +Versioning +---------- +VES OpenAPI Manager keeps its Changelog in the repository. It's available here: +`Changelog `_ + +Use *latest* image tag to get the most recent version of VES OpenAPI Manager. + +Repository +---------- +Repository with the code of VES OpenAPI Manager is available on ONAP Gerrit: +`Gerrit `_ + diff --git a/docs/sections/services/ves-openapi-manager/deployment.rst b/docs/sections/services/ves-openapi-manager/deployment.rst new file mode 100644 index 00000000..fdec791a --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/deployment.rst @@ -0,0 +1,69 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-deployment: + +VES OpenAPI Manager deployment +============================== +VES OpenAPI Manager is a simple Java application which can be started by using only Java 11+, yet it has some +prerequisites to work correctly: + +1) File with OpenAPI schemas mappings. +2) Access to two ONAP services: SDC BE and Message Router. + +These prerequisites are met by default when using Helm charts created for VES OpenAPI Manager in OOM. It's described in +more detail in *Helm chart* section. + +There is also available a simple configuration via environment variables which are optional. It's described in more +detail in *Environment variables* section. + +File with OpenAPI schemas mappings +---------------------------------- +VES OpenAPI checks whether schemaReferences of distributed service align with stndDefined schemas from VES Collector. +To achieve that application should receive a file with mappings used by VES. Because there are few ways to run the +application, it contains its own default file which assure that application will work. Default file may be overwritten +or edited any time, even during application runtime. + +Helm charts which are used to deploy application in ONAP cluster are configured to overwrite default mapping file with +file from predefined ConfigMap (named *dcae-external-repo-configmap-schema-map*), which is also used by VES Collector. +Using ConfigMap ensures that both: VES OpenAPI Manager and VES Collector use the exact same file. + +.. warning:: + VES OpenAPI Manager does not check if the used mapping file is the same file that VES uses. Within ONAP, the working + assumption is, that both: VES openAPI Manager and VES Collector leverage the same Kubernetes ConfigMaps, which + contain the schema-mapping file and respective openAPI descriptions. + +VES OpenAPI Manager has a configurable property which contains path to the mapping file. It has to be set before the +application startup. It can be done by setting environment variable *SCHEMA_MAP_PATH*. Helm charts are preconfigured to +set this variable. + +Environment variables +--------------------- +There are environment variables which must be used for configuration. Helm chart contain predefined values which are +valid when running VES OpenAPI Manager from its released image in the ONAP cluster. + ++-----------------+---------------------------+----------------------+ +| Variable name | Description | Helm chart values | ++=================+===========================+======================+ +| SCHEMA_MAP_PATH | Path to the mapping file. | /app/schema-map.json | ++-----------------+---------------------------+----------------------+ +| ASDC_ADDRESS | URL to SDC BE. | sdc-be:8443 | ++-----------------+---------------------------+----------------------+ + + +Helm chart +---------- +By default VES OpenAPI Manger is deployed via Helm as the DCAE subcomponent in the ONAP cluster. Helm chart is +configured to deploy application with all prerequisites met. It achieves that by: + +1) Mounting ConfigMap with mapping file under */app/schema-map.json* path. +2) Proper setting environment variables to values described in section *Environment variables*. Mapping file path is set to point to mounted file and SDC BE URL is set to internal port available only from Kubernetes cluster. +3) Setting Readiness check. It waits for other ONAP components to start: SDC BE, Message Router. VES OpenAPI Manager Pod will not start until they are not ready. + +Local deployment +---------------- +It's possible to run VES OpenAPI Manager in local environment which connects to external lab with ONAP. This way +requires exposing ports of some services on lab, creating local port tunneling and running VES OpenAPI Manager (using +docker-compose or IDE e.g. IntelliJ). + +It's described in more detail in the README in project repository (`README `_). \ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/index.rst b/docs/sections/services/ves-openapi-manager/index.rst new file mode 100644 index 00000000..e6cc15c3 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/index.rst @@ -0,0 +1,27 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +VES OpenAPI Manager +=================== + +VES OpenAPI Manager has been created to validate the presence of OpenAPI schemas declared in *VES_EVENT* type artifacts, +within the DCAE run-time environment during Service Model distribution in SDC. When deployed, it automatically listens +to events of Service Models distributions by using SDC Distribution Client in order to read the declared OpenAPI +descriptions. Purpose of this component is to partially validate artifacts of type *VES_EVENT* from Resources of +distributed services. During validation phase it checks whether *stndDefined* events defined in VES_EVENT type artifact, +contain only *schemaReferences* that local copies are accessible by DCAE VES Collector. If any of schemaReference is +absent in local externalSchema repository, the VES OpenAPI Manager informs ONAP user which schemas need to be uploaded +to the DCAE run-time environment. + + +VES OpenAPI Manager overview and functions +------------------------------------------ + +.. toctree:: + :maxdepth: 1 + + ./architecture.rst + ./artifacts.rst + ./deployment.rst + ./use-cases.rst diff --git a/docs/sections/services/ves-openapi-manager/resources/architecture.png b/docs/sections/services/ves-openapi-manager/resources/architecture.png new file mode 100644 index 00000000..5df1198f Binary files /dev/null and b/docs/sections/services/ves-openapi-manager/resources/architecture.png differ diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml new file mode 100644 index 00000000..73c6919e --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml @@ -0,0 +1,37 @@ +--- +event: + presence: required + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: notification} + eventName: {presence: required, value: Noti_MyPnf-Acme_FileReady} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.0.1} + vesEventListenerVersion: {presence: required, value: 7.0.1} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + notificationFields: + presence: required + structure: + changeIdentifier: {presence: required, value: PM_MEAS_FILES} + changeType: {presence: required, value: fileReady} + notificationFieldsVersion: {presence: required, value: 2.0} + arrayOfNamedHashMap: + presence: required + array: + - name: {presence: required} + hashMap: {presence: required, structure: { + keyValuePair: {presence: required, structure: {key: {presence: required, value: location}, value: {presence: required}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: compression}, value: {presence: required, value: gzip}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatType}, value: {presence: required, value: org.3GPP.32.435}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatVersion}, value: {presence: required, value: V10}}}} + } +... \ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml new file mode 100644 index 00000000..6bba640e --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml @@ -0,0 +1,29 @@ +--- +event: + presence: required + comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification" + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: stndDefined} + eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.1} + vesEventListenerVersion: {presence: required, value: 7.2} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"} + stndDefinedFields: + presence: required + structure: + data: {presence: required} + stndDefinedFieldsVersion: {presence: required, value: "1.0"} + +... \ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml new file mode 100644 index 00000000..ad3ca469 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml @@ -0,0 +1,30 @@ +--- +event: + presence: required + comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification" + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: stndDefined} + eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.1} + vesEventListenerVersion: {presence: required, value: 7.2} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"} + stndDefinedFields: + presence: required + structure: + schemaReference: { presence: required, value: "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm" } + data: {presence: required} + stndDefinedFieldsVersion: {presence: required, value: "1.0"} + +... \ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json new file mode 100644 index 00000000..e11851c0 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json @@ -0,0 +1,14 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml" + }, + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml" + }, + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml" + } +] \ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json new file mode 100644 index 00000000..fb34f5ae --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json @@ -0,0 +1,6 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml" + } +] \ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map.json b/docs/sections/services/ves-openapi-manager/resources/schema-map.json new file mode 100644 index 00000000..97883fbc --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map.json @@ -0,0 +1,6 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml" + } +] \ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/workflow.png b/docs/sections/services/ves-openapi-manager/resources/workflow.png new file mode 100644 index 00000000..423792b6 Binary files /dev/null and b/docs/sections/services/ves-openapi-manager/resources/workflow.png differ diff --git a/docs/sections/services/ves-openapi-manager/use-cases.rst b/docs/sections/services/ves-openapi-manager/use-cases.rst new file mode 100644 index 00000000..aa3a4bf8 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/use-cases.rst @@ -0,0 +1,117 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-use-cases: + +VES OpenAPI Manager validation use-cases +======================================== +The main VES OpenAPI Manager use case is to verify if the schemaReferences declared in *VES_EVENT* type artifacts are +present in the local DCAE run-time externalSchemaRepo and show validation results to user in SDC UI. + +The general flow of VES OpenAPI Manager is available here :ref:`ves-openapi-manager-flow`. + +Based on the referenced flow, there are few possible behaviours of VES OpenAPI Manager. In this section two main flows: +successful and unsuccessful validation will be described step by step. + +Validation prerequisites +------------------------ +Validation phase takes place only when specific conditions are met. + +1) VES OpenAPI Manager is properly configured: client is connected to SDC and mapping file is present and pointed in configuration. Configuration is described in detail here: :ref:`ves-openapi-manager-deployment`. +2) Distribution of a Service Model takes place in SDC. +3) Service contains an *VES_EVENT* type artifact. +4) Artifact content is correctly downloaded. + +Validation description +---------------------- +When *schemaReference* field from artifact is being validated, only the part of the URI that indicates public openAPI +description file location is taken into consideration. + +For example when *schemaReference* with value +*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm* +is found in artifact, then only the part before # sign (public openAPI description file location URI part) is being +validated. This way part which would be validated is +*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml*. + +Mapping file must have a predefined JSON format of list of objects (mappings) with publicURL and localURL fields. +Example with 3 mappings: + +.. literalinclude:: resources/schema-map-example.json + :language: json + +When *schemaReference* is split, it's compared to each publicURL from mapping file. If there is no publicURL in mapping +file which matches schemaReference, then schemaReference is marked as invalid. This process is executed for all +stndDefined events defined in *VES_EVENT* artifact, which declare a schemaReference. All invalid references are returned +to user via SDC UI when validation of a complete artifact ends. + +Based on returned information with invalid references user can take action and e.g. add mappings and schemas to DCAE +run-time environment by editing ConfigMaps which store them. + ++-----------------------------------------+-------------------------+ +| ConfigMap name | Content | ++=========================================+=========================+ +| dcae-external-repo-configmap-schema-map | Mapping file | ++-----------------------------------------+-------------------------+ +| dcae-external-repo-configmap-sa88-rel16 | OpenAPI schemas content,| +| | example stores 3GPP | +| | sa88-rel16 schemas | ++-----------------------------------------+-------------------------+ + + +Successful validation case +-------------------------- +There are few ways to get a successful validation status - *DEPLOY_OK*. + +1) When artifact *VES_EVENT* does not contain *stndDefined* events definitions. Only *stndDefined* event are validated. +2) When artifact *VES_EVENT* contains *stndDefined* events definitions but *schemaReference* fields are not present. +3) When artifact *VES_EVENT* contains *stndDefined* events definitions and each *schemaReference* of the event is present in the mapping file. + + +*VES_EVENT* artifact may contain more than one event definition. Examples of valid artifacts with single events are +below. + +Example of valid artifact without *stndDefined* event definition (case 1): + +.. literalinclude:: resources/artifact-no-stndDefined.yaml + :language: yaml + +Example of valid artifact with *stndDefined* event definition, but without schemaReference field (case 2): + +.. literalinclude:: resources/artifact-stndDefined-no-schemaReference.yaml + :language: yaml + +Example of artifact with *stndDefined* event definition (case 3): + +.. literalinclude:: resources/artifact-stndDefined.yaml + :language: yaml + +which is valid when mapping file contains a mapping of schemaReference field. +Example of mapping file content which makes example artifact valid: + +.. literalinclude:: resources/schema-map.json + :language: json + +Unsuccessful validation case +---------------------------- +Another case is an unsuccessful validation case which sends status *DEPLOY_ERROR* with error message containing listed +*schemaReference* that are missing from mapping file. Fail case might occur: + +1) When artifact *VES_EVENT* contains *stndDefined* events definitions and any of *schemaReference* is not present in mapping file. + +Example of artifact with *stndDefined* event definition: + +.. literalinclude:: resources/artifact-stndDefined.yaml + :language: yaml + +which is invalid when mapping file does not contain a mapping of schemaReference field. +Example of mapping file which makes example artifact invalid: + +.. literalinclude:: resources/schema-map-invalid.json + :language: json + +Validation results +------------------ +There are two ways to receive validation results. + +1) Via SDC UI. Results are available in *Service->Distributions* view. To see results in SDC UI user has to wait up to few minutes. +2) In VES OpenAPI Manager logs. They are printed right after validation. \ No newline at end of file -- cgit 1.2.3-korg