Merge "Add ves-openapi-manager documentation"

13 files changed, 391 insertions, 0 deletions

diff --git a/docs/sections/services/ves-openapi-manager/architecture.rst b/docs/sections/services/ves-openapi-manager/architecture.rst
new file mode 100644
index 00000000..2e4c4276
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/architecture.rst
@@ -0,0 +1,30 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. _ves-openapi-manager-architecture:
+
+VES OpenAPI Manager architecture
+================================
+Functionalities of VES OpenAPI Manager require communication with other ONAP components. Because of that, SDC
+Distribution Client has been used as a library to achieve such a communication. There are two components required by
+application to work: SDC BE and Message Router. SDC Distribution Client provides communication with both of them when
+it's properly configured (for application configuration instruction refer to: :ref:`ves-openapi-manager-deployment` ).
+
+.. image:: resources/architecture.png
+
+
+.. _ves-openapi-manager-flow:
+
+VES OpenAPI Manager workflow
+----------------------------
+VES OpenAPI Manager workflow can be split into phases:
+
+1) Listening for Service Model distribution events
+2) Optional downloading of artifacts depending on Service Model contents. At least one Service Model resource must contain *VES_EVENT* type artifacts.
+3) Optional validation of artifacts depending on content of downloaded artifacts. Artifact must contain stndDefined events declaration.
+
+
+VES OpenAPI Manager workflow is presented on the diagram below.
+
+.. image:: resources/workflow.png
+
diff --git a/docs/sections/services/ves-openapi-manager/artifacts.rst b/docs/sections/services/ves-openapi-manager/artifacts.rst
new file mode 100644
index 00000000..f42a470a
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/artifacts.rst
@@ -0,0 +1,26 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. _ves-openapi-manager-artifacts:
+
+VES OpenAPI Manager artifacts and delivery
+==========================================
+VES OpenAPI Manager is delivered as a docker container and published in ONAP Nexus repository following image naming
+convention.
+
+Image
+-----
+Full image name is *onap/org.onap.dcaegen2.platform.ves-openapi-manager*.
+
+Versioning
+----------
+VES OpenAPI Manager keeps its Changelog in the repository. It's available here:
+`Changelog <https://github.com/onap/dcaegen2-platform-ves-openapi-manager/blob/master/Changelog.md>`_
+
+Use *latest* image tag to get the most recent version of VES OpenAPI Manager.
+
+Repository
+----------
+Repository with the code of VES OpenAPI Manager is available on ONAP Gerrit:
+`Gerrit <https://gerrit.onap.org/r/admin/repos/dcaegen2/platform/ves-openapi-manager>`_
+
diff --git a/docs/sections/services/ves-openapi-manager/deployment.rst b/docs/sections/services/ves-openapi-manager/deployment.rst
new file mode 100644
index 00000000..fdec791a
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/deployment.rst
@@ -0,0 +1,69 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. _ves-openapi-manager-deployment:
+
+VES OpenAPI Manager deployment
+==============================
+VES OpenAPI Manager is a simple Java application which can be started by using only Java 11+, yet it has some
+prerequisites to work correctly:
+
+1) File with OpenAPI schemas mappings.
+2) Access to two ONAP services: SDC BE and Message Router.
+
+These prerequisites are met by default when using Helm charts created for VES OpenAPI Manager in OOM. It's described in
+more detail in *Helm chart* section.
+
+There is also available a simple configuration via environment variables which are optional. It's described in more
+detail in *Environment variables* section.
+
+File with OpenAPI schemas mappings
+----------------------------------
+VES OpenAPI checks whether schemaReferences of distributed service align with stndDefined schemas from VES Collector.
+To achieve that application should receive a file with mappings used by VES. Because there are few ways to run the
+application, it contains its own default file which assure that application will work. Default file may be overwritten
+or edited any time, even during application runtime.
+
+Helm charts which are used to deploy application in ONAP cluster are configured to overwrite default mapping file with
+file from predefined ConfigMap (named *dcae-external-repo-configmap-schema-map*), which is also used by VES Collector.
+Using ConfigMap ensures that both: VES OpenAPI Manager and VES Collector use the exact same file.
+
+.. warning::
+    VES OpenAPI Manager does not check if the used mapping file is the same file that VES uses. Within ONAP, the working
+    assumption is, that both: VES openAPI Manager and VES Collector leverage the same Kubernetes ConfigMaps, which
+    contain the schema-mapping file and respective openAPI descriptions.
+
+VES OpenAPI Manager has a configurable property which contains path to the mapping file. It has to be set before the
+application startup. It can be done by setting environment variable *SCHEMA_MAP_PATH*. Helm charts are preconfigured to
+set this variable.
+
+Environment variables
+---------------------
+There are environment variables which must be used for configuration. Helm chart contain predefined values which are
+valid when running VES OpenAPI Manager from its released image in the ONAP cluster.
+
++-----------------+---------------------------+----------------------+
+| Variable name   | Description               | Helm chart values    |
++=================+===========================+======================+
+| SCHEMA_MAP_PATH | Path to the mapping file. | /app/schema-map.json |
++-----------------+---------------------------+----------------------+
+| ASDC_ADDRESS    | URL to SDC BE.            | sdc-be:8443          |
++-----------------+---------------------------+----------------------+
+
+
+Helm chart
+----------
+By default VES OpenAPI Manger is deployed via Helm as the DCAE subcomponent in the ONAP cluster. Helm chart is
+configured to deploy application with all prerequisites met. It achieves that by:
+
+1) Mounting ConfigMap with mapping file under */app/schema-map.json* path.
+2) Proper setting environment variables to values described in section *Environment variables*. Mapping file path is set to point to mounted file and SDC BE URL is set to internal port available only from Kubernetes cluster.
+3) Setting Readiness check. It waits for other ONAP components to start: SDC BE, Message Router. VES OpenAPI Manager Pod will not start until they are not ready.
+
+Local deployment
+----------------
+It's possible to run VES OpenAPI Manager in local environment which connects to external lab with ONAP. This way
+requires exposing ports of some services on lab, creating local port tunneling and running VES OpenAPI Manager (using
+docker-compose or IDE e.g. IntelliJ).
+
+It's described in more detail in the README in project repository (`README <https://github.com/onap/dcaegen2-platform-ves-openapi-manager/blob/master/README.md>`_).
\ No newline at end of file
diff --git a/docs/sections/services/ves-openapi-manager/index.rst b/docs/sections/services/ves-openapi-manager/index.rst
new file mode 100644
index 00000000..e6cc15c3
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/index.rst
@@ -0,0 +1,27 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+VES OpenAPI Manager
+===================
+
+VES OpenAPI Manager has been created to validate the presence of OpenAPI schemas declared in *VES_EVENT* type artifacts,
+within the DCAE run-time environment during Service Model distribution in SDC. When deployed, it automatically listens
+to events of Service Models distributions by using SDC Distribution Client in order to read the declared OpenAPI
+descriptions. Purpose of this component is to partially validate artifacts of type *VES_EVENT* from Resources of
+distributed services. During validation phase it checks whether *stndDefined* events defined in VES_EVENT type artifact,
+contain only *schemaReferences* that local copies are accessible by DCAE VES Collector. If any of schemaReference is
+absent in local externalSchema repository, the VES OpenAPI Manager informs ONAP user which schemas need to be uploaded
+to the DCAE run-time environment.
+
+
+VES OpenAPI Manager overview and functions
+------------------------------------------
+
+.. toctree::
+  :maxdepth: 1
+
+  ./architecture.rst
+  ./artifacts.rst
+  ./deployment.rst
+  ./use-cases.rst
diff --git a/docs/sections/services/ves-openapi-manager/resources/architecture.png b/docs/sections/services/ves-openapi-manager/resources/architecture.png
new file mode 100644
index 00000000..5df1198f
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/architecture.png
diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml
new file mode 100644
index 00000000..73c6919e
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml
@@ -0,0 +1,37 @@
+---
+event:
+  presence: required
+  structure:
+    commonEventHeader:
+      presence: required
+      structure:
+        domain: {presence: required, value: notification}
+        eventName: {presence: required, value: Noti_MyPnf-Acme_FileReady}
+        priority: {presence: required, value: Normal}
+        eventId: {presence: required}
+        reportingEntityId: {presence: required}
+        reportingEntityName: {presence: required}
+        sequence: {presence: required, value: 0}
+        sourceId: {presence: required}
+        sourceName: {presence: required}
+        version: {presence: required, value: 4.0.1}
+        vesEventListenerVersion: {presence: required, value: 7.0.1}
+        startEpochMicrosec: {presence: required}
+        lastEpochMicrosec: {presence: required}
+    notificationFields:
+      presence: required
+      structure:
+        changeIdentifier: {presence: required, value: PM_MEAS_FILES}
+        changeType: {presence: required, value: fileReady}
+        notificationFieldsVersion: {presence: required, value: 2.0}
+        arrayOfNamedHashMap:
+          presence: required
+          array:
+            - name: {presence: required}
+              hashMap: {presence: required, structure: {
+                keyValuePair: {presence: required, structure: {key: {presence: required, value: location}, value: {presence: required}}},
+                keyValuePair: {presence: required, structure: {key: {presence: required, value: compression}, value: {presence: required, value: gzip}}},
+                keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatType}, value: {presence: required, value: org.3GPP.32.435}}},
+                keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatVersion}, value: {presence: required, value: V10}}}}
+              }
+...
\ No newline at end of file
diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml
new file mode 100644
index 00000000..6bba640e
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml
@@ -0,0 +1,29 @@
+---
+event:
+  presence: required
+  comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification"
+  structure:
+    commonEventHeader:
+      presence: required
+      structure:
+        domain: {presence: required, value: stndDefined}
+        eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification}
+        priority: {presence: required, value: Normal}
+        eventId: {presence: required}
+        reportingEntityId: {presence: required}
+        reportingEntityName: {presence: required}
+        sequence: {presence: required, value: 0}
+        sourceId: {presence: required}
+        sourceName: {presence: required}
+        version: {presence: required, value: 4.1}
+        vesEventListenerVersion: {presence: required, value: 7.2}
+        startEpochMicrosec: {presence: required}
+        lastEpochMicrosec: {presence: required}
+        stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"}
+    stndDefinedFields:
+      presence: required
+      structure:
+        data: {presence: required}
+        stndDefinedFieldsVersion: {presence: required, value: "1.0"}
+
+...
\ No newline at end of file
diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml
new file mode 100644
index 00000000..ad3ca469
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml
@@ -0,0 +1,30 @@
+---
+event:
+  presence: required
+  comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification"
+  structure:
+    commonEventHeader:
+      presence: required
+      structure:
+        domain: {presence: required, value: stndDefined}
+        eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification}
+        priority: {presence: required, value: Normal}
+        eventId: {presence: required}
+        reportingEntityId: {presence: required}
+        reportingEntityName: {presence: required}
+        sequence: {presence: required, value: 0}
+        sourceId: {presence: required}
+        sourceName: {presence: required}
+        version: {presence: required, value: 4.1}
+        vesEventListenerVersion: {presence: required, value: 7.2}
+        startEpochMicrosec: {presence: required}
+        lastEpochMicrosec: {presence: required}
+        stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"}
+    stndDefinedFields:
+      presence: required
+      structure:
+        schemaReference: { presence: required, value: "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm" }
+        data: {presence: required}
+        stndDefinedFieldsVersion: {presence: required, value: "1.0"}
+
+...
\ No newline at end of file
diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json
new file mode 100644
index 00000000..e11851c0
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json
@@ -0,0 +1,14 @@
+[
+  {
+    "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml",
+    "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml"
+  },
+  {
+    "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml",
+    "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml"
+  },
+  {
+    "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml",
+    "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml"
+  }
+]
\ No newline at end of file
diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json
new file mode 100644
index 00000000..fb34f5ae
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json
@@ -0,0 +1,6 @@
+[
+  {
+    "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml",
+    "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml"
+  }
+]
\ No newline at end of file
diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map.json b/docs/sections/services/ves-openapi-manager/resources/schema-map.json
new file mode 100644
index 00000000..97883fbc
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/schema-map.json
@@ -0,0 +1,6 @@
+[
+  {
+    "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml",
+    "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml"
+  }
+]
\ No newline at end of file
diff --git a/docs/sections/services/ves-openapi-manager/resources/workflow.png b/docs/sections/services/ves-openapi-manager/resources/workflow.png
new file mode 100644
index 00000000..423792b6
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/resources/workflow.png
diff --git a/docs/sections/services/ves-openapi-manager/use-cases.rst b/docs/sections/services/ves-openapi-manager/use-cases.rst
new file mode 100644
index 00000000..aa3a4bf8
--- /dev/null
+++ b/docs/sections/services/ves-openapi-manager/use-cases.rst
@@ -0,0 +1,117 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. _ves-openapi-manager-use-cases:
+
+VES OpenAPI Manager validation use-cases
+========================================
+The main VES OpenAPI Manager use case is to verify if the schemaReferences declared in *VES_EVENT* type artifacts are
+present in the local DCAE run-time externalSchemaRepo and show validation results to user in SDC UI.
+
+The general flow of VES OpenAPI Manager is available here :ref:`ves-openapi-manager-flow`.
+
+Based on the referenced flow, there are few possible behaviours of VES OpenAPI Manager. In this section two main flows:
+successful and unsuccessful validation will be described step by step.
+
+Validation prerequisites
+------------------------
+Validation phase takes place only when specific conditions are met.
+
+1) VES OpenAPI Manager is properly configured: client is connected to SDC and mapping file is present and pointed in configuration. Configuration is described in detail here: :ref:`ves-openapi-manager-deployment`.
+2) Distribution of a Service Model takes place in SDC.
+3) Service contains an *VES_EVENT* type artifact.
+4) Artifact content is correctly downloaded.
+
+Validation description
+----------------------
+When *schemaReference* field from artifact is being validated, only the part of the URI that indicates public openAPI
+description file location is taken into consideration.
+
+For example when *schemaReference* with value
+*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm*
+is found in artifact, then only the part before # sign (public openAPI description file location URI part) is being
+validated. This way part which would be validated is
+*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml*.
+
+Mapping file must have a predefined JSON format of list of objects (mappings) with publicURL and localURL fields.
+Example with 3 mappings:
+
+.. literalinclude:: resources/schema-map-example.json
+    :language: json
+
+When *schemaReference* is split, it's compared to each publicURL from mapping file. If there is no publicURL in mapping
+file which matches schemaReference, then schemaReference is marked as invalid. This process is executed for all
+stndDefined events defined in *VES_EVENT* artifact, which declare a schemaReference. All invalid references are returned
+to user via SDC UI when validation of a complete artifact ends.
+
+Based on returned information with invalid references user can take action and e.g. add mappings and schemas to DCAE
+run-time environment by editing ConfigMaps which store them.
+
++-----------------------------------------+-------------------------+
+| ConfigMap name                          | Content                 |
++=========================================+=========================+
+| dcae-external-repo-configmap-schema-map | Mapping file            |
++-----------------------------------------+-------------------------+
+| dcae-external-repo-configmap-sa88-rel16 | OpenAPI schemas content,|
+|                                         | example stores 3GPP     |
+|                                         | sa88-rel16 schemas      |
++-----------------------------------------+-------------------------+
+
+
+Successful validation case
+--------------------------
+There are few ways to get a successful validation status - *DEPLOY_OK*.
+
+1) When artifact *VES_EVENT* does not contain *stndDefined* events definitions. Only *stndDefined* event are validated.
+2) When artifact *VES_EVENT* contains *stndDefined* events definitions but *schemaReference* fields are not present.
+3) When artifact *VES_EVENT* contains *stndDefined* events definitions and each *schemaReference* of the event is present in the mapping file.
+
+
+*VES_EVENT* artifact may contain more than one event definition. Examples of valid artifacts with single events are
+below.
+
+Example of valid artifact without *stndDefined* event definition (case 1):
+
+.. literalinclude:: resources/artifact-no-stndDefined.yaml
+    :language: yaml
+
+Example of valid artifact with *stndDefined* event definition, but without schemaReference field (case 2):
+
+.. literalinclude:: resources/artifact-stndDefined-no-schemaReference.yaml
+    :language: yaml
+
+Example of artifact with *stndDefined* event definition (case 3):
+
+.. literalinclude:: resources/artifact-stndDefined.yaml
+    :language: yaml
+
+which is valid when mapping file contains a mapping of schemaReference field.
+Example of mapping file content which makes example artifact valid:
+
+.. literalinclude:: resources/schema-map.json
+    :language: json
+
+Unsuccessful validation case
+----------------------------
+Another case is an unsuccessful validation case which sends status *DEPLOY_ERROR* with error message containing listed
+*schemaReference* that are missing from mapping file. Fail case might occur:
+
+1) When artifact *VES_EVENT* contains *stndDefined* events definitions and any of *schemaReference* is not present in mapping file.
+
+Example of artifact with *stndDefined* event definition:
+
+.. literalinclude:: resources/artifact-stndDefined.yaml
+    :language: yaml
+
+which is invalid when mapping file does not contain a mapping of schemaReference field.
+Example of mapping file which makes example artifact invalid:
+
+.. literalinclude:: resources/schema-map-invalid.json
+    :language: json
+
+Validation results
+------------------
+There are two ways to receive validation results.
+
+1) Via SDC UI. Results are available in *Service->Distributions* view. To see results in SDC UI user has to wait up to few minutes.
+2) In VES OpenAPI Manager logs. They are printed right after validation.
\ No newline at end of file