Merge "dcae r6 doc updates cont."

4 files changed, 36 insertions, 189 deletions

diff --git a/docs/sections/services/ves-http/VES-processingFlow.png b/docs/sections/services/ves-http/VES-processingFlow.png
index 4c53a2e9..2c765937 100644
--- a/docs/sections/services/ves-http/VES-processingFlow.png
+++ b/docs/sections/services/ves-http/VES-processingFlow.png
diff --git a/docs/sections/services/ves-http/architecture.rst b/docs/sections/services/ves-http/architecture.rst
index 960c9cc5..30145509 100644
--- a/docs/sections/services/ves-http/architecture.rst
+++ b/docs/sections/services/ves-http/architecture.rst
@@ -9,6 +9,14 @@ VES Architecture
 VES Processing Flow
 -------------------
 
+1) Collector supports different URI based on single or batch event to be received
+2) Post authentication – events are validated for schema. At this point – appropriate return code is sent to client (success/failure)
+3) Event Processor check against transformation rules (if enabled) and handles VES output standardization (e.g. VES 7.x input to VES5.4 output)
+4) Based on domain, events are distributed to configurable topics
+5) Post to outbound topic(s). If DMaaP publish is unsuccessful, messages will be queued per topic within VESCollector
+
+Note: As the collector is deployed as micro-service, all configuration parameters (including DMaaP topics) are passed to the collector dynamically. VEScollector refreshes the configuration from CBS every 5 minutes
+
 .. image:: ./VES-processingFlow.png
 
 
@@ -20,13 +28,12 @@ VES Collector is configured to support below VES Version; the corresponding API
 ===========     ================    ==================================
 VES Version     API version         Schema Definition
 ===========     ================    ==================================
-VES 1.2         eventListener/v1    CommonEventFormat_Vendors_v25.json   
-VES 4.1         eventListener/v4    CommonEventFormat_27.2.json
-VES 5.4         eventListener/v5    CommonEventFormat_28.4.1.json
-VES 7.0         eventListener/v7    CommonEventFormat_30.0.1.json
+VES 1.2         eventListener/v1    `CommonEventFormat_Vendors_v25.json <https://git.onap.org/dcaegen2/collectors/ves/tree/etc/CommonEventFormat_Vendors_v25.json>`_  
+VES 4.1         eventListener/v4    `CommonEventFormat_27.2.json <https://git.onap.org/dcaegen2/collectors/ves/tree/etc/CommonEventFormat_27.2.json>`_
+VES 5.4         eventListener/v5    `CommonEventFormat_28.4.1.json <https://git.onap.org/dcaegen2/collectors/ves/tree/etc/CommonEventFormat_28.4.1.json>`_
+VES 7.1.1       eventListener/v7    `CommonEventFormat_30.1.1.json <https://git.onap.org/vnfrqts/requirements/tree/docs/Chapter8/CommonEventFormat_30.1.1_ONAP.json>`_
 ===========     ================    ==================================
-
-Schema definition files are contained within VES collector gerrit repo - https://git.onap.org/dcaegen2/collectors/ves/tree/etc
+ 
 
 
 Features Supported
@@ -53,8 +60,8 @@ Dynamic configuration fed into Collector via DCAEPlatform
 
 POST requests result in standard HTTP status codes:
 
-200-299  Success
-400-499  Client request has a problem (data error)
-500-599  Collector service has a problem
+- 200-299  Success
+- 400-499  Client request has a problem (data error)
+- 500-599  Collector service has a problem
  
 
diff --git a/docs/sections/services/ves-http/configuration.rst b/docs/sections/services/ves-http/configuration.rst
index eaadf277..7cf9831f 100644
--- a/docs/sections/services/ves-http/configuration.rst
+++ b/docs/sections/services/ves-http/configuration.rst
@@ -12,10 +12,10 @@ VES expects to be able to fetch configuration directly from consul service in fo
         "collector.dynamic.config.update.frequency": "5",
         "event.transform.flag": "0",
         "collector.schema.checkflag": "1",
-        "collector.dmaap.streamid": "fault=ves-fault|syslog=ves-syslog|heartbeat=ves-heartbeat|measurementsForVfScaling=ves-measurement|mobileFlow=ves-mobileflow|other=ves-other|stateChange=ves-statechange|thresholdCrossingAlert=ves-thresholdCrossingAlert|voiceQuality=ves-voicequality|sipSignaling=ves-sipsignaling|notification=ves-notification|pnfRegistration=ves-pnfRegistration",
+        "collector.dmaap.streamid": "fault=ves-fault|syslog=ves-syslog|heartbeat=ves-heartbeat|measurementsForVfScaling=ves-measurement|measurement=ves-measurement|mobileFlow=ves-mobileflow|other=ves-other|stateChange=ves-statechange|thresholdCrossingAlert=ves-thresholdCrossingAlert|voiceQuality=ves-voicequality|sipSignaling=ves-sipsignaling|notification=ves-notification|pnfRegistration=ves-pnfRegistration",
         "collector.service.port": "8080",
-        "collector.schema.file": "{\"v1\":\"./etc/CommonEventFormat_27.2.json\",\"v2\":\"./etc/CommonEventFormat_27.2.json\",\"v3\":\"./etc/CommonEventFormat_27.2.json\",\"v4\":\"./etc/CommonEventFormat_27.2.json\",\"v5\":\"./etc/CommonEventFormat_28.4.1.json\",\"v7\":\"./etc/CommonEventFormat_30.0.1.json\"}",
-        "collector.keystore.passwordfile": "/opt/app/VESCollector/etc/passwordfile",
+        "collector.service.secure.port": "8443",
+        "collector.schema.file": "{\"v1\":\"./etc/CommonEventFormat_27.2.json\",\"v2\":\"./etc/CommonEventFormat_27.2.json\",\"v3\":\"./etc/CommonEventFormat_27.2.json\",\"v4\":\"./etc/CommonEventFormat_27.2.json\",\"v5\":\"./etc/CommonEventFormat_28.4.1.json\",\"v7\":\"./etc/CommonEventFormat_30.1.1.json\"}",
         "streams_publishes": {
             "ves-measurement": {
                 "type": "message_router",
@@ -55,9 +55,12 @@ VES expects to be able to fetch configuration directly from consul service in fo
             }
         },
         "collector.service.secure.port": "8443",
-        "auth.method": "noAuth",
-        "collector.keystore.file.location": "/opt/app/VESCollector/etc/keystore",
-        "header.authlist": "sample1,$2a$10$0buh.2WeYwN868YMwnNNEuNEAMNYVU9.FSMJGyIKV3dGET/7oGOi6"
+        "auth.method": "certBasicAuth",
+        "collector.keystore.file.location": "/opt/app/dcae-certificate/cert.jks",
+        "collector.keystore.passwordfile": "/opt/app/dcae-certificate/jks.pass",
+        "collector.truststore.file.location": "/opt/app/dcae-certificate/trust.jks",
+        "collector.truststore.passwordfile": "/opt/app/dcae-certificate/trust.pass",        
+        "header.authlist": "sample1,$2a$10$0buh.2WeYwN868YMwnNNEuNEAMNYVU9.FSMJGyIKV3dGET/7oGOi6|demouser,$2a$10$1cc.COcqV/d3iT2N7BjPG.S6ZKv2jpb9a5MV.o7lMih/GpjJRX.Ce"
     }
 
 
diff --git a/docs/sections/services/ves-http/installation.rst b/docs/sections/services/ves-http/installation.rst
index 0e399301..b21ca919 100644
--- a/docs/sections/services/ves-http/installation.rst
+++ b/docs/sections/services/ves-http/installation.rst
@@ -36,198 +36,35 @@ VES Collector support following authentication types
     * *auth.method=noAuth* default option - no security (http)
     * *auth.method=certBasicAuth* is used to enable mutual TLS authentication or/and basic HTTPs authentication
 
-Default ONAP deployed VESCOllector is configured for "noAuth". If VESCollector instance need to be deployed with authentication enabled, follow below setup
+The blueprint is same for both deployments - based on the input configuration, VESCollector can be set for required authentication type.
+Default ONAP deployed VESCollector is configured for "certBasicAuth".
 
+If VESCollector instance need to be deployed with authentication disabled, follow below setup
 
-- Update existing VESCollector deployment to remove nodeport conflict by editing service definition
-    .. code-block:: bash
-
-        kubectl edit svc -n onap xdcae-ves-collector
-
-and remove following entry and save the changes; K8S will update the  service definition default VES instance
-
-    .. code-block:: bash
-
-              - name: xport-t-8443
-                nodePort: 30417
-                port: 8443
-                protocol: TCP
-                targetPort: 8443
 
 - Execute into Bootstrap POD using kubectl command
 
-- Copy blueprint content into DCAE bootstrap POD under /blueprints directory under same file name.
-
-``k8s-ves-tls.yaml``
---------------------
-
-
-::
-
-    # ============LICENSE_START====================================================
-    # =============================================================================
-    # Copyright (c) 2019 AT&T Intellectual Property. All rights reserved.
-    # =============================================================================
-    # Licensed under the Apache License, Version 2.0 (the "License");
-    # you may not use this file except in compliance with the License.
-    # You may obtain a copy of the License at
-    #
-    #      http://www.apache.org/licenses/LICENSE-2.0
-    #
-    # Unless required by applicable law or agreed to in writing, software
-    # distributed under the License is distributed on an "AS IS" BASIS,
-    # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-    # See the License for the specific language governing permissions and
-    # limitations under the License.
-    # ============LICENSE_END======================================================
-
-    tosca_definitions_version: cloudify_dsl_1_3
-
-    imports:
-      - "http://www.getcloudify.org/spec/cloudify/3.4/types.yaml"
-      - https://nexus.onap.org/service/local/repositories/raw/content/org.onap.dcaegen2.platform.plugins/R4/k8splugin/1.4.13/k8splugin_types.yaml
-
-    inputs:
-      ves_other_publish_url:
-        type: string
-        default: "http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.SEC_OTHER_OUTPUT"
-      ves_heartbeat_publish_url:
-        type: string
-        default: "http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.SEC_HEARTBEAT_OUTPUT"
-      ves_fault_publish_url:
-        type: string
-        default: "http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.SEC_FAULT_OUTPUT"
-      ves_measurement_publish_url:
-        type: string
-        default: "http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.VES_MEASUREMENT_OUTPUT"
-      ves_notification_publish_url:
-        type: string
-        default: "http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.VES_NOTIFICATION_OUTPUT"
-      ves_pnfRegistration_publish_url:
-        type: string
-        default: "http://message-router.onap.svc.cluster.local:3904/events/unauthenticated.VES_PNFREG_OUTPUT"
-      tag_version:
-        type: string
-        default: "nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.4.5"
-      external_port:
-        type: string
-        description: Kubernetes node port on which collector is exposed
-        default: "30235"
-      external_tls_port:
-        type: string
-        description: Kubernetes node port on which collector is exposed for https
-        default: "30417"
-      replicas:
-        type: integer
-        description: number of instances
-        default: 1
-    node_templates:
-      ves:
-        interfaces:
-          cloudify.interfaces.lifecycle:
-            start:
-              inputs:
-               ports:
-                 - concat: ["8443:", { get_input: external_tls_port }]
-        properties:
-          application_config:
-            collector.dmaap.streamid: fault=ves-fault|syslog=ves-syslog|heartbeat=ves-heartbeat|measurementsForVfScaling=ves-measurement|measurement=ves-measurement|mobileFlow=ves-mobileflow|other=ves-other|stateChange=ves-statechange|thresholdCrossingAlert=ves-thresholdCrossingAlert|voiceQuality=ves-voicequality|sipSignaling=ves-sipsignaling|notification=ves-notification|pnfRegistration=ves-pnfRegistration
-            collector.keystore.file.location: /opt/app/VESCollector/etc/keystore
-            collector.keystore.passwordfile: /opt/app/VESCollector/etc/passwordfile
-            collector.schema.checkflag: "1"
-            collector.schema.file: "{\"v1\":\"./etc/CommonEventFormat_27.2.json\",\"v2\":\"./etc/CommonEventFormat_27.2.json\",\"v3\":\"./etc/CommonEventFormat_27.2.json\",\"v4\":\"./etc/CommonEventFormat_27.2.json\",\"v5\":\"./etc/CommonEventFormat_28.4.1.json\",\"v7\":\"./etc/CommonEventFormat_30.0.1.json\"}"
-            collector.service.port: "8080"
-            collector.service.secure.port: "8443"
-            event.transform.flag: "0"
-            auth.method: certBasicAuth
-            header.authlist: "sample1,$2a$10$0buh.2WeYwN868YMwnNNEuNEAMNYVU9.FSMJGyIKV3dGET/7oGOi6"
-            streams_publishes:
-                ves-fault:
-                  dmaap_info:
-                    topic_url:
-                      get_input: ves_fault_publish_url
-                  type: message_router
-                ves-measurement:
-                  dmaap_info:
-                    topic_url:
-                      get_input: ves_measurement_publish_url
-                  type: message_router
-                ves-notification:
-                  dmaap_info:
-                    topic_url:
-                      get_input: ves_notification_publish_url
-                  type: message_router
-                ves-pnfRegistration:
-                  dmaap_info:
-                    topic_url:
-                      get_input: ves_pnfRegistration_publish_url
-                  type: message_router
-                ves-heartbeat:
-                  dmaap_info:
-                    topic_url:
-                      get_input: ves_heartbeat_publish_url
-                  type: message_router
-                ves-other:
-                  dmaap_info:
-                    topic_url:
-                      get_input: ves_other_publish_url
-                  type: message_router
-            collector.dynamic.config.update.frequency: "5"
-          #docker_config:
-          #  healthcheck:
-          #    endpoint: /healthcheck
-          #    interval: 15s
-          #    timeout: 1s
-          #    type: https
-          image:
-            get_input: tag_version
-          replicas: {get_input: replicas}
-          name: 'dcae-ves-collector-tls'
-          dns_name: 'dcae-ves-collector-tls'
-          log_info:
-            log_directory: "/opt/app/VESCollector/logs/ecomp"
-        type: dcae.nodes.ContainerizedPlatformComponent
-
-
-
-- Validate blueprint
-    .. code-block:: bash
-
-        cfy blueprints validate /blueprints/k8s-ves-tls.yaml
+- VES blueprint is available under  /blueprints directory ``k8s-ves-tls.yaml``. A corresponding input files is also pre-loaded into bootstrap
+pod under /inputs/k8s-ves-inputs.yaml
 
 - Deploy blueprint
     .. code-block:: bash
 
-        cfy install -b ves-tls -d ves-tls /blueprints/k8s-ves-tls.yaml
+        cfy install -b ves-http -d ves-http -i /inputs/k8s-ves-inputs.yaml /blueprints/k8s-ves-tls.yaml
 
-To undeploy ves-tls, steps are noted below
+To undeploy ves-http, steps are noted below
 
-- Uninstall running ves-tls and delete deployment
+- Uninstall running ves-http and delete deployment
     .. code-block:: bash
 
-        cfy uninstall ves-tls
+        cfy uninstall ves-http
 
 The deployment uninstall will also delete the blueprint. In some case you might notice 400 error reported indicating active deployment exist such as below
-** An error occurred on the server: 400: Can't delete blueprint ves-tls - There exist deployments for this blueprint; Deployments ids: ves-tls**
+** An error occurred on the server: 400: Can't delete blueprint ves-http - There exist deployments for this blueprint; Deployments ids: ves-http**
 
 In this case blueprint can be deleted explicitly using this command.
 
     .. code-block:: bash
 
-        cfy blueprint delete ves-tls
-
-Known Issue : When VESCollector is required to be deployed with authentication enabled *auth.method: certBasicAuth*
-the blueprint currently disables healthcheck parameters configuration (below). This causes no readiness probe to be deployed in K8S when VES Collector is deployed with authentication enabled.
-
-    
-        .. code-block:: bash
-
-            docker_config:
-                healthcheck:
-                  endpoint: /healthcheck
-                  interval: 15s
-                  timeout: 1s
-                  type: https
-
+        cfy blueprint delete ves-http
 
-The healthcheck support when VESauthentication is enabled needs a different solution to be worked. This will be worked as future enhancement  (DCAEGEN2-1594)