Merge "Update HV-VES documentation"

11 files changed, 113 insertions, 80 deletions

diff --git a/docs/sections/services/ves-hv/architecture.rst b/docs/sections/services/ves-hv/architecture.rst
index 986e8bb3..1812f4ee 100644
--- a/docs/sections/services/ves-hv/architecture.rst
+++ b/docs/sections/services/ves-hv/architecture.rst
@@ -12,7 +12,7 @@ High Volume Collector is deployed with DCAEGEN2 via OOM Helm charts and Cloudify
 Input messages come from TCP interface and Wire Transfer Protocol. Each frame includes Google Protocol Buffers (GPB) encoded payload.
 Based on information provided in CommonEventHeader, domain messages are validated and published to specific Kafka topic in DMaaP.
 
-.. image:: ONAP_VES_HV_Architecture.png
+.. image:: resources/ONAP_VES_HV_Architecture.png
 
 Messages published in DMaaP's Kafka topic will be consumed by DCAE analytics application or other ONAP component that consumes messages from DMaaP/Kafka.
 DMaaP serves direct access to Kafka allowing other analytics applications to utilize its data.
diff --git a/docs/sections/services/ves-hv/deployment.rst b/docs/sections/services/ves-hv/deployment.rst
index 07d26b94..3a14eb18 100644
--- a/docs/sections/services/ves-hv/deployment.rst
+++ b/docs/sections/services/ves-hv/deployment.rst
@@ -7,83 +7,106 @@
 Deployment
 ============
 
-To run HV-VES Collector container, you need to specify required parameters by passing them as command
-line arguments either by using long form (--long-form) or short form (-s) followed by argument if needed.
+To run HV-VES Collector container you need to specify required command line options and environment variables.
 
-All parameters can also be configured by specifying environment variables. These variables have to be named after command line option name
-rewritten using `UPPER_SNAKE_CASE` and prepended with `VESHV_` prefix, for example `VESHV_LISTEN_PORT`.
+Command line parameters can be specified either by using long form (--long-form) or short form (-s) followed by argument if needed (see `Arg` column in table below). These parameters can be omitted if corresponding environment variables are set. These variables are named after command line option name rewritten using `UPPER_SNAKE_CASE` and prepended with `VESHV_` prefix, for example `VESHV_CONFIGURATION_FILE`.
 
-Command line options have precedence over environment variables.
+Command line options have precedence over environment variables in cases when both are present.
 
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
-| Long form   | Short form | Env form          | Required | Arg | Description                                     |
-+=============+============+===================+==========+=====+=================================================+
-| listen-port | p          | VESHV_LISTEN_PORT | yes      | yes | Port on which HV-VES listens internally         |
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
-| config-url  | c          | VESHV_CONFIG_URL  | yes      | yes | URL of HV-VES configuration on Consul service   |
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
+Currently HV-VES requires single command line parameter which points to base configuration file.
 
-HV-VES requires also to specify if SSL should be used when handling incoming TCP connections.
-This can be done by passing the flag below to the command line.
+.. csv-table::
+    :widths: auto
+    :delim: ;
+    :header: Long form , Short form , Arg , Env form , Description
 
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
-| Long form   | Short form | Env form          | Required | Arg | Description                                     |
-+=============+============+===================+==========+=====+=================================================+
-| ssl-disable | l          | VESHV_SSL_DISABLE | no       | no  | Disables SSL encryption                         |
-+-------------+------------+-------------------+----------+-----+-------------------------------------------------+
+    configuration-file ; c ; yes ; VESHV_CONFIGURATION_FILE  ; Path to JSON file containing HV-VES configuration
 
+Environment variables that are required by HV-VES are used by collector for provisioning of run-time configuration and are usually provided by DCAE platform.
 
-Minimal command for running the container:
+.. csv-table::
+    :widths: auto
+    :delim: ;
+    :header: Environment variable name , Description
 
-.. code-block:: bash
+    CONSUL_HOST            ; Hostname under which Consul service is available
+    CONFIG_BINDING_SERVICE ; Hostname under which Config Binding Service is available
+    HOSTNAME               ; Configuration key of HV-VES as seen by CBS, usually *dcae-hv-ves-collector*
+
+There is also optional command line parameter which configures container-internal port for Healthcheck Server API (see :ref:`healthcheck_and_monitoring`).
+
+.. csv-table::
+    :widths: auto
+    :delim: ;
+    :header: Long form , Short form , Arg , Env form , Description
+
+    health-check-api-port ; H ; yes ; VESHV_HEALTH_CHECK_API_PORT  ; Health check rest api listen port
+
+.. _configuration_file:
+
+Configuration file
+------------------
+
+File must provide base configuration for HV-VES Collector in JSON format.
+
+Some entries in configuration can also be obtained from Config Binding Service (see :ref:`run_time_configuration`). **Every entry defined in configuration file will be OVERRIDEN if it is also present in run-time configuration.**
+
+Following JSON shows every possible configuration option. Default file shipped with HV-VES container can be found in the collector's repository (see :ref:`repositories`).
+
+.. literalinclude:: resources/base-configuration.json
+    :language: json
+
+
+The configuration is split into smaller sections.
+Tables show restrictions on fields in file configuration and short description.
+
+.. csv-table::
+    :widths: auto
+    :delim: ;
+    :header-rows: 2
+
+    Server
+    Key                         ; Value type ; Description
+    server.listenPort           ; number     ; Port on which HV-VES listens internally
+    server.idleTimeoutSec       ; number     ; Idle timeout for remote hosts. After given time without any data exchange, the connection might be closed
+
+.. csv-table::
+    :widths: auto
+    :delim: ;
+    :header-rows: 2
+
+    Config Binding Service
+    Key                      ; Value type ; Description
+    cbs.firstRequestDelaySec ; number     ; Delay of first request to Config Binding Service in seconds
+    cbs.requestIntervalSec   ; number     ; Interval of configuration requests in seconds
+
+.. csv-table::
+    :widths: auto
+    :delim: ;
+    :header-rows: 2
+
+    Security
+    Key                                   ; Value type ; Description
+    security.sslDisable                   ; boolean    ; Disables SSL encryption
+    security.keys.keyStoreFile            ; String     ; Key store path used in HV-VES incoming connections
+    security.keys.keyStorePasswordFile    ; String     ; Key store password file used in HV-VES incoming connections
+    security.keys.trustStoreFile          ; String     ; Path to file with trusted certificates bundle used in HV-VES incoming connections
+    security.keys.trustStorePasswordFile  ; String     ; Trust store password file used in HV-VES incoming connections
+
+All security entries are mandatory with `security.sslDisable` set to `false`. Otherwise only `security.sslDisable` needs to be specified. If `security.sslDisable` flag is missing, then it is  interpreted same as it would be set to `false`.
+
+.. csv-table::
+    :widths: auto
+    :delim: ;
+    :header-rows: 2
+
+    Uncategorized
+    Key                  ; Value type ; Description
+    logLevel             ; String     ; Log level on which HV-VES publishes all log messages. Valid argument values are (case insensitive): ERROR, WARN, INFO, DEBUG, TRACE.
 
-    docker run nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main --listen-port 6061 --config-url http://consul:8500/v1/kv/dcae-hv-ves-collector --ssl-disable
-
-Optional configuration parameters:
-
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| Long form             | Short form | Env form                   | Required | Arg | Default         | Description                                           |
-+=======================+============+============================+==========+=====+=================+=======================================================+
-| health-check-api-port | H          | VESHV_HEALTHCHECK_API_PORT | no       | yes | 6060            | Health check REST API listen port                     |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| first-request-delay   | d          | VESHV_FIRST_REQUEST_DELAY  | no       | yes | 10              | Delay of first request to Consul service in seconds   |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| request-interval      | I          | VESHV_REQUEST_INTERVAL     | no       | yes | 5               | Interval of Consul configuration requests in seconds  |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| idle-timeout-sec      | i          | VESHV_IDLE_TIMEOUT_SEC     | no       | yes | 60              | Idle timeout for remote hosts. After given time       |
-|                       |            |                            |          |     |                 | without any data exchange, the connection             |
-|                       |            |                            |          |     |                 | might be closed.                                      |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| max-payload-size      | m          | VESHV_MAX_PAYLOAD_SIZE     | no       | yes | 1048576 (1 MiB) | Maximum supported payload size in bytes               |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-| log-level             | ll         | VESHV_LOG_LEVEL            | no       | yes | INFO            | Log level on which HV-VES publishes all log messages  |
-|                       |            |                            |          |     |                 | Valid argument values are (case insensitive): ERROR,  |
-|                       |            |                            |          |     |                 | WARN, INFO, DEBUG, TRACE.                             |
-+-----------------------+------------+----------------------------+----------+-----+-----------------+-------------------------------------------------------+
-
-As part of experimental API if you do not specify `ssl-disable` flag, there is need to specify additional
-parameters for security configuration.
-
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| Long form             | Short form | Env form                   | Required | Arg | Default                | Description                                                  |
-+=======================+============+============================+==========+=====+========================+==============================================================+
-| key-store             | k          | VESHV_KEY_STORE            | no       | yes | /etc/ves-hv/server.p12 | Key store in PKCS12 format path                              |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| key-store-password    | kp         | VESHV_KEY_STORE_PASSWORD   | no       | yes |                        | Key store password                                           |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| trust-store           | t          | VESHV_TRUST_STORE          | no       | yes | /etc/ves-hv/trust.p12  | File with trusted certificate bundle in PKCS12 format path   |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-| trust-store-password  | tp         | VESHV_TRUST_STORE_PASSWORD | no       | yes |                        | Trust store password                                         |
-+-----------------------+------------+----------------------------+----------+-----+------------------------+--------------------------------------------------------------+
-
-Passwords are mandatory without ssl-disable flag. If key-store or trust-store location is not specified, HV-VES will try to read them from default locations.
-
-These parameters can be configured either by passing command line option during `docker run` call or
-by specifying environment variables named after command line option name
-rewritten using `UPPER_SNAKE_CASE` and prepended with `VESHV_` prefix e.g. `VESHV_LISTEN_PORT`.
 
 Horizontal Scaling
-==================
+------------------
 
 Kubernetes command line tool (`kubectl`) is recommended for manual horizontal scaling of HV-VES Collector.
 
diff --git a/docs/sections/services/ves-hv/design.rst b/docs/sections/services/ves-hv/design.rst
index a6c2b864..fb4fa2c7 100644
--- a/docs/sections/services/ves-hv/design.rst
+++ b/docs/sections/services/ves-hv/design.rst
@@ -33,13 +33,13 @@ The proto file (with the VES CommonHeader) comes with a binary-type **Payload**
 Domain-specific data are encoded as well with GPB. A domain-specific proto file is required to decode the data.
 This domain-specific proto has to be shared with analytics applications - HV-VES does not analyze domain-specific data.
 
-In order to support the RT-PM use-case, HV-VES includes a **perf3gpp** domain proto file. Within this domain, high volume data are expected to be reported to HV-VES collector.
+In order to support the RT-PM use-case, HV-VES uses a **perf3gpp** domain proto file. Within this domain, high volume data are expected to be reported to HV-VES collector.
 Additional domains can be defined based on existing VES domains (like Fault, Heartbeat) or completely new domains. New domains can be added when needed.
 
 GPB proto files are backwards compatible, and a new domain can be added without affecting existing systems.
 
 Analytics applications have to be equipped with the new domain-specific proto file as well.
-Currently, these additional, domain specific proto files can be added to respective repos of HV-VES collector.
+Currently, these additional, domain specific proto files can be added to hv-ves-client protobuf library repository (artifactId: hvvesclient-protobuf).
 
 Implementation details
 ----------------------
diff --git a/docs/sections/services/ves-hv/example-event.rst b/docs/sections/services/ves-hv/example-event.rst
index 3a335395..a413d401 100644
--- a/docs/sections/services/ves-hv/example-event.rst
+++ b/docs/sections/services/ves-hv/example-event.rst
@@ -11,5 +11,5 @@ The message consists of several parts. Each part is encoded in a different way.
 
 Values of fields can be changed according to types specified in noted definition files.
 
-.. literalinclude:: WTP.yaml
+.. literalinclude:: resources/WTP.yaml
     :language: yaml
diff --git a/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst b/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst
index 18333778..9d35e1ef 100644
--- a/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst
+++ b/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst
@@ -9,7 +9,7 @@ Healthcheck and Monitoring
 Healthcheck
 -----------
 Inside HV-VES docker container runs a small HTTP service for healthcheck. Port for healthchecks can be configured
-at deployment using ``--health-check-api-port`` command line option or via `VESHV_HEALTHCHECK_API_PORT` environment variable (for details see :ref:`deployment`).
+at deployment using command line (for details see :ref:`deployment`).
 
 This service exposes endpoint **GET /health/ready** which returns a **HTTP 200 OK** when HV-VES is healthy
 and ready for connections. Otherwise it returns a **HTTP 503 Service Unavailable** message with a short reason of unhealthiness.
@@ -90,4 +90,4 @@ JVM metrics:
 
 Sample response for **GET monitoring/prometheus**:
 
-.. literalinclude:: metrics_sample_response.txt
+.. literalinclude:: resources/metrics_sample_response.txt
diff --git a/docs/sections/services/ves-hv/index.rst b/docs/sections/services/ves-hv/index.rst
index 5bb83ddc..144f557e 100644
--- a/docs/sections/services/ves-hv/index.rst
+++ b/docs/sections/services/ves-hv/index.rst
@@ -29,11 +29,11 @@ High Volume VES Collector overview and functions
 
     architecture
     design
-    run-time-configuration
     repositories
     deployment
-    troubleshooting
+    run-time-configuration
     HV-VES Offered APIs <../../apis/ves-hv/index>
     authorization
     example-event
     healthcheck-and-monitoring
+    troubleshooting
diff --git a/docs/sections/services/ves-hv/ONAP_VES_HV_Architecture.png b/docs/sections/services/ves-hv/resources/ONAP_VES_HV_Architecture.png
index 7652b970..7652b970 100644
--- a/docs/sections/services/ves-hv/ONAP_VES_HV_Architecture.png
+++ b/docs/sections/services/ves-hv/resources/ONAP_VES_HV_Architecture.png
diff --git a/docs/sections/services/ves-hv/WTP.yaml b/docs/sections/services/ves-hv/resources/WTP.yaml
index 835ab309..835ab309 100644
--- a/docs/sections/services/ves-hv/WTP.yaml
+++ b/docs/sections/services/ves-hv/resources/WTP.yaml
diff --git a/docs/sections/services/ves-hv/resources/base-configuration.json b/docs/sections/services/ves-hv/resources/base-configuration.json
new file mode 100644
index 00000000..6580287d
--- /dev/null
+++ b/docs/sections/services/ves-hv/resources/base-configuration.json
@@ -0,0 +1,12 @@
+{
+  "logLevel": "INFO",
+  "server.listenPort": 6061,
+  "server.idleTimeoutSec": 60,
+  "cbs.firstRequestDelaySec": 10,
+  "cbs.requestIntervalSec": 5,
+  "security.sslDisable": false,
+  "security.keys.keyStoreFile": "/etc/ves-hv/ssl/server.p12",
+  "security.keys.keyStorePasswordFile": "/etc/ves-hv/ssl/server.pass",
+  "security.keys.trustStoreFile": "/etc/ves-hv/ssl/trust.p12",
+  "security.keys.trustStorePasswordFile": "/etc/ves-hv/ssl/trust.pass"
+}
\ No newline at end of file
diff --git a/docs/sections/services/ves-hv/metrics_sample_response.txt b/docs/sections/services/ves-hv/resources/metrics_sample_response.txt
index da54e3ea..da54e3ea 100644
--- a/docs/sections/services/ves-hv/metrics_sample_response.txt
+++ b/docs/sections/services/ves-hv/resources/metrics_sample_response.txt
diff --git a/docs/sections/services/ves-hv/run-time-configuration.rst b/docs/sections/services/ves-hv/run-time-configuration.rst
index 76d622c6..5efc6e31 100644
--- a/docs/sections/services/ves-hv/run-time-configuration.rst
+++ b/docs/sections/services/ves-hv/run-time-configuration.rst
@@ -8,7 +8,7 @@ Run-Time configuration
 
 (see :ref:`deployment`)
 
-HV-VES can fetch configuration directly from Consul service in the following JSON format:
+HV-VES fetches configuration from Config Binding Service in the following JSON format:
 
 .. code-block:: json
 
@@ -50,10 +50,8 @@ The configuration is created from HV-VES Cloudify blueprint by specifying **appl
               fromDomain: perf3gpp
               toTopic: HV_VES_PERF3GPP
 
-Endpoint on which HV-VES seeks configuration can be set during deployment as described in :ref:`deployment`.
+HV-VES waits 10 seconds (default, configurable during deployment with **firstRequestDelay** option, see :ref:`configuration_file`) before the first attempt to retrieve configuration from CBS. This is to prevent possible synchronization issues. During that time HV-VES declines any connection attempts from xNF (VNF/PNF).
 
-HV-VES waits 10 seconds (default, configurable during deplyoment with **first-request-delay** option, see :ref:`deployment`) before the first attempt to retrieve configuration from Consul. This is to prevent possible synchronization issues. During that time HV-VES declines any connection attempts from xNF (VNF/PNF).
+After first request, HV-VES asks for configuration in fixed intervals, configurable from file configuration (**requestInterval**). By default interval is set to 5 seconds.
 
-After first request, HV-VES asks for configuration in fixed intervals, configurable from command line (**request-interval**). By defualt interval is set to 5 seconds.
-
-In case of failing to retrieve configuration, collector temporarily extends this interval and retries. After five unsuccessfull attempts, container becomes unhealthy and cannot recover. HV-VES in this state is unusable and the container should be restarted.
+In case of failing to retrieve configuration, collector retries the action. After five unsuccessful attempts, container becomes unhealthy and cannot recover. HV-VES in this state is unusable and the container should be restarted.