From 7f278e30bef4a7a7cc7a4744d4bd3b3beb4ea5c1 Mon Sep 17 00:00:00 2001 From: Chengkai Yan Date: Tue, 6 Nov 2018 16:32:03 +0100 Subject: Update DFC documentation Change-Id: I95befc4a6b597f67672797d44561de41af9c480d Signed-off-by: Chengkai Yan Issue-ID: DCAEGEN2-876 --- docs/sections/apis/DFC.rst | 24 +------ docs/sections/services/dfc/administration.rst | 15 ++++ docs/sections/services/dfc/architecture.rst | 31 +++++++++ docs/sections/services/dfc/configuration.rst | 98 +++++++++++++++++++++++++++ docs/sections/services/dfc/delivery.rst | 12 ++++ docs/sections/services/dfc/index.rst | 19 ++++++ docs/sections/services/dfc/installation.rst | 25 +++++++ docs/sections/services/dfc/logging.rst | 13 ++++ docs/sections/services/dfc/release-notes.rst | 41 +++++++++++ 9 files changed, 256 insertions(+), 22 deletions(-) create mode 100644 docs/sections/services/dfc/administration.rst create mode 100644 docs/sections/services/dfc/architecture.rst create mode 100644 docs/sections/services/dfc/configuration.rst create mode 100644 docs/sections/services/dfc/delivery.rst create mode 100644 docs/sections/services/dfc/index.rst create mode 100644 docs/sections/services/dfc/installation.rst create mode 100644 docs/sections/services/dfc/logging.rst create mode 100644 docs/sections/services/dfc/release-notes.rst diff --git a/docs/sections/apis/DFC.rst b/docs/sections/apis/DFC.rst index 68326b5d..d7892a14 100644 --- a/docs/sections/apis/DFC.rst +++ b/docs/sections/apis/DFC.rst @@ -11,19 +11,9 @@ DFC (DataFile Collector) Overview ======== -DFC will orchestrate the collection of bulk PM data flow: - 1. Subscribes to fileReady DMaaP topic - 2. Collects the file from the xNF - 3. Sends new event to DataRouter with file. +Component description can be found under `DFC`_. -Introduction -============ - -DFC is delivered as one **Docker container** which hosts application server and can be started by `docker-compose`. - -Functionality -============= -.. image:: ../images/DFC.png +.. _DFC: ../../services/dfc/index.html Paths @@ -79,16 +69,6 @@ Whole project (top level of DFC directory) and each module (sub module directory Configuration file: Config/datafile_endpoints.json -Main API Endpoints -================== - -Running with dev-mode of DFC - - Heartbeat: **http://:8100/heartbeat** or **https://:8433/heartbeat** - - Start DFC: **http://:8100/start** or **https://:8433/start** - - Stop DFC: **http://:8100/stopDatafile** or **https://:8433/stopDatafile** - -The external port allocated for 8100 (http) is 30245. - Maven GroupId: ============== diff --git a/docs/sections/services/dfc/administration.rst b/docs/sections/services/dfc/administration.rst new file mode 100644 index 00000000..43845a0e --- /dev/null +++ b/docs/sections/services/dfc/administration.rst @@ -0,0 +1,15 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Administration +============== +DFC has a healthcheck functionality. The service can then be started and stopped through an API. One can also check the liveliness of the service. + +Main API Endpoints +"""""""""""""""""" +Running with dev-mode of DFC + - Heartbeat: **http://:8100/heartbeat** or **https://:8433/heartbeat** + - Start DFC: **http://:8100/start** or **https://:8433/start** + - Stop DFC: **http://:8100/stopDatafile** or **https://:8433/stopDatafile** + +The external port allocated for 8100 (http) is 30245. diff --git a/docs/sections/services/dfc/architecture.rst b/docs/sections/services/dfc/architecture.rst new file mode 100644 index 00000000..1ee922af --- /dev/null +++ b/docs/sections/services/dfc/architecture.rst @@ -0,0 +1,31 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Architecture +============ + +Introduction +"""""""""""" +DataFile Collector (DFC) is a part of DCAEGEN2. Some information about DFC and the reasons of its implementation can be found here: `5G bulk PM wiki page`_. + +.. _5G bulk PM wiki page: https://wiki.onap.org/display/DW/5G+-+Bulk+PM + +DFC will handle the collection of bulk PM data flow: + 1. Subscribes to fileReady DMaaP topic + 2. Collects the file from the xNF + 3. Sends new event to DataRouter with file. + + +DFC is delivered as one **Docker container** which hosts application server and can be started by `docker-compose`. +See `Delivery`_ for more information about the docker container. + +.. _Delivery: ./delivery.html + +Functionality +""""""""""""" +.. image:: ../../images/DFC.png + +Interaction +""""""""""" +DFC will interact with the DMaaP Message Router and with the Data Router via secured protocol, using json files. +So far, the implemented protocols are sftp and ftp(e)s. diff --git a/docs/sections/services/dfc/configuration.rst b/docs/sections/services/dfc/configuration.rst new file mode 100644 index 00000000..d57a85f1 --- /dev/null +++ b/docs/sections/services/dfc/configuration.rst @@ -0,0 +1,98 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Configuration +============= + +**datafile** configuration is controlled via a single JSON file called datafile_endpoints.json. +This is located under datafile-app-server/config. + +JSON CONFIGURATION EXPLAINED +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Variables of interest (e.g. variables that should be inspected/modifed for a specific runtime environment) are listed below for convenience. The entire file is provided later in this page for reference. + +dmaapConsumerConfiguration +"""""""""""""""""""""""""" + +.. code-block:: json + + "dmaapHostName": + "dmaapPortNumber": + "dmaapTopicName": "/events/unauthenticated.VES_NOTIFICATION_OUTPUT" + "dmaapProtocol": "http" + "dmaapUserName": "" + "dmaapUserPassword": "" + "dmaapContentType": "application/json" + "consumerId": "C12" + "consumerGroup": "OpenDcae-c12" + "timeoutMS": -1 + "messageLimit": 1 + +dmaapProducerConfiguration +"""""""""""""""""""""""""" + +.. code-block:: json + + "dmaapHostName": + "dmaapPortNumber": + "dmaapTopicName": "publish" + "dmaapProtocol": "httpa" + "dmaapUserName": "dradmin" + "dmaapUserPassword": "dradmin" + "dmaapContentType": "application/octet-stream" + +ftpesConfiguration +"""""""""""""""""" + +.. code-block:: json + + "keyCert": + "keyPassword": + "trustedCA": + "trustedCAPassword": + + +Sample JSON configuration +""""""""""""""""""""""""" + +The format of the JSON configuration that drives all behavior of DFC is probably best described using an example: + +.. code-block:: json + + { + "configs": { + "dmaap": { + "dmaapConsumerConfiguration": { + "dmaapHostName": "localhost", + "dmaapPortNumber": 2222, + "dmaapTopicName": "/events/unauthenticated.VES_NOTIFICATION_OUTPUT", + "dmaapProtocol": "http", + "dmaapUserName": "", + "dmaapUserPassword": "", + "dmaapContentType": "application/json", + "consumerId": "C12", + "consumerGroup": "OpenDcae-c12", + "timeoutMS": -1, + "messageLimit": 1 + }, + "dmaapProducerConfiguration": { + "dmaapHostName": "localhost", + "dmaapPortNumber": 3907, + "dmaapTopicName": "publish", + "dmaapProtocol": "https", + "dmaapUserName": "dradmin", + "dmaapUserPassword": "dradmin", + "dmaapContentType": "application/octet-stream" + } + }, + "ftp": { + "ftpesConfiguration": { + "keyCert": "config/ftpKey.jks", + "keyPassword": "secret", + "trustedCA": "config/cacerts", + "trustedCAPassword": "secret" + } + } + } + } diff --git a/docs/sections/services/dfc/delivery.rst b/docs/sections/services/dfc/delivery.rst new file mode 100644 index 00000000..233e2543 --- /dev/null +++ b/docs/sections/services/dfc/delivery.rst @@ -0,0 +1,12 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Delivery +======== + +Docker Container +---------------- + +**datafile** is delivered as a docker container that can be downloaded from onap: + + ``docker run -d -p 8100:8100 -p 8433:8433 nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile`` diff --git a/docs/sections/services/dfc/index.rst b/docs/sections/services/dfc/index.rst new file mode 100644 index 00000000..176c403c --- /dev/null +++ b/docs/sections/services/dfc/index.rst @@ -0,0 +1,19 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +DATAFILE COLLECTOR MS (DFC) +============================= + +.. Add or remove sections below as appropriate for the platform component. + +.. toctree:: + :maxdepth: 1 + + ./architecture.rst + ./delivery.rst + ./logging.rst + ./installation.rst + ./configuration.rst + ./administration.rst + ./release-notes.rst diff --git a/docs/sections/services/dfc/installation.rst b/docs/sections/services/dfc/installation.rst new file mode 100644 index 00000000..ffaa580e --- /dev/null +++ b/docs/sections/services/dfc/installation.rst @@ -0,0 +1,25 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Installation +============ + +An environment suitable for running docker containers is recommended. + +As a docker container +--------------------- + +**datafile** is delivered as a docker container based on openjdk:8-jre-alpine. The +host or VM that will run this container must have the docker application +loaded and available to the userID that will be running the DFC container. + +Also required is a working DMAAP/MR and DMAAP/DR environment. datafile +subscribes to DMAAP/MR fileReady event as JSON messages and publishes the downloaded files to the DMAAP/DR. + +Installation +^^^^^^^^^^^^ + +The following command will download the latest datafile container from +nexus and launch it in the container named "datafile": + + ``docker run -d -p 8100:8100 -p 8433:8433 nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.datafile`` diff --git a/docs/sections/services/dfc/logging.rst b/docs/sections/services/dfc/logging.rst new file mode 100644 index 00000000..b6ac7930 --- /dev/null +++ b/docs/sections/services/dfc/logging.rst @@ -0,0 +1,13 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Logging +======= + +Logging is controlled by the configuration provided to **datafile** in the application.yaml +file located in datafile-app-server/config folder. + + +**Where is the log file?** + +The log file is located under /opt/log and called application.log. diff --git a/docs/sections/services/dfc/release-notes.rst b/docs/sections/services/dfc/release-notes.rst new file mode 100644 index 00000000..fb66ebb6 --- /dev/null +++ b/docs/sections/services/dfc/release-notes.rst @@ -0,0 +1,41 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Release Notes +============= + +Version: 1.0.4 +-------------- + +:Release Date: 2018-11-08 (Casablanca) + + +**New Features** + +All DFC features from v1.0.4 are new. + + +**Bug Fixes** + +This is the initial release. + + +**Known Issues** + +No known issues. + + +**Known limitations** + + - DFC has only be tested successfully with one node. + - The certificates are distributed hand to hand, no automated process. + + +**Security Issues** + +No known security issues. + + +**Upgrade Notes** + +This is the initial release. -- cgit 1.2.3-korg