Merge "Adding PNDA documentation"

7 files changed, 215 insertions, 6 deletions

diff --git a/docs/sections/apis/PNDA.rst b/docs/sections/apis/PNDA.rst
new file mode 100644
index 00000000..f93ee2f1
--- /dev/null
+++ b/docs/sections/apis/PNDA.rst
@@ -0,0 +1,10 @@
+PNDA – Platform for Network Data Analytics
+==========================================
+
+
+PNDA has several APIs that are documented as part of the PNDA project.
+
+* https://github.com/pndaproject/platform-package-repository#repository-api
+* https://github.com/pndaproject/platform-deployment-manager#api-documentation
+* https://github.com/pndaproject/platform-data-mgmnt/blob/develop/data-service/README.md#dataset-apis
+
diff --git a/docs/sections/architecture.rst b/docs/sections/architecture.rst
index a53480b0..62b4e9b5 100644
--- a/docs/sections/architecture.rst
+++ b/docs/sections/architecture.rst
@@ -36,6 +36,7 @@ The following list displays the details of what are included in ONAP DCAE R3.  A
         - DCAE Inventory-API: API for DCAE's TOSCA model store.
     - Platform services
         - Consul: Distributed service discovery service and KV store.
+        - PNDA: Platform for Network Data Analytics.
         - Postgres Database: DCAE's TOSCA model store.
         - Redis Database: DCAE's transactional state store, used by TCA for supporting persistence and seamless scaling.
 
@@ -55,7 +56,7 @@ The following list displays the details of what are included in ONAP DCAE R3.  A
         - Universal Data Mapper service
 
 
-The figure below shows the DCAE R3 architecture and how the components work with each other.  The components on the right constitute the Platform/controller components which are statically deployed. The components on the right represent the services which can be both deployed statically or dynamically (via CLAMP)  
+The figure below shows the DCAE R3 architecture and how the components work with each other.  The components on the right constitute the Platform/controller components which are statically deployed. The components on the right represent the services which can be both deployed statically or dynamically (via CLAMP)
 
 .. image:: images/R3_architecture_diagram.gif
  
@@ -68,6 +69,8 @@ Because DCAE service components are deployed on-demand following the control loo
 
 For R3, ONAP supports two deployment methodologies: Heat Orchestration Template method, or Helm Chart method. No matter which method, DCAE is deployed following the same flow.  At its minimum, only the TOSCA model executor, the DCAE Cloudify Manager, needs to be deployed through the ONAP deployment process.  Once the Cloudify Manager is up and running, all the rest of DCAE platform can be deployed by a bootstrap script, which makes a number of calls into the Cloudify Manager API with Blueprints for various DCAE components, first the DCAE Platform components, then the service components that are needed for the built-in control loops, such as vFW/vDNS traffic throttling.  It is also possible that additional DCAE components are also launched as part of the ONAP deployment process using the ONAP level method instead of TOSCA model based method.
 
+The PNDA platform service is an optional component that can be installed when using the OOM Helm Chart installation method on Openstack based Kubernetes infrastructure.
+
 More details of the DCAE R3 deployment will be covered by the Installation section.
 
 
@@ -78,7 +81,7 @@ For ONAP R3 DCAE participates in the following use cases.
 
 - vDNS:  VES collector, TCA analytics
 
-- vFW:  VES collector, TCA analytics
+- vFW:  VES collector, TCA analytics, PNDA based analytics
 
 - vCPE:  VES collector, TCA analytics
 
diff --git a/docs/sections/humaninterfaces.rst b/docs/sections/humaninterfaces.rst
index 3afcac30..c83d1f1c 100644
--- a/docs/sections/humaninterfaces.rst
+++ b/docs/sections/humaninterfaces.rst
@@ -30,4 +30,6 @@ DCAE provides a number of interfaces for users to interact with the DCAE system.
     * The CDAP VMs accept ssh connection with the standard access key.
     * The CDAP GUI can be accessed from http://{{CDAP02_VM_IP}}:11011 .
 
-
+6. DCAE PNDA
+    * The PNDA Console can be accessed from http://knox.service.dc1.pnda.local:8443/gateway/pnda/console .
+    * The PNDA UIs can be accessed from the PNDA Console which is described here: http://pnda.io/pnda-guide/console/ .
diff --git a/docs/sections/installation.rst b/docs/sections/installation.rst
index eec69c38..da87f529 100644
--- a/docs/sections/installation.rst
+++ b/docs/sections/installation.rst
@@ -10,5 +10,6 @@ DCAE Deployment (Installation)
 
    ./installation_heat.rst
    ./installation_oom.rst
+   ./installation_pnda.rst
    ./installation_test.rst
 
diff --git a/docs/sections/installation_oom.rst b/docs/sections/installation_oom.rst
index b8926d55..9691dce6 100644
--- a/docs/sections/installation_oom.rst
+++ b/docs/sections/installation_oom.rst
@@ -18,6 +18,10 @@ At deployment time, with a single **helm install** command, Helm resolves all th
 
 All ONAP Helm Charts are organized under the **kubernetes** directory of the **OOM** project, where roughly each ONAP component occupied a subdirectory.  DCAE charts are placed under the **dcaegen2** directory.  DCAE Kubernetes deployment is based on the same set of Docker containers that the Heat based deployment uses, with the exception of bootstrap container and health check container are only used in Kubernetes deployment.
 
+The PNDA data platform is an optional DCAE component that is placed under the **pnda**
+directory. Details for how to configure values to enable PNDA installation during Helm install
+are described in `Installing PNDA During Helm Chart Based DCAE Deployment
+<installation_pnda.html>`_.
 
 DCAE Chart Organization
 -----------------------
@@ -43,15 +47,17 @@ DCAE Deployment
 ---------------
 
 At deployment time, when the **helm install** command is executed, all DCAE resources defined within charts under the OOM Chart hierarchy are deployed.  They are the 1st order components, namely the Cloudify Manager deployment, the Health Check deployment, the Redis cluster deployment, and the Bootstrap job.  In addition, a Postgres database deployment is also launched, which is specified as a dependency of the DCAE Bootstrap job.  These resources will show up as the following, where the name before / indicates resource type and the term "dev" is a tag that **helm install** command uses as "release name":
+
   * deploy/dev-dcae-cloudify-manager;
   * deploy/dev-dcae-healthcheck;
   * statefulsets/dev-dcae-redis;
   * statefulsets/dev-dcae-db;
   * job/dev-dcae-bootstrap.
 
-In addition, DCAE operations depends on a Consul server cluster.  For ONAP OOM deployment, since Consul cluster is provided as a shared resource, its charts are defined under the consul direcory, not part of DCAE charts. 
+In addition, DCAE operations depends on a Consul server cluster.  For ONAP OOM deployment, since Consul cluster is provided as a shared resource, its charts are defined under the consul direcory, not part of DCAE charts.
 
 The dcae-bootstrap job has a number of prerequisites because the subsequently deployed DCAE components depends on a number of resources having entered their normal operation state.  DCAE bootstrap job will not start before these resources are ready.  They are:
+
   * dcae-cloudify-manager;
   * consul-server;
   * msb-discovery;
@@ -60,6 +66,7 @@ The dcae-bootstrap job has a number of prerequisites because the subsequently de
 Once started, the DCAE bootstrap job will call Cloudify Manager to deploy a series of Blueprints which specify the additional DCAE R3 components.  These Blueprints are almost identical to the Docker container Blueprints used by DACE R1 and Heat based R2 deployment, except that they are using the k8splugin instead of dockerplugin.  The k8splugin is a major contribution of DCAE R2.  It is a Cloudify Manager plugin that is capable of expanding a Docker container node definition into a Kubernetes deployment definition, with enhancements such as replica scaling, ONAP logging sidecar, MSB registration, etc.
 
 The additional DCAE components launched into ONAP deployment are:
+
   * deploy/dep-config-binding-service;
   * deploy/dep-dcae-tca-analytics;
   * deploy/dep-dcae-ves-collector;
@@ -90,6 +97,7 @@ Deployment time configuration of DCAE components are defined in several places.
      * Helm command line supplied values supersedes values defined in any values.yaml files.
 
 In addition, for DCAE components deployed through Cloudify Manager Blueprints, their configuration parameters are defined in the following places:
+
      * The Blueprint files can contain static values for configuration parameters;
         * The Blueprint files are defined under the blueprints directory of the dcaegen2/platform/blueprints repo, named with "k8s" prefix.
      * The Blueprint files can specify input parameters and the values of these parameters will be used for configuring parameters in Blueprints.  The values for these input parameters can be supplied in several ways as listed below in the order of precedence (low to high):
@@ -139,6 +147,7 @@ Below is a table of default hostnames and ports for DCAE component service endpo
 
 In addition, a number of ONAP service endpoints that are used by DCAE components are listed as follows
 for reference by DCAE developers and testers:
+
     ====================   ============================      ================================
     Component              Cluster Internal (host:port)      Cluster external (svc_name:port)
     ====================   ============================      ================================
diff --git a/docs/sections/installation_pnda.rst b/docs/sections/installation_pnda.rst
new file mode 100644
index 00000000..d1c0a383
--- /dev/null
+++ b/docs/sections/installation_pnda.rst
@@ -0,0 +1,185 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+Installing PNDA During Helm Chart Based DCAE Deployment
+=======================================================
+
+PNDA is integrated into ONAP as a component system of DCAE. It is possible to deploy PNDA as
+part of an ONAP OOM deployment on Openstack infrastructure. This is achieved by using a
+pnda-bootstrap container in kubernetes to deploy Openstack VMs and then install a PNDA cluster
+onto those VMs.
+
+Requirements
+------------
+
+PNDA deployment within ONAP requires the following:
+
+* Openstack based infrastructure. Follow the `PNDA guide <http://pnda.io/pnda-guide/provisioning/openstack/PREPARE.html>`_
+  for more information.
+* OOM / Helm template based installation of ONAP on Kubernetes.
+* Kubernetes and PNDA need to share the same Openstack network subnet.
+
+The dcae-pnda-bootstrap Helm chart currently deploys a PNDA cluster with the following resource
+requirements:
+
++--------------------+--------------------+--------------------+--------------------+
+|Node                |        CPU         |       RAM GB       |      Disk GB       |
++====================+====================+====================+====================+
+|Gateway             |         2          |         4          |         20         |
++--------------------+--------------------+--------------------+--------------------+
+|Edge                |         8          |         32         |         50         |
++--------------------+--------------------+--------------------+--------------------+
+|Hadoop Manager      |         4          |         16         |         50         |
++--------------------+--------------------+--------------------+--------------------+
+|Hadoop Data x 2     |         4          |         8          |      20 + 45       |
++--------------------+--------------------+--------------------+--------------------+
+|Kafka               |         2          |         8          |         50         |
++--------------------+--------------------+--------------------+--------------------+
+
+There are several parameters that need to be defined with values from your environment for a
+successful PNDA deployment within ONAP. These parameters can be grouped in separate files that
+are passed as parameters to helm install. With the following files, the command to install ONAP
+with PNDA would then look like this:
+
+::
+
+    helm install -f ~/pnda.yaml -f ~/pnda-openstack.yaml -f ~/pnda-pem.yaml local/onap -n dev --namespace onap
+
+
+
+``pnda.yaml``
+-------------
+
+This file contains settings to enable PNDA install and the evironment settings required by PNDA.
+
+::
+
+    pnda:
+      dcae-pnda-bootstrap:
+        enabled: true
+        pnda:
+          osUser: {guest-os-username}        # e.g. centos
+          nameserver: nn.nn.nn.nn            # IP address of a reachable nameserver
+          ntp: nn.nn.nn.nn                   # IP address of a reachable NTP server
+          apps:
+            fsType: local
+          networkCidr: 192.168.0.0/24
+          outboundCidr: 0.0.0.0/0
+
+Description
+~~~~~~~~~~~
+
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.enabled         |boolean   |Deploy PNDA or not.                                     |
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda.osUser     |string    |Login used during PNDA installation process to connect  |
+|                                         |          |to newly created Openstack PNDA instances.              |
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda.nameserver |string    |IP address of the nameserver that will be used by all   |
+|                                         |          |Openstack PNDA instances.                               |
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda.ntp        |string    |Hostname or IP address of a NTP server. This NTP server |
+|                                         |          |MUST be reachable from the Openstack PNDA instances.    |
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda.apps.fsType|string    |Type of storage used to store PNDA application packages |
+|                                         |          |(s3, sshfs, local, swift). Set it to local.             |
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda.networkCidr|string    |CIDR specifying the address range for the network       |
+|                                         |          |containing all PNDA instances.                          |
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda.dataNodes  |number    |Number of data nodes to deploy. Defaults to 2.          |
++-----------------------------------------+----------+--------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda.kafkaNodes |number    |Number of kafka nodes to deploy. Defaults to 1.         |
++-----------------------------------------+----------+--------------------------------------------------------+
+
+
+``pnda-openstack.yaml``
+-----------------------
+
+This file contains the Openstack connection settings and Openstack entity identifiers.
+
+::
+
+    pnda:
+      dcae-pnda-bootstrap:
+        openstack:
+          keystoneUser: {openstack-username}
+          keystonePassword: {openstack-password}
+          keystoneTenant: {openstack-tenant-name}
+          keystoneAuthUrl: {openstack-auth-url}             # e.g. http://openstack-host:5000/v2.0/
+          keystoneRegion: {openstack-region-name}
+          useExistingNetwork: true
+          existingNetworkId: {kubernetes-cluster-network}
+          existingSubnetId: {kubernetes-cluster-subnet}
+          imageId: {pnda-vm-image-id}                       # Openstack imageId to use for PNDA VMs
+          publicNetworkId: {public-network-id}              # Openstack networkId of public facing network
+          publicSubnetCidr: nn.nn.nn.nn/nn                  # CIDR address of the public subnet e.g. 10.40.10.0/24
+
+Description
+~~~~~~~~~~~
+
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.keystoneUser      |string    |Openstack user.                                |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.keystonePassword  |string    |Openstack password.                            |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.keystoneTenant    |string    |Openstack tenant.                              |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.keystoneAuthUrl   |string    |Openstack authentication url.                  |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.keystoneRegion    |string    |Openstack region.                              |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.useExistingNetwork|boolean   |Do you want to create a new Openstack network  |
+|                                                     |          |or do you want to use an already existing one? |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.existingNetworkId |string    |If you want to use an already existing         |
+|                                                     |          |Openstack network, specify its UUID.           |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.existingSubnetId  |string    |If you want to use an already existing         |
+|                                                     |          |Openstack network, specify the UUID of the     |
+|                                                     |          |Openstack subnet to use.                       |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.imageId           |string    |Base image to use for the created Openstack    |
+|                                                     |          |PNDA instances.                                |
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.publicNetworkId   |string    |UUID of the public network in openstack to use.|
++-----------------------------------------------------+----------+-----------------------------------------------+
+|pnda.dcae-pnda-bootstrap.openstack.publicSubnetCidr  |string    |CIDR specifying the address range for the      |
+|                                                     |          |public subnet.                                 |
++-----------------------------------------------------+----------+-----------------------------------------------+
+
+
+``pnda-pem.yaml``
+-----------------
+
+This file contains the private secret and Openstack keypair name to use for the guest VMs in Openstack.
+
+::
+
+    pnda:
+      dcae-pnda-bootstrap:
+        pnda_keypair_name: {keypair-name}    # Name of the keypair you have created in Openstack, e.g. pnda-key
+        pnda_secret: |
+          -----BEGIN DSA PRIVATE KEY-----
+          MIIBugIBAAKBgQCLLCmDJdxCxOOmaSGoH0WUyoiGUJiE0JnzEEhXd4SZWmoxo7yn
+          9d3iA9z2OiOnbQ4s5tAOWknpFEnIwtFb+L2x3Fzv7yEdHBk0tgC8c91sIxb72SVp
+          RgJDaseL2C5RRdWqDiKlXnA1iY7H5z/k/bZD61/4eGdAojVxroQX2H6uLQIVAIrG
+          WVGBudBQmfwKzyP5eO1nevNFAoGAALSGjzP0/yVnEKZ8JO8Vw/eC1YDmTG1IwBdN
+          Rm8fXqyUjd03ijpmOoPISPM6Jt0TrJTvGNVfqSI1mKVznJ+5B7y5M9qvjzHQW5hL
+          GkXHL57mN2QkaJE/m4ilKr/p5RzTBk6c/zhvxg/8DEne6klQ3NQhapzY4cL9aLK6
+          hrR4T94CgYAhGFZI5buQUk8MtG9nac9hEQuYw+lrdjy+y1F8XIjq8+ZLAdCY5gw/
+          ph+4di/R8MyesqG5AtqUqXQY3tibya3YrqyNZ5mTz6kMN1lT26QHwYMBF1IymMWV
+          lq5wzjpctVovYchCLagrh7KepetNigni0Vrjc0TLPbvXQkoLG5JifgIURmDpblYA
+          HSNN6un6nlCoGnm07SA=
+          -----END DSA PRIVATE KEY-----
+
+Description
+~~~~~~~~~~~
+
++------------------------------------------+----------+-------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda_keypair_name|string    |Name of the keypair to use to spawn the Openstack PNDA |
+|                                          |          |instances.Note that the private key above is a         |
+|                                          |          |generated sample that is unusable.                     |
++------------------------------------------+----------+-------------------------------------------------------+
+|pnda.dcae-pnda-bootstrap.pnda_secret      |string    |Secret key of the pnda_keypair_name keypair.           |
++------------------------------------------+----------+-------------------------------------------------------+
diff --git a/docs/sections/offeredapis.rst b/docs/sections/offeredapis.rst
index fc9cb773..2b26d36d 100644
--- a/docs/sections/offeredapis.rst
+++ b/docs/sections/offeredapis.rst
@@ -13,5 +13,4 @@ DCAEGEN2 Components Offered APIs
    apis/dcaecdap.rst
    apis/PRH.rst
    apis/DFC.rst
-
-
+   apis/PNDA.rst