summaryrefslogtreecommitdiffstats
path: root/docs/distribution
diff options
context:
space:
mode:
authorramverma <ram.krishna.verma@est.tech>2019-07-10 11:25:37 +0000
committerramverma <ram.krishna.verma@est.tech>2019-07-10 11:25:37 +0000
commit3b71c9772e0f86b5cef9840668a36569540a037c (patch)
treee960a7651237b6dbb63beacb344e679ca16a2e1d /docs/distribution
parent0c56e114555b17818d2ee03bb680f27df8158ade (diff)
Adding apex-pdp & distribution documents to parent
Change-Id: If8039f68a1669787685d4a7bdc37dec203901062 Issue-ID: POLICY-1898 Signed-off-by: ramverma <ram.krishna.verma@est.tech>
Diffstat (limited to 'docs/distribution')
-rw-r--r--docs/distribution/Distribution-Introduction.rst51
-rw-r--r--docs/distribution/Distribution-User-Manual.rst648
-rw-r--r--docs/distribution/distribution.rst11
3 files changed, 710 insertions, 0 deletions
diff --git a/docs/distribution/Distribution-Introduction.rst b/docs/distribution/Distribution-Introduction.rst
new file mode 100644
index 00000000..6fb0d979
--- /dev/null
+++ b/docs/distribution/Distribution-Introduction.rst
@@ -0,0 +1,51 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+Introduction to Policy-Distribution
+***********************************
+
+ .. container:: paragraph
+
+ The main job of policy distribution component is to receive
+ incoming notifications, download artifacts, decode policies
+ from downloaded artifacts & forward the decoded policies to
+ all configured policy engines.
+
+|
+
+ .. container:: paragraph
+
+ The current implementation of distribution component comes
+ with built-in SDC reception handler for receiving incoming
+ distribution notifications from SDC using SDC client library.
+ Upon receiving the notification, the corresponding CSAR artifacts
+ are downloaded using SDC client library.The downloaded CSAR is
+ then given to the configured policy decoder for decoding and
+ generating policies. The generated policies are then forwarded
+ to all configured policy engines. Related distribution status
+ is sent to SDC at each step (download/deploy/done) during the
+ entire flow.
+
+|
+
+ .. container:: paragraph
+
+ The distribution component also comes with built-in REST based
+ endpoints for fetching health check status & statistical data
+ of running distribution system.
+
+|
+
+ .. container:: paragraph
+
+ The distribution component is designed using plugin based architecture.
+ All the handlers, decoders & forwarders are basically plugins to
+ the running distribution engine. The plugins are configured in the
+ configuration JSON file provided during startup of distribution engine.
+ Adding a new plugin is simply implementing the related interfaces,
+ adding them to the configuration JSON file & making the classes available
+ in the classpath while starting distribution engine. There is no need
+ to edit anything in the distribution core engine.
+ Refer to distribution user manual for more details about the system and
+ the configuration.
diff --git a/docs/distribution/Distribution-User-Manual.rst b/docs/distribution/Distribution-User-Manual.rst
new file mode 100644
index 00000000..12f75b9a
--- /dev/null
+++ b/docs/distribution/Distribution-User-Manual.rst
@@ -0,0 +1,648 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+
+Distribution User Manual
+************************
+
+.. contents::
+ :depth: 3
+
+Installation
+^^^^^^^^^^^^
+
+Requirements
+------------
+
+ .. container:: paragraph
+
+ Distribution is 100% written in Java and runs on any platform
+ that supports a JVM, e.g. Windows, Unix, Cygwin.
+
+Installation Requirements
+#########################
+
+ .. container:: ulist
+
+ - Downloaded distribution: JAVA runtime environment
+ (JRE, Java 8 or later, Distribution is tested with the
+ OpenJDK)
+
+ - Building from source: JAVA development kit (JDK,
+ Java 8 or later, Distribution is tested with the OpenJDK)
+
+ - Sufficient rights to install Distribution on the system
+
+ - Installation tools
+
+ .. container:: ulist
+
+ - TAR and GZ to extract from that TAR.GZ
+ distribution
+
+ .. container:: ulist
+
+ - Windows for instance
+ `7Zip <http://www.7-zip.org/>`__
+
+ - `Docker <https://www.docker.com/>`__ to run Distribution
+ inside a Docker container
+
+
+Build (Install from Source) Requirements
+########################################
+
+ .. container:: paragraph
+
+ Installation from source requires a few development
+ tools
+
+ .. container:: ulist
+
+ - GIT to retrieve the source code
+
+ - Java SDK, Java version 8 or later
+
+ - Apache Maven 3 (the Distribution build environment)
+
+Get the Distribution Source Code
+--------------------------------
+
+ .. container:: paragraph
+
+ The Distribution source code is hosted in ONAP as project distribution.
+ The current stable version is in the master branch.
+ Simply clone the master branch from ONAP using HTTPS.
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ .. code:: text
+
+ git clone https://gerrit.onap.org/r/policy/distribution
+
+Build Distribution
+------------------
+
+ .. container:: paragraph
+
+ The examples in this document assume that the distribution source
+ repositories are cloned to:
+
+ .. container:: ulist
+
+ - Unix, Cygwin: ``/usr/local/src/distribution``
+
+ - Windows: ``C:\dev\distribution``
+
+ - Cygwin: ``/cygdrive/c/dev/distribution``
+
+ .. important::
+ A Build requires ONAP Nexus
+ Distribution has a dependency to ONAP parent projects. You might need to adjust your Maven M2 settings. The most current
+ settings can be found in the ONAP oparent repo: `Settings <https://git.onap.org/oparent/plain/settings.xml>`__.
+
+ .. important::
+ A Build needs Space
+ Building distribution requires approximately 1-2 GB of hard disc space, 100 MB for the actual build with full
+ distribution and around 1 GB for the downloaded dependencies.
+
+ .. important::
+ A Build requires Internet (for first build)
+ During the build, several (a lot) of Maven dependencies will be downloaded and stored in the configured local Maven
+ repository. The first standard build (and any first specific build) requires Internet access to download those
+ dependencies.
+
+ .. container:: paragraph
+
+ Use Maven for a standard build without any tests.
+
+ +-------------------------------------------------------+--------------------------------------------------------+
+ | Unix, Cygwin | Windows |
+ +=======================================================+========================================================+
+ | .. container:: | .. container:: |
+ | | |
+ | .. container:: content | .. container:: content |
+ | | |
+ | .. code:: text | .. code:: text |
+ | | |
+ | # cd /usr/local/src/distribution | >c: |
+ | # mvn clean install -DskipTest | >cd \dev\distribution |
+ | | >mvn clean install -DskipTests |
+ +-------------------------------------------------------+--------------------------------------------------------+
+
+.. container:: paragraph
+
+ The build takes 2-3 minutes on a standard development laptop. It
+ should run through without errors, but with a lot of messages from
+ the build process.
+
+|
+
+.. container:: paragraph
+
+ When Maven is finished with the build, the final screen should look
+ similar to this (omitting some ``success`` lines):
+
+.. container:: listingblock
+
+ .. container:: content
+
+ .. code:: text
+
+ [INFO] ------------------------------------------------------------------------
+ [INFO] Reactor Summary:
+ [INFO]
+ [INFO] policy-distribution ................................ SUCCESS [ 3.666 s]
+ [INFO] distribution-model ................................. SUCCESS [ 11.234 s]
+ [INFO] forwarding ......................................... SUCCESS [ 7.611 s]
+ [INFO] reception .......................................... SUCCESS [ 7.072 s]
+ [INFO] main ............................................... SUCCESS [ 21.017 s]
+ [INFO] plugins ............................................ SUCCESS [ 0.453 s]
+ [INFO] forwarding-plugins ................................. SUCCESS [01:20 min]
+ [INFO] reception-plugins .................................. SUCCESS [ 18.545 s]
+ [INFO] Policy Distribution Packages ....................... SUCCESS [ 0.419 s]
+ [INFO] ------------------------------------------------------------------------
+ [INFO] BUILD SUCCESS
+ [INFO] ------------------------------------------------------------------------
+ [INFO] Total time: 02:39 min
+ [INFO] Finished at: 2018-11-15T13:59:09Z
+ [INFO] Final Memory: 73M/1207M
+ [INFO] ------------------------------------------------------------------------
+
+.. container:: paragraph
+
+ The build will have created all artifacts required for distribution
+ installation. The following example show how to change to the target
+ directory and how it should look like.
+
++----------------------------------------------------------------------------------------------------------------------------+
+| Unix, Cygwin |
++============================================================================================================================+
+| .. container:: |
+| |
+| .. container:: listingblock |
+| |
+| .. container:: content |
+| |
+| .. code:: text |
+| |
+| -rw-r--r-- 1 user 1049089 10616 Oct 31 13:35 checkstyle-checker.xml |
+| -rw-r--r-- 1 user 1049089 609 Oct 31 13:35 checkstyle-header.txt |
+| -rw-r--r-- 1 user 1049089 245 Oct 31 13:35 checkstyle-result.xml |
+| -rw-r--r-- 1 user 1049089 89 Oct 31 13:35 checkstyle-cachefile |
+| drwxr-xr-x 1 user 1049089 0 Oct 31 13:35 maven-archiver/ |
+| -rw-r--r-- 1 user 1049089 7171 Oct 31 13:35 policy-distribution-tarball-2.0.1-SNAPSHOT.jar |
+| drwxr-xr-x 1 user 1049089 0 Oct 31 13:35 archive-tmp/ |
+| -rw-r--r-- 1 user 1049089 66296012 Oct 31 13:35 policy-distribution-tarball-2.0.1-SNAPSHOT-tarball.tar.gz |
+| drwxr-xr-x 1 user 1049089 0 Nov 12 10:56 test-classes/ |
+| drwxr-xr-x 1 user 1049089 0 Nov 20 14:31 classes/ |
++----------------------------------------------------------------------------------------------------------------------------+
+
++-------------------------------------------------------------------------------------------------------------------+
+| Windows |
++===================================================================================================================+
+| .. container:: |
+| |
+| .. container:: listingblock |
+| |
+| .. container:: content |
+| |
+| .. code:: text |
+| |
+| 11/12/2018 10:56 AM <DIR> . |
+| 11/12/2018 10:56 AM <DIR> .. |
+| 10/31/2018 01:35 PM <DIR> archive-tmp |
+| 10/31/2018 01:35 PM 89 checkstyle-cachefile |
+| 10/31/2018 01:35 PM 10,616 checkstyle-checker.xml |
+| 10/31/2018 01:35 PM 609 checkstyle-header.txt |
+| 10/31/2018 01:35 PM 245 checkstyle-result.xml |
+| 11/20/2018 02:31 PM <DIR> classes |
+| 10/31/2018 01:35 PM <DIR> maven-archiver |
+| 10/31/2018 01:35 PM 66,296,012 policy-distribution-tarball-2.0.1-SNAPSHOT-tarball.tar.gz |
+| 10/31/2018 01:35 PM 7,171 policy-distribution-tarball-2.0.1-SNAPSHOT.jar |
+| 11/12/2018 10:56 AM <DIR> test-classes |
++-------------------------------------------------------------------------------------------------------------------+
+
+Install Distribution
+--------------------
+
+ .. container:: paragraph
+
+ Distribution can be installed in different ways:
+
+ .. container:: ulist
+
+ - Windows, Unix, Cygwin: manually from a ``.tar.gz`` archive
+
+ - Windows, Unix, Cygwin: build from source using Maven, then
+ install manually
+
+Install Manually from Archive (Windows, 7Zip, GUI)
+##################################################
+
+ .. container:: paragraph
+
+ Download a ``tar.gz`` archive and copy the file into the install
+ folder (in this example ``C:\distribution``). Assuming you are using 7Zip,
+ right click on the file and extract the ``tar`` archive.
+
+|
+
+ .. container:: content
+
+ Extract the TAR archive
+
+ .. container:: paragraph
+
+ The right-click on the new created TAR file and extract the actual
+ distribution.
+
+|
+
+ .. container:: content
+
+ Extract the distribution
+
+ .. container:: paragraph
+
+ Inside the new distribution folder you see the main directories: ``bin``,
+ ``etc``and ``lib``
+
+|
+
+ .. container:: paragraph
+
+ Once extracted, please rename the created folder to
+ ``distribution-full-2.0.2-SNAPSHOT``. This will keep the directory name in
+ line with the rest of this documentation.
+
+Build from Source
+-----------------
+
+Build and Install Manually (Unix, Windows, Cygwin)
+##################################################
+
+ .. container:: paragraph
+
+ Clone the Distribution GIT repositories into a directory. Go to that
+ directory. Use Maven to build Distribution (all details on building
+ Distribution from source can be found in *Distribution HowTo: Build*).
+
+ .. container:: paragraph
+
+ Now, take the ``.tar.gz`` file and install distribution.
+
+Installation Layout
+-------------------
+
+ .. container:: paragraph
+
+ A full installation of distribution comes with the following layout.
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ - bin
+ - etc
+ - lib
+
+Running Distribution in Docker
+------------------------------
+
+Run in ONAP
+###########
+
+ .. container:: paragraph
+
+ Running distribution from the ONAP docker repository only requires 2
+ commands:
+
+ .. container:: olist arabic
+
+ #. Log into the ONAP docker repo
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ ::
+
+ docker login -u docker -p docker nexus3.onap.org:10003
+
+ .. container:: olist arabic
+
+ #. Run the distribution docker image
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ ::
+
+ docker run -it --rm nexus3.onap.org:10003/onap/policy-distribution:latest
+
+Build a Docker Image
+####################
+
+ .. container:: paragraph
+
+ Alternatively, one can use the Dockerfile defined in the Docker
+ package to build an image.
+
+Distribution Configurations Explained
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Introduction to Distribution Configuration
+------------------------------------------
+
+ .. container:: paragraph
+
+ A distribution engine can be configured to use various combinations
+ of policy reception handlers, policy decoders and policy forwarders.
+ The system is build using a plugin architecture. Each configuration
+ option is realized by a plugin, which can be loaded and
+ configured when the engine is started. New plugins can be
+ added to the system at any time, though to benefit from a
+ new plugin an engine will need to be restarted.
+
+ |
+
+ .. container:: paragraph
+
+ The distribution already comes with sdc reception handler,
+ file reception handler, xacml policy decoder, file in csar policy decoder,
+ xacml policy forwarder, apex policy forwarder.
+
+General Configuration Format
+----------------------------
+
+ .. container:: paragraph
+
+ The distribution configuration file is a JSON file containing a few
+ main blocks for different parts of the configuration. Each
+ block then holds the configuration details. The following
+ code shows the main blocks:
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ .. code:: text
+
+ {
+ "restServerParameters":{
+ ... (1)
+ },
+ "receptionHandlerParameters":{ (2)
+ "pluginHandlerParameters":{ (3)
+ "policyDecoders":{...}, (4)
+ "policyForwarders":{...} (5)
+ }
+ },
+ "receptionHandlerConfigurationParameters":{
+ ... (6)
+ }
+ ,
+ "policyForwarderConfigurationParameters":{
+ ... (7)
+ }
+ ,
+ "policyDecoderConfigurationParameters":{
+ ... (8)
+ }
+ }
+
+ .. container:: colist arabic
+
+ +-----------------------------------+-----------------------------------+
+ | **1** | rest server configuration |
+ +-----------------------------------+-----------------------------------+
+ | **2** | reception handler plugin |
+ | | configurations |
+ +-----------------------------------+-----------------------------------+
+ | **3** | plugin handler parameters |
+ | | configuration |
+ +-----------------------------------+-----------------------------------+
+ | **4** | policy decoder plugin |
+ | | configuration |
+ +-----------------------------------+-----------------------------------+
+ | **5** | policy forwarder plugin |
+ | | configuration |
+ +-----------------------------------+-----------------------------------+
+ | **6** | reception handler plugin |
+ | | parameters |
+ +-----------------------------------+-----------------------------------+
+ | **7** | policy forwarder plugin |
+ | | parameters |
+ +-----------------------------------+-----------------------------------+
+ | **8** | policy decoder plugin |
+ | | parameters |
+ +-----------------------------------+-----------------------------------+
+
+A configuration example
+-----------------------
+
+ .. container:: paragraph
+
+ The following example loads HPA use case related plug-ins.
+
+ .. container:: paragraph
+
+ Notifications are consumed from SDC through SDC client.
+ Consumed artifacts format is CSAR.
+
+ .. container:: paragraph
+
+ Generated policies are forwarded to XACML policy engine.
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ .. code:: text
+
+ {
+ "name":"SDCDistributionGroup",
+ "restServerParameters":{
+ "host":"0.0.0.0",
+ "port":6969,
+ "userName":"healthcheck",
+ "password":"zb!XztG34"
+ },
+ "receptionHandlerParameters":{
+ "SDCReceptionHandler":{
+ "receptionHandlerType":"SDC",
+ "receptionHandlerClassName":"org.onap.policy.distribution.reception.handling.sdc.SdcReceptionHandler",
+ "receptionHandlerConfigurationName":"sdcConfiguration",
+ "pluginHandlerParameters":{
+ "policyDecoders":{
+ "CsarDecoder":{
+ "decoderType":"CsarDecoder",
+ "decoderClassName":"org.onap.policy.distribution.reception.decoding.pdpx.PolicyDecoderCsarPdpx",
+ "decoderConfigurationName": "csarToOptimizationPolicyConfiguration"
+ }
+ },
+ "policyForwarders":{
+ "PAPEngineForwarder":{
+ "forwarderType":"PAPEngine",
+ "forwarderClassName":"org.onap.policy.distribution.forwarding.xacml.pdp.XacmlPdpPolicyForwarder",
+ "forwarderConfigurationName": "xacmlPdpConfiguration"
+ }
+ }
+ }
+ }
+ },
+ "receptionHandlerConfigurationParameters":{
+ "sdcConfiguration":{
+ "parameterClassName":"org.onap.policy.distribution.reception.handling.sdc.SdcReceptionHandlerConfigurationParameterGroup",
+ "parameters":{
+ "asdcAddress": "sdc-be.onap:8443",
+ "messageBusAddress": [
+ "message-router.onap"
+ ],
+ "user": "policy",
+ "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U",
+ "pollingInterval":20,
+ "pollingTimeout":30,
+ "consumerId": "policy-id",
+ "artifactTypes": [
+ "TOSCA_CSAR",
+ "HEAT"
+ ],
+ "consumerGroup": "policy-group",
+ "environmentName": "AUTO",
+ "keystorePath": "null",
+ "keystorePassword": "null",
+ "activeserverTlsAuth": false,
+ "isFilterinEmptyResources": true,
+ "isUseHttpsWithDmaap": false
+ }
+ }
+ },
+ "policyDecoderConfigurationParameters":{
+ "csarToOptimizationPolicyConfiguration":{
+ "parameterClassName":"org.onap.policy.distribution.reception.decoding.pdpx.PolicyDecoderCsarPdpxConfigurationParameterGroup",
+ "parameters":{
+ "policyNamePrefix": "oofCasablanca",
+ "onapName": "OOF",
+ "version": "1.0",
+ "priority": "3",
+ "riskType": "Test",
+ "riskLevel": "2"
+ }
+ }
+ },
+ "policyForwarderConfigurationParameters":{
+ "xacmlPdpConfiguration":{
+ "parameterClassName":"org.onap.policy.distribution.forwarding.xacml.pdp.XacmlPdpPolicyForwarderParameterGroup",
+ "parameters":{
+ "useHttps": false,
+ "hostname": "pdp",
+ "port": 8081,
+ "userName": "testpdp",
+ "password": "alpha123",
+ "clientAuth": "cHl0aG9uOnRlc3Q=",
+ "isManaged": true,
+ "pdpGroup": "pdpGroup"
+ }
+ }
+ }
+ }
+
+
+The Distribution Engine
+^^^^^^^^^^^^^^^^^^^^^^^
+
+ .. container:: paragraph
+
+ The Distribution engine can be started using ``policy-dist.sh`` script.
+ The script is located in the source code at
+ *distribution/packages/policy-distribution-docker/src/main/docker*
+ directory
+
+ |
+
+ .. container:: paragraph
+
+ On UNIX and Cygwin systems use ``policy-dist.sh`` script.
+
+ |
+
+ .. container:: paragraph
+
+ On Windows systems navigate to the distribution installation directory.
+ Run the following command
+ ``java -cp "etc:lib\*" org.onap.policy.distribution.main.startstop.Main -c <config-file-path>``
+
+ |
+
+ .. container:: paragraph
+
+ The Distribution engine comes with CLI arguments for setting
+ configuration. The configuration file is always required.
+ The option ``-h`` prints a help screen.
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ .. code:: text
+
+ usage: org.onap.policy.distribution.main.startstop.Main [options...]
+ options
+ -c,--config-file <CONFIG_FILE> the full path to the configuration file to use, the configuration file must be a Json file
+ containing the distribution configuration parameters
+ -h,--help outputs the usage of this command
+ -v,--version outputs the version of distribution system
+
+
+The Distribution REST End-points
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+ .. container:: paragraph
+
+ The distribution engine comes with built-in REST based
+ endpoints for fetching health check status & statistical data
+ of running distribution system.
+
+ .. container:: listingblock
+
+ .. container:: content
+
+ .. code:: text
+
+ # Example Output from curl http -a '{user}:{password}' :6969/healthcheck
+
+ HTTP/1.1 200 OK
+ Content-Length: XXX
+ Content-Type: application/json
+ Date: Tue, 17 Apr 2018 10:51:14 GMT
+ Server: Jetty(9.3.20.v20170531)
+ {
+ "code":200,
+ "healthy":true,
+ "message":"alive",
+ "name":"Policy SSD",
+ "url":"self"
+ }
+
+ # Example Output from curl http -a '{user}:{password}' :6969/statistics
+
+ HTTP/1.1 200 OK
+ Content-Length: XXX
+ Content-Type: application/json
+ Date: Tue, 17 Apr 2018 10:51:14 GMT
+ Server: Jetty(9.3.20.v20170531)
+ {
+ "code":200,
+ "distributions":10,
+ "distribution_complete_ok":8,
+ "distribution_complete_fail":2,
+ "downloads":15,
+ "downloads_ok"; 10,
+ "downloads_error": 5
+ }
+
diff --git a/docs/distribution/distribution.rst b/docs/distribution/distribution.rst
new file mode 100644
index 00000000..cd2f90b7
--- /dev/null
+++ b/docs/distribution/distribution.rst
@@ -0,0 +1,11 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+
+.. _policy-ssd-doc:
+
+POLICY DISTRIBUTION
+------------------------------------------------
+.. toctree::
+ :maxdepth: 1
+
+ Distribution-Introduction.rst
+ Distribution-User-Manual.rst