From 592f9d090e4351a27aa6f283f5fb2b0b3dc7255b Mon Sep 17 00:00:00 2001 From: ramverma Date: Thu, 22 Nov 2018 11:33:03 +0000 Subject: policy/distribution documents Change-Id: I1c3e083a9ce050ce67acd5f68d0dcf498927592a Issue-ID: POLICY-1260 Signed-off-by: ramverma --- docs/Distribution-Introduction.rst | 51 +++ docs/Distribution-User-Manual.rst | 648 +++++++++++++++++++++++++++++++++++++ docs/index.rst | 9 + 3 files changed, 708 insertions(+) create mode 100644 docs/Distribution-Introduction.rst create mode 100644 docs/Distribution-User-Manual.rst create mode 100644 docs/index.rst (limited to 'docs') diff --git a/docs/Distribution-Introduction.rst b/docs/Distribution-Introduction.rst new file mode 100644 index 00000000..6fb0d979 --- /dev/null +++ b/docs/Distribution-Introduction.rst @@ -0,0 +1,51 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +Introduction to Policy-Distribution +*********************************** + + .. container:: paragraph + + The main job of policy distribution component is to receive + incoming notifications, download artifacts, decode policies + from downloaded artifacts & forward the decoded policies to + all configured policy engines. + +| + + .. container:: paragraph + + The current implementation of distribution component comes + with built-in SDC reception handler for receiving incoming + distribution notifications from SDC using SDC client library. + Upon receiving the notification, the corresponding CSAR artifacts + are downloaded using SDC client library.The downloaded CSAR is + then given to the configured policy decoder for decoding and + generating policies. The generated policies are then forwarded + to all configured policy engines. Related distribution status + is sent to SDC at each step (download/deploy/done) during the + entire flow. + +| + + .. container:: paragraph + + The distribution component also comes with built-in REST based + endpoints for fetching health check status & statistical data + of running distribution system. + +| + + .. container:: paragraph + + The distribution component is designed using plugin based architecture. + All the handlers, decoders & forwarders are basically plugins to + the running distribution engine. The plugins are configured in the + configuration JSON file provided during startup of distribution engine. + Adding a new plugin is simply implementing the related interfaces, + adding them to the configuration JSON file & making the classes available + in the classpath while starting distribution engine. There is no need + to edit anything in the distribution core engine. + Refer to distribution user manual for more details about the system and + the configuration. diff --git a/docs/Distribution-User-Manual.rst b/docs/Distribution-User-Manual.rst new file mode 100644 index 00000000..12f75b9a --- /dev/null +++ b/docs/Distribution-User-Manual.rst @@ -0,0 +1,648 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +Distribution User Manual +************************ + +.. contents:: + :depth: 3 + +Installation +^^^^^^^^^^^^ + +Requirements +------------ + + .. container:: paragraph + + Distribution is 100% written in Java and runs on any platform + that supports a JVM, e.g. Windows, Unix, Cygwin. + +Installation Requirements +######################### + + .. container:: ulist + + - Downloaded distribution: JAVA runtime environment + (JRE, Java 8 or later, Distribution is tested with the + OpenJDK) + + - Building from source: JAVA development kit (JDK, + Java 8 or later, Distribution is tested with the OpenJDK) + + - Sufficient rights to install Distribution on the system + + - Installation tools + + .. container:: ulist + + - TAR and GZ to extract from that TAR.GZ + distribution + + .. container:: ulist + + - Windows for instance + `7Zip `__ + + - `Docker `__ to run Distribution + inside a Docker container + + +Build (Install from Source) Requirements +######################################## + + .. container:: paragraph + + Installation from source requires a few development + tools + + .. container:: ulist + + - GIT to retrieve the source code + + - Java SDK, Java version 8 or later + + - Apache Maven 3 (the Distribution build environment) + +Get the Distribution Source Code +-------------------------------- + + .. container:: paragraph + + The Distribution source code is hosted in ONAP as project distribution. + The current stable version is in the master branch. + Simply clone the master branch from ONAP using HTTPS. + + .. container:: listingblock + + .. container:: content + + .. code:: text + + git clone https://gerrit.onap.org/r/policy/distribution + +Build Distribution +------------------ + + .. container:: paragraph + + The examples in this document assume that the distribution source + repositories are cloned to: + + .. container:: ulist + + - Unix, Cygwin: ``/usr/local/src/distribution`` + + - Windows: ``C:\dev\distribution`` + + - Cygwin: ``/cygdrive/c/dev/distribution`` + + .. important:: + A Build requires ONAP Nexus + Distribution has a dependency to ONAP parent projects. You might need to adjust your Maven M2 settings. The most current + settings can be found in the ONAP oparent repo: `Settings `__. + + .. important:: + A Build needs Space + Building distribution requires approximately 1-2 GB of hard disc space, 100 MB for the actual build with full + distribution and around 1 GB for the downloaded dependencies. + + .. important:: + A Build requires Internet (for first build) + During the build, several (a lot) of Maven dependencies will be downloaded and stored in the configured local Maven + repository. The first standard build (and any first specific build) requires Internet access to download those + dependencies. + + .. container:: paragraph + + Use Maven for a standard build without any tests. + + +-------------------------------------------------------+--------------------------------------------------------+ + | Unix, Cygwin | Windows | + +=======================================================+========================================================+ + | .. container:: | .. container:: | + | | | + | .. container:: content | .. container:: content | + | | | + | .. code:: text | .. code:: text | + | | | + | # cd /usr/local/src/distribution | >c: | + | # mvn clean install -DskipTest | >cd \dev\distribution | + | | >mvn clean install -DskipTests | + +-------------------------------------------------------+--------------------------------------------------------+ + +.. container:: paragraph + + The build takes 2-3 minutes on a standard development laptop. It + should run through without errors, but with a lot of messages from + the build process. + +| + +.. container:: paragraph + + When Maven is finished with the build, the final screen should look + similar to this (omitting some ``success`` lines): + +.. container:: listingblock + + .. container:: content + + .. code:: text + + [INFO] ------------------------------------------------------------------------ + [INFO] Reactor Summary: + [INFO] + [INFO] policy-distribution ................................ SUCCESS [ 3.666 s] + [INFO] distribution-model ................................. SUCCESS [ 11.234 s] + [INFO] forwarding ......................................... SUCCESS [ 7.611 s] + [INFO] reception .......................................... SUCCESS [ 7.072 s] + [INFO] main ............................................... SUCCESS [ 21.017 s] + [INFO] plugins ............................................ SUCCESS [ 0.453 s] + [INFO] forwarding-plugins ................................. SUCCESS [01:20 min] + [INFO] reception-plugins .................................. SUCCESS [ 18.545 s] + [INFO] Policy Distribution Packages ....................... SUCCESS [ 0.419 s] + [INFO] ------------------------------------------------------------------------ + [INFO] BUILD SUCCESS + [INFO] ------------------------------------------------------------------------ + [INFO] Total time: 02:39 min + [INFO] Finished at: 2018-11-15T13:59:09Z + [INFO] Final Memory: 73M/1207M + [INFO] ------------------------------------------------------------------------ + +.. container:: paragraph + + The build will have created all artifacts required for distribution + installation. The following example show how to change to the target + directory and how it should look like. + ++----------------------------------------------------------------------------------------------------------------------------+ +| Unix, Cygwin | ++============================================================================================================================+ +| .. container:: | +| | +| .. container:: listingblock | +| | +| .. container:: content | +| | +| .. code:: text | +| | +| -rw-r--r-- 1 user 1049089 10616 Oct 31 13:35 checkstyle-checker.xml | +| -rw-r--r-- 1 user 1049089 609 Oct 31 13:35 checkstyle-header.txt | +| -rw-r--r-- 1 user 1049089 245 Oct 31 13:35 checkstyle-result.xml | +| -rw-r--r-- 1 user 1049089 89 Oct 31 13:35 checkstyle-cachefile | +| drwxr-xr-x 1 user 1049089 0 Oct 31 13:35 maven-archiver/ | +| -rw-r--r-- 1 user 1049089 7171 Oct 31 13:35 policy-distribution-tarball-2.0.1-SNAPSHOT.jar | +| drwxr-xr-x 1 user 1049089 0 Oct 31 13:35 archive-tmp/ | +| -rw-r--r-- 1 user 1049089 66296012 Oct 31 13:35 policy-distribution-tarball-2.0.1-SNAPSHOT-tarball.tar.gz | +| drwxr-xr-x 1 user 1049089 0 Nov 12 10:56 test-classes/ | +| drwxr-xr-x 1 user 1049089 0 Nov 20 14:31 classes/ | ++----------------------------------------------------------------------------------------------------------------------------+ + ++-------------------------------------------------------------------------------------------------------------------+ +| Windows | ++===================================================================================================================+ +| .. container:: | +| | +| .. container:: listingblock | +| | +| .. container:: content | +| | +| .. code:: text | +| | +| 11/12/2018 10:56 AM . | +| 11/12/2018 10:56 AM .. | +| 10/31/2018 01:35 PM archive-tmp | +| 10/31/2018 01:35 PM 89 checkstyle-cachefile | +| 10/31/2018 01:35 PM 10,616 checkstyle-checker.xml | +| 10/31/2018 01:35 PM 609 checkstyle-header.txt | +| 10/31/2018 01:35 PM 245 checkstyle-result.xml | +| 11/20/2018 02:31 PM classes | +| 10/31/2018 01:35 PM maven-archiver | +| 10/31/2018 01:35 PM 66,296,012 policy-distribution-tarball-2.0.1-SNAPSHOT-tarball.tar.gz | +| 10/31/2018 01:35 PM 7,171 policy-distribution-tarball-2.0.1-SNAPSHOT.jar | +| 11/12/2018 10:56 AM test-classes | ++-------------------------------------------------------------------------------------------------------------------+ + +Install Distribution +-------------------- + + .. container:: paragraph + + Distribution can be installed in different ways: + + .. container:: ulist + + - Windows, Unix, Cygwin: manually from a ``.tar.gz`` archive + + - Windows, Unix, Cygwin: build from source using Maven, then + install manually + +Install Manually from Archive (Windows, 7Zip, GUI) +################################################## + + .. container:: paragraph + + Download a ``tar.gz`` archive and copy the file into the install + folder (in this example ``C:\distribution``). Assuming you are using 7Zip, + right click on the file and extract the ``tar`` archive. + +| + + .. container:: content + + Extract the TAR archive + + .. container:: paragraph + + The right-click on the new created TAR file and extract the actual + distribution. + +| + + .. container:: content + + Extract the distribution + + .. container:: paragraph + + Inside the new distribution folder you see the main directories: ``bin``, + ``etc``and ``lib`` + +| + + .. container:: paragraph + + Once extracted, please rename the created folder to + ``distribution-full-2.0.2-SNAPSHOT``. This will keep the directory name in + line with the rest of this documentation. + +Build from Source +----------------- + +Build and Install Manually (Unix, Windows, Cygwin) +################################################## + + .. container:: paragraph + + Clone the Distribution GIT repositories into a directory. Go to that + directory. Use Maven to build Distribution (all details on building + Distribution from source can be found in *Distribution HowTo: Build*). + + .. container:: paragraph + + Now, take the ``.tar.gz`` file and install distribution. + +Installation Layout +------------------- + + .. container:: paragraph + + A full installation of distribution comes with the following layout. + + .. container:: listingblock + + .. container:: content + + - bin + - etc + - lib + +Running Distribution in Docker +------------------------------ + +Run in ONAP +########### + + .. container:: paragraph + + Running distribution from the ONAP docker repository only requires 2 + commands: + + .. container:: olist arabic + + #. Log into the ONAP docker repo + + .. container:: listingblock + + .. container:: content + + :: + + docker login -u docker -p docker nexus3.onap.org:10003 + + .. container:: olist arabic + + #. Run the distribution docker image + + .. container:: listingblock + + .. container:: content + + :: + + docker run -it --rm nexus3.onap.org:10003/onap/policy-distribution:latest + +Build a Docker Image +#################### + + .. container:: paragraph + + Alternatively, one can use the Dockerfile defined in the Docker + package to build an image. + +Distribution Configurations Explained +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Introduction to Distribution Configuration +------------------------------------------ + + .. container:: paragraph + + A distribution engine can be configured to use various combinations + of policy reception handlers, policy decoders and policy forwarders. + The system is build using a plugin architecture. Each configuration + option is realized by a plugin, which can be loaded and + configured when the engine is started. New plugins can be + added to the system at any time, though to benefit from a + new plugin an engine will need to be restarted. + + | + + .. container:: paragraph + + The distribution already comes with sdc reception handler, + file reception handler, xacml policy decoder, file in csar policy decoder, + xacml policy forwarder, apex policy forwarder. + +General Configuration Format +---------------------------- + + .. container:: paragraph + + The distribution configuration file is a JSON file containing a few + main blocks for different parts of the configuration. Each + block then holds the configuration details. The following + code shows the main blocks: + + .. container:: listingblock + + .. container:: content + + .. code:: text + + { + "restServerParameters":{ + ... (1) + }, + "receptionHandlerParameters":{ (2) + "pluginHandlerParameters":{ (3) + "policyDecoders":{...}, (4) + "policyForwarders":{...} (5) + } + }, + "receptionHandlerConfigurationParameters":{ + ... (6) + } + , + "policyForwarderConfigurationParameters":{ + ... (7) + } + , + "policyDecoderConfigurationParameters":{ + ... (8) + } + } + + .. container:: colist arabic + + +-----------------------------------+-----------------------------------+ + | **1** | rest server configuration | + +-----------------------------------+-----------------------------------+ + | **2** | reception handler plugin | + | | configurations | + +-----------------------------------+-----------------------------------+ + | **3** | plugin handler parameters | + | | configuration | + +-----------------------------------+-----------------------------------+ + | **4** | policy decoder plugin | + | | configuration | + +-----------------------------------+-----------------------------------+ + | **5** | policy forwarder plugin | + | | configuration | + +-----------------------------------+-----------------------------------+ + | **6** | reception handler plugin | + | | parameters | + +-----------------------------------+-----------------------------------+ + | **7** | policy forwarder plugin | + | | parameters | + +-----------------------------------+-----------------------------------+ + | **8** | policy decoder plugin | + | | parameters | + +-----------------------------------+-----------------------------------+ + +A configuration example +----------------------- + + .. container:: paragraph + + The following example loads HPA use case related plug-ins. + + .. container:: paragraph + + Notifications are consumed from SDC through SDC client. + Consumed artifacts format is CSAR. + + .. container:: paragraph + + Generated policies are forwarded to XACML policy engine. + + .. container:: listingblock + + .. container:: content + + .. code:: text + + { + "name":"SDCDistributionGroup", + "restServerParameters":{ + "host":"0.0.0.0", + "port":6969, + "userName":"healthcheck", + "password":"zb!XztG34" + }, + "receptionHandlerParameters":{ + "SDCReceptionHandler":{ + "receptionHandlerType":"SDC", + "receptionHandlerClassName":"org.onap.policy.distribution.reception.handling.sdc.SdcReceptionHandler", + "receptionHandlerConfigurationName":"sdcConfiguration", + "pluginHandlerParameters":{ + "policyDecoders":{ + "CsarDecoder":{ + "decoderType":"CsarDecoder", + "decoderClassName":"org.onap.policy.distribution.reception.decoding.pdpx.PolicyDecoderCsarPdpx", + "decoderConfigurationName": "csarToOptimizationPolicyConfiguration" + } + }, + "policyForwarders":{ + "PAPEngineForwarder":{ + "forwarderType":"PAPEngine", + "forwarderClassName":"org.onap.policy.distribution.forwarding.xacml.pdp.XacmlPdpPolicyForwarder", + "forwarderConfigurationName": "xacmlPdpConfiguration" + } + } + } + } + }, + "receptionHandlerConfigurationParameters":{ + "sdcConfiguration":{ + "parameterClassName":"org.onap.policy.distribution.reception.handling.sdc.SdcReceptionHandlerConfigurationParameterGroup", + "parameters":{ + "asdcAddress": "sdc-be.onap:8443", + "messageBusAddress": [ + "message-router.onap" + ], + "user": "policy", + "password": "Kp8bJ4SXszM0WXlhak3eHlcse2gAw84vaoGGmJvUy2U", + "pollingInterval":20, + "pollingTimeout":30, + "consumerId": "policy-id", + "artifactTypes": [ + "TOSCA_CSAR", + "HEAT" + ], + "consumerGroup": "policy-group", + "environmentName": "AUTO", + "keystorePath": "null", + "keystorePassword": "null", + "activeserverTlsAuth": false, + "isFilterinEmptyResources": true, + "isUseHttpsWithDmaap": false + } + } + }, + "policyDecoderConfigurationParameters":{ + "csarToOptimizationPolicyConfiguration":{ + "parameterClassName":"org.onap.policy.distribution.reception.decoding.pdpx.PolicyDecoderCsarPdpxConfigurationParameterGroup", + "parameters":{ + "policyNamePrefix": "oofCasablanca", + "onapName": "OOF", + "version": "1.0", + "priority": "3", + "riskType": "Test", + "riskLevel": "2" + } + } + }, + "policyForwarderConfigurationParameters":{ + "xacmlPdpConfiguration":{ + "parameterClassName":"org.onap.policy.distribution.forwarding.xacml.pdp.XacmlPdpPolicyForwarderParameterGroup", + "parameters":{ + "useHttps": false, + "hostname": "pdp", + "port": 8081, + "userName": "testpdp", + "password": "alpha123", + "clientAuth": "cHl0aG9uOnRlc3Q=", + "isManaged": true, + "pdpGroup": "pdpGroup" + } + } + } + } + + +The Distribution Engine +^^^^^^^^^^^^^^^^^^^^^^^ + + .. container:: paragraph + + The Distribution engine can be started using ``policy-dist.sh`` script. + The script is located in the source code at + *distribution/packages/policy-distribution-docker/src/main/docker* + directory + + | + + .. container:: paragraph + + On UNIX and Cygwin systems use ``policy-dist.sh`` script. + + | + + .. container:: paragraph + + On Windows systems navigate to the distribution installation directory. + Run the following command + ``java -cp "etc:lib\*" org.onap.policy.distribution.main.startstop.Main -c `` + + | + + .. container:: paragraph + + The Distribution engine comes with CLI arguments for setting + configuration. The configuration file is always required. + The option ``-h`` prints a help screen. + + .. container:: listingblock + + .. container:: content + + .. code:: text + + usage: org.onap.policy.distribution.main.startstop.Main [options...] + options + -c,--config-file the full path to the configuration file to use, the configuration file must be a Json file + containing the distribution configuration parameters + -h,--help outputs the usage of this command + -v,--version outputs the version of distribution system + + +The Distribution REST End-points +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + + .. container:: paragraph + + The distribution engine comes with built-in REST based + endpoints for fetching health check status & statistical data + of running distribution system. + + .. container:: listingblock + + .. container:: content + + .. code:: text + + # Example Output from curl http -a '{user}:{password}' :6969/healthcheck + + HTTP/1.1 200 OK + Content-Length: XXX + Content-Type: application/json + Date: Tue, 17 Apr 2018 10:51:14 GMT + Server: Jetty(9.3.20.v20170531) + { + "code":200, + "healthy":true, + "message":"alive", + "name":"Policy SSD", + "url":"self" + } + + # Example Output from curl http -a '{user}:{password}' :6969/statistics + + HTTP/1.1 200 OK + Content-Length: XXX + Content-Type: application/json + Date: Tue, 17 Apr 2018 10:51:14 GMT + Server: Jetty(9.3.20.v20170531) + { + "code":200, + "distributions":10, + "distribution_complete_ok":8, + "distribution_complete_fail":2, + "downloads":15, + "downloads_ok"; 10, + "downloads_error": 5 + } + diff --git a/docs/index.rst b/docs/index.rst new file mode 100644 index 00000000..112548b7 --- /dev/null +++ b/docs/index.rst @@ -0,0 +1,9 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. + +POLICY DISTRIBUTION +------------------------------------------------ +.. toctree:: + :maxdepth: 1 + + Distribution-Introduction.rst + Distribution-User-Manual.rst \ No newline at end of file -- cgit 1.2.3-korg