diff options
| -rwxr-xr-x | docs/drools/ctrlog_config.png | bin | 7721 -> 0 bytes | |||
| -rwxr-xr-x | docs/drools/ctrlog_enablefeature.png | bin | 61614 -> 0 bytes | |||
| -rwxr-xr-x | docs/drools/ctrlog_logback.png | bin | 11467 -> 0 bytes | |||
| -rwxr-xr-x | docs/drools/ctrlog_view.png | bin | 12464 -> 0 bytes | |||
| -rw-r--r-- | docs/drools/drools.rst | 24 | ||||
| -rw-r--r-- | docs/drools/feature_activestdbymgmt.rst | 109 | ||||
| -rw-r--r-- | docs/drools/feature_controllerlogging.rst | 48 | ||||
| -rw-r--r-- | docs/drools/feature_eelf.rst | 47 | ||||
| -rw-r--r-- | docs/drools/feature_mdcfilters.rst | 117 | ||||
| -rw-r--r-- | docs/drools/feature_nolocking.rst | 11 | ||||
| -rw-r--r-- | docs/drools/feature_pooling.rst | 44 | ||||
| -rw-r--r-- | docs/drools/feature_sesspersist.rst | 49 | ||||
| -rw-r--r-- | docs/drools/feature_statemgmt.rst | 310 | ||||
| -rw-r--r-- | docs/drools/feature_testtransaction.rst | 26 | ||||
| -rw-r--r-- | docs/drools/mdc_enablefeature.png | bin | 27429 -> 0 bytes | |||
| -rwxr-xr-x | docs/drools/mdc_properties.png | bin | 24399 -> 0 bytes | |||
| -rw-r--r-- | docs/drools/pdpdApps.rst | 179 | ||||
| -rw-r--r-- | docs/drools/pdpdEngine.rst | 766 | ||||
| -rw-r--r-- | docs/drools/poolingDesign.png | bin | 61828 -> 0 bytes | |||
| -rw-r--r-- | docs/pap/pap.rst | 8 |
20 files changed, 421 insertions, 1317 deletions
diff --git a/docs/drools/ctrlog_config.png b/docs/drools/ctrlog_config.png Binary files differdeleted file mode 100755 index 8d5aeb65..00000000 --- a/docs/drools/ctrlog_config.png +++ /dev/null diff --git a/docs/drools/ctrlog_enablefeature.png b/docs/drools/ctrlog_enablefeature.png Binary files differdeleted file mode 100755 index dc1abf34..00000000 --- a/docs/drools/ctrlog_enablefeature.png +++ /dev/null diff --git a/docs/drools/ctrlog_logback.png b/docs/drools/ctrlog_logback.png Binary files differdeleted file mode 100755 index 252f3fe1..00000000 --- a/docs/drools/ctrlog_logback.png +++ /dev/null diff --git a/docs/drools/ctrlog_view.png b/docs/drools/ctrlog_view.png Binary files differdeleted file mode 100755 index 118bd64d..00000000 --- a/docs/drools/ctrlog_view.png +++ /dev/null diff --git a/docs/drools/drools.rst b/docs/drools/drools.rst index 1bcbda9a..447102df 100644 --- a/docs/drools/drools.rst +++ b/docs/drools/drools.rst @@ -9,7 +9,7 @@ Policy Drools PDP Engine :depth: 1 The Drools PDP, aka PDP-D, is the PDP in the Policy Framework that uses the -`Drools BRMS <https://www.drools.org/>`__ to enforce policies. +`Drools BRMS <https://www.drools.org/>`_ to enforce policies. The PDP-D functionality has been partitioned into two functional areas: @@ -18,8 +18,8 @@ The PDP-D functionality has been partitioned into two functional areas: **PDP-D Engine** -The PDP-D Engine is the infrastructure that *policy applications* use. -It provides networking services, resource grouping, and diagnostics. +The PDP-D Engine is the infrastructure that *policy applications* use. It provides networking +services, resource grouping, and diagnostics. The PDP-D Engine supports the following Tosca Native Policy Types: @@ -28,18 +28,16 @@ The PDP-D Engine supports the following Tosca Native Policy Types: These types are used to dynamically add and configure new application controllers. -The PDP-D Engine hosts applications by means of *controllers*. -*Controllers* may support other Tosca Policy Types. The -types supported by the *Control Loop* applications are: +The PDP-D Engine hosts applications by means of *controllers*. *Controllers* may support other +Tosca Policy Types. The types supported by the *Control Loop* applications are: - onap.policies.controlloop.operational.common.Drools **PDP-D Applications** -A PDP-D application, ie. a *controller*, contains references to the -resources that the application needs. These include networked endpoint references, -and maven coordinates. +A PDP-D application, ie. a *controller*, contains references to the resources that the application +needs. These include networked endpoint references, and maven coordinates. *Control Loop* applications are used in ONAP to enforce operational policies. @@ -52,3 +50,11 @@ The following guides offer more information in these two functional areas. pdpdEngine.rst pdpdApps.rst + +Additional information +====================== + +For additional information, please see the +`Drools PDP Development and Testing (In Depth) <https://wiki.onap.org/display/DW/2020-08+Frankfurt+Tutorials>`_ page. + +End of Document diff --git a/docs/drools/feature_activestdbymgmt.rst b/docs/drools/feature_activestdbymgmt.rst deleted file mode 100644 index 193c331f..00000000 --- a/docs/drools/feature_activestdbymgmt.rst +++ /dev/null @@ -1,109 +0,0 @@ - -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _feature-asm-label: - -********************************** -Feature: Active/Standby Management -********************************** - -.. contents:: - :depth: 3 - -When the Feature Session Persistence is enabled, there can only be one active/providing service Drools PDP due to the behavior of Drools persistence. The Active/Standby Management Feature controls the selection of the Drools PDP that is providing service. It utilizes its own database and the State Management Feature database in the election algorithm. All Drools PDP nodes periodically run the election algorithm and, since they all use the same data, all nodes come to the same conclusion with the "elected" node assuming an active/providingservice state. Thus, the algorithm is distributed and has no single point of failure - assuming the database is configured for high availability. - -When the algorithm selects a Drools PDP to be active/providing service the controllers and topic endpoints are unlocked and allowed to process transactions. When a Drools PDP transitions to a hotstandby or coldstandby state, the controllers and topic endpoints are locked, preventing the Drools PDP from handling transactions. - - -Enabling and Disabling Feature State Management -=============================================== - -The Active/Standby Management Feature is enabled from the command line when logged in as policy after configuring the feature properties file (see Description Details section). From the command line: - -- > features status - Lists the status of features -- > features enable active-standby-management - Enables the Active-Standby Management Feature -- > features disable active-standby-management - Disables the Active-Standby Management Feature - -The Drools PDP must be stopped prior to enabling/disabling features and then restarted after the features have been enabled/disabled. - - .. code-block:: bash - :caption: Enabling Active/Standby Management Feature - - policy@hyperion-4:/opt/app/policy$ policy stop - [drools-pdp-controllers] - L []: Stopping Policy Management... Policy Management (pid=354) is stopping... Policy Management has stopped. - policy@hyperion-4:/opt/app/policy$ features enable active-standby-management - name version status - ---- ------- ------ - controlloop-utils 1.1.0-SNAPSHOT disabled - healthcheck 1.1.0-SNAPSHOT disabled - test-transaction 1.1.0-SNAPSHOT disabled - eelf 1.1.0-SNAPSHOT disabled - state-management 1.1.0-SNAPSHOT disabled - active-standby-management 1.1.0-SNAPSHOT enabled - session-persistence 1.1.0-SNAPSHOT disabled - - -Description Details -~~~~~~~~~~~~~~~~~~~ - -Election Algorithm ------------------- - -The election algorithm selects the active/providingservice Drools PDP. The algorithm on each node reads the *standbystatus* from the *StateManagementEntity* table for all other nodes to determine if they are providingservice or in a hotstandby state and able to assume an active status. It uses the *DroolsPdpEntity* table to verify that other node election algorithms are currently functioning and when the other nodes were last designated as the active Drools PDP. - -In general terms, the election algorithm periodically gathers the standbystatus and designation status for all the Drools PDPs. If the node which is currently designated as providingservice is "current" in updating its status, no action is required. If the designated node is either not current or has a standbystatus other than providingservice, it is time to choose another designated *DroolsPDP*. The algorithm will build a list of all DroolsPDPs that are current and have a *standbystatus* of *hotstandby*. It will then give preference to DroolsPDPs within the same site, choosing the DroolsPDP with the lowest lexicographic value to the droolsPdpId (resourceName). If the chosen DroolsPDP is itself, it will promote its standbystatus from hotstandby to providingservice. If the chosen DroolsPDP is other than itself, it will do nothing. - -When the DroolsPDP promotes its *standbystatus* from hotstandby to providing service, a state change notification will occur and the Standby State Change Handler will take appropriate action. - - -Standby State Change Handler ----------------------------- - -The Standby State Change Handler (*PMStandbyStateChangeHandler* class) extends the IntegrityMonitor StateChangeNotifier class which implements the Observer class. When the DroolsPDP is constructed, an instance of the handler is constructed and registered with StateManagement. Whenever StateManagement implements a state transition, it calls the *handleStateChange()* method of the handler. If the StandbyStatus transitions to hot or cold standby, the handler makes a call into the lower level management layer to lock the application controllers and topic endpoints, preventing it from handling transactions. If the StandbyStatus transitions to providingservice, the handler makes a call into the lower level management layer to unlock the application controllers and topic endpoints, allowing it to handle transactions. - - -Database --------- - -The Active/Standby Feature creates a database named activestandbymanagement with a single table, **droolspdpentity**. The election handler uses that table to determine which DroolsPDP was/is designated as the active DroolsPDP and which DroolsPDP election handlers are healthy enough to periodically update their status. - -The **droolspdpentity** table has the following columns: - - **pdpId** - The unique indentifier for the DroolsPDP. It is the same as the resourceName - - **designated** - Has a value of 1 if the DroolsPDP is designated as active/providingservice. It has a value of 0 otherwise - - **priority** - Indicates the priority level of the DroolsPDP for the election handler. In general, this is ignore and all have the same priority. - - **updatedDate** - This is the timestamp for the most recent update of the record. - - **designatedDate** - This is the timestamp that indicates when the designated column was most recently set to a value of 1 - - **site** - This is the name of the site - -Properties ----------- - -The properties are found in the feature-active-standby-management.properties file. In general, the properties are adequately described in the properties file. Parameters which must be replaced prior to usage are indicated thus: ${{parameter to be replaced}} - - .. code-block:: bash - :caption: feature-active-standby-mangement.properties - - # DB properties - javax.persistence.jdbc.driver=org.mariadb.jdbc.Driver - javax.persistence.jdbc.url=jdbc:mariadb://${{SQL_HOST}}:3306/activestandbymanagement - javax.persistence.jdbc.user=${{SQL_USER}} - javax.persistence.jdbc.password=${{SQL_PASSWORD}} - - # Must be unique across the system - resource.name=pdp1 - # Name of the site in which this node is hosted - site_name=site1 - - # Needed by DroolsPdpsElectionHandler - pdp.checkInterval=1500 # The interval in ms between updates of the updatedDate - pdp.updateInterval=1000 # The interval in ms between executions of the election handler - #pdp.timeout=3000 - # Need long timeout, because testTransaction is only run every 10 seconds. - pdp.timeout=15000 - #how long do we wait for the pdp table to populate on initial startup - pdp.initialWait=20000 - - -End of Document diff --git a/docs/drools/feature_controllerlogging.rst b/docs/drools/feature_controllerlogging.rst deleted file mode 100644 index fc8d6dab..00000000 --- a/docs/drools/feature_controllerlogging.rst +++ /dev/null @@ -1,48 +0,0 @@ - -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _feature_controllerlogging-label: - -*************************** -Feature: Controller Logging -*************************** - -.. contents:: - :depth: 3 - -The controller logging feature provides a way to log network topic messages to a separate controller log file for each controller. This allows a clear separation of network traffic between all of the controllers. - -Type "features enable controller-logging". The feature will now display as "enabled". - - .. image:: ctrlog_enablefeature.png - -When the feature's enable script is executed, it will search the $POLICY_HOME/config directory for any logback files containing the prefix "logback-include-". These logger configuration files are typically provided with a feature that installs a controlloop (ex: controlloop-amsterdam and controlloop-casablanca features). Once these configuration files are found by the enable script, the logback.xml config file will be updated to include the configurations. - - .. image:: ctrlog_logback.png - - -Controller Logger Configuration -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -The contents of a logback-include-``*``.xml file follows the same configuration syntax as the logback.xml file. It will contain the configurations for the logger associated with the given controller. - - .. note:: A controller logger MUST be configured with the same name as the controller (ex: a controller named "casablanca" will have a logger named "casablanca"). - - .. image:: ctrlog_config.png - - -Viewing the Controller Logs -^^^^^^^^^^^^^^^^^^^^^^^^^^^ - -Once a logger for the controller is configured, start the drools-pdp and navigate to the $POLICY_LOGS directory. A new controller specific network log will be added that contains all the network topic traffic of the controller. - - .. image:: ctrlog_view.png - -The original network log remains and will append traffic information from all topics regardless of which controller it is for. To abbreviate and customize messages for the network log, refer to the -:ref:`Feature MDC Filters <feature_mdcfilters-label>` documentation. - - -End of Document - - diff --git a/docs/drools/feature_eelf.rst b/docs/drools/feature_eelf.rst deleted file mode 100644 index a505490c..00000000 --- a/docs/drools/feature_eelf.rst +++ /dev/null @@ -1,47 +0,0 @@ - -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -************************************************* -Feature: EELF (Event and Error Logging Framework) -************************************************* - -.. contents:: - :depth: 3 - -The EELF feature provides backwards compatibility with R0 logging functionality. It supports the use of EELF/Common Framework style logging at the same time as traditional logging. - -.. seealso:: Additional information for EELF logging can be found at `EELF wiki`_. - -.. _EELF wiki: https://github.com/att/EELF/wiki - - -To utilize the eelf logging capabilities, first stop policy engine and then enable the feature using the "*features*" command. - - .. code-block:: bash - :caption: Enabling EELF Feature - - policy@hyperion-4:/opt/app/policy$ policy stop - [drools-pdp-controllers] - L []: Stopping Policy Management... Policy Management (pid=354) is stopping... Policy Management has stopped. - policy@hyperion-4:/opt/app/policy$ features enable eelf - name version status - ---- ------- ------ - controlloop-utils 1.1.0-SNAPSHOT disabled - healthcheck 1.1.0-SNAPSHOT disabled - test-transaction 1.1.0-SNAPSHOT disabled - eelf 1.1.0-SNAPSHOT enabled - state-management 1.1.0-SNAPSHOT disabled - active-standby-management 1.1.0-SNAPSHOT disabled - session-persistence 1.1.0-SNAPSHOT disabled - -The output of the enable command will indicate whether or not the feature was enabled successfully. - -Policy engine can then be started as usual. - - - -End of Document - -.. SSNote: Wiki page ref. https://wiki.onap.org/display/DW/Feature+EELF - diff --git a/docs/drools/feature_mdcfilters.rst b/docs/drools/feature_mdcfilters.rst deleted file mode 100644 index b7077138..00000000 --- a/docs/drools/feature_mdcfilters.rst +++ /dev/null @@ -1,117 +0,0 @@ - -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _feature_mdcfilters-label: - -******************** -Feature: MDC Filters -******************** - -.. contents:: - :depth: 3 - -The MDC Filter Feature provides configurable properties for network topics to extract fields from JSON strings and place them in a mapped diagnostic context (MDC). - -Before enabling the feature, the network log contains the entire content of each message received on a topic. Below is a sample message from the network log. Note that the topic used for this tutorial is DCAE-CL. - - .. code-block:: bash - - [2019-03-22T16:36:42.942+00:00|DMAAP-source-DCAE-CL][IN|DMAAP|DCAE-CL] - {"closedLoopControlName":"ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e","closedLoopAlarmStart":1463679805324,"closedLoopEventClient":"DCAE_INSTANCE_ID.dcae-tca","closedLoopEventStatus":"ONSET","requestID":"664be3d2-6c12-4f4b-a3e7-c349acced200","target_type":"VNF","target":"generic-vnf.vnf-id","AAI":{"vserver.is-closed-loop-disabled":"false","vserver.prov-status":"ACTIVE","generic-vnf.vnf-id":"vCPE_Infrastructure_vGMUX_demo_app"},"from":"DCAE","version":"1.0.2"} - -The network log can become voluminous if messages received from various topics carry large messages for various controllers. With the MDC Filter Feature, users can define keywords in JSON messages to extract and structure according to a desired format. This is done through configuring the feature's properties. - -Configuring the MDC Filter Feature -================================== - -To configure the feature, the feature must be enabled using the following command: - - .. code-block:: bash - - features enable mdc-filters - - - .. image:: mdc_enablefeature.png - -Once the feature is enabled, there will be a new properties file in *$POLICY_HOME/config* called **feature-mdc-filters.properties**. - - .. image:: mdc_properties.png - -The properties file contains filters to extract key data from messages on the network topics that are saved in an MDC, which can be referenced in logback.xml. The configuration format is as follows: - - .. code-block:: bash - - <protocol>.<type>.topics.<topic-name>.mdcFilters=<filters> - - Where: - <protocol> = ueb, dmaap, noop - <type> = source, sink - <topic-name> = Name of DMaaP or UEB topic - <filters> = Comma separated list of key/json-path(s) - -The filters consist of an MDC key used by **logback.xml** (see below) and the JSON path(s) to the desired data. The path always begins with '$', which signifies the root of the JSON document. The underlying library, JsonPath, uses a query syntax for searching through a JSON file. The query syntax and some examples can be found at https://github.com/json-path/JsonPath. An example filter for the *DCAE-CL* is provided below: - - .. code-block:: bash - - dmaap.source.topics.DCAE-CL.mdcFilters=requestID=$.requestID - -This filter is specifying that the dmaap source topic *DCAE-CL* will search each message received for requestID by following the path starting at the root ($) and searching for the field *requestID*. If the field is found, it is placed in the MDC with the key "requestID" as signified by the left hand side of the filter before the "=". - - -Configuring Multiple Filters and Paths -====================================== - -Multiple fields can be found for a given JSON document by a comma separated list of <mdcKey,jsonPath> pairs. For the previous example, another filter is added by adding a comma and specifying the filter as follows: - - .. code-block:: bash - - dmaap.source.topics.DCAE-CL.mdcFilters=requestID=$.requestID,closedLoopName=$.closedLoopControlName - -The feature will now search for both requestID and closedLoopControlName in a JSON message using the specified "$." path notations and put them in the MDC using the keys "requestID" and "closedLoopName" respectively. To further refine the filter, if a topic receives different message structures (ex: a response message structure vs an error message structure) the "|" notation allows multiple paths to a key to be defined. The feature will search through each specified path until a match is found. An example can be found below: - - .. code-block:: bash - - dmaap.source.topics.DCAE-CL.mdcFilters=requestID=$.requestID,closedLoopName=$.closedLoopControlName|$.AAI.closedLoopControlName - -Now when the filter is searching for closedLoopControlName it will check the first path "$.closedLoopControlName", if it is not present then it will try the second path "$.AAI.closedLoopControlName". If the user is unsure of the path to a field, JsonPath supports a deep scan by using the ".." notation. This will search the entire JSON document for the field without specifying the path. - - -Accessing the MDC Values in logback.xml -======================================= - -Once the feature properties have been defined, logback.xml contains a "abstractNetworkPattern" property that will hold the desired message structure defined by the user. The user has the flexibility to define the message structure however they choose but for this tutorial the following pattern is used: - - .. code-block:: bash - - <property name="abstractNetworkPattern" value="[%d{yyyy-MM-dd'T'HH:mm:ss.SSS+00:00, UTC}] [%X{networkEventType:-NULL}|%X{networkProtocol:-NULL}|%X{networkTopic:-NULL}|%X{requestID:-NULL}|%X{closedLoopName:-NULL}]%n" /> - -The "value" portion consists of two headers in bracket notation, the first header defines the timestamp while the second header references the keys from the MDC filters defined in the feature properties. The standard logback syntax is used and more information on the syntax can be found here. Note that some of the fields here were not defined in the feature properties file. The feature automatically puts the network infrastructure information in the keys that are prepended with "network". The current supported network infrastructure information is listed below. - - +-------------------+-------------------------------------------------+ - | Field | Values | - +===================+=================================================+ - | networkEventType | IN, OUT | - +-------------------+-------------------------------------------------+ - | networkProtocol | DMAAP, UEB, NOOP | - +-------------------+-------------------------------------------------+ - | networkTopic | The name of the topic that received the message | - +-------------------+-------------------------------------------------+ - - -To reference the keys from the feature properties the syntax "%X{KEY_DEFINED_IN_PROPERTIES}" provides access to the value. An optional addition is to append ":-", which specifies a default value to display in the log if the field was not found in the message received. For this tutorial, a default of "NULL" is displayed for any of the fields that were not found while filtering. The "|" has no special meaning and is just used as a field separator for readability; the user can decorate the log format to their desired visual appeal. - -Network Log Structure After Feature Enabled -=========================================== - -Once the feature and logback.xml is configured to the user's desired settings, start the PDP-D by running "policy start". Based on the configurations from the previous sections of this tutorial, the following log message is written to network log when a message is received on the DCAE-CL topic: - - .. code-block:: bash - - [2019-03-22T16:38:23.884+00:00] [IN|DMAAP|DCAE-CL|664be3d2-6c12-4f4b-a3e7-c349acced200|ControlLoop-vCPE-48f0c2c3-a172-4192-9ae3-052274181b6e] - -The message has now been filtered to display the network infrastructure information and the extracted data from the JSON message based on the feature properties. In order to view the entire message received from a topic, a complementary feature was developed to display the entire message on a per controller basis while preserving the compact network log. Refer to the -:ref:`Feature Controller Logging <feature_controllerlogging-label>` documentation for details. - -End of Document - diff --git a/docs/drools/feature_nolocking.rst b/docs/drools/feature_nolocking.rst index e98cc8ee..12c6570a 100644 --- a/docs/drools/feature_nolocking.rst +++ b/docs/drools/feature_nolocking.rst @@ -9,11 +9,11 @@ Feature: no locking .. contents:: :depth: 3 -The no-locking feature allows applications to use a Lock Manager that always succeeds. It does not deny -acquiring resource locks. +The no-locking feature allows applications to use a Lock Manager that always succeeds. It does not +deny acquiring resource locks. -To utilize the no-locking feature, first stop policy engine, disable other locking features, and then enable it -using the "*features*" command. +To utilize the no-locking feature, first stop policy engine, disable other locking features, and +then enable it using the "*features*" command. In an official OOM installation, place a script with a .pre.sh suffix: @@ -35,6 +35,7 @@ under the directory: and rebuild the policy charts. -At container initialization, the distributed-locking will be disabled, and the no-locking feature will be enabled. +At container initialization, the distributed-locking will be disabled, and the no-locking feature +will be enabled. End of Document diff --git a/docs/drools/feature_pooling.rst b/docs/drools/feature_pooling.rst index ba950a3d..705c98e3 100644 --- a/docs/drools/feature_pooling.rst +++ b/docs/drools/feature_pooling.rst @@ -8,7 +8,9 @@ Feature: Pooling **************** -The Pooling feature provides the ability to load-balance work across a “pool” of active-active Drools-PDP hosts. This particular implementation uses a DMaaP topic for communication between the hosts within the pool. +The Pooling feature provides the ability to load-balance work across a “pool” of active-active +Drools-PDP hosts. This particular implementation uses a kafka topic for communication between the +hosts within the pool. The pool is adjusted automatically, with no manual intervention when: * a new host is brought online @@ -18,35 +20,36 @@ Assumptions and Limitations =========================== * Session persistence is not required * Data may be lost when processing is moved from one host to another - * The entire pool may shut down if the inter-host DMaaP topic becomes inaccessible - - .. image:: poolingDesign.png + * The entire pool may shut down if the kafka topic becomes inaccessible Key Points ========== - * Requests are received on a common DMaaP topic - - DMaaP distributes the requests randomly to the hosts - - The request topic should have at least as many partitions as there are hosts - * Uses a single, internal DMaaP topic for all inter-host communication + * Requests are received on a common kafka topic + * Uses a single, kafka topic for all inter-host communication * Allocates buckets to each host - Requests are assigned to buckets based on their respective “request IDs” * No session persistence * No objects copied between hosts * Requires feature(s): distributed-locking - * Precludes feature(s): session-persistence, active-standby, state-management Example Scenario ================ - 1. Incoming DMaaP message is received on a topic — all hosts are listening, but only one random host receives the message + 1. Incoming message is received on a topic — all hosts are listening, but only one random host + receives the message 2. Decode message to determine “request ID” key (message-specific operation) 3. Hash request ID to determine the bucket number 4. Look up host associated with hash bucket (most likely remote) - 5. Publish “forward” message to internal DMaaP topic, including remote host, bucket number, DMaaP topic information, and message body - 6. Remote host verifies ownership of bucket, and routes the DMaaP message to its own rule engine for processing + 5. Publish “forward” message to internal topic, including remote host, bucket number, topic + information, and message body + 6. Remote host verifies ownership of bucket, and routes the message to its own rule engine for + processing - The figure below shows several different hosts in a pool. Each host has a copy of the bucket assignments, which specifies which buckets are assigned to which hosts. Incoming requests are mapped to a bucket, and a bucket is mapped to a host, to which the request is routed. The host table includes an entry for each active host in the pool, to which one or more buckets are mapped. +The figure below shows several different hosts in a pool. Each host has a copy of the bucket +assignments, which specifies which buckets are assigned to which hosts. Incoming requests are mapped +to a bucket, and a bucket is mapped to a host, to which the request is routed. The host table +includes an entry for each active host in the pool, to which one or more buckets are mapped. .. image:: poolingPdps.png @@ -58,7 +61,12 @@ Bucket Reassignment * Leaves buckets with their current owner, where possible * Takes a few buckets from each host to assign to new hosts - For example, in the diagram below, the left side shows how 32 buckets might be assigned among four different hosts. When the first host fails, the buckets from host 1 would be reassigned among the remaining hosts, similar to what is shown on the right side of the diagram. Any requests that were being processed by host 1 will be lost and must be restarted. However, the buckets that had already been assigned to the remaining hosts are unchanged, thus requests associated with those buckets are not impacted by the loss of host 1. +For example, in the diagram below, the left side shows how 32 buckets might be assigned among four +different hosts. When the first host fails, the buckets from host 1 would be reassigned among the +remaining hosts, similar to what is shown on the right side of the diagram. Any requests that were +being processed by host 1 will be lost and must be restarted. However, the buckets that had already +been assigned to the remaining hosts are unchanged, thus requests associated with those buckets are +not impacted by the loss of host 1. .. image:: poolingBuckets.png @@ -73,11 +81,11 @@ For pooling to be enabled, the distributed-locking feature must be also be enabl policy stop features enable distributed-locking - features enable pooling-dmaap + features enable pooling-messages The configuration is located at: - * $POLICY_HOME/config/feature-pooling-dmaap.properties + * $POLICY_HOME/config/feature-pooling-messages.properties .. code-block:: bash @@ -90,12 +98,10 @@ For pooling to be enabled, the distributed-locking feature must be also be enabl :caption: Disable the pooling feature policy stop - features disable pooling-dmaap + features disable pooling-messages policy start End of Document .. SSNote: Wiki page ref. https://wiki.onap.org/display/DW/Feature+Pooling - - diff --git a/docs/drools/feature_sesspersist.rst b/docs/drools/feature_sesspersist.rst deleted file mode 100644 index 4bb5ef62..00000000 --- a/docs/drools/feature_sesspersist.rst +++ /dev/null @@ -1,49 +0,0 @@ - -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -************************************ -Feature: Session Persistence -************************************ - -The session persistence feature allows drools kie sessions to be persisted in a database surviving pdp-d restarts. - - .. code-block:: bash - :caption: Enable session persistence - :linenos: - - policy stop - features enable session-persistence - -The configuration is located at: - - - *$POLICY_HOME/config/feature-session-persistence.properties* - -Each controller that wants to be started with persistence should contain the following line in its *<controller-name>-controller.properties* - - - *persistence.type=auto* - - .. code-block:: bash - :caption: Start the PDP-D using session-persistence - :linenos: - - db-migrator -o upgrade -s ALL - policy start - -Facts will survive PDP-D restart using the native drools capabilities and introduce a performance overhead. - - .. code-block:: bash - :caption: Disable the session-persistence feature - :linenos: - - policy stop - features disable session-persistence - sed -i "/persistence.type=auto/d" <controller-name>-controller.properties - db-migrator -o erase -s sessionpersistence # delete all its database data (optional) - policy start - -End of Document - -.. SSNote: Wiki page ref. https://wiki.onap.org/display/DW/Feature+Session+Persistence - - diff --git a/docs/drools/feature_statemgmt.rst b/docs/drools/feature_statemgmt.rst deleted file mode 100644 index 29497003..00000000 --- a/docs/drools/feature_statemgmt.rst +++ /dev/null @@ -1,310 +0,0 @@ - -.. This work is licensed under a Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _feature-sm-label: - -************************* -Feature: State Management -************************* - -.. contents:: - :depth: 2 - -The State Management Feature provides: - -- Node-level health monitoring -- Monitoring the health of dependency nodes - nodes on which a particular node is dependent -- Ability to lock/unlock a node and suspend or resume all application processing -- Ability to suspend application processing on a node that is disabled or in a standby state -- Interworking/Coordination of state values -- Support for ITU X.731 states and state transitions for: - - Administrative State - - Operational State - - Availability Status - - Standby Status - - -Enabling and Disabling Feature State Management -=============================================== - -The State Management Feature is enabled from the command line when logged in as policy after configuring the feature properties file (see Description Details section). From the command line: - -- > features status - Lists the status of features -- > features enable state-management - Enables the State Management Feature -- > features disable state-management - Disables the State Management Feature - -The Drools PDP must be stopped prior to enabling/disabling features and then restarted after the features have been enabled/disabled. - - .. code-block:: bash - :caption: Enabling State Management Feature - - policy@hyperion-4:/opt/app/policy$ policy stop - [drools-pdp-controllers] - L []: Stopping Policy Management... Policy Management (pid=354) is stopping... Policy Management has stopped. - policy@hyperion-4:/opt/app/policy$ features enable state-management - name version status - ---- ------- ------ - controlloop-utils 1.1.0-SNAPSHOT disabled - healthcheck 1.1.0-SNAPSHOT disabled - test-transaction 1.1.0-SNAPSHOT disabled - eelf 1.1.0-SNAPSHOT disabled - state-management 1.1.0-SNAPSHOT enabled - active-standby-management 1.1.0-SNAPSHOT disabled - session-persistence 1.1.0-SNAPSHOT disabled - -Description Details -~~~~~~~~~~~~~~~~~~~ - -State Model -""""""""""" - -The state model follows the ITU X.731 standard for state management. The supported state values are: - **Administrative State:** - - Locked - All application transaction processing is prohibited - - Unlocked - Application transaction processing is allowed - - **Administrative State Transitions:** - - The transition from Unlocked to Locked state is triggered with a Lock operation - - The transition from the Locked to Unlocked state is triggered with an Unlock operation - - **Operational State:** - - Enabled - The node is healthy and able to process application transactions - - Disabled - The node is not healthy and not able to process application transactions - - **Operational State Transitions:** - - The transition from Enabled to Disabled is triggered with a disableFailed or disableDependency operation - - The transition from Disabled to Enabled is triggered with an enableNotFailed and enableNoDependency operation - - **Availability Status:** - - Null - The Operational State is Enabled - - Failed - The Operational State is Disabled because the node is no longer healthy - - Dependency - The Operational State is Disabled because all members of a dependency group are disabled - - Dependency.Failed - The Operational State is Disabled because the node is no longer healthy and all members of a dependency group are disabled - - **Availability Status Transitions:** - - The transition from Null to Failed is triggered with a disableFailed operation - - The transtion from Null to Dependency is triggered with a disableDependency operation - - The transition from Failed to Dependency.Failed is triggered with a disableDependency operation - - The transition from Dependency to Dependency.Failed is triggered with a disableFailed operation - - The transition from Dependency.Failed to Failed is triggered with an enableNoDependency operation - - The transition from Dependency.Failed to Dependency is triggered with an enableNotFailed operation - - The transition from Failed to Null is triggered with an enableNotFailed operation - - The transition from Dependency to Null is triggered with an enableNoDependency operation - - **Standby Status:** - - Null - The node does not support active-standby behavior - - ProvidingService - The node is actively providing application transaction service - - HotStandby - The node is capable of providing application transaction service, but is currently waiting to be promoted - - ColdStandby - The node is not capable of providing application service because of a failure - - **Standby Status Transitions:** - - The transition from Null to HotStandby is triggered by a demote operation when the Operational State is Enabled - - The transition for Null to ColdStandby is triggered is a demote operation when the Operational State is Disabled - - The transition from ColdStandby to HotStandby is triggered by a transition of the Operational State from Disabled to Enabled - - The transition from HotStandby to ColdStandby is triggered by a transition of the Operational State from Enabled to Disabled - - The transition from ProvidingService to ColdStandby is triggered by a transition of the Operational State from Enabled to Disabled - - The transition from HotStandby to ProvidingService is triggered by a Promote operation - - The transition from ProvidingService to HotStandby is triggered by a Demote operation - -Database -~~~~~~~~ - -The State Management feature creates a StateManagement database having three tables: - - **StateManagementEntity** - This table has the following columns: - - **id** - Automatically created unique identifier - - **resourceName** - The unique identifier for a node - - **adminState** - The Administrative State - - **opState** - The Operational State - - **availStatus** - The Availability Status - - **standbyStatus** - The Standby Status - - **created_Date** - The timestamp the resource entry was created - - **modifiedDate** - The timestamp the resource entry was last modified - - **ForwardProgressEntity** - This table has the following columns: - - **forwardProgressId** - Automatically created unique identifier - - **resourceName** - The unique identifier for a node - - **fpc_count** - A forward progress counter which is periodically incremented if the node is healthy - - **created_date** - The timestamp the resource entry was created - - **last_updated** - The timestamp the resource entry was last updated - - **ResourceRegistrationEntity** - This table has the following columns: - - **ResourceRegistrationId** - Automatically created unique identifier - - **resourceName** - The unique identifier for a node - - **resourceUrl** - The JMX URL used to check the health of a node - - **site** - The name of the site in which the resource resides - - **nodeType** - The type of the node (i.e, pdp_xacml, pdp_drools, pap, pap_admin, logparser, brms_gateway, astra_gateway, elk_server, pypdp) - - **created_date** - The timestamp the resource entry was created - - **last_updated** - The timestamp the resource entry was last updated - -Node Health Monitoring -~~~~~~~~~~~~~~~~~~~~~~ - -**Application Monitoring** - - Application monitoring can be implemented using the *startTransaction()* and *endTransaction()* methods. Whenever a transaction is started, the *startTransaction()* method is called. If the node is locked, disabled or in a hot/cold standby state, the method will throw an exception. Otherwise, it resets the timer which triggers the default *testTransaction()* method. - - When a transaction completes, calling *endTransaction()* increments the forward process counter in the *ForwardProgressEntity* DB table. As long as this counter is updating, the integrity monitor will assume the node is healthy/sane. - - If the *startTransaction()* method is not called within a provisioned period of time, a timer will expire which calls the *testTransaction()* method. The default implementation of this method simply increments the forward progress counter. The *testTransaction()* method may be overwritten to perform a more meaningful test of system sanity, if desired. - - If the forward progress counter stops incrementing, the integrity monitoring routine will assume the node application has lost sanity and it will trigger a *statechange* (disableFailed) to cause the operational state to become disabled and the availability status attribute to become failed. Once the forward progress counter again begins incrementing, the operational state will return to enabled. - -**Application Monitoring with AllSeemsWell** - - The IntegrityMonitor class provides a facility for applications to directly control updates of the forwardprogressentity table. As previously described, *startTransaction()* and *endTransaction()* are provided to monitor the forward progress of transactions. This, however, does not monitor things such as internal threads that may be blocked or die. An example is the feature-state-management *DroolsPdpElectionHandler.run()* method. - - The *run()* method is monitored by a timer task, *checkWaitTimer()*. If the *run()* method is stalled an extended period of time, the *checkWaitTimer()* method will call *StateManagementFeature.allSeemsWell(<className>, <AllSeemsWell State>, <String message>)* with the AllSeemsWell state of Boolean.FALSE. - - The IntegrityMonitor instance owned by StateManagementFeature will then store an entry in the allSeemsWellMap and block updates of the forwardprogressentity table. This in turn, will cause the Drools PDP operational state to be set to “disabled” and availability status to be set to “failed”. - - Once the blocking condition is cleared, the *checkWaiTimer()* will again call the *allSeemsWell()* method and include an AllSeemsWell state of Boolean.True. This will cause the IntegrityMonitor to remove the entry for that className from the allSeemsWellMap and allow updating of the forwardprogressentity table, so long as there are no other entries in the map. - -**Dependency Monitoring** - - When a Drools PDP (or other node using the *IntegrityMonitor* policy/common module) is dependent upon other nodes to perform its function, those other nodes can be defined as dependencies in the properties file. In order for the dependency algorithm to function, the other nodes must also be running the *IntegrityMonitor*. Periodically the Drools PDP will check the state of dependencies. If all of a node type have failed, the Drools PDP will declare that it can no longer function and change the operational state to disabled and the availability status to dependency. - - In addition to other policy node types, there is a *subsystemTest()* method that is periodically called by the *IntegrityMonitor*. In Drools PDP, *subsystemTest* has been overwritten to execute an audit of the Database and of the Maven Repository. If the audit is unable to verify the function of either the DB or the Maven Repository, he Drools PDP will declare that it can no longer function and change the operational state to disabled and the availability status to dependency. - - When a failed dependency returns to normal operation, the *IntegrityMontor* will change the operational state to enabled and availability status to null. - -**External Health Monitoring Interface** - - The Drools PDP has a http test interface which, when called, will return 200 if all seems well and 500 otherwise. The test interface URL is defined in the properties file. - - -Site Manager -~~~~~~~~~~~~ - -The Site Manager is not deployed with the Drools PDP, but it is available in the policy/common repository in the site-manager directory. -The Site Manager provides a lock/unlock interface for nodes and a way to display node information and status. - -The following is from the README file included with the Site Manager. - - .. code-block:: bash - :caption: Site Manager README extract - - Before using 'siteManager', the file 'siteManager.properties' needs to be - edited to configure the parameters used to access the database: - - javax.persistence.jdbc.driver - typically 'org.mariadb.jdbc.Driver' - - javax.persistence.jdbc.url - URL referring to the database, - which typically has the form: 'jdbc:mariadb://<host>:<port>/<db>' - ('<db>' is probably 'xacml' in this case) - - javax.persistence.jdbc.user - the user id for accessing the database - - javax.persistence.jdbc.password - password for accessing the database - - Once the properties file has been updated, the 'siteManager' script can be - invoked as follows: - - siteManager show [ -s <site> | -r <resourceName> ] : - display node information (Site, NodeType, ResourceName, AdminState, - OpState, AvailStatus, StandbyStatus) - - siteManager setAdminState { -s <site> | -r <resourceName> } <new-state> : - update admin state on selected nodes - - siteManager lock { -s <site> | -r <resourceName> } : - lock selected nodes - - siteManager unlock { -s <site> | -r <resourceName> } : - unlock selected nodes - -Note that the 'siteManager' script assumes that the script, -'site-manager-${project.version}.jar' file and 'siteManager.properties' file -are all in the same directory. If the files are separated, the 'siteManager' -script will need to be modified so it can locate the jar and properties files. - - -Properties -~~~~~~~~~~ - -The feature-state-mangement.properties file controls the function of the State Management Feature. In general, the properties have adequate descriptions in the file. Parameters which must be replaced prior to usage are indicated thus: ${{parameter to be replaced}}. - - .. code-block:: bash - :caption: feature-state-mangement.properties - - # DB properties - javax.persistence.jdbc.driver=org.mariadb.jdbc.Driver - javax.persistence.jdbc.url=jdbc:mariadb://${{SQL_HOST}}:3306/statemanagement - javax.persistence.jdbc.user=${{SQL_USER}} - javax.persistence.jdbc.password=${{SQL_PASSWORD}} - - # DroolsPDPIntegrityMonitor Properties - # Test interface host and port defaults may be overwritten here - http.server.services.TEST.host=0.0.0.0 - http.server.services.TEST.port=9981 - #These properties will default to the following if no other values are provided: - # http.server.services.TEST.restClasses=org.onap.policy.drools.statemanagement.IntegrityMonitorRestManager - # http.server.services.TEST.managed=false - # http.server.services.TEST.swagger=true - - #IntegrityMonitor Properties - - # Must be unique across the system - resource.name=pdp1 - # Name of the site in which this node is hosted - site_name=site1 - # Forward Progress Monitor update interval seconds - fp_monitor_interval=30 - # Failed counter threshold before failover - failed_counter_threshold=3 - # Interval between test transactions when no traffic seconds - test_trans_interval=10 - # Interval between writes of the FPC to the DB seconds - write_fpc_interval=5 - # Node type Note: Make sure you don't leave any trailing spaces, or you'll get an 'invalid node type' error! - node_type=pdp_drools - # Dependency groups are groups of resources upon which a node operational state is dependent upon. - # Each group is a comma-separated list of resource names and groups are separated by a semicolon. For example: - # dependency_groups=site_1.astra_1,site_1.astra_2;site_1.brms_1,site_1.brms_2;site_1.logparser_1;site_1.pypdp_1 - dependency_groups= - # When set to true, dependent health checks are performed by using JMX to invoke test() on the dependent. - # The default false is to use state checks for health. - test_via_jmx=true - # This is the max number of seconds beyond which a non incrementing FPC is considered a failure - max_fpc_update_interval=120 - # Run the state audit every 60 seconds (60000 ms). The state audit finds stale DB entries in the - # forwardprogressentity table and marks the node as disabled/failed in the statemanagemententity - # table. NOTE! It will only run on nodes that have a standbystatus = providingservice. - # A value of <= 0 will turn off the state audit. - state_audit_interval_ms=60000 - # The refresh state audit is run every (default) 10 minutes (600000 ms) to clean up any state corruption in the - # DB statemanagemententity table. It only refreshes the DB state entry for the local node. That is, it does not - # refresh the state of any other nodes. A value <= 0 will turn the audit off. Any other value will override - # the default of 600000 ms. - refresh_state_audit_interval_ms=600000 - - # Repository audit properties - # Assume it's the releaseRepository that needs to be audited, - # because that's the one BRMGW will publish to. - repository.audit.id=${{releaseRepositoryID}} - repository.audit.url=${{releaseRepositoryUrl}} - repository.audit.username=${{repositoryUsername}} - repository.audit.password=${{repositoryPassword}} - repository2.audit.id=${{releaseRepository2ID}} - repository2.audit.url=${{releaseRepository2Url}} - repository2.audit.username=${{repositoryUsername2}} - repository2.audit.password=${{repositoryPassword2}} - - # Repository Audit Properties - # Flag to control the execution of the subsystemTest for the Nexus Maven repository - repository.audit.is.active=false - repository.audit.ignore.errors=true - repository.audit.interval_sec=86400 - repository.audit.failure.threshold=3 - - # DB Audit Properties - # Flag to control the execution of the subsystemTest for the Database - db.audit.is.active=false - - -End of Document - -.. SSNote: Wiki page ref. https://wiki.onap.org/display/DW/Feature+State+Management - - diff --git a/docs/drools/feature_testtransaction.rst b/docs/drools/feature_testtransaction.rst index 8bec1421..8e99f0b6 100644 --- a/docs/drools/feature_testtransaction.rst +++ b/docs/drools/feature_testtransaction.rst @@ -11,15 +11,24 @@ Feature: Test Transaction .. contents:: :depth: 3 -The Test Transaction feature provides a mechanism by which the health of drools policy controllers can be tested. +The Test Transaction feature provides a mechanism by which the health of drools policy controllers +can be tested. -When enabled, the feature functions by injecting an event object (identified by a UUID) into the drools session of each policy controller that is active in the system. Only an object with this UUID can trigger the Test Transaction-specific drools logic to execute. +When enabled, the feature functions by injecting an event object (identified by a UUID) into the +drools session of each policy controller that is active in the system. Only an object with this UUID +can trigger the Test Transaction-specific drools logic to execute. -The injection of the event triggers the "TT" rule (see *TestTransactionTemplate.drl* below) to fire. The "TT" rule simply increments a ForwardProgress counter object, thereby confirming that the drools session for this particular controller is active and firing its rules accordingly. This cycle repeats at 20 second intervals. +The injection of the event triggers the "TT" rule (see *TestTransactionTemplate.drl* below) to fire. +The "TT" rule simply increments a ForwardProgress counter object, thereby confirming that the drools +session for this particular controller is active and firing its rules accordingly. This cycle +repeats at 20 second intervals. -If it is ever the case that a drools controller does not have the "TT" rule present in its *.drl*, or that the forward progress counter is not incremented, the Test Transaction thread for that particular drools session (i.e. controller) is terminated and a message is logged to *error.log*. +If it is ever the case that a drools controller does not have the "TT" rule present in its *.drl*, +or that the forward progress counter is not incremented, the Test Transaction thread for that +particular drools session (i.e. controller) is terminated and a message is logged to *error.log*. -Prior to being enabled, the following drools rules need to be appended to the rules templates of any use-case that is to be monitored by the feature. +Prior to being enabled, the following drools rules need to be appended to the rules templates of any +use-case that is to be monitored by the feature. .. code-block:: java :caption: TestTransactionTemplate.drl @@ -73,7 +82,8 @@ Prior to being enabled, the following drools rules need to be appended to the ru ForwardProgress(counter >= 0, $ttc : counter) end -Once the proper artifacts are built and deployed with the addition of the TestTransactionTemplate rules, the feature can then be enabled by entering the following commands: +Once the proper artifacts are built and deployed with the addition of the TestTransactionTemplate +rules, the feature can then be enabled by entering the following commands: .. code-block:: bash :caption: PDPD Features Command @@ -87,10 +97,6 @@ Once the proper artifacts are built and deployed with the addition of the TestTr controlloop-utils 1.1.0-SNAPSHOT disabled healthcheck 1.1.0-SNAPSHOT disabled test-transaction 1.1.0-SNAPSHOT enabled - eelf 1.1.0-SNAPSHOT disabled - state-management 1.1.0-SNAPSHOT disabled - active-standby-management 1.1.0-SNAPSHOT disabled - session-persistence 1.1.0-SNAPSHOT disabled The output of the enable command will indicate whether or not the feature was enabled successfully. diff --git a/docs/drools/mdc_enablefeature.png b/docs/drools/mdc_enablefeature.png Binary files differdeleted file mode 100644 index 26ae55a4..00000000 --- a/docs/drools/mdc_enablefeature.png +++ /dev/null diff --git a/docs/drools/mdc_properties.png b/docs/drools/mdc_properties.png Binary files differdeleted file mode 100755 index 63cea92e..00000000 --- a/docs/drools/mdc_properties.png +++ /dev/null diff --git a/docs/drools/pdpdApps.rst b/docs/drools/pdpdApps.rst index 6dceee5f..abcf2e69 100644 --- a/docs/drools/pdpdApps.rst +++ b/docs/drools/pdpdApps.rst @@ -28,47 +28,48 @@ Software Source Code repositories ~~~~~~~~~~~~~~~~~~~~~~~~ -The PDP-D Applications software resides on the `policy/drools-applications <https://git.onap.org/policy/drools-applications>`__ repository. The actor libraries introduced in the *frankfurt* release reside in -the `policy/models repository <https://git.onap.org/policy/models>`__. +The PDP-D Applications software resides on the +`policy/drools-applications <https://git.onap.org/policy/drools-applications>`_ repository. +The actor libraries introduced in the *Frankfurt* release reside in the +`policy/models repository <https://git.onap.org/policy/models>`_. At this time, the *control loop* application is the only application supported in ONAP. All the application projects reside under the -`controlloop directory <https://git.onap.org/policy/drools-applications/tree/controlloop>`__. +`controlloop directory <https://git.onap.org/policy/drools-applications/tree/controlloop>`_. Docker Image ~~~~~~~~~~~~ -See the *drools-applications* -`released versions <https://wiki.onap.org/display/DW/Policy+Framework+Project%3A+Component+Versions>`__ -for the latest images: +Check the *drools-applications* `released versions <https://github.com/onap/policy-parent/tree/master/integration/src/main/resources/release>`_ +page for the latest versions. At the time of this writing *3.0.1* is the latest version. .. code-block:: bash - docker pull onap/policy-pdpd-cl:3.0.0 + docker pull nexus3.onap.org:10001/onap/policy-pdpd-cl:3.0.1 -At the time of this writing *3.0.0* is the latest version. +At the time of this writing *3.0.1* is the latest version. -The *onap/policy-pdpd-cl* image extends the *onap/policy-drools* image with -the *usecases* controller that realizes the *control loop* application. +The *onap/policy-pdpd-cl* image extends the *onap/policy-drools* image with the *usecases* +controller that realizes the *control loop* application. Usecases Controller =================== -The `usecases <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases>`__ +The `usecases <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases>`_ controller is the *control loop* application in ONAP. There are three parts in this controller: -* The `drl rules <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/usecases.drl>`__. -* The `kmodule.xml <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/META-INF/kmodule.xml>`__. -* The `dependencies <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/pom.xml>`__. +* The `drl rules <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/usecases.drl>`_. +* The `kmodule.xml <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/META-INF/kmodule.xml>`_. +* The `dependencies <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/pom.xml>`_. -The `kmodule.xml` specifies only one session, and declares in the *kbase* section the two operational policy types that -it supports. +The `kmodule.xml` specifies only one session, and declares in the *kbase* section the two +operational policy types that it supports. -The Usecases controller relies on the new Actor framework to interact with remote -components, part of a control loop transaction. The reader is referred to the -*Policy Platform Actor Development Guidelines* in the documentation for further information. +The Usecases controller relies on the new Actor framework to interact with remote components, part +of a control loop transaction. The reader is referred to the *Policy Platform Actor Development +Guidelines* in the documentation for further information. Operational Policy Types ======================== @@ -77,19 +78,20 @@ The *usecases* controller supports the following policy type: - *onap.policies.controlloop.operational.common.Drools*. -The *onap.policies.controlloop.operational.common.Drools* -is the Tosca compliant policy type introduced in *frankfurt*. +The *onap.policies.controlloop.operational.common.Drools* is the Tosca compliant policy type +introduced in *Frankfurt*. The Tosca Compliant Operational Policy Type is defined at the -`onap.policies.controlloop.operational.common.Drools <https://git.onap.org/policy/models/tree/models-examples/src/main/resources/policytypes/onap.policies.controlloop.operational.common.Drools.yaml>`__. +`onap.policies.controlloop.operational.common.Drools <https://git.onap.org/policy/models/tree/models-examples/src/main/resources/policytypes/onap.policies.controlloop.operational.common.Drools.yaml>`_. -An example of a Tosca Compliant Operational Policy can be found -`here <https://git.onap.org/policy/models/tree/models-examples/src/main/resources/policies/vDNS.policy.operational.input.tosca.json>`__. +An example of a Tosca Compliant Operational Policy: +`vDNS <https://git.onap.org/policy/models/tree/models-examples/src/main/resources/policies/vDNS.policy.operational.input.tosca.json>`_. Policy Chaining =============== -The *usecases* controller supports chaining of multiple operations inside a Tosca Operational Policy. The next operation can be chained based on the result/output from an operation. +The *usecases* controller supports chaining of multiple operations inside a Tosca Operational +Policy. The next operation can be chained based on the result/output from an operation. The possibilities available for chaining are: - *success: chain after the result of operation is success* @@ -99,17 +101,17 @@ The possibilities available for chaining are: - *failure_exception: chain after the result of operation is failure due to exception* - *failure_guard: chain after the result of operation is failure due to guard not allowing the operation* -An example of policy chaining for VNF can be found -`here <https://github.com/onap/policy-models/blob/master/models-examples/src/main/resources/policies/vFirewall.cds.policy.operational.chaining.yaml>`__. +An example of policy chaining for VNF: +`vFirewall <https://github.com/onap/policy-models/blob/master/models-examples/src/main/resources/policies/vFirewall.cds.policy.operational.chaining.yaml>`_. -An example of policy chaining for PNF can be found -`here <https://github.com/onap/policy-models/blob/master/models-examples/src/main/resources/policies/pnf.cds.policy.operational.chaining.yaml>`__. +An example of policy chaining for PNF: +`pnf <https://github.com/onap/policy-models/blob/master/models-examples/src/main/resources/policies/pnf.cds.policy.operational.chaining.yaml>`_. Features ======== -Since the PDP-D Control Loop Application image was created from the PDP-D Engine one (*onap/policy-drools*), -it inherits all features and functionality. +Since the PDP-D Control Loop Application image was created from the PDP-D Engine one +(*onap/policy-drools*), it inherits all features and functionality. The enabled features in the *onap/policy-pdpd-cl* image are: @@ -118,29 +120,27 @@ The enabled features in the *onap/policy-pdpd-cl* image are: - **lifecycle**: enables the lifecycle APIs. - **controlloop-trans**: control loop transaction tracking. - **controlloop-management**: generic controller capabilities. -- **controlloop-usecases**: new *controller* introduced in the guilin release to realize the ONAP use cases. +- **controlloop-usecases**: new *controller* introduced in the Guilin release to realize the ONAP + use cases. Control Loops Transaction (controlloop-trans) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -It tracks Control Loop Transactions and Operations. These are recorded in -the *$POLICY_LOGS/audit.log* and *$POLICY_LOGS/metrics.log*, and accessible -through the telemetry APIs. +It tracks Control Loop Transactions and Operations. These are recorded in the +*$POLICY_LOGS/audit.log* and *$POLICY_LOGS/metrics.log*, and accessible through the telemetry APIs. Control Loops Management (controlloop-management) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -It installs common control loop application resources, and provides -telemetry API extensions. *Actor* configurations are packaged in this -feature. +It installs common control loop application resources, and provides telemetry API extensions. +*Actor* configurations are packaged in this feature. Usecases Controller (controlloop-usecases) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -It is the *guilin* release implementation of the ONAP use cases. -It relies on the new *Actor* model framework to carry out a policy's -execution. +It is the *Guilin* release implementation of the ONAP use cases. It relies on the new *Actor* model +framework to carry out a policy's execution. Utilities (controlloop-utils) @@ -151,16 +151,15 @@ Enables *actor simulators* for testing purposes. Offline Mode ============ -The default ONAP installation in *onap/policy-pdpd-cl:1.8.2* is *OFFLINE*. -In this configuration, the *rules* artifact and the *dependencies* are all in the local -maven repository. This requires that the maven dependencies are preloaded in the local -repository. +The default ONAP installation in *onap/policy-pdpd-cl:1.8.2* is *OFFLINE*. In this configuration, +the *rules* artifact and the *dependencies* are all in the local maven repository. This requires +that the maven dependencies are preloaded in the local repository. An offline configuration requires two configuration items: -- *OFFLINE* environment variable set to true (see `values.yaml <https://git.onap.org/oom/tree/kubernetes/policy/values.yaml>`__. -- override of the default *settings.xml* (see - `settings.xml <https://git.onap.org/oom/tree/kubernetes/policy/components/policy-drools-pdp/resources/configmaps/settings.xml>`__) override. +- *OFFLINE* environment variable set to true (see `values.yaml <https://git.onap.org/oom/tree/kubernetes/policy/values.yaml>`_. +- override of the default *settings.xml* (see `settings.xml <https://git.onap.org/oom/tree/kubernetes/policy/components/policy-drools-pdp/resources/configmaps/settings.xml>`_) + override. Running the PDP-D Control Loop Application in a single container ================================================================ @@ -205,13 +204,7 @@ First create an environment file (in this example *env.conf*) to configure the P SQL_USER= SQL_PASSWORD= - # AAF - - AAF=false - AAF_NAMESPACE=org.onap.policy - AAF_HOST=aaf.api.simpledemo.onap.org - - # PDP-D DMaaP configuration channel + # PDP-D configuration channel PDPD_CONFIGURATION_TOPIC=PDPD-CONFIGURATION PDPD_CONFIGURATION_API_KEY= @@ -258,7 +251,7 @@ First create an environment file (in this example *env.conf*) to configure the P PDP_PASSWORD=password GUARD_DISABLED=true - # DCAE DMaaP + # DCAE Topic DCAE_TOPIC=unauthenticated.DCAE_CL_OUTPUT DCAE_SERVERS=localhost @@ -301,7 +294,8 @@ Configuration features.pre.sh """"""""""""""" -We can enable the *controlloop-utils* and disable the *distributed-locking* feature to avoid using the database. +We can enable the *controlloop-utils* and disable the *distributed-locking* feature to avoid using +the database. .. code-block:: bash @@ -310,22 +304,11 @@ We can enable the *controlloop-utils* and disable the *distributed-locking* feat bash -c "/opt/app/policy/bin/features disable distributed-locking" bash -c "/opt/app/policy/bin/features enable controlloop-utils" -active.post.sh -"""""""""""""" - -The *active.post.sh* script makes the PDP-D active. - -.. code-block:: bash - - #!/bin/bash -x - - bash -c "http --verify=no -a ${TELEMETRY_USER}:${TELEMETRY_PASSWORD} PUT https://localhost:9696/policy/pdp/engine/lifecycle/state/ACTIVE" - Actor Properties """""""""""""""" -In the *guilin* release, some *actors* configurations need to be overridden to support *http* for compatibility -with the *controlloop-utils* feature. +In the *Guilin* release, some *actors* configurations need to be overridden to support *http* for +compatibility with the *controlloop-utils* feature. AAI-http-client.properties """""""""""""""""""""""""" @@ -420,8 +403,9 @@ Bring up the PDP-D Control Loop Application To run the container in detached mode, add the *-d* flag. -Note that we are opening the *9696* telemetry API port to the outside world, mounting the *config* host directory, -and setting environment variables. +.. note:: + The *9696* telemetry API port is open to the outside world, the *config* host directory is mounted + as a volume and environment variables are set with an env-file option. To open a shell into the PDP-D: @@ -429,13 +413,12 @@ To open a shell into the PDP-D: docker exec -it pdp-d bash -Once in the container, run tools such as *telemetry*, *db-migrator*, *policy* to look at the system state: +Once in the container, run tools such as *telemetry*, *policy* to look at the system state: .. code-block:: bash docker exec -it PDPD bash -c "/opt/app/policy/bin/telemetry" docker exec -it PDPD bash -c "/opt/app/policy/bin/policy status" - docker exec -it PDPD bash -c "/opt/app/policy/bin/db-migrator -s ALL -o report" Controlled instantiation of the PDP-D Control Loop Appplication ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -565,11 +548,12 @@ To initiate a control loop transaction, simulate a DCAE ONSET to Policy: http --verify=no -a "${TELEMETRY_USER}:${TELEMETRY_PASSWORD}" PUT https://localhost:9696/policy/pdp/engine/topics/sources/noop/DCAE_TOPIC/events @dcae.vdns.onset.json Content-Type:'text/plain' -This will trigger the scale out control loop transaction that will interact with the *SO* -simulator to complete the transaction. +This will trigger the scale out control loop transaction that will interact with the *SO* simulator +to complete the transaction. -Verify in *$POLICY_LOGS/network.log* that a *FINAL: SUCCESS* notification is sent over the POLICY-CL-MGT channel. -An entry in the *$POLICY_LOGS/audit.log* should indicate successful completion as well. +Verify in *$POLICY_LOGS/network.log* that a *FINAL: SUCCESS* notification is sent over the +POLICY-CL-MGT channel. An entry in the *$POLICY_LOGS/audit.log* should indicate successful +completion as well. vCPE use case testing ===================== @@ -661,8 +645,8 @@ To initiate a control loop transaction, simulate a DCAE ONSET to Policy: http --verify=no -a "${TELEMETRY_USER}:${TELEMETRY_PASSWORD}" PUT https://localhost:9696/policy/pdp/engine/topics/sources/noop/DCAE_TOPIC/events @dcae.vcpe.onset.json Content-Type:'text/plain' -This will spawn a vCPE control loop transaction in the PDP-D. Policy will send a *restart* message over the -*APPC-LCM-READ* channel to APPC and wait for a response. +This will spawn a vCPE control loop transaction in the PDP-D. Policy will send a *restart* message +over the *APPC-LCM-READ* channel to APPC and wait for a response. Verify that you see this message in the network.log by looking for *APPC-LCM-READ* messages. @@ -705,8 +689,9 @@ Send a simulated APPC response back to the PDP-D over the *APPC-LCM-WRITE* chann http --verify=no -a "${TELEMETRY_USER}:${TELEMETRY_PASSWORD}" PUT https://localhost:9696/policy/pdp/engine/topics/sources/noop/APPC-LCM-WRITE/events @appc.vcpe.success.json Content-Type:'text/plain' -Verify in *$POLICY_LOGS/network.log* that a *FINAL: SUCCESS* notification is sent over the *POLICY-CL-MGT* channel, -and an entry is added to the *$POLICY_LOGS/audit.log* indicating successful completion. +Verify in *$POLICY_LOGS/network.log* that a *FINAL: SUCCESS* notification is sent over the +*POLICY-CL-MGT* channel, and an entry is added to the *$POLICY_LOGS/audit.log* indicating successful +completion. vFirewall use case testing ========================== @@ -807,13 +792,14 @@ To initiate a control loop transaction, simulate a DCAE ONSET to Policy: http --verify=no -a "${TELEMETRY_USER}:${TELEMETRY_PASSWORD}" PUT https://localhost:9696/policy/pdp/engine/topics/sources/noop/DCAE_TOPIC/events @dcae.vfw.onset.json Content-Type:'text/plain' -This will spawn a vFW control loop transaction in the PDP-D. Policy will send a *ModifyConfig* message over the -*APPC-CL* channel to APPC and wait for a response. This can be seen by searching the network.log for *APPC-CL*. +This will spawn a vFW control loop transaction in the PDP-D. Policy will send a *ModifyConfig* +message over the *APPC-CL* channel to APPC and wait for a response. This can be seen by searching +the network.log for *APPC-CL*. Note the *SubRequestId* field in the *ModifyConfig* message in the *APPC-CL* topic in the network.log -Send a simulated APPC response back to the PDP-D over the *APPC-CL* channel. -To do this, change the *REPLACEME* text in the *appc.vcpe.success.json* with this *SubRequestId*. +Send a simulated APPC response back to the PDP-D over the *APPC-CL* channel. To do this, change the +*REPLACEME* text in the *appc.vcpe.success.json* with this *SubRequestId*. appc.vcpe.success.json ~~~~~~~~~~~~~~~~~~~~~~ @@ -842,24 +828,19 @@ appc.vcpe.success.json http --verify=no -a "${TELEMETRY_USER}:${TELEMETRY_PASSWORD}" PUT https://localhost:9696/policy/pdp/engine/topics/sources/noop/APPC-CL/events @appc.vcpe.success.json Content-Type:'text/plain' -Verify in *$POLICY_LOGS/network.log* that a *FINAL: SUCCESS* notification is sent over the POLICY-CL-MGT channel, -and an entry is added to the *$POLICY_LOGS/audit.log* indicating successful completion. +Verify in *$POLICY_LOGS/network.log* that a *FINAL: SUCCESS* notification is sent over the +POLICY-CL-MGT channel, and an entry is added to the *$POLICY_LOGS/audit.log* indicating successful +completion. Running PDP-D Control Loop Application with other components ============================================================ -The reader can also look at the `policy/docker repository <https://github.com/onap/policy-docker/tree/master/csit>`__. +The reader can also look at the `policy/docker repository <https://github.com/onap/policy-docker/tree/master/csit>`_. More specifically, these directories have examples of other PDP-D Control Loop configurations: -* `plans <https://github.com/onap/policy-docker/tree/master/compose>`__: startup & teardown scripts. -* `scripts <https://github.com/onap/policy-docker/blob/master/compose/compose.yaml>`__: docker-compose file. -* `tests <https://github.com/onap/policy-docker/blob/master/csit/resources/tests/drools-applications-test.robot>`__: test plan. - -Additional information -====================== - -For additional information, please see the -`Drools PDP Development and Testing (In Depth) <https://wiki.onap.org/display/DW/2020-08+Frankfurt+Tutorials>`__ page. - +* `plans <https://github.com/onap/policy-docker/tree/master/compose>`_: startup & teardown scripts. +* `scripts <https://github.com/onap/policy-docker/blob/master/compose/compose.yaml>`_: docker-compose file. +* `tests <https://github.com/onap/policy-docker/blob/master/csit/resources/tests/drools-applications-test.robot>`_: test plan. +End of Document diff --git a/docs/drools/pdpdEngine.rst b/docs/drools/pdpdEngine.rst index 0ee4fc28..6397dd86 100644 --- a/docs/drools/pdpdEngine.rst +++ b/docs/drools/pdpdEngine.rst @@ -12,21 +12,21 @@ PDP-D Engine Overview ======== -The PDP-D Core Engine provides an infrastructure and services for `drools <https://www.drools.org/>`__ based applications -in the context of Policies and ONAP. +The PDP-D Core Engine provides an infrastructure and services for `drools <https://www.drools.org/>`_ +based applications in the context of Policies and ONAP. -A PDP-D supports applications by means of *controllers*. A *controller* is a named -grouping of resources. These typically include references to communication endpoints, -maven artifact coordinates, and *coders* for message mapping. +A PDP-D supports applications by means of *controllers*. A *controller* is a named grouping of +resources. These typically include references to communication endpoints, maven artifact +coordinates, and *coders* for message mapping. -*Controllers* use *communication endpoints* to interact -with remote networked entities typically using messaging (dmaap or ueb), -or http. +*Controllers* use *communication endpoints* to interact with remote networked entities typically +using kafka messaging or http. -PDP-D Engine capabilities can be extended via *features*. Integration with other +PDP-D Engine capabilities can be extended via *features*. Integration with other Policy Framework components (API, PAP, and PDP-X) is through one of them (*feature-lifecycle*). -The PDP-D Engine infrastructure provides mechanisms for data migration, diagnostics, and application management. +The PDP-D Engine infrastructure provides mechanisms for data migration, diagnostics, and application +management. Software ======== @@ -34,25 +34,29 @@ Software Source Code repositories ~~~~~~~~~~~~~~~~~~~~~~~~ -The PDP-D software is mainly located in the `policy/drools repository <https://git.onap.org/policy/drools-pdp>`__ with the *communication endpoints* software residing in the `policy/common repository <https://git.onap.org/policy/common>`__ and Tosca policy models in the `policy/models repository <https://git.onap.org/policy/models>`__. +The PDP-D software is mainly located in the `policy/drools repository <https://git.onap.org/policy/drools-pdp>`_ +with the *communication endpoints* software residing in the +`policy/common repository <https://git.onap.org/policy/common>`_ and Tosca policy models in the +`policy/models repository <https://git.onap.org/policy/models>`_. Docker Image ~~~~~~~~~~~~ -Check the *drools-pdp* `released versions <https://wiki.onap.org/display/DW/Policy+Framework+Project%3A+Component+Versions>`__ page for the latest versions. -At the time of this writing *3.0.0* is the latest version. +Check the *drools-pdp* `released versions <https://github.com/onap/policy-parent/tree/master/integration/src/main/resources/release>`_ +page for the latest versions. At the time of this writing *3.0.1* is the latest version. .. code-block:: bash - docker pull onap/policy-drools:3.0.0 + docker pull nexus3.onap.org:10001/onap/policy-drools:3.0.1 -A container instantiated from this image will run under the non-priviledged *policy* account. +A container instantiated from this image will run under the non-privileged *policy* account. The PDP-D root directory is located at the */opt/app/policy* directory (or *$POLICY_HOME*), with the exception of the *$HOME/.m2* which contains the local maven repository. The PDP-D configuration resides in the following directories: -- **/opt/app/policy/config**: (*$POLICY_HOME/config* or *$POLICY_CONFIG*) contains *engine*, *controllers*, and *endpoint* configuration. +- **/opt/app/policy/config**: (*$POLICY_HOME/config* or *$POLICY_CONFIG*) contains *engine*, + *controllers*, and *endpoint* configuration. - **/home/policy/.m2**: (*$HOME/.m2*) maven repository configuration. - **/opt/app/policy/etc/**: (*$POLICY_HOME/etc*) miscellaneous configuration such as certificate stores. @@ -65,114 +69,91 @@ The following command can be used to explore the directory layout. Communication Endpoints ======================= -PDP-D supports the following networked infrastructures. This is also referred to as +PDP-D supports the following networked infrastructures. This is also referred to as *communication infrastructures* in the source code. -- DMaaP -- UEB +- Kafka - NOOP - Http Servers - Http Clients The source code is located at -`the policy-endpoints module <https://git.onap.org/policy/common/tree/policy-endpoints>`__ +`the policy-endpoints module <https://git.onap.org/policy/common/tree/policy-endpoints>`_ in the *policy/commons* repository. -These network resources are *named* and typically have a *global* scope, therefore typically visible to -the PDP-D engine (for administration purposes), application *controllers*, -and *features*. +These network resources are *named* and typically have a *global* scope, therefore typically visible +to the PDP-D engine (for administration purposes), application *controllers*, and *features*. -DMaaP, UEB, and NOOP are message-based communication infrastructures, hence the terminology of +Kafka and NOOP are message-based communication infrastructures, hence the terminology of source and sinks, to denote their directionality into or out of the *controller*, respectively. An endpoint can either be *managed* or *unmanaged*. The default for an endpoint is to be *managed*, meaning that they are globally accessible by name, and managed by the PDP-D engine. -*Unmanaged* topics are used when neither global visibility, or centralized PDP-D management is desired. -The software that uses *unmanaged* topics is responsible for their lifecycle management. +*Unmanaged* topics are used when neither global visibility, or centralized PDP-D management is +desired. The software that uses *unmanaged* topics is responsible for their lifecycle management. -DMaaP Endpoints +Kafka Topics ~~~~~~~~~~~~~~~ -These are messaging enpoints that use DMaaP as the communication infrastructure. - -Typically, a *managed* endpoint configuration is stored in the *<topic-name>-topic.properties* files. +Typically, a *managed* topic configuration is stored in the *<topic-name>-topic.properties* files. For example, the -`DCAE_TOPIC-topic.properties <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/DCAE_TOPIC-topic.properties>`__ is defined as +`dcae_topic-topic.properties <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/DCAE_TOPIC-topic.properties>`_ is defined as .. code-block:: bash - dmaap.source.topics=DCAE_TOPIC - - dmaap.source.topics.DCAE_TOPIC.effectiveTopic=${env:DCAE_TOPIC} - dmaap.source.topics.DCAE_TOPIC.servers=${env:DMAAP_SERVERS} - dmaap.source.topics.DCAE_TOPIC.consumerGroup=${env:DCAE_CONSUMER_GROUP} - dmaap.source.topics.DCAE_TOPIC.https=true + kafka.source.topics=dcae_topic + kafka.source.topics.dcae_topic.effectiveTopic=${env:dcae_topic} + kafka.source.topics.dcae_topic.servers=${env:KAFKA_SERVERS} + kafka.source.topics.dcae_topic.consumerGroup=${env:DCAE_CONSUMER_GROUP} + kafka.source.topics.dcae_topic.https=false -In this example, the generic name of the *source* endpoint -is *DCAE_TOPIC*. This is known as the *canonical* name. -The actual *topic* used in communication exchanges in a physical lab is contained -in the *$DCAE_TOPIC* environment variable. This environment variable is usually -set up by *devops* on a per installation basis to meet the needs of each -lab spec. +In this example, the generic name of the *source* topic is *dcae_topic*. This is known as the +*canonical* name. The actual *topic* used in communication exchanges in a physical lab is contained +in the *$dcae_topic* environment variable. This environment variable is usually set up by *devops* +on a per installation basis to meet the needs of each lab spec. -In the previous example, *DCAE_TOPIC* is a source-only topic. +In the previous example, *dcae_topic* is a source-only topic. -Sink topics are similarly specified but indicating that are sink endpoints -from the perspective of the *controller*. For example, the *APPC-CL* topic -is configured as +Sink topics are similarly specified but indicating that are sink endpoints from the perspective of +the *controller*. For example, the *appc-cl* topic is configured as: .. code-block:: bash - dmaap.source.topics=APPC-CL - dmaap.sink.topics=APPC-CL + kafka.source.topics=appc-cl + kafka.sink.topics=appc-cl - dmaap.source.topics.APPC-CL.servers=${env:DMAAP_SERVERS} - dmaap.source.topics.APPC-CL.https=true + kafka.source.topics.appc-cl.servers=${env:KAFKA_SERVERS} + kafka.source.topics.appc-cl.https=false - dmaap.sink.topics.APPC-CL.servers=${env:DMAAP_SERVERS} - dmaap.sink.topics.APPC-CL.https=true + kafka.sink.topics.appc-cl.servers=${env:KAFKA_SERVERS} + kafka.sink.topics.appc-cl.https=false -Although not shown in these examples, additional configuration options are available such as *user name*, -*password*, *security keys*, *consumer group* and *consumer instance*. - -UEB Endpoints -~~~~~~~~~~~~~ +Although not shown in these examples, additional configuration options are available such as +*user name*, *password*, *security keys*, *consumer group* and *consumer instance*. -Similary, UEB endpoints are messaging endpoints, similar to the DMaaP ones. - -For example, the -`DCAE_TOPIC-topic.properties <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/DCAE_TOPIC-topic.properties>`__ can be converted to an *UEB* one, by replacing the -*dmaap* prefix with *ueb*. For example: - -.. code-block:: bash - - ueb.source.topics=DCAE_TOPIC - - ueb.source.topics.DCAE_TOPIC.effectiveTopic=${env:DCAE_TOPIC} - ueb.source.topics.DCAE_TOPIC.servers=${env:DMAAP_SERVERS} - ueb.source.topics.DCAE_TOPIC.consumerGroup=${env:DCAE_CONSUMER_GROUP} - ueb.source.topics.DCAE_TOPIC.https=true NOOP Endpoints ~~~~~~~~~~~~~~ NOOP (no-operation) endpoints are messaging endpoints that don't have any network attachments. They are used for testing convenience. -To convert the -`DCAE_TOPIC-topic.properties <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/DCAE_TOPIC-topic.properties>`__ to a *NOOP* endpoint, simply replace the *dmaap* prefix with *noop*: +To convert the dcae_topic-topic.properties to a *NOOP* endpoint, simply replace the *kafka* prefix +with *noop*: .. code-block:: bash - noop.source.topics=DCAE_TOPIC - noop.source.topics.DCAE_TOPIC.effectiveTopic=${env:DCAE_TOPIC} + noop.source.topics=dcae_topic + noop.source.topics.dcae_topic.effectiveTopic=${env:dcae_topic} HTTP Clients ~~~~~~~~~~~~ -HTTP Clients are typically stored in files following the naming convention: *<name>-http-client.properties* convention. -One such example is -the `AAI HTTP Client <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/AAI-http-client.properties>`__: +HTTP Clients are typically stored in files following the naming convention: +*<name>-http-client.properties* convention. + +One such example is the +`AAI HTTP Client <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/AAI-http-client.properties>`_: .. code-block:: bash @@ -189,9 +170,9 @@ the `AAI HTTP Client <https://git.onap.org/policy/drools-applications/tree/contr HTTP Servers ~~~~~~~~~~~~ -HTTP Servers are stored in files that follow a similar naming convention *<name>-http-server.properties*. -The following is an example of a server named *CONFIG*, getting most of its configuration from -environment variables. +HTTP Servers are stored in files that follow a similar naming convention +*<name>-http-server.properties*. The following is an example of a server named *CONFIG*, getting +most of its configuration from environment variables. .. code-block:: bash @@ -204,21 +185,22 @@ environment variables. http.server.services.CONFIG.restPackages=org.onap.policy.drools.server.restful http.server.services.CONFIG.managed=false http.server.services.CONFIG.swagger=true - http.server.services.CONFIG.https=true - http.server.services.CONFIG.aaf=${envd:AAF:false} + http.server.services.CONFIG.https=false -*Endpoints* configuration resides in the *$POLICY_HOME/config* (or *$POLICY_CONFIG*) directory in a container. +*Endpoints* configuration resides in the *$POLICY_HOME/config* (or *$POLICY_CONFIG*) directory in a +container. Controllers =========== -*Controllers* are the means for the PDP-D to run *applications*. Controllers are -defined in *<name>-controller.properties* files. +*Controllers* are the means for the PDP-D to run *applications*. Controllers are defined in +*<name>-controller.properties* files. For example, see the -`usecases controller configuration <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-usecases/src/main/feature/config/usecases-controller.properties>`__. +`usecases controller configuration <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-usecases/src/main/feature/config/usecases-controller.properties>`_. -This configuration file has two sections: *a)* application maven coordinates, and *b)* endpoint references and coders. +This configuration file has two sections: *a)* application maven coordinates, and *b)* endpoint +references and coders. Maven Coordinates ~~~~~~~~~~~~~~~~~ @@ -236,7 +218,8 @@ It is the *brain* of the control loop application. ..... This *kjar* contains the -`usecases DRL <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/usecases.drl>`__ file (there may be more than one DRL file included). +`usecases DRL <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/usecases.drl>`_ +file (there may be more than one DRL file included). .. code-block:: bash @@ -255,10 +238,10 @@ This *kjar* contains the end ... -The DRL in conjuction with the dependent java libraries in the kjar -`pom <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/pom.xml>`__ -realizes the application's function. For intance, it realizes the -vFirewall, vCPE, and vDNS use cases in ONAP. +The DRL in conjunction with the dependent java libraries in the kjar +`pom <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/pom.xml>`_ +realizes the application's function. For instance, it realizes the vFirewall, vCPE, and vDNS use +cases in ONAP. .. code-block:: bash @@ -274,66 +257,64 @@ vFirewall, vCPE, and vDNS use cases in ONAP. Endpoints References and Coders ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The *usecases-controller.properties* configuration also contains a mix of -source (of incoming controller traffic) and sink (of outgoing controller traffic) -configuration. This configuration also contains specific -filtering and mapping rules for incoming and outgoing dmaap messages -known as *coders*. +The *usecases-controller.properties* configuration also contains a mix of source (of incoming +controller traffic) and sink (of outgoing controller traffic) configuration. This configuration also +contains specific filtering and mapping rules for incoming and outgoing messages known as *coders*. .. code-block:: bash ... - dmaap.source.topics=DCAE_TOPIC,APPC-CL,APPC-LCM-WRITE,SDNR-CL-RSP - dmaap.sink.topics=APPC-CL,APPC-LCM-READ,POLICY-CL-MGT,SDNR-CL,DCAE_CL_RSP - + kafka.source.topics=dcae_topic,appc-cl,appc-lcm-write,sdnr-cl-rsp + kafka.sink.topics=appc-cl,appc-lcm-read,policy-cl-mgt,sdnr-cl,dcae_cl_rsp - dmaap.source.topics.APPC-LCM-WRITE.events=org.onap.policy.appclcm.AppcLcmDmaapWrapper - dmaap.source.topics.APPC-LCM-WRITE.events.org.onap.policy.appclcm.AppcLcmDmaapWrapper.filter=[?($.type == 'response')] - dmaap.source.topics.APPC-LCM-WRITE.events.custom.gson=org.onap.policy.appclcm.util.Serialization,gson + kafka.source.topics.appc-lcm-write.events=org.onap.policy.appclcm.AppcLcmMessageWrapper + kafka.source.topics.appc-lcm-write.events.org.onap.policy.appclcm.AppcLcmMessageWrapper.filter=[?($.type == 'response')] + kafka.source.topics.appc-lcm-write.events.custom.gson=org.onap.policy.appclcm.util.Serialization,gson - dmaap.sink.topics.APPC-CL.events=org.onap.policy.appc.Request - dmaap.sink.topics.APPC-CL.events.custom.gson=org.onap.policy.appc.util.Serialization,gsonPretty + kafka.sink.topics.appc-cl.events=org.onap.policy.appc.Request + kafka.sink.topics.appc-cl.events.custom.gson=org.onap.policy.appc.util.Serialization,gsonPretty ... -In this example, the *coders* specify that incoming messages over the DMaaP endpoint -reference *APPC-LCM-WRITE*, that have a field called *type* under the root JSON object with -value *response* are allowed into the *controller* application. In this case, the incoming -message is converted into an object (fact) of type *org.onap.policy.appclcm.AppcLcmDmaapWrapper*. -The *coder* has attached a custom implementation provided by the *application* with class -*org.onap.policy.appclcm.util.Serialization*. Note that the *coder* filter is expressed in JSONPath notation. +In this example, the *coders* specify that incoming messages reference *appc-lcm-write*, that have a +field called *type* under the root JSON object with value *response* are allowed into the +*controller* application. In this case, the incoming message is converted into an object (fact) of +type *org.onap.policy.appclcm.AppcLcmMessageWrapper*. The *coder* has attached a custom +implementation provided by the *application* with class +*org.onap.policy.appclcm.util.Serialization*. Note that the *coder* filter is expressed in JSONPath +notation. Note that not all the communication endpoint references need to be explicitly referenced within the -*controller* configuration file. For example, *Http clients* do not. -The reasons are historical, as the PDP-D was initially intended to only communicate -through messaging-based protocols such as UEB or DMaaP in asynchronous unidirectional mode. -The introduction of *Http* with synchronous bi-directional communication with remote endpoints made -it more convenient for the application to manage each network exchange. +*controller* configuration file. For example, *Http clients* do not. The reasons are historical, as +the PDP-D was initially intended to only communicate through messaging-based protocols such as UEB +or DMaaP in asynchronous unidirectional mode. The introduction of *Http* with synchronous +bi-directional communication with remote endpoints made it more convenient for the application to +manage each network exchange. UEB and DMaaP have been replaced by Kafka messaging since Kohn release. -*Controllers* configuration resides in the *$POLICY_HOME/config* (or *$POLICY_CONFIG*) directory in a container. +*Controllers* configuration resides in the *$POLICY_HOME/config* (or *$POLICY_CONFIG*) directory in +a container. Other Configuration Files ~~~~~~~~~~~~~~~~~~~~~~~~~ -There are other types of configuration files that *controllers* can use, for example *.environment* files -that provides a means to share data across applications. The -`controlloop.properties.environment <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/controlloop.properties.environment>`__ is one such example. +There are other types of configuration files that *controllers* can use, for example *.environment* +files that provides a means to share data across applications. The +`controlloop.properties.environment <https://git.onap.org/policy/drools-applications/tree/controlloop/common/feature-controlloop-management/src/main/feature/config/controlloop.properties.environment>`_ +is one such example. Tosca Policies ============== -PDP-D supports Tosca Policies through the *feature-lifecycle*. The *PDP-D* receives its policy set -from the *PAP*. A policy conforms to its Policy Type specification. -Policy Types and policy creation is done by the *API* component. -Policy deployments are orchestrated by the *PAP*. +PDP-D supports Tosca Policies through the *feature-lifecycle*. The *PDP-D* receives its policy set +from the *PAP*. A policy conforms to its Policy Type specification. Policy Types and policy creation +is done by the *API* component. Policy deployments are orchestrated by the *PAP*. -All communication between *PAP* and PDP-D is over the DMaaP *POLICY-PDP-PAP* topic. +All communication between *PAP* and PDP-D is over the Kafka *policy-pdp-pap* topic. Native Policy Types ~~~~~~~~~~~~~~~~~~~ -The PDP-D Engine supports two (native) Tosca policy types by means of the *lifecycle* -feature: +The PDP-D Engine supports two (native) Tosca policy types by means of the *lifecycle* feature: - *onap.policies.native.drools.Controller* - *onap.policies.native.drools.Artifact* @@ -342,7 +323,8 @@ These types can be used to dynamically deploy or undeploy application *controlle assign policy types, and upgrade or downgrade their attached maven artifact versions. For instance, an -`example native controller <https://git.onap.org/policy/drools-pdp/tree/feature-lifecycle/src/test/resources/tosca-policy-native-controller-example.json>`__ policy is shown below. +`example native controller <https://git.onap.org/policy/drools-pdp/tree/feature-lifecycle/src/test/resources/tosca-policy-native-controller-example.json>`_ +policy is shown below. .. code-block:: bash @@ -363,7 +345,7 @@ For instance, an "controllerName": "lifecycle", "sourceTopics": [ { - "topicName": "DCAE_TOPIC", + "topicName": "dcae_topic", "events": [ { "eventClass": "java.util.HashMap", @@ -378,7 +360,7 @@ For instance, an ], "sinkTopics": [ { - "topicName": "APPC-CL", + "topicName": "appc-cl", "events": [ { "eventClass": "java.util.HashMap", @@ -397,8 +379,9 @@ For instance, an } } -The actual application coordinates are provided with a policy of type onap.policies.native.drools.Artifact, -see the `example native artifact <https://git.onap.org/policy/drools-pdp/tree/feature-lifecycle/src/test/resources/tosca-policy-native-artifact-example.json>`__ +The actual application coordinates are provided with a policy of type +onap.policies.native.drools.Artifact, see the +`example native artifact <https://git.onap.org/policy/drools-pdp/tree/feature-lifecycle/src/test/resources/tosca-policy-native-artifact-example.json>`_ .. code-block:: bash @@ -434,16 +417,15 @@ see the `example native artifact <https://git.onap.org/policy/drools-pdp/tree/fe Operational Policy Types ~~~~~~~~~~~~~~~~~~~~~~~~ -The PDP-D also recognizes Tosca Operational Policies, although it needs an -application *controller* that understands them to execute them. These are: +The PDP-D also recognizes Tosca Operational Policies, although it needs an application *controller* +that understands them to execute them. These are: - *onap.policies.controlloop.operational.common.Drools* -A minimum of one application *controller* that supports these capabilities -must be installed in order to honor the *operational policy types*. -One such controller is the *usecases* controller residing in the -`policy/drools-applications <https://git.onap.org/policy/drools-applications>`__ -repository. +A minimum of one application *controller* that supports these capabilities must be installed in +order to honor the *operational policy types*. One such controller is the *usecases* controller +residing in the +`policy/drools-applications <https://git.onap.org/policy/drools-applications>`_ repository. Controller Policy Type Support ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -459,15 +441,14 @@ explicitly in a native *onap.policies.native.drools.Controller* policy. The *controller* application could declare its supported policy types in the *kjar*. For example, the *usecases controller* packages this information in the -`kmodule.xml <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/META-INF/kmodule.xml>`__. One advantage of this approach is that the PDP-D would only +`kmodule.xml <https://git.onap.org/policy/drools-applications/tree/controlloop/common/controller-usecases/src/main/resources/META-INF/kmodule.xml>`_. One advantage of this approach is that the PDP-D would only commit to execute policies against these policy types if a supporting controller is up and running. .. code-block:: bash - <kmodule xmlns="http://jboss.org/kie/6.0.0/kmodule"> - <kbase name="onap.policies.controlloop.operational.common.Drools" default="false" equalsBehavior="equality"/> - <kbase name="onap.policies.controlloop.Operational" equalsBehavior="equality" - packages="org.onap.policy.controlloop" includes="onap.policies.controlloop.operational.common.Drools"> + <kmodule xmlns="http://www.drools.org/xsd/kmodule"> + <kbase name="onap.policies.controlloop.operational.common.Drools" equalsBehavior="equality" + packages="org.onap.policy.controlloop"> <ksession name="usecases"/> </kbase> </kmodule> @@ -477,23 +458,23 @@ Software Architecture PDP-D is divided into 2 layers: -- core (`policy-core <https://git.onap.org/policy/drools-pdp/tree/policy-core>`__) -- management (`policy-management <https://git.onap.org/policy/drools-pdp/tree/policy-management>`__) +- core (`policy-core <https://git.onap.org/policy/drools-pdp/tree/policy-core>`_) +- management (`policy-management <https://git.onap.org/policy/drools-pdp/tree/policy-management>`_) Core Layer ~~~~~~~~~~ The core layer directly interfaces with the *drools* libraries with 2 main abstractions: -* `PolicyContainer <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicyContainer.java>`__, and -* `PolicySession <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicySession.java>`__. +* `PolicyContainer <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicyContainer.java>`_, and +* `PolicySession <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicySession.java>`_. Policy Container and Sessions """"""""""""""""""""""""""""" -The *PolicyContainer* abstracts the drools *KieContainer*, while a *PolicySession* abstracts a drools *KieSession*. -PDP-D uses stateful sessions in active mode (*fireUntilHalt*) (please visit the `drools <https://www.drools.org/>`__ -website for additional documentation). +The *PolicyContainer* abstracts the drools *KieContainer*, while a *PolicySession* abstracts a +drools *KieSession*. PDP-D uses stateful sessions in active mode (*fireUntilHalt*) (please visit the +`drools <https://www.drools.org/>`_ website for additional documentation). Management Layer ~~~~~~~~~~~~~~~~ @@ -503,53 +484,59 @@ The management layer manages the PDP-D and builds on top of the *core* capabilit PolicyEngine """""""""""" -The PDP-D `PolicyEngine <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyEngine.java>`__ is the top abstraction and abstracts away the PDP-D and all the -resources it holds. The reader looking at the source code can start looking at this component -in a top-down fashion. Note that the *PolicyEngine* abstraction should not be confused with the -sofware in the *policy/engine* repository, there is no relationship whatsoever other than in the naming. +The PDP-D `PolicyEngine <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyEngine.java>`_ is the top abstraction and abstracts away the PDP-D and all the +resources it holds. The reader looking at the source code can start looking at this component in a +top-down fashion. Note that the *PolicyEngine* abstraction should not be confused with the software +in the *policy/engine* repository, there is no relationship whatsoever other than in the naming. -The *PolicyEngine* represents the PDP-D, holds all PDP-D resources, and orchestrates activities among those. +The *PolicyEngine* represents the PDP-D, holds all PDP-D resources, and orchestrates activities +among those. -The *PolicyEngine* manages applications via the `PolicyController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyController.java>`__ abstractions in the base code. The +The *PolicyEngine* manages applications via the `PolicyController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyController.java>`_ abstractions in the base code. The relationship between the *PolicyEngine* and *PolicyController* is one to many. -The *PolicyEngine* holds other global resources such as a *thread pool*, *policies validator*, *telemetry* server, -and *unmanaged* topics for administration purposes. +The *PolicyEngine* holds other global resources such as a *thread pool*, *policies validator*, +*telemetry* server, and *unmanaged* topics for administration purposes. The *PolicyEngine* has interception points that allow -`*features* <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/features/PolicyEngineFeatureApi.java>`__ +`*features* <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/features/PolicyEngineFeatureApi.java>`_ to observe and alter the default *PolicyEngine* behavior. -The *PolicyEngine* implements the `*Startable* <https://git.onap.org/policy/common/tree/capabilities/src/main/java/org/onap/policy/common/capabilities/Startable.java>`__ and `*Lockable* <https://git.onap.org/policy/common/tree/capabilities/src/main/java/org/onap/policy/common/capabilities/Lockable.java>`__ interfaces. These operations -have a cascading effect on the resources the *PolicyEngine* holds, as it is the top level entity, thus -affecting *controllers* and *endpoints*. These capabilities are intended to be used for extensions, -for example active/standby multi-node capabilities. This programmability is -exposed via the *telemetry* API, and *feature* hooks. +The *PolicyEngine* implements the `*Startable* <https://git.onap.org/policy/common/tree/capabilities/src/main/java/org/onap/policy/common/capabilities/Startable.java>`_ and `*Lockable* <https://git.onap.org/policy/common/tree/capabilities/src/main/java/org/onap/policy/common/capabilities/Lockable.java>`_ interfaces. These operations +have a cascading effect on the resources the *PolicyEngine* holds, as it is the top level entity, +thus affecting *controllers* and *endpoints*. These capabilities are intended to be used for +extensions, for example active/standby multi-node capabilities. This programmability is exposed via +the *telemetry* API, and *feature* hooks. Configuration ^^^^^^^^^^^^^ *PolicyEngine* related configuration is located in the -`engine.properties <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server/config/engine.properties>`__, -and `engine-system.properties <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server/config/engine.properties>`__. +`engine.properties <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server/config/engine.properties>`_, +and `engine-system.properties <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server/config/engine.properties>`_. The *engine* configuration files reside in the *$POLICY_CONFIG* directory. PolicyController """""""""""""""" -A *PolicyController* represents an application. Each *PolicyController* has an instance of a -`DroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyController.java>`__. The *PolicyController* provides the means to group application specific resources -into a single unit. Such resources include the application's *maven coordinates*, *endpoint references*, and *coders*. +A *PolicyController* represents an application. Each +`PolicyController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyController.java>`_ has an instance of a +has an instance of a +`DroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/system/PolicyController.java>`_. +The *PolicyController* provides the means to group application specific resources into a single +unit. Such resources include the application's *maven coordinates*, *endpoint references*, and +*coders*. -A *PolicyController* uses a -`DroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/DroolsController.java>`__ to interface with the *core* layer (*PolicyContainer* and *PolicySession*). +A *PolicyController* uses a *DroolsController* to interface with the *core* layer (*PolicyContainer* +and *PolicySession*). The relationship between the *PolicyController* and the *DroolsController* is one-to-one. The *DroolsController* currently supports 2 implementations, the -`MavenDroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/internal/MavenDroolsController.java>`__, and the -`NullDroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/internal/NullDroolsController.java>`__. -The *DroolsController*'s polymorphic behavior depends on whether a maven artifact is attached to the controller or not. +`MavenDroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/internal/MavenDroolsController.java>`_, and the +`NullDroolsController <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/controller/internal/NullDroolsController.java>`_. +The *DroolsController*'s polymorphic behavior depends on whether a maven artifact is attached to the +controller or not. Configuration ^^^^^^^^^^^^^ @@ -569,35 +556,35 @@ Using Features and Listeners Features hook into the interception points provided by the the *PDP-D* main entities. -*Endpoint Listeners*, see `here <https://git.onap.org/policy/common/tree/message-bus/src/main/java/org/onap/policy/common/message/bus/event/TopicListener.java>`__ -and `here <https://git.onap.org/policy/common/tree/policy-endpoints/src/main/java/org/onap/policy/common/endpoints/listeners>`__, can be used in conjunction with features for additional capabilities. +`TopicListener <https://git.onap.org/policy/common/tree/message-bus/src/main/java/org/onap/policy/common/message/bus/event/TopicListener.java>`_ +and `other listeners <https://git.onap.org/policy/common/tree/policy-endpoints/src/main/java/org/onap/policy/common/endpoints/listeners>`_ +, can be used in conjunction with features for additional capabilities. Using Maven-Drools applications """"""""""""""""""""""""""""""" -Maven-based drools applications can run any arbitrary functionality structured with rules and java logic. +Maven-based drools applications can run any arbitrary functionality structured with rules and java +logic. Recommended Flow """""""""""""""" -Whenever possible it is suggested that PDP-D related operations flow through the -*PolicyEngine* downwards in a top-down manner. This imposed order implies that -all the feature hooks are always invoked in a deterministic fashion. It is also -a good mechanism to safeguard against deadlocks. +Whenever possible it is suggested that PDP-D related operations flow through the *PolicyEngine* +downwards in a top-down manner. This imposed order implies that all the feature hooks are always +invoked in a deterministic fashion. It is also a good mechanism to safeguard against deadlocks. Telemetry Extensions """""""""""""""""""" -It is recommended to *features* (extensions) to offer a diagnostics REST API -to integrate with the telemetry API. This is done by placing JAX-RS files under -the package *org.onap.policy.drools.server.restful*. The root context path -for all the telemetry services is */policy/pdp/engine*. +It is recommended to *features* (extensions) to offer a diagnostics REST API to integrate with the +telemetry API. This is done by placing JAX-RS files under the package +*org.onap.policy.drools.server.restful*. The root context path for all the telemetry services is +*/policy/pdp/engine*. Features ======== -*Features* is an extension mechanism for the PDP-D functionality. -Features can be toggled on and off. +*Features* is an extension mechanism for the PDP-D functionality. Features can be toggled on and off. A feature is composed of: - Java libraries. @@ -607,13 +594,13 @@ Java Extensions ~~~~~~~~~~~~~~~ Additional functionality can be provided in the form of java libraries that hook into the -*PolicyEngine*, *PolicyController*, *DroolsController*, and *PolicySession* interception -points to observe or alter the PDP-D logic. +*PolicyEngine*, *PolicyController*, *DroolsController*, and *PolicySession* interception points to +observe or alter the PDP-D logic. See the Feature APIs available in the -`management <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/features>`__ +`management <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/java/org/onap/policy/drools/features>`_ and -`core <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicySessionFeatureApi.java>`__ layers. +`core <https://git.onap.org/policy/drools-pdp/tree/policy-core/src/main/java/org/onap/policy/drools/core/PolicySessionFeatureApi.java>`_ layers. The convention used for naming these extension modules are *api-<name>* for interfaces, and *feature-<name>* for the actual java extensions. @@ -623,9 +610,9 @@ Configuration Items Installation items such as scripts, SQL, maven artifacts, and configuration files. -The reader can refer to the `policy/drools-pdp repository <https://git.onap.org/policy/drools-pdp>`__ -and the <https://git.onap.org/policy/drools-applications>`__ repository for miscellaneous feature -implementations. +The reader can refer to the `policy/drools-pdp repository <https://git.onap.org/policy/drools-pdp>`_ +and the `policy/drools-applications repository <https://git.onap.org/policy/drools-applications>`_ +repository for miscellaneous feature implementations. Layout """""" @@ -649,10 +636,6 @@ A feature is packaged in a *feature-<name>.zip* and has this internal layout: # | | L─ <dependent-jar>+ # │ L─ feature/ # │ L─ <feature-jar> - # L─ [db]/ - # │ L─ <db-name>/+ - # │ L─ sql/ - # │ L─ <sql-scripts>* # L─ [artifacts]/ # L─ <artifact>+ # L─ [install] @@ -689,7 +672,7 @@ A feature is packaged in a *feature-<name>.zip* and has this internal layout: # by the feature designer. # ######################################################################################## -The `features <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server-gen/bin/features>`__ +The `features <https://git.onap.org/policy/drools-pdp/tree/policy-management/src/main/server-gen/bin/features>`_ is the tool used for administration purposes: .. code-block:: bash @@ -720,18 +703,22 @@ The following features are included in the image but disabled. Healthcheck """"""""""" -The Healthcheck feature provides reports used to verify the health of *PolicyEngine.manager* in addition to the construction, operation, and deconstruction of HTTP server/client objects. +The Healthcheck feature provides reports used to verify the health of *PolicyEngine.manager* in +addition to the construction, operation, and deconstruction of HTTP server/client objects. When enabled, the feature takes as input a properties file named "*feature-healtcheck.properties*. -This file should contain configuration properties necessary for the construction of HTTP client and server objects. +This file should contain configuration properties necessary for the construction of HTTP client and +server objects. -Upon initialization, the feature first constructs HTTP server and client objects using the properties -from its properties file. A healthCheck operation is then triggered. The logic of the healthCheck verifies -that *PolicyEngine.manager* is alive, and iteratively tests each HTTP server object by sending HTTP GET -requests using its respective client object. If a server returns a "200 OK" message, it is marked as "healthy" -in its individual report. Any other return code results in an "unhealthy" report. +Upon initialization, the feature first constructs HTTP server and client objects using the +properties from its properties file. A healthCheck operation is then triggered. The logic of the +healthCheck verifies that *PolicyEngine.manager* is alive, and iteratively tests each HTTP server +object by sending HTTP GET requests using its respective client object. If a server returns a"200 +OK" message, it is marked as "healthy" in its individual report. Any other return code results in an +"unhealthy" report. -After the testing of the server objects has completed, the feature returns a single consolidated report. +After the testing of the server objects has completed, the feature returns a single consolidated +report. Lifecycle """"""""" @@ -744,41 +731,38 @@ The PAP interacts with the lifecycle feature to put a PDP-D in PASSIVE or ACTIVE The PASSIVE state allows for Tosca Operational policies to be deployed. Policy execution is enabled when the PDP-D transitions to the ACTIVE state. -This feature can coexist side by side with the legacy mode of operation that pre-dates the Dublin release. +This feature can coexist side by side with the legacy mode of operation that pre-dates the Dublin +release. Distributed Locking """"""""""""""""""" -The Distributed Locking Feature provides locking of resources across a pool of PDP-D hosts. -The list of locks is maintained in a database, where each record includes a resource identifier, -an owner identifier, and an expiration time. Typically, a drools application will unlock the resource -when it's operation completes. However, if it fails to do so, then the resource will be automatically +The Distributed Locking Feature provides locking of resources across a pool of PDP-D hosts. The list +of locks is maintained in a database, where each record includes a resource identifier, an owner +identifier, and an expiration time. Typically, a drools application will unlock the resource when +it's operation completes. However, if it fails to do so, then the resource will be automatically released when the lock expires, thus preventing a resource from becoming permanently locked. Other features ~~~~~~~~~~~~~~ -The following features have been contributed to the *policy/drools-pdp* but are either -unnecessary or have not been thoroughly tested: +The following features have been contributed to the *policy/drools-pdp* but are either unnecessary +or have not been thoroughly tested: .. toctree:: :maxdepth: 1 - feature_activestdbymgmt.rst - feature_controllerlogging.rst - feature_eelf.rst - feature_mdcfilters.rst feature_pooling.rst - feature_sesspersist.rst - feature_statemgmt.rst feature_testtransaction.rst feature_nolocking.rst Data Migration ============== -PDP-D data is migrated across releases with the -`db-migrator <https://git.onap.org/policy/docker/tree/policy-db-migrator/src/main/docker/db-migrator>`__. +PDP-D data used to be migrated across releases with its own db-migrator until Kohn release. Since +Oslo, the main policy database manager, +`db-migrator <https://git.onap.org/policy/docker/tree/policy-db-migrator/src/main/docker/db-migrator>`_ +has been in use. The migration occurs when different release data is detected. *db-migrator* will look under the *$POLICY_HOME/etc/db/migration* for databases and SQL scripts to migrate. @@ -793,37 +777,7 @@ where *<sql-file>* is of the form: <VERSION>-<pdp|feature-name>[-description](.upgrade|.downgrade).sql -The db-migrator tool syntax is - -.. code-block:: bash - - syntax: db-migrator - -s <schema-name> - [-b <migration-dir>] - [-f <from-version>] - [-t <target-version>] - -o <operations> - - where <operations>=upgrade|downgrade|auto|version|erase|report - - Configuration Options: - -s|--schema|--database: schema to operate on ('ALL' to apply on all) - -b|--basedir: overrides base DB migration directory - -f|--from: overrides current release version for operations - -t|--target: overrides target release to upgrade/downgrade - - Operations: - upgrade: upgrade operation - downgrade: performs a downgrade operation - auto: autonomous operation, determines upgrade or downgrade - version: returns current version, and in conjunction if '-f' sets the current version - erase: erase all data related <schema> (use with care) - report: migration detailed report on an schema - ok: is the migration status valid - -See the -`feature-distributed-locking sql directory <https://git.onap.org/policy/docker/tree/policy-db-migrator/src/main/docker/config/pooling/sql>`__ -for an example of upgrade/downgrade scripts. +More information on DB Migrator, check :ref:`Policy DB Migrator <policy-db-migrator-label>` page. Maven Repositories @@ -831,15 +785,15 @@ Maven Repositories The drools libraries in the PDP-D uses maven to fetch rules artifacts and software dependencies. -The default *settings.xml* file specifies the repositories to search. This configuration -can be overriden with a custom copy that would sit in a mounted configuration -directory. See an example of the OOM override +The default *settings.xml* file specifies the repositories to search. This configuration can be +overwritten with a custom copy that would sit in a mounted configuration directory. See an example +of the OOM override `settings.xml <https://github.com/onap/oom/blob/master/kubernetes/policy/components/policy-drools-pdp/resources/configmaps/settings.xml>`_. -The default ONAP installation of the *control loop* child image *onap/policy-pdpd-cl:1.6.4* is *OFFLINE*. -In this configuration, the *rules* artifact and the *dependencies* retrieves all the artifacts from the local -maven repository. Of course, this requires that the maven dependencies are preloaded in the local -repository for it to work. +The default ONAP installation of the *control loop* child image *onap/policy-pdpd-cl:3.0.1* is +*OFFLINE*. In this configuration, the *rules* artifact and the *dependencies* retrieves all the +artifacts from the local maven repository. Of course, this requires that the maven dependencies are +preloaded in the local repository for it to work. An offline configuration requires two items: @@ -847,13 +801,13 @@ An offline configuration requires two items: - override *settings.xml* customization, see `settings.xml <https://github.com/onap/oom/blob/master/kubernetes/policy/components/policy-drools-pdp/resources/configmaps/settings.xml>`_. -The default mode in the *onap/policy-drools:1.6.3* is ONLINE instead. +The default mode in the *onap/policy-drools:3.0.1* is ONLINE instead. In *ONLINE* mode, the *controller* initialization can take a significant amount of time. -The Policy ONAP installation includes a *nexus* repository component that can be used to host any arbitrary -artifacts that an PDP-D application may require. -The following environment variables configure its location: +The Policy ONAP installation includes a *nexus* repository component that can be used to host any +arbitrary artifacts that an PDP-D application may require. The following environment variables +configure its location: .. code-block:: bash @@ -863,10 +817,10 @@ The following environment variables configure its location: RELEASE_REPOSITORY_URL=http://nexus:8080/nexus/content/repositories/releases/ REPOSITORY_OFFLINE=false -The *deploy-artifact* tool is used to deploy artifacts to the local or remote maven repositories. -It also allows for dependencies to be installed locally. The *features* tool invokes it when artifacts are -to be deployed as part of a feature. The tool can be useful for developers to test a new application -in a container. +The *deploy-artifact* tool is used to deploy artifacts to the local or remote maven repositories. It +also allows for dependencies to be installed locally. The *features* tool invokes it when artifacts +are to be deployed as part of a feature. The tool can be useful for developers to test a new +application in a container. .. code-block:: bash @@ -903,30 +857,29 @@ The *status* option provides generic status of the system. [features] name version status ---- ------- ------ - healthcheck 1.6.3 enabled - distributed-locking 1.6.3 enabled - lifecycle 1.6.3 enabled - controlloop-management 1.6.4 enabled - controlloop-utils 1.6.4 enabled - controlloop-trans 1.6.4 enabled - controlloop-usecases 1.6.4 enabled + healthcheck 3.0.1 enabled + distributed-locking 3.0.1 enabled + lifecycle 3.0.1 enabled + controlloop-management 3.0.1 enabled + controlloop-utils 3.0.1 enabled + controlloop-trans 3.0.1 enabled + controlloop-usecases 3.0.1 enabled - [migration] - pooling: OK @ 1811 It contains 3 sections: - *PDP-D* running status - *features* applied -The *start* and *stop* commands are useful for developers testing functionality on a docker container instance. +The *start* and *stop* commands are useful for developers testing functionality on a docker +container instance. Telemetry Shell =============== -*PDP-D* offers an ample set of REST APIs to debug, introspect, and change state on a running PDP-D. This is known as the -*telemetry* API. The *telemetry* shell wraps these APIs for shell-like access using -`http-prompt <http://http-prompt.com/>`__. +*PDP-D* offers an ample set of REST APIs to debug, introspect, and change state on a running PDP-D. +This is known as the *telemetry* API. The *telemetry* shell wraps these APIs for shell-like access +using `http-prompt <http://http-prompt.com/>`_. .. code-block:: bash @@ -956,7 +909,8 @@ Refer to the *$POLICY_HOME/bin/* directory for additional tooling. PDP-D Docker Container Configuration ==================================== -Both the PDP-D *onap/policy-drools* and *onap/policy-pdpd-cl* images can be used without other components. +Both the PDP-D *onap/policy-drools* and *onap/policy-pdpd-cl* images can be used without other +components. There are 2 types of configuration data provided to the container: @@ -966,9 +920,9 @@ There are 2 types of configuration data provided to the container: Environment variables ~~~~~~~~~~~~~~~~~~~~~ -As it was shown in the *controller* and *endpoint* sections, PDP-D configuration can rely -on environment variables. In a container environment, these variables are set up by the user -in the host environment. +As it was shown in the *controller* and *endpoint* sections, PDP-D configuration can rely on +environment variables. In a container environment, these variables are set up by the user in the +host environment. Configuration Files and Shell Scripts ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -981,15 +935,15 @@ These are the configuration items that can reside externally and override the de - **settings.xml** if working with external nexus repositories. - **standalone-settings.xml** if an external *policy* nexus repository is not available. -- ***.conf** files containing environment variables. This is an alternative to use environment variables, - as these files will be sourced in before the PDP-D starts. +- ***.conf** files containing environment variables. This is an alternative to use environment + variables, as these files will be sourced in before the PDP-D starts. - **features*.zip** to load any arbitrary feature not present in the image. - ***.pre.sh** scripts that will be executed before the PDP-D starts. - ***.post.sh** scripts that will be executed after the PDP-D starts. - **policy-keystore** to override the default PDP-D java keystore. - **policy-truststore** to override the default PDP-D java truststore. -- ***.properties** to override or add any properties file for the PDP-D, this includes *controller*, *endpoint*, - *engine* or *system* configurations. +- ***.properties** to override or add any properties file for the PDP-D, this includes *controller*, + *endpoint*, *engine* or *system* configurations. - **logback*.xml** to override the default logging configuration. - ***.xml** to override other .xml configuration that may be used for example by an *application*. - ***.json** *json* configuration that may be used by an *application*. @@ -1035,13 +989,7 @@ First create an environment file (in this example *env.conf*) to configure the P SQL_USER= SQL_PASSWORD= - # AAF - - AAF=false - AAF_NAMESPACE=org.onap.policy - AAF_HOST=aaf.api.simpledemo.onap.org - - # PDP-D DMaaP configuration channel + # PDP-D configuration channel PDPD_CONFIGURATION_TOPIC=PDPD-CONFIGURATION PDPD_CONFIGURATION_API_KEY= @@ -1052,42 +1000,30 @@ First create an environment file (in this example *env.conf*) to configure the P # PAP-PDP configuration channel - POLICY_PDP_PAP_TOPIC=POLICY-PDP-PAP + POLICY_PDP_PAP_TOPIC=policy-pdp-pap POLICY_PDP_PAP_API_KEY= POLICY_PDP_PAP_API_SECRET= -Note that *SQL_HOST*, and *REPOSITORY* are empty, so the PDP-D does not attempt -to integrate with those components. +Note that *SQL_HOST*, and *REPOSITORY* are empty, so the PDP-D does not attempt to integrate with +those components. Configuration ~~~~~~~~~~~~~ - -active.post.sh -"""""""""""""" - -To put the controller directly in active mode at initialization, place an *active.post.sh* script under the -mounted host directory: - -.. code-block:: bash - - #!/bin/bash -x - - bash -c "http --verify=no -a ${TELEMETRY_USER}:${TELEMETRY_PASSWORD} PUT https://localhost:9696/policy/pdp/engine/lifecycle/state/ACTIVE" - Bring up the PDP-D ~~~~~~~~~~~~~~~~~~ .. code-block:: bash - docker run --rm -p 9696:9696 -v ${PWD}/config:/tmp/policy-install/config --env-file ${PWD}/env/env.conf -it --name PDPD -h pdpd nexus3.onap.org:10001/onap/policy-drools:1.6.3 + docker run --rm -p 9696:9696 -v ${PWD}/config:/tmp/policy-install/config --env-file ${PWD}/env/env.conf -it --name PDPD -h pdpd nexus3.onap.org:10001/onap/policy-drools:3.0.1 To run the container in detached mode, add the *-d* flag. -Note that in this command, we are opening the *9696* telemetry API port to the outside world, the config directory -(where the *noop.pre.sh* customization script resides) is mounted as /tmp/policy-install/config, -and the customization environment variables (*env/env.conf*) are passed into the container. +Note that in this command, we are opening the *9696* telemetry API port to the outside world, the +config directory (where the *noop.pre.sh* customization script resides) is mounted as +/tmp/policy-install/config, and the customization environment variables (*env/env.conf*) are passed +into the container. To open a shell into the PDP-D: @@ -1095,7 +1031,7 @@ To open a shell into the PDP-D: docker exec -it pdp-d bash -Once in the container, run tools such as *telemetry*, *db-migrator*, *policy* to look at the system state: +Once in the container, run tools such as *telemetry*, *policy* to look at the system state: To run the *telemetry shell* and other tools from the host: @@ -1113,7 +1049,7 @@ Sometimes a developer may want to start and stop the PDP-D manually: # start a bash - docker run --rm -p 9696:9696 -v ${PWD}/config:/tmp/policy-install/config --env-file ${PWD}/env/env.conf -it --name PDPD -h pdpd nexus3.onap.org:10001/onap/policy-drools:1.6.3 bash + docker run --rm -p 9696:9696 -v ${PWD}/config:/tmp/policy-install/config --env-file ${PWD}/env/env.conf -it --name PDPD -h pdpd nexus3.onap.org:10001/onap/policy-drools:3.0.1 bash # use this command to start policy applying host customizations from /tmp/policy-install/config @@ -1131,188 +1067,34 @@ Sometimes a developer may want to start and stop the PDP-D manually: policy start -Running PDP-D with nexus and mariadb -==================================== - -*docker-compose* can be used to test the PDP-D with other components. This is an example configuration -that brings up *nexus*, *mariadb* and the PDP-D (*docker-compose-pdp.yml*) - -docker-compose-pdp.yml -~~~~~~~~~~~~~~~~~~~~~~ - -.. code-block:: bash - - version: '3' - services: - mariadb: - image: mariadb:10.2.25 - container_name: mariadb - hostname: mariadb - command: ['--lower-case-table-names=1', '--wait_timeout=28800'] - env_file: - - ${PWD}/db/db.conf - volumes: - - ${PWD}/db:/docker-entrypoint-initdb.d - ports: - - "3306:3306" - nexus: - image: sonatype/nexus:2.14.8-01 - container_name: nexus - hostname: nexus - ports: - - "8081:8081" - drools: - image: nexus3.onap.org:10001/onap/policy-drools:1.6.3 - container_name: drools - depends_on: - - mariadb - - nexus - hostname: drools - ports: - - "9696:9696" - volumes: - - ${PWD}/config:/tmp/policy-install/config - env_file: - - ${PWD}/env/env.conf - -with *${PWD}/db/db.conf*: - -db.conf -~~~~~~~ - -.. code-block:: bash - - MYSQL_ROOT_PASSWORD=secret - MYSQL_USER=policy_user - MYSQL_PASSWORD=policy_user - -and *${PWD}/db/db.sh*: - -db.sh -~~~~~ - -.. code-block:: bash - - for db in support onap_sdk log migration operationshistory10 pooling policyadmin operationshistory - do - mysql -uroot -p"${MYSQL_ROOT_PASSWORD}" --execute "CREATE DATABASE IF NOT EXISTS ${db};" - mysql -uroot -p"${MYSQL_ROOT_PASSWORD}" --execute "GRANT ALL PRIVILEGES ON \`${db}\`.* TO '${MYSQL_USER}'@'%' ;" - done - - mysql -uroot -p"${MYSQL_ROOT_PASSWORD}" --execute "FLUSH PRIVILEGES;" - -env.conf -~~~~~~~~ - -The environment file *env/env.conf* for *PDP-D* can be set up with appropriate variables to point to the *nexus* instance -and the *mariadb* database: - -.. code-block:: bash - - # SYSTEM software configuration - - POLICY_HOME=/opt/app/policy - POLICY_LOGS=/var/log/onap/policy/pdpd - KEYSTORE_PASSWD=Pol1cy_0nap - TRUSTSTORE_PASSWD=Pol1cy_0nap - - # Telemetry credentials - - TELEMETRY_PORT=9696 - TELEMETRY_HOST=0.0.0.0 - TELEMETRY_USER=demo@people.osaaf.org - TELEMETRY_PASSWORD=demo123456! - - # nexus repository - - SNAPSHOT_REPOSITORY_ID=policy-nexus-snapshots - SNAPSHOT_REPOSITORY_URL=http://nexus:8081/nexus/content/repositories/snapshots/ - RELEASE_REPOSITORY_ID=policy-nexus-releases - RELEASE_REPOSITORY_URL=http://nexus:8081/nexus/content/repositories/releases/ - REPOSITORY_USERNAME=admin - REPOSITORY_PASSWORD=admin123 - REPOSITORY_OFFLINE=false - - MVN_SNAPSHOT_REPO_URL=https://nexus.onap.org/content/repositories/snapshots/ - MVN_RELEASE_REPO_URL=https://nexus.onap.org/content/repositories/releases/ - - # Relational (SQL) DB access - - SQL_HOST=mariadb - SQL_USER=policy_user - SQL_PASSWORD=policy_user +Running PDP-D with docker compose +================================= - # AAF - - AAF=false - AAF_NAMESPACE=org.onap.policy - AAF_HOST=aaf.api.simpledemo.onap.org - - # PDP-D DMaaP configuration channel - - PDPD_CONFIGURATION_TOPIC=PDPD-CONFIGURATION - PDPD_CONFIGURATION_API_KEY= - PDPD_CONFIGURATION_API_SECRET= - PDPD_CONFIGURATION_CONSUMER_GROUP= - PDPD_CONFIGURATION_CONSUMER_INSTANCE= - PDPD_CONFIGURATION_PARTITION_KEY= - - # PAP-PDP configuration channel - - POLICY_PDP_PAP_TOPIC=POLICY-PDP-PAP - POLICY_PDP_PAP_API_KEY= - POLICY_PDP_PAP_API_SECRET= - - -active.post.sh -~~~~~~~~~~~~~~ - -A post-start script *config/active.post.sh* can place PDP-D in *active* mode at initialization: - - .. code-block:: bash - - bash -c "http --verify=no -a ${TELEMETRY_USER}:${TELEMETRY_PASSWORD} PUT <http|https>://localhost:9696/policy/pdp/engine/lifecycle/state/ACTIVE" - -Bring up the PDP-D, nexus, and mariadb -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To bring up the containers: - -.. code-block:: bash - - docker-compose -f docker-compose-pdpd.yaml up -d - -To take it down: - -.. code-block:: bash - - docker-compose -f docker-compose-pdpd.yaml down -v +*docker-compose* can be used to test the PDP-D with other components. +Refer to the docker usage documentation at +`policy-docker <https://github.com/onap/policy-docker/tree/master/compose>`_ Other examples ~~~~~~~~~~~~~~ -The reader can also look at the `policy/docker repository <https://github.com/onap/policy-docker/tree/master/csit>`__. +The reader can also look at the `policy/docker repository <https://github.com/onap/policy-docker/tree/master/csit>`_. More specifically, these directories have examples of other PDP-D configurations: -* `plans <https://github.com/onap/policy-docker/tree/master/compose>`__: startup & teardown scripts. -* `scripts <https://github.com/onap/policy-docker/blob/master/compose/compose.yaml>`__: docker-compose file. -* `tests <https://github.com/onap/policy-docker/blob/master/csit/resources/tests/drools-pdp-test.robot>`__: test plan. +* `plans <https://github.com/onap/policy-docker/tree/master/compose>`_: startup & teardown scripts. +* `scripts <https://github.com/onap/policy-docker/blob/master/compose/compose.yaml>`_: docker-compose file. +* `tests <https://github.com/onap/policy-docker/blob/master/csit/resources/tests/drools-pdp-test.robot>`_: test plan. Configuring the PDP-D in an OOM Kubernetes installation ======================================================= -The `PDP-D OOM chart <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp>`__ can be -customized at the following locations: - -* `values.yaml <https://github.com/onap/oom/blob/master/kubernetes/policy/components/policy-drools-pdp/values.yaml>`__: custom values for your installation. -* `configmaps <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp/resources/configmaps>`__: place in this directory any configuration extensions or overrides to customize the PDP-D that does not contain sensitive information. -* `secrets <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp/resources/secrets>`__: place in this directory any configuration extensions or overrides to customize the PDP-D that does contain sensitive information. +The `PDP-D OOM chart <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp>`_ +can be customized at the following locations: -The same customization techniques described in the docker sections for PDP-D, fully apply here, by placing the corresponding -files or scripts in these two directories. +* `values.yaml <https://github.com/onap/oom/blob/master/kubernetes/policy/components/policy-drools-pdp/values.yaml>`_: custom values for your installation. +* `configmaps <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp/resources/configmaps>`_: place in this directory any configuration extensions or overrides to customize the PDP-D that does not contain sensitive information. +* `secrets <https://github.com/onap/oom/tree/master/kubernetes/policy/components/policy-drools-pdp/resources/secrets>`_: place in this directory any configuration extensions or overrides to customize the PDP-D that does contain sensitive information. -Additional information -====================== +The same customization techniques described in the docker sections for PDP-D, fully apply here, by +placing the corresponding files or scripts in these two directories. -For additional information, please see the -`Drools PDP Development and Testing (In Depth) <https://wiki.onap.org/display/DW/2020-08+Frankfurt+Tutorials>`__ page. +End of Document diff --git a/docs/drools/poolingDesign.png b/docs/drools/poolingDesign.png Binary files differdeleted file mode 100644 index 8040e809..00000000 --- a/docs/drools/poolingDesign.png +++ /dev/null diff --git a/docs/pap/pap.rst b/docs/pap/pap.rst index c2e38e53..ac21c786 100644 --- a/docs/pap/pap.rst +++ b/docs/pap/pap.rst @@ -97,7 +97,7 @@ The purpose of this API is to support CRUD of PDP groups and subgroups and to su policies on PDP sub groups and PDPs. This API is provided by the *PolicyAdministration* component (PAP) of the Policy Framework, see the :ref:`ONAP Policy Framework Architecture <architecture-label>` page. -PDP groups and subgroups may be prefedined in the system. Predefined groups and subgroups may be modified or deleted +PDP groups and subgroups may be predefined in the system. Predefined groups and subgroups may be modified or deleted over this API. The policies running on predefined groups or subgroups as well as the instance counts and properties may also be modified. @@ -150,8 +150,10 @@ Here is a sample notification: 2 PAP REST API Swagger ====================== -It is worth noting that we use basic authorization for access with user name and password set to *policyadmin* and -*zb!XztG34*, respectively. +.. note:: + PF uses basic authorization for access with user name and password, to be set on application.yaml + properties file. An example can be seen at + `the docker configuration papParameters.yaml <https://github.com/onap/policy-docker/blob/master/compose/config/pap/papParameters.yaml>`_ For every call, the client is encouraged to insert a uuid-type *requestID* as parameter. It is helpful for tracking each http transaction and facilitates debugging. More importantly, it complies with Logging requirements v1.2. If the client |
