diff options
Diffstat (limited to 'docs/development')
6 files changed, 151 insertions, 186 deletions
diff --git a/docs/development/devtools/smoke/acm-participants-smoke.rst b/docs/development/devtools/smoke/acm-participants-smoke.rst index 0b8061e2..ad377768 100644 --- a/docs/development/devtools/smoke/acm-participants-smoke.rst +++ b/docs/development/devtools/smoke/acm-participants-smoke.rst @@ -26,92 +26,71 @@ The procedure documented in this article has been verified using Ubuntu 20.04 LT 2.1 Prerequisites ================= -- Java 11 +- Java 17 - Docker -- Maven 3 +- Maven 3.9 - Git - helm3 - k8s cluster - Refer to this guide for basic environment setup `Setting up dev environment <https://wiki.onap.org/display/DW/Setting+Up+Your+Development+Environment>`_ -2.2 Assumptions -=============== -- You are accessing the policy repositories through gerrit. +2.2 Cloning CLAMP automation composition +======================================== -The following repositories are required for development in this project. These repositories should be present on your machine and you should run "mvn clean install" on all of them so that the packages are present in your .m2 repository. +Run a script such as the script below to clone the required modules from the `ONAP git repository <https://gerrit.onap.org/r/admin/repos/q/filter:policy>`_. This script clones CLAMP automation composition and all dependency. -- policy/parent -- policy/common -- policy/models -- policy/clamp +.. code-block:: bash -In this setup guide, we will be setting up all the components technically required for a working dev environment. + cd ~/git + git clone https://gerrit.onap.org/r/policy/clamp clamp -2.3 Setting up the components -============================= -2.3.1 MariaDB Setup -^^^^^^^^^^^^^^^^^^^ -We will be using Docker to run our mariadb instance. It will have the acm-runtime database running in it. +Execution of the command above results in the following directory hierarchy in your *~/git* directory: -- AutomationComposition: the runtime-acm db + * ~/git/clamp -The easiest way to do this is to perform a SQL script. Create the *mariadb.sql* file in the directory *~/git*. -.. code-block:: mysql +2.3 Setting up the components +============================= - CREATE DATABASE `clampacm`; - USE `clampacm`; - CREATE USER 'policy'; - GRANT ALL on clampacm.* to 'policy' identified by 'P01icY' with GRANT OPTION; +2.3.1 Running MariaDb and Kafka +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +We will be using Docker to run our mariadb instance and Kafka. It will have the acm-runtime database running in it. +The easiest way to do this is to perform a SQL script. Create the *mariadb.sql* file in the directory *~/git*. -Execution of the command above results in the creation and start of the *mariadb-smoke-test* container. +.. literalinclude:: files/mariadb.sql + :language: SQL - .. code-block:: bash +Create the '*docker-compose.yaml*' using following code: - cd ~/git - docker run --name mariadb-smoke-test \ - -p 3306:3306 \ - -e MYSQL_ROOT_PASSWORD=my-secret-pw \ - --mount type=bind,source=$HOME/git/mariadb.sql,target=/docker-entrypoint-initdb.d/data.sql \ - -d mariadb:10.10.2 \ - --lower-case-table-names=1 +.. literalinclude:: files/docker-compose-local.yaml + :language: yaml -The database will be exposed locally on port 3306 and will be backed by an anonymous docker volume. +Run the docker composition: -2.3.2 DMAAP Simulator -^^^^^^^^^^^^^^^^^^^^^ -For convenience, a dmaap simulator has been provided in the policy/models repository. To start the simulator, you can do the following: + .. code-block:: bash -1. Navigate to /models-sim/policy-models-simulators in the policy/models repository. -2. Add a configuration file to src/test/resources with the following contents: + cd ~/git/ + docker compose up -.. code-block:: json +2.3.2 Setting topicParameterGroup for kafka localhost +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +It needs to set 'kafka' as topicCommInfrastructure and 'localhost:29092' as server. +In the clamp repo, you should find the file 'runtime-acm/src/main/resources/application.yaml'. This file (in the 'runtime' parameters section) may need to be altered as below: - { - "dmaapProvider":{ - "name":"DMaaP simulator", - "topicSweepSec":900 - }, - "restServers":[ - { - "name":"DMaaP simulator", - "providerClass":"org.onap.policy.models.sim.dmaap.rest.DmaapSimRestControllerV1", - "host":"localhost", - "port":3904, - "https":false - } - ] - } - -3. You can then start dmaap with: +.. literalinclude:: files/runtime-application.yaml + :language: yaml -.. code-block:: bash +Same changes (in the 'participant' parameters section) +may need to be apply into the file 'participant/participant-impl/participant-impl-http/src/main/resources/config/application.yaml'. - mvn exec:java -Dexec.mainClass=org.onap.policy.models.simulators.Main -Dexec.args="src/test/resources/YOUR_CONF_FILE.json" +.. literalinclude:: files/participant-http-application.yaml + :language: yaml -At this stage the dmaap simulator should be running on your local machine on port 3904. +And into the file 'participant/participant-impl/participant-impl-kubernetes/src/main/resources/config/application.yaml'. +.. literalinclude:: files/participant-kubernetes-application.yaml + :language: yaml 2.3.3 Automation composition Runtime ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ @@ -245,7 +224,7 @@ Verify automation composition state: GET: https://<Runtime ACM IP> : <Port>/onap/policy/clamp/acm/v2/compositions/{compositionId}/instances/{instanceId} -3.4 Automation Compositions can be "UNDEPLOYED" after deployment +3.5 Automation Compositions can be "UNDEPLOYED" after deployment ================================================================ By changing the state to "UNDEPLOYED", all the helm deployments under the corresponding automation composition will be uninstalled from the cluster. diff --git a/docs/development/devtools/smoke/distribution-smoke.rst b/docs/development/devtools/smoke/distribution-smoke.rst index c0e3cb1d..ad57ee15 100644 --- a/docs/development/devtools/smoke/distribution-smoke.rst +++ b/docs/development/devtools/smoke/distribution-smoke.rst @@ -7,131 +7,72 @@ Policy Distribution Smoke Test ################################ -The policy-distribution smoke testing is executed against a custom ONAP docker installation as defined in the docker compose file in "policy/docker/csit/". +The policy-distribution smoke testing is executed against a custom ONAP docker or Kubernetes installation as defined in the docker compose file in "policy/docker/csit/". The policy-distribution configuration file is located in "docker/csit/config/distribution/". -This test verifies the execution of the REST api's exposed by the component to make sure the CSAR Decoding and Forwarding works as expected. +This test verifies the execution of the REST API's exposed by the component to make sure the CSAR Decoding and Forwarding works as expected. +Also, the deployment of the policy is checked automatically. An event is then sent to the deployed policy, and thus, its correct operation is verified. -General Setup -***************** -In policy/docker/csit/ - -.. code-block:: bash - - ./start-grafana.sh distribution - -This script will compose the ONAP components used during the smoke tests are: - -- Policy API to perform CRUD of policies. -- Policy DB to store the policies, and DB Migrator to start the db. -- DMAAP Simulator for the communication between components. -- Policy PAP to perform runtime administration (deploy/undeploy/status/statistics/etc). -- Policy Apex-PDP to deploy & undeploy policies. And send heartbeats to PAP. -- Policy Drools-PDP to deploy & undeploy policies. And send heartbeats to PAP. -- Policy Xacml-PDP to deploy & undeploy policies. And send heartbeats to PAP. - -- Policy Distribution to test the Decoding and Forwarding functions. - -Use this script to easily bring down the containers : - -.. code-block:: bash - - ./stop-grafana.sh - -Testing procedure -********************** - -The test set is focused on the following use cases: - -- Wait until Distribution starts and reach the built-in REST endpoints for fetching healthcheck & statistics. -- Execute some of the REST api's exposed by policy-pap component. - -Starting Policy Distribution ------------------------------------- - -Check the docker logs to see when Distribution service is up and running. - -Get the ips of distribution and pap services: +There are 2 alternative ways to carry out the tests. In Docker and in Kubernetes -.. code:: - :number-lines: +Docker Setup +************ +In policy/docker/csit/ - ./get-instance-ip.sh policy-distribution - ./get-instance-ip.sh policy-pap +More detailed setup for docker CSIT is here -Health check status & statistical data of running distribution system. +`Policy CSIT Test Install Docker <https://docs.onap.org/projects/onap-policy-parent/en/latest/development/devtools/testing/csit.html>`_ .. code-block:: bash - curl -u 'healthcheck:zb!XztG34' --basic http://{POLICY_DISTRIBUTION_IP}:6969/healthcheck - curl -u 'healthcheck:zb!XztG34' --basic http://{POLICY_DISTRIBUTION_IP}:6969/statistics - -Expected result for healthcheck - -.. code-block:: json - - {"name":"Policy SSD","url":"policy-distribution","healthy":true,"code":200,"message":"alive"} - -Expected result for statistics + ./run-project-csit.sh distribution -.. code-block:: json +This script will do the following: - {"code":200,"totalDistributionCount":0,"distributionSuccessCount":0,"distributionFailureCount":0,"totalDownloadCount":0,"downloadSuccessCount":0,"downloadFailureCount":0} +- Deploys all required components - including api, pap and distribution +- Runs distributions CSIT tests. These tests: -Trigger Policy Distribution Core ------------------------------------------- + - Take a policy from a CSAR file in distribution. + - Distribution sends and deploys it in API and PAP. + - Send an event to the deployed policy - if the policy is NOT deployed successfully, this will FAIL with a 404. -In order to test policy-distribution, we need to trigger the decoding copying a .csar in the mapped volume, -defined in the docker-compose-distribution-smoke.yml as : +- Takes down all the components. +- Saves the results in a .html file in the csit/archives/distribution directory +- Saves the docker compose logs in the same directory -.. code-block:: yaml - - volumes: - - ./distribution/config/temp/:/opt/app/policy/distribution/etc/temp/ - -So now copy the "sample_csar_with_apex_policy.csar" from ./distribution/config/csar/ to ./distribution/config/temp/ - -If the commissioning is successful we should see from the logs this message +K8S Setup +********* +In policy/docker/csit/ -.. image:: images/message-commissioning-participant.png +More detailed setup for K8S CSIT is here -So if we check the distribution statistics again +`Policy CSIT Test Install Kubernetes <https://docs.onap.org/projects/onap-policy-parent/en/latest/development/devtools/testing/csit.html>`_ .. code-block:: bash - {"code":200,"totalDistributionCount":1,"distributionSuccessCount":1,"distributionFailureCount":0,"totalDownloadCount":1,"downloadSuccessCount":1,"downloadFailureCount":0} + ./run-k8s-csit.sh install distribution -Execute policy-pap testing ------------------------------------- -.. note:: - The user for pap is different. +This script will do the following: -Check the details of policies deployed +- Installs microk8s and configures it. +- Deploys all required components - including api, pap and distribution as helm charts +- Waits for all the charts to come up +- Runs distributions CSIT tests. These tests: -.. code-block:: bash - - curl -k --user 'policyadmin:zb!XztG34' http://{POLICY_PAP_IP}:6969/policy/pap/v1/policies/status - -Expected SUCCESS result - -.. code-block:: json + - Take a policy from a CSAR file in distribution. + - Distribution sends and deploys it in API and PAP. + - Sends an event to the deployed policy - if the policy is NOT deployed successfully, this will FAIL with a 404. - [{"pdpGroup":"defaultGroup","pdpType":"apex","pdpId":"apex-91fa25a1-0456-42fa-9556-6a4d2bd613fc","policy":{"name":"operational.apex.sampledomain","version":"1.0.0"},"policyType":{"name":"onap.policies.native.Apex","version":"1.0.0"},"deploy":true,"state":"SUCCESS"},{"pdpGroup":"defaultGroup","pdpType":"xacml","pdpId":"xacml-83e19452-0854-41dd-9f17-8b0a68f11813","policy":{"name":"SDNC_Policy.ONAP_NF_NAMING_TIMESTAMP","version":"1.0.0"},"policyType":{"name":"onap.policies.Naming","version":"1.0.0"},"deploy":true,"state":"SUCCESS"}] +- Saves the results in a .html file in the csit/archives/distribution directory +- The pods are not automatically taken down. -Check number of policies deployed +To uninstall policy helm deployment and/or the microk8s cluster, use `run-k8s-csit.sh` .. code-block:: bash - curl -k --user 'policyadmin:zb!XztG34' http://{POLICY_PAP_IP}:6969/policy/pap/v1/policies/deployed - -Expected success-count result - -.. code-block:: json - - [{"policy-type":"onap.policies.native.Apex","policy-type-version":"1.0.0","policy-id":"operational.apex.sampledomain","policy-version":"1.0.0","success-count":1,"failure-count":0,"incomplete-count":0},{"policy-type":"onap.policies.Naming","policy-type-version":"1.0.0","policy-id":"SDNC_Policy.ONAP_NF_NAMING_TIMESTAMP","policy-version":"1.0.0","success-count":1,"failure-count":0,"incomplete-count":0}] - -Or download & execute the steps in postman collection for verifying policy-pap component. -The steps need to be performed sequentially one after another. And no input is required from user. + # to uninstall deployment + ./run-k8s-csit.sh uninstall -`Policy Framework Administration API <https://github.com/onap/policy-pap/blob/master/postman/pap-api-collection.json>`_ + # to remove cluster + ./run-k8s-csit.sh clean -Make sure to execute the delete steps in order to clean the setup after testing. +End of document
\ No newline at end of file diff --git a/docs/development/devtools/smoke/files/participant-http-application.yaml b/docs/development/devtools/smoke/files/participant-http-application.yaml new file mode 100644 index 00000000..142c24e5 --- /dev/null +++ b/docs/development/devtools/smoke/files/participant-http-application.yaml @@ -0,0 +1,23 @@ + +participant: + intermediaryParameters: + reportingTimeIntervalMs: 120000 + description: Participant Description + participantId: 101c62b3-8918-41b9-a747-d21eb79c6c01 + clampAutomationCompositionTopics: + topicSources: + - topic: policy-acruntime-participant + servers: + - ${topicServer:localhost:29092} + topicCommInfrastructure: kafka + fetchTimeout: 15000 + topicSinks: + - topic: policy-acruntime-participant + servers: + - ${topicServer:localhost:29092} + topicCommInfrastructure: kafka + participantSupportedElementTypes: + - + typeName: org.onap.policy.clamp.acm.HttpAutomationCompositionElement + typeVersion: 1.0.0 + diff --git a/docs/development/devtools/smoke/files/participant-kubernetes-application.yaml b/docs/development/devtools/smoke/files/participant-kubernetes-application.yaml new file mode 100644 index 00000000..9b25c615 --- /dev/null +++ b/docs/development/devtools/smoke/files/participant-kubernetes-application.yaml @@ -0,0 +1,26 @@ + +participant: + localChartDirectory: /home/policy/local-charts + infoFileName: CHART_INFO.json + intermediaryParameters: + reportingTimeIntervalMs: 120000 + description: Participant Description + participantId: 101c62b3-8918-41b9-a747-d21eb79c6c02 + clampAutomationCompositionTopics: + topicSources: + - + topic: policy-acruntime-participant + servers: + - ${topicServer:localhost:29092} + topicCommInfrastructure: kafka + fetchTimeout: 15000 + topicSinks: + - + topic: policy-acruntime-participant + servers: + - ${topicServer:localhost:29092} + topicCommInfrastructure: kafka + participantSupportedElementTypes: + - + typeName: org.onap.policy.clamp.acm.K8SMicroserviceAutomationCompositionElement + typeVersion: 1.0.0 diff --git a/docs/development/devtools/smoke/xacml-smoke.rst b/docs/development/devtools/smoke/xacml-smoke.rst index 2a0be4b6..61f3551f 100644 --- a/docs/development/devtools/smoke/xacml-smoke.rst +++ b/docs/development/devtools/smoke/xacml-smoke.rst @@ -10,37 +10,33 @@ XACML PDP Smoke Test ~~~~~~~~~~~~~~~~~~~~ -The policy-xacml-pdp smoke testing can be executed against a kubernetes based ONAP installation, +The policy-xacml-pdp smoke testing can be executed against a kubernetes based policy framework installation, and/or a docker-compose set up similar to the one executed by CSIT tests. General Setup ************* -ONAP OOM kubernetes -------------------- +PF kubernetes Install +--------------------- For installation instructions, please refer to the following documentation: -`OOM Documentation <https://docs.onap.org/projects/onap-oom/en/latest/>`_ +`Policy Framework K8S Install <https://docs.onap.org/projects/onap-policy-parent/en/latest/development/devtools/testing/csit.html>`_ -At a minimum policy needs the following components installed: - -- onap base charts -- AAF for certificate generation -- DMaaP message-router for communication among policy components. +The script referred to in the above link should handle the install of the of microk8s, docker and other required components for the install of the policy framework and clamp components. The scripts are used by policy as a means to run the CSIT tests in Kubernetes. docker-compose based -------------------- -A smaller testing environment can be put together by replicating the CSIT test environment: +A smaller testing environment can be put together by replicating the docker-based CSIT test environment. Details are on the same page as K8s setup: -`Policy CSIT Test infrastructure <https://git.onap.org/policy/docker/tree/csit>`_ +`Policy CSIT Test Install Docker <https://docs.onap.org/projects/onap-policy-parent/en/latest/development/devtools/testing/csit.html>`_ Testing procedures ****************** The smoke tests should be focused on verifying the proper workings of the xacml -PDP and dependent components. The following scenarios should be considered: +PDP and dependent components. The following scenarios should be considered: - PDP-X registration with PAP. - PDP-X restarts and re-registration with PAP. diff --git a/docs/development/property-configuration.rst b/docs/development/property-configuration.rst index 39691ade..d89b4769 100644 --- a/docs/development/property-configuration.rst +++ b/docs/development/property-configuration.rst @@ -10,13 +10,13 @@ Property-configuration mechanisms .. contents:: :depth: 3 -This article explains how to implement handling and validation of common parameter into the Policy Framework Components. +This article explains how to implement handling and validation of common parameters into the Policy Framework Components. Not Spring boot framework ************************* -The application should have a ParameterHandler class to support the map values from Json to a POJO, so it should be load the file, convert it performing all type conversion. +The application should have a ParameterHandler class to support the map of values from Json to a POJO; so it should load the file and convert it; performing all type conversion. -The code below shown an example of ParameterHandler: +The code below shown is an example of ParameterHandler: .. code-block:: java @@ -59,9 +59,9 @@ The code below shown an example of ParameterHandler: } -The POJO have to implement **org.onap.policy.common.parameters.ParameterGroup** interface or eventually extend **org.onap.policy.common.parameters.ParameterGroupImpl**. The last one already implements **validate()** method that performs error checking using validation **org.onap.policy.common.parameters.annotations**. +The POJO has to implement the **org.onap.policy.common.parameters.ParameterGroup** interface or eventually extend **org.onap.policy.common.parameters.ParameterGroupImpl**. The last one already implements the **validate()** method that performs error checking using validation **org.onap.policy.common.parameters.annotations**. -The code below shown an example of POJO: +The code below shows an example of the POJO: .. code-block:: java @@ -87,7 +87,7 @@ The code below shown an example of POJO: } -The code shows below, is an example of Unit Test validation of the POJO PapParameterGroup: +The code shown below, is an example of Unit Test validation of the POJO PapParameterGroup: .. code-block:: java @@ -106,13 +106,13 @@ The code shows below, is an example of Unit Test validation of the POJO PapParam Using Spring boot framework *************************** -Spring loads automatically the property file and put it available under the **org.springframework.core.env.Environment** Spring component. +Spring loads the property file automatically and makes it available under the **org.springframework.core.env.Environment** Spring component. Environment +++++++++++ A component can use Environment component directly. -Environment component is not a good approach because there is not type conversion and error checking, but it could be useful when the name of the property you need to access changes dynamically. +The Environment component is not a good approach because there is no type conversion or error checking, but it could be useful when the name of the property you need to access changes dynamically. .. code-block:: java @@ -134,7 +134,7 @@ Annotation-based Spring configuration All annotation-based Spring configurations support the Spring Expression Language (SpEL), a powerful expression language that supports querying and manipulating an object graph at runtime. A documentation about SpEL could be found here: https://docs.spring.io/spring-framework/docs/3.0.x/reference/expressions.html. -A component can use **org.springframework.beans.factory.annotation.Value**, which reads from properties, performs a type conversion and injects the value into the filed. There is not error checking, but it can assign default value if the property is not defined. +A component can use **org.springframework.beans.factory.annotation.Value**, which reads from properties, performs a type conversion and injects the value into the field. There is no error checking, but it can assign a default value if the property is not defined. .. code-block:: java @@ -178,9 +178,9 @@ ConfigurationProperties private String description; } -In a scenario that we need to include into a POJO shown before, a class that implement **ParameterGroup** interface, we need to add the **org.onap.policy.common.parameters.validation.ParameterGroupConstraint** annotation. That annotation is configured to use **ParameterGroupValidator** that handles the conversion of a **org.onap.policy.common.parameters.BeanValidationResult** to a Spring validation. +In a scenario where we need to include the properties in a POJO, as shown before, in a class that implements **ParameterGroup** interface, we need to add the **org.onap.policy.common.parameters.validation.ParameterGroupConstraint** annotation. That annotation is configured to use **ParameterGroupValidator**, which handles the conversion of a **org.onap.policy.common.parameters.BeanValidationResult** to a Spring validation. -The code below shown how to add TopicParameterGroup parameter into ClRuntimeParameterGroup: +The code below shows how to add the TopicParameterGroup parameter into acRuntimeParameterGroup: .. code-block:: java @@ -197,7 +197,7 @@ A bean configured with ConfigurationProperties, is automatically a Spring compon @RequiredArgsConstructor public class Example { - private ClRuntimeParameterGroup parameters; + private acRuntimeParameterGroup parameters; .... public void method() { @@ -206,7 +206,7 @@ A bean configured with ConfigurationProperties, is automatically a Spring compon ..... } -The code shows below, is an example of Unit Test validation of the POJO ClRuntimeParameterGroup: +The code shown below, is an example of Unit Test validation of the POJO acRuntimeParameterGroup: .. code-block:: java @@ -214,7 +214,7 @@ The code shows below, is an example of Unit Test validation of the POJO ClRuntim @Test void testParameters_NullTopicParameterGroup() { - final ClRuntimeParameterGroup parameters = CommonTestData.geParameterGroup(); + final acRuntimeParameterGroup parameters = CommonTestData.geParameterGroup(); parameters.setTopicParameterGroup(null); assertThat(validatorFactory.getValidator().validate(parameters)).isNotEmpty(); } |