diff options
Diffstat (limited to 'docs/development/devtools/smoke/policy-participant-smoke.rst')
| -rw-r--r-- | docs/development/devtools/smoke/policy-participant-smoke.rst | 581 |
1 files changed, 324 insertions, 257 deletions
diff --git a/docs/development/devtools/smoke/policy-participant-smoke.rst b/docs/development/devtools/smoke/policy-participant-smoke.rst index 58e34494..ba677e11 100644 --- a/docs/development/devtools/smoke/policy-participant-smoke.rst +++ b/docs/development/devtools/smoke/policy-participant-smoke.rst @@ -11,21 +11,23 @@ CLAMP Policy Participant Smoke Tests The Smoke testing of the policy participant is executed in a local CLAMP/Policy environment. The CLAMP-ACM interfaces interact with the Policy Framework to perform actions based on the state of the policy participant. The goal of the Smoke tests is the ensure that CLAMP Policy Participant and Policy Framework work together as expected. All applications will be running by console, so they need to run with different ports. Configuration files should be changed accordingly. -+------------------------------+------+ -| Application | port | -+==============================+======+ -| MariDB | 3306 | -+------------------------------+------+ -| DMaaP simulator | 3904 | -+------------------------------+------+ -| policy-api | 6968 | -+------------------------------+------+ -| policy-pap | 6970 | -+------------------------------+------+ -| policy-clamp-runtime-acm | 6969 | -+------------------------------+------+ -| onap/policy-clamp-ac-pf-ppnt | 8085 | -+------------------------------+------+ ++------------------------------+-------+ +| Application | port | ++==============================+=======+ +| MariDB | 3306 | ++------------------------------+-------+ +| Zookeeper | 2181 | ++------------------------------+-------+ +| Kafka | 29092 | ++------------------------------+-------+ +| policy-api | 6968 | ++------------------------------+-------+ +| policy-pap | 6970 | ++------------------------------+-------+ +| policy-clamp-runtime-acm | 6969 | ++------------------------------+-------+ +| onap/policy-clamp-ac-pf-ppnt | 8085 | ++------------------------------+-------+ 2. Setup Guide @@ -36,301 +38,366 @@ This section will show the developer how to set up their environment to start te 2.1 Prerequisites ================= -- Java 11 -- Maven 3 +- Java 17 +- Maven 3.9 - Git - Refer to this guide for basic environment setup `Setting up dev environment <https://wiki.onap.org/display/DW/Setting+Up+Your+Development+Environment>`_ -2.2 Assumptions -=============== +2.2 Cloning CLAMP automation composition and all dependency +=========================================================== -- You are accessing the policy repositories through gerrit -- You are using "git review". +Run a script such as the script below to clone the required modules from the `ONAP git repository <https://gerrit.onap.org/r/admin/repos/q/filter:policy>`_. This script clones CLAMP automation composition and all dependency. -The following repositories are required for development in this project. These repositories should be present on your machine and you should run "mvn clean install" on all of them so that the packages are present in your .m2 repository. +.. code-block:: bash + :caption: Typical ONAP Policy Framework Clone Script + :linenos: -- policy/parent -- policy/common -- policy/models -- policy/clamp -- policy/api -- policy/pap + #!/usr/bin/env bash -In this setup guide, we will be setting up all the components technically required for a working convenient dev environment. + ## script name for output + MOD_SCRIPT_NAME='basename $0' -2.3 Setting up the components -============================= + ## the ONAP clone directory, defaults to "onap" + clone_dir="onap" -2.3.1 MariaDB Setup -^^^^^^^^^^^^^^^^^^^ + ## the ONAP repos to clone + onap_repos="\ + policy/api \ + policy/clamp \ + policy/pap " -We will be using Docker to run our mariadb instance. It will have a total of two databases running in it. + ## + ## Help screen and exit condition (i.e. too few arguments) + ## + Help() + { + echo "" + echo "$MOD_SCRIPT_NAME - clones all required ONAP git repositories" + echo "" + echo " Usage: $MOD_SCRIPT_NAME [-options]" + echo "" + echo " Options" + echo " -d - the ONAP clone directory, defaults to '.'" + echo " -h - this help screen" + echo "" + exit 255; + } -- clampacm: the policy-clamp-runtime-acm db -- policyadmin: the policy-api db + ## + ## read command line + ## + while [ $# -gt 0 ] + do + case $1 in + #-d ONAP clone directory + -d) + shift + if [ -z "$1" ]; then + echo "$MOD_SCRIPT_NAME: no clone directory" + exit 1 + fi + clone_dir=$1 + shift + ;; -A sql such as the one below can be used to build the SQL initialization. Create the *mariadb.sql* file in a directory *PATH_DIRECTORY*. + #-h prints help and exists + -h) + Help;exit 0;; - .. code-block:: SQL + *) echo "$MOD_SCRIPT_NAME: undefined CLI option - $1"; exit 255;; + esac + done + + if [ -f "$clone_dir" ]; then + echo "$MOD_SCRIPT_NAME: requested clone directory '$clone_dir' exists as file" + exit 2 + fi + if [ -d "$clone_dir" ]; then + echo "$MOD_SCRIPT_NAME: requested clone directory '$clone_dir' exists as directory" + exit 2 + fi + + mkdir $clone_dir + if [ $? != 0 ] + then + echo cannot clone ONAP repositories, could not create directory '"'$clone_dir'"' + exit 3 + fi + + for repo in $onap_repos + do + repoDir=`dirname "$repo"` + repoName=`basename "$repo"` + + if [ ! -z $dirName ] + then + mkdir "$clone_dir/$repoDir" + if [ $? != 0 ] + then + echo cannot clone ONAP repositories, could not create directory '"'$clone_dir/repoDir'"' + exit 4 + fi + fi + + git clone https://gerrit.onap.org/r/${repo} $clone_dir/$repo + done + + echo ONAP has been cloned into '"'$clone_dir'"' + + +Execution of the script above results in the following directory hierarchy in your *~/git* directory: + + * ~/git/onap + * ~/git/onap/policy + * ~/git/onap/policy/api + * ~/git/onap/policy/clamp + * ~/git/onap/policy/pap + + +2.3 Building CLAMP automation composition and all dependency +============================================================ + +**Step 1:** Setting topicParameterGroup for kafka localhost in clamp and policy-participant. +It needs to set 'kafka' as topicCommInfrastructure and 'localhost:29092' as server. +In the clamp repo, you should find the file 'runtime-acm/src/main/resources/application.yaml'. This file (in the 'runtime' parameters section) may need to be altered as below: + +.. literalinclude:: files/runtime-application.yaml + :language: yaml + +Setting topicParameterGroup for kafka localhost and api/pap http client (in the 'participant' parameters section) may need to be apply into the file 'participant/participant-impl/participant-impl-policy/src/main/resources/config/application.yaml'. + +.. literalinclude:: files/participant-policy-application.yaml + :language: yaml + + +**Step 2:** Setting datasource.url, hibernate.ddl-auto and server.port in policy-api. +In the api repo, you should find the file 'main/src/main/resources/application.yaml'. This file may need to be altered as below: + +.. literalinclude:: files/api-application.yaml + :language: yaml - create database clampacm; - CREATE USER 'policy'@'%' IDENTIFIED BY 'P01icY'; - GRANT ALL PRIVILEGES ON clampacm.* TO 'policy'@'%'; - CREATE DATABASE `policyadmin`; - CREATE USER 'policy_user'@'%' IDENTIFIED BY 'policy_user'; - GRANT ALL PRIVILEGES ON policyadmin.* to 'policy_user'@'%'; - FLUSH PRIVILEGES; +**Step 3:** Setting datasource.url, server.port, and api http client in policy-pap. +In the pap repo, you should find the file 'main/src/main/resources/application.yaml'. This file may need to be altered as below: -Execution of the command above results in the creation and start of the *mariadb-smoke-test* container. +.. literalinclude:: files/pap-application.yaml + :language: yaml + + +**Step 4:** Optionally, for a completely clean build, remove the ONAP built modules from your local repository. .. code-block:: bash - docker run --name mariadb-smoke-test \ - -p 3306:3306 \ - -e MYSQL_ROOT_PASSWORD=my-secret-pw \ - --mount type=bind,source=PATH_DIRECTORY/mariadb.sql,target=/docker-entrypoint-initdb.d/data.sql \ - -d mariadb:10.10.2 \ - --lower-case-table-names=1 + rm -fr ~/.m2/repository/org/onap -This will setup the two databases needed. The database will be exposed locally on port 3306. -2.3.2 DMAAP Simulator -^^^^^^^^^^^^^^^^^^^^^ +**Step 5:** A pom such as the one below can be used to build the ONAP Policy Framework modules. Create the *pom.xml* file in the directory *~/git/onap/policy*. -For convenience, a dmaap simulator has been provided in the policy/models repository. To start the simulator, you can do the following: +.. code-block:: xml + :caption: Typical pom.xml to build the ONAP Policy Framework + :linenos: -#. Navigate to models-sim/policy-models-simulators in the policy/models repository. -#. Add a configuration file to src/test/resources with the following contents: + <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> + <modelVersion>4.0.0</modelVersion> + <groupId>org.onap</groupId> + <artifactId>onap-policy</artifactId> + <version>1.0.0-SNAPSHOT</version> + <packaging>pom</packaging> + <name>${project.artifactId}</name> + <inceptionYear>2024</inceptionYear> + <organization> + <name>ONAP</name> + </organization> -.. code-block:: json + <modules> + <module>api</module> + <module>clamp</module> + <module>pap</module> + </modules> + </project> - { - "dmaapProvider":{ - "name":"DMaaP simulator", - "topicSweepSec":900 - }, - "restServers":[ - { - "name":"DMaaP simulator", - "providerClass":"org.onap.policy.models.sim.dmaap.rest.DmaapSimRestControllerV1", - "host":"localhost", - "port":3904, - "https":false - } - ] - } -3. You can then start dmaap with: +**Step 6:** You can now build the Policy framework. -.. code-block:: bash +Build java artifacts only: + + .. code-block:: bash + + cd ~/git/onap/policy + mvn clean install -DskipTests + +Build with docker images: + + .. code-block:: bash + + cd ~/git/onap/policy/clamp/packages/ + mvn clean install -P docker + + cd ~/git/onap/policy/api/packages/ + mvn clean install -P docker + + cd ~/git/onap/policy/pap/packages/ + mvn clean install -P docker + +2.4 Setting up the components +============================= - mvn exec:java -Dexec.mainClass=org.onap.policy.models.simulators.Main -Dexec.args="src/test/resources/YOUR_CONF_FILE.json" +2.4.1 MariaDB and Kafka Setup +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -At this stage the dmaap simulator should be running on your local machine on port 3904. +We will be using Docker to run our mariadb instance`and Zookeeper/Kafka. It will have a total of two databases running in mariadb. -2.3.3 Policy API +- clampacm: the policy-clamp-runtime-acm db +- policyadmin: the policy-api db + +**Step 1:** Create the *mariadb.sql* file in a directory *~/git*. + + .. code-block:: SQL + + create database clampacm; + CREATE USER 'policy'@'%' IDENTIFIED BY 'P01icY'; + GRANT ALL PRIVILEGES ON clampacm.* TO 'policy'@'%'; + CREATE DATABASE `policyadmin`; + CREATE USER 'policy_user'@'%' IDENTIFIED BY 'policy_user'; + GRANT ALL PRIVILEGES ON policyadmin.* to 'policy_user'@'%'; + CREATE DATABASE `migration`; + GRANT ALL PRIVILEGES ON migration.* to 'policy_user'@'%'; + FLUSH PRIVILEGES; + + +**Step 2:** Create the *init.sh* file in a directory *~/git* with execution permission. + + .. code-block:: sh + + #!/bin/sh + + export POLICY_HOME=/opt/app/policy + export SQL_USER=${MYSQL_USER} + export SQL_PASSWORD=${MYSQL_PASSWORD} + export SCRIPT_DIRECTORY=sql + + /opt/app/policy/bin/prepare_upgrade.sh ${SQL_DB} + /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o report + /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o upgrade + rc=$? + /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o report + nc -l -p 6824 + exit $rc + + +**Step 3:** Create the *wait_for_port.sh* file in a directory *~/git* with execution permission. + + .. code-block:: sh + + #!/bin/sh + + usage() { + echo args: [-t timeout] [-c command] hostname1 port1 hostname2 port2 ... >&2 + exit 1 + } + tmout=300 + cmd= + while getopts c:t: opt + do + case "$opt" in + c) + cmd="$OPTARG" + ;; + t) + tmout="$OPTARG" + ;; + *) + usage + ;; + esac + done + nargs=$((OPTIND-1)) + shift "$nargs" + even_args=$(($#%2)) + if [ $# -lt 2 ] || [ "$even_args" -ne 0 ] + then + usage + fi + while [ $# -ge 2 ] + do + export host="$1" + export port="$2" + shift + shift + echo "Waiting for $host port $port..." + + while [ "$tmout" -gt 0 ] + do + if command -v docker > /dev/null 2>&1 + then + docker ps --format "table {{ .Names }}\t{{ .Status }}" + fi + nc -vz "$host" "$port" + rc=$? + if [ $rc -eq 0 ] + then + break + else + tmout=$((tmout-1)) + sleep 1 + fi + done + if [ $rc -ne 0 ] + then + echo "$host port $port cannot be reached" + exit $rc + fi + done + $cmd + exit 0 + + +**Step 4:** Create the '*docker-compose.yaml*' using following code: + +.. literalinclude:: files/docker-compose-policy.yaml + :language: yaml + + +**Step 5:** Run the docker composition: + + .. code-block:: bash + + cd ~/git/ + docker compose up + + +2.4.2 Policy API ^^^^^^^^^^^^^^^^ -Navigate to the "/main" directory. You can then run the following command to start the policy api: +In the policy-api repo, navigate to the "/main" directory. You can then run the following command to start the policy api: .. code-block:: bash - java -jar target/api-main-2.8.2-SNAPSHOT.jar --spring.datasource.url=jdbc:mariadb://localhost:3306/policyadmin --spring.jpa.hibernate.ddl-auto=update --server.port=6968 + mvn spring-boot:run -2.3.4 Policy PAP +2.4.3 Policy PAP ^^^^^^^^^^^^^^^^ -In the policy-pap repo, you should find the file 'main\src\main\resources\application.yaml'. This file may need to be altered slightly as below: - -.. code-block:: yaml - - spring: - security: - user: - name: policyadmin - password: zb!XztG34 - http: - converters: - preferred-json-mapper: gson - datasource: - url: jdbc:mariadb://localhost:3306/policyadmin - driverClassName: org.mariadb.jdbc.Driver - username: policy_user - password: policy_user - jpa: - properties: - hibernate: - dialect: org.hibernate.dialect.MariaDB103Dialect - hibernate: - ddl-auto: update - naming: - physical-strategy: org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl - implicit-strategy: org.onap.policy.common.spring.utils.CustomImplicitNamingStrategy - - server: - port: 6970 - servlet: - context-path: /policy/pap/v1 - pap: - name: PapGroup - aaf: false - topic: - pdp-pap.name: POLICY-PDP-PAP - notification.name: POLICY-NOTIFICATION - heartbeat.name: POLICY-HEARTBEAT - pdpParameters: - heartBeatMs: 120000 - updateParameters: - maxRetryCount: 1 - maxWaitMs: 30000 - stateChangeParameters: - maxRetryCount: 1 - maxWaitMs: 30000 - savePdpStatisticsInDb: true - topicParameterGroup: - topicSources: - - topic: ${pap.topic.pdp-pap.name} - servers: - - localhost - topicCommInfrastructure: dmaap - fetchTimeout: 15000 - - topic: ${pap.topic.heartbeat.name} - effectiveTopic: ${pap.topic.pdp-pap.name} - consumerGroup: policy-pap - servers: - - localhost - topicCommInfrastructure: dmaap - fetchTimeout: 15000 - topicSinks: - - topic: ${pap.topic.pdp-pap.name} - servers: - - localhost - topicCommInfrastructure: dmaap - - topic: ${pap.topic.notification.name} - servers: - - localhost - topicCommInfrastructure: dmaap - healthCheckRestClientParameters: - - clientName: api - hostname: localhost - port: 6968 - userName: policyadmin - password: zb!XztG34 - useHttps: false - basePath: policy/api/v1/healthcheck - - clientName: distribution - hostname: policy-distribution - port: 6969 - userName: healthcheck - password: zb!XztG34 - useHttps: false - basePath: healthcheck - - clientName: dmaap - hostname: localhost - port: 3904 - useHttps: false - basePath: topics - - management: - endpoints: - web: - base-path: / - exposure: - include: health, metrics, prometheus - path-mapping.metrics: plain-metrics - path-mapping.prometheus: metrics - -Next, navigate to the "/main" directory. You can then run the following command to start the policy pap +In the policy-pap repo, navigate to the "/main" directory. You can then run the following command to start the policy pap: .. code-block:: bash mvn spring-boot:run -2.3.5 ACM Runtime +2.4.4 ACM Runtime ^^^^^^^^^^^^^^^^^ -To start the clampacm runtime we need to go the "runtime-acm" directory in the clamp repo. There is a config file that is used, by default, for the clampacm runtime. That config file is here: "src/main/resources/application.yaml". For development in your local environment, it shouldn't need any adjustment and we can just run the clampacm runtime with: +To start the clampacm runtime we need to go the "runtime-acm" directory in the clamp repo. You can then run the following command to start the clampacm runtime: .. code-block:: bash mvn spring-boot:run -2.3.6 ACM Policy Participant +2.4.5 ACM Policy Participant ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -To start the policy participant we need to go to the "participant/participant-impl/participant-impl-policy" directory in the clamp repo. There is a config file under "src/main/resources/config/application.yaml". For development in your local environment, we will need to adjust this file slightly: - -.. code-block:: yaml - - spring: - security: - user: - name: participantUser - password: zb!XztG34 - autoconfigure: - exclude: - - org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration - - org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration - - org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration - - org.springframework.boot.autoconfigure.data.web.SpringDataWebAutoConfiguration - - participant: - pdpGroup: defaultGroup - pdpType: apex - policyApiParameters: - clientName: api - hostname: localhost - port: 6968 - userName: policyadmin - password: zb!XztG34 - useHttps: false - allowSelfSignedCerts: true - policyPapParameters: - clientName: pap - hostname: localhost - port: 6970 - userName: policyadmin - password: zb!XztG34 - useHttps: false - allowSelfSignedCerts: true - intermediaryParameters: - reportingTimeIntervalMs: 120000 - description: Participant Description - participantId: 101c62b3-8918-41b9-a747-d21eb79c6c03 - clampAutomationCompositionTopics: - topicSources: - - - topic: POLICY-ACRUNTIME-PARTICIPANT - servers: - - ${topicServer:localhost} - topicCommInfrastructure: dmaap - fetchTimeout: 15000 - topicSinks: - - - topic: POLICY-ACRUNTIME-PARTICIPANT - servers: - - ${topicServer:localhost} - topicCommInfrastructure: dmaap - participantSupportedElementTypes: - - - typeName: org.onap.policy.clamp.acm.PolicyAutomationCompositionElement - typeVersion: 1.0.0 - - management: - endpoints: - web: - base-path: / - exposure: - include: health, metrics, prometheus - server: - port: 8085 - servlet: - context-path: /onap/policy/clamp/acm/policyparticipant - - -Navigate to the "participant/participant-impl/participant-impl-policy" directory. We can then run the policy-participant with the following command: +To start the policy participant we need to go to the "participant/participant-impl/participant-impl-policy" directory in the clamp repo. You can then run the following command to start the policy-participant: .. code-block:: bash |