summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
Diffstat
-rw-r--r--docs/development/devtools/apex-s3p-results/apex_perf_jmeter_results.pngbin135217 -> 110730 bytes
-rw-r--r--docs/development/devtools/apex-s3p-results/apex_stability_jmeter_results.pngbin134635 -> 109911 bytes
-rw-r--r--docs/development/devtools/apex-s3p.rst16
-rw-r--r--docs/development/devtools/clamp-s3p-results/Stability_after_stats.pngbin24482 -> 123032 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/Stability_before_stats.pngbin24290 -> 0 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/acm_performance_jmeter.pngbin0 -> 229066 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/acm_stability_jmeter.pngbin79956 -> 218877 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/acm_stability_table.pngbin200091 -> 454197 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.pngbin79731 -> 0 bytes
-rw-r--r--docs/development/devtools/clamp-s3p.rst132
10 files changed, 105 insertions, 43 deletions
diff --git a/docs/development/devtools/apex-s3p-results/apex_perf_jmeter_results.png b/docs/development/devtools/apex-s3p-results/apex_perf_jmeter_results.png
index c17d3a6c..0fa35c0b 100644
--- a/docs/development/devtools/apex-s3p-results/apex_perf_jmeter_results.png
+++ b/docs/development/devtools/apex-s3p-results/apex_perf_jmeter_results.png
Binary files differ
diff --git a/docs/development/devtools/apex-s3p-results/apex_stability_jmeter_results.png b/docs/development/devtools/apex-s3p-results/apex_stability_jmeter_results.png
index e092d77c..585f99c5 100644
--- a/docs/development/devtools/apex-s3p-results/apex_stability_jmeter_results.png
+++ b/docs/development/devtools/apex-s3p-results/apex_stability_jmeter_results.png
Binary files differ
diff --git a/docs/development/devtools/apex-s3p.rst b/docs/development/devtools/apex-s3p.rst
index 9575277c..6a3be847 100644
--- a/docs/development/devtools/apex-s3p.rst
+++ b/docs/development/devtools/apex-s3p.rst
@@ -24,11 +24,13 @@ At a minimum, the following ONAP components are needed: policy, mariadb-galera,
Before deploying, the values.yaml files are changed to use NodePort instead of ClusterIP for policy-api,
policy-pap, and policy-apex-pdp, so that they are accessible from jmeter::
- policy-apex-pdp NodePort 10.43.70.176 <none> 6969:30237/TCP
- policy-api NodePort 10.43.2.99 <none> 6969:30240/TCP
- policy-pap NodePort 10.43.203.178 <none> 6969:30442/TCP
+ policy-apex-pdp NodePort 10.43.131.43 <none> 6969:31739/TCP
+ policy-api NodePort 10.43.67.153 <none> 6969:30430/TCP
+ policy-pap NodePort 10.43.200.57 <none> 6969:30585/TCP
-The node ports (30237, 30240 and 30442 above) are used in JMeter. The HOSTNAME for JMeter is set to the IP returned by running kubectl cluster-info.
+The node ports (31739, 30430 and 30585 above) are used in JMeter.
+The HOSTNAMEs for JMeter are set to the IPs returned by running "kubectl get node -o wide"
+and to find the applications for each node by running "kubectl describe node <node-name>".
Set up policy-models-simulator
------------------------------
@@ -41,7 +43,7 @@ It is run as a docker image from a node accessible to the kubernetes cluster::
docker run -d --rm --publish 6680:6680 --publish 31054:3905 \
--volume "apex-pdp/testsuites/apex-pdp-stability/src/main/resources/simulatorConfig:/opt/app/policy/simulators/etc/mounted" \
- nexus3.onap.org:10001/onap/policy-models-simulator:2.6-SNAPSHOT-latest
+ nexus3.onap.org:10001/onap/policy-models-simulator:2.7-SNAPSHOT-latest
The published ports 6680 and 31054 are used in JMeter for CDS and DMaaP simulators.
@@ -173,7 +175,7 @@ Stability test plan was triggered for 72 hours. There were no failures during th
======================= ================= ================== ==================================
**Total # of requests** **Success %** **Error %** **Average time taken per request**
======================= ================= ================== ==================================
-430331 100 % 0.00 % 152 ms
+430397 100 % 0.00 % 151.694 ms
======================= ================= ================== ==================================
.. Note::
@@ -242,7 +244,7 @@ Test results are shown as below.
======================= ================= ================== ==================================
**Total # of requests** **Success %** **Error %** **Average time taken per request**
======================= ================= ================== ==================================
-47586 100 % 0.00 % 163 ms
+47567 100 % 0.00 % 163.841 ms
======================= ================= ================== ==================================
**JMeter Screenshot**
diff --git a/docs/development/devtools/clamp-s3p-results/Stability_after_stats.png b/docs/development/devtools/clamp-s3p-results/Stability_after_stats.png
index 8370f0df..38242866 100644
--- a/docs/development/devtools/clamp-s3p-results/Stability_after_stats.png
+++ b/docs/development/devtools/clamp-s3p-results/Stability_after_stats.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/Stability_before_stats.png b/docs/development/devtools/clamp-s3p-results/Stability_before_stats.png
deleted file mode 100644
index 46eaddf6..00000000
--- a/docs/development/devtools/clamp-s3p-results/Stability_before_stats.png
+++ /dev/null
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/acm_performance_jmeter.png b/docs/development/devtools/clamp-s3p-results/acm_performance_jmeter.png
new file mode 100644
index 00000000..bad1cf71
--- /dev/null
+++ b/docs/development/devtools/clamp-s3p-results/acm_performance_jmeter.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/acm_stability_jmeter.png b/docs/development/devtools/clamp-s3p-results/acm_stability_jmeter.png
index abe15373..2f576505 100644
--- a/docs/development/devtools/clamp-s3p-results/acm_stability_jmeter.png
+++ b/docs/development/devtools/clamp-s3p-results/acm_stability_jmeter.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/acm_stability_table.png b/docs/development/devtools/clamp-s3p-results/acm_stability_table.png
index 70c14feb..28942eff 100644
--- a/docs/development/devtools/clamp-s3p-results/acm_stability_table.png
+++ b/docs/development/devtools/clamp-s3p-results/acm_stability_table.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.png b/docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.png
deleted file mode 100644
index a9a8e01d..00000000
--- a/docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.png
+++ /dev/null
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p.rst b/docs/development/devtools/clamp-s3p.rst
index 639b295d..eb17d894 100644
--- a/docs/development/devtools/clamp-s3p.rst
+++ b/docs/development/devtools/clamp-s3p.rst
@@ -13,8 +13,54 @@ Policy Clamp Automation Composition
Both the Performance and the Stability tests were executed by performing requests
against acm components installed as docker images in local environment.
-Setup Details
-+++++++++++++
+
+ACM Deployment
+++++++++++++++
+
+The docker containers can be deployed via Policy CSIT script.
+Clone the Policy/docker repo to the local vm
+
+.. code-block:: bash
+
+ git clone "https://gerrit.onap.org/r/policy/docker"
+
+Set the following environment variables on the system before deploying the containers.
+
+.. code-block:: bash
+
+ export CONTAINER_LOCATION=nexus3.onap.org:10001/
+ export PROJECT=clamp
+
+Invoke the following script from the ~/docker/csit folder.
+
+.. code-block:: bash
+
+ ./start-all.sh
+
+This script installs the docker containers of ACM and Policy components required for running the tests.
+
+
+Jmeter setup
+++++++++++++
+
+Apache jmeter tool is installed either on the same virtual machine or on a different virtual machine.
+
+.. code-block:: bash
+
+ # Install required packages
+ sudo apt install -y wget unzip
+
+ # Install JMeter
+ mkdir -p jmeter
+ cd jmeter
+ wget https://dlcdn.apache.org//jmeter/binaries/apache-jmeter-5.5.zip # check if valid version
+ unzip -q apache-jmeter-5.5.zip
+ rm apache-jmeter-5.5.zip
+
+
+Setup Verification
+++++++++++++++++++
+Ensure the following components are up and running before executing the test.
- acm runtime component docker image is started and running.
- Participant docker images policy-clamp-cl-pf-ppnt, policy-clamp-cl-http-ppnt, policy-clamp-cl-k8s-ppnt are started and running.
@@ -32,7 +78,7 @@ The 72 hours stability test ran the following steps sequentially in a single thr
- **Create Policy defaultDomain** - creates an operational policy using policy/api component
- **Delete Policy sampleDomain** - deletes the operational policy sampleDomain using policy/api component
-- **Commission Contorlloop definition** - commissions the acm definition in runtime
+- **Commission AC definition** - commissions the acm definition in runtime
- **Instantiate acm** - Instantiate the acm towards participants
- **Check acm state** - check the current state of acm
- **Change State to PASSIVE** - change the state of the acm to PASSIVE
@@ -42,7 +88,7 @@ The 72 hours stability test ran the following steps sequentially in a single thr
- **Delete instantiated acm** - delete the instantiated acm from all participants
- **Delete ACM Definition** - delete the acm definition on runtime
-The following steps can be used to configure the parameters of test plan.
+The following parameters can be configured on the JMX file for the test.
- **HTTP Authorization Manager** - used to store user/password authentication details.
- **HTTP Header Manager** - used to store headers which will be used for making HTTP requests.
@@ -57,11 +103,19 @@ The following steps can be used to configure the parameters of test plan.
POLICY_PARTICIPANT_HOST_PORT Port number of policy participant
============================= ========================================================================
+Download the ACM stability.jmx and performance.jmx files from the Policy-Clamp repo.
+
+Stability jmx file
+
+.. code-block:: bash
+
+ ~/clamp/testsuites/stability/src/main/resources/testplans/stability.jmx
+
The test was run in the background via "nohup", to prevent it from being interrupted:
.. code-block:: bash
- nohup ./jMeter/apache-jmeter-5.2.1/bin/jmeter -n -t stability.jmx -l testresults.jtl
+ nohup ./jmeter/apache-jmeter-5.5/bin/jmeter -n -t stability.jmx -l testresults.jtl
Test Results
------------
@@ -83,22 +137,24 @@ Stability test plan was triggered for 72 hours.
======================= ================= ================== ==================================
**Total # of requests** **Success %** **Error %** **Average time taken per request**
======================= ================= ================== ==================================
-99992 100.00 % 0.00 % 192 ms
+97916 100.00 % 0.00 % 246 ms
======================= ================= ================== ==================================
**ACM component Setup**
-================ ========================================================= =========================================== =========================
-**CONTAINER ID** **IMAGE** **PORTS** **NAMES**
-================ ========================================================= =========================================== =========================
- a9cb0cd103cf onap/policy-clamp-cl-runtime:latest 6969/tcp policy-clamp-cl-runtime
- 886e572b8438 onap/policy-clamp-cl-pf-ppnt:latest 6973/tcp policy-clamp-cl-pf-ppnt
- 035707b1b95f nexus3.onap.org:10001/onap/policy-api:latest 6969/tcp policy-api
- d34204f95ff3 onap/policy-clamp-cl-http-ppnt:latest 6971/tcp policy-clamp-cl-http-ppnt
- 4470e608c9a8 onap/policy-clamp-cl-k8s-ppnt:latest 6972/tcp, 8083/tcp policy-clamp-cl-k8s-ppnt
- 62229d46b79c nexus3.onap.org:10001/onap/policy-models-simulator:latest 3905/tcp, 6666/tcp, 6668-6670/tcp, 6680/tcp simulator
- efaf0ca5e1f0 nexus3.onap.org:10001/mariadb:10.5.8 3306/tcp mariadb
-================ ========================================================= =========================================== =========================
+================ ============================================================ =========================================== =========================
+**CONTAINER ID** **IMAGE** **PORT** **NAME**
+================ ============================================================ =========================================== =========================
+ a9cb0cd103cf nexus3.onap.org:10001/onap/policy-clamp-runtime-acm:latest 6969/tcp policy-clamp-runtime-acm
+ 886e572b8438 nexus3.onap.org:10001/onap/policy-clamp-ac-pf-ppnt:latest 6969/tcp policy-clamp-ac-pf-ppnt
+ 035707b1b95f nexus3.onap.org:10001/onap/policy-api:latest 6969/tcp policy-api
+ d34204f95ff3 nexus3.onap.org:10001/onap/policy-clamp-ac-http-ppnt:latest 6969/tcp policy-clamp-ac-http-ppnt
+ 4470e608c9a8 nexus3.onap.org:10001/onap/policy-clamp-ac-k8s-ppnt:latest 6969/tcp policy-clamp-ac-k8s-ppnt
+ 62229d46b79c nexus3.onap.org:10001/onap/policy-models-simulator:latest 3905/tcp, 6666/tcp, 6668-6670/tcp, 6680/tcp simulator
+ efaf0ca5e1f0 nexus3.onap.org:10001/mariadb:10.5.8 3306/tcp mariadb
+ e84cf17db2a4 nexus3.onap.org:10001/onap/policy-pap:latest 6969/tcp policy-pap
+ 0a16eecd13c9 nexus3.onap.org:10001/onap/policy-apex-pdp:latest 6969/tcp policy-apex-pdp
+================ ============================================================ =========================================== =========================
.. Note::
@@ -116,11 +172,7 @@ Stability test plan was triggered for 72 hours.
**Memory and CPU usage**
-The memory and CPU usage can be monitored by running "docker stats" command. A snapshot is taken before and after test execution to monitor the changes in resource utilization.
-
-Memory and CPU usage before test execution:
-
-.. image:: clamp-s3p-results/Stability_before_stats.png
+The memory and CPU usage can be monitored by running "docker stats" command.
Memory and CPU usage after test execution:
@@ -152,11 +204,17 @@ Performance test plan is the same as the stability test plan above except for th
Run Test
--------
+Performance jmx file
+
+.. code-block:: bash
+
+ ~/clamp/testsuites/performance/src/main/resources/testplans/performance.jmx
+
Running/Triggering the performance test will be the same as the stability test. That is, launch JMeter pointing to corresponding *.jmx* test plan. The *RUNTIME_HOST*, *RUNTIME_PORT*, *POLICY_PARTICIPANT_HOST*, *POLICY_PARTICIPANT_HOST_PORT* are already set up in *.jmx*
.. code-block:: bash
- nohup ./jMeter/apache-jmeter-5.2.1/bin/jmeter -n -t performance.jmx -l testresults.jtl
+ nohup ./jmeter/apache-jmeter-5.5/bin/jmeter -n -t performance.jmx -l testresults.jtl
Once the test execution is completed, execute the below script to get the statistics:
@@ -175,23 +233,25 @@ Test results are shown as below.
======================= ================= ================== ==================================
**Total # of requests** **Success %** **Error %** **Average time taken per request**
======================= ================= ================== ==================================
-13809 100 % 0.00 % 206 ms
+13591 100 % 0.00 % 249 ms
======================= ================= ================== ==================================
**ACM component Setup**
-================ ========================================================= =========================================== =========================
-**CONTAINER ID** **IMAGE** **PORTS** **NAMES**
-================ ========================================================= =========================================== =========================
- a9cb0cd103cf onap/policy-clamp-cl-runtime:latest 6969/tcp policy-clamp-cl-runtime
- 886e572b8438 onap/policy-clamp-cl-pf-ppnt:latest 6973/tcp policy-clamp-cl-pf-ppnt
- 035707b1b95f nexus3.onap.org:10001/onap/policy-api:latest 6969/tcp policy-api
- d34204f95ff3 onap/policy-clamp-cl-http-ppnt:latest 6971/tcp policy-clamp-cl-http-ppnt
- 4470e608c9a8 onap/policy-clamp-cl-k8s-ppnt:latest 6972/tcp, 8083/tcp policy-clamp-cl-k8s-ppnt
- 62229d46b79c nexus3.onap.org:10001/onap/policy-models-simulator:latest 3905/tcp, 6666/tcp, 6668-6670/tcp, 6680/tcp simulator
- efaf0ca5e1f0 nexus3.onap.org:10001/mariadb:10.5.8 3306/tcp mariadb
-================ ========================================================= =========================================== =========================
+================ ============================================================ =========================================== =========================
+**CONTAINER ID** **IMAGE** **PORT** **NAME**
+================ ============================================================ =========================================== =========================
+ a9cb0cd103cf nexus3.onap.org:10001/onap/policy-clamp-runtime-acm:latest 6969/tcp policy-clamp-runtime-acm
+ 886e572b8438 nexus3.onap.org:10001/onap/policy-clamp-ac-pf-ppnt:latest 6969/tcp policy-clamp-ac-pf-ppnt
+ 035707b1b95f nexus3.onap.org:10001/onap/policy-api:latest 6969/tcp policy-api
+ d34204f95ff3 nexus3.onap.org:10001/onap/policy-clamp-ac-http-ppnt:latest 6969/tcp policy-clamp-ac-http-ppnt
+ 4470e608c9a8 nexus3.onap.org:10001/onap/policy-clamp-ac-k8s-ppnt:latest 6969/tcp policy-clamp-ac-k8s-ppnt
+ 62229d46b79c nexus3.onap.org:10001/onap/policy-models-simulator:latest 3905/tcp, 6666/tcp, 6668-6670/tcp, 6680/tcp simulator
+ efaf0ca5e1f0 nexus3.onap.org:10001/mariadb:10.5.8 3306/tcp mariadb
+ e84cf17db2a4 nexus3.onap.org:10001/onap/policy-pap:latest 6969/tcp policy-pap
+ 0a16eecd13c9 nexus3.onap.org:10001/onap/policy-apex-pdp:latest 6969/tcp policy-apex-pdp
+================ ============================================================ =========================================== =========================
**JMeter Screenshot**
-.. image:: clamp-s3p-results/cl-s3p-performance-result-jmeter.png
+.. image:: clamp-s3p-results/acm_performance_jmeter.png