aboutsummaryrefslogtreecommitdiffstats
path: root/docs/development/devtools
diff options
context:
space:
mode:
authorSirisha_Manchikanti <sirisha.manchikanti@est.tech>2021-10-12 17:04:33 +0100
committerRam Krishna Verma <ram_krishna.verma@bell.ca>2021-10-25 15:12:42 +0000
commit3b2c0abcfda9f83a911e9a5e7b071b456ce89306 (patch)
tree497aa3eae48130832422a2e6b82a361777c7a8e2 /docs/development/devtools
parentd68ec8874d66958b87be4c83d88151166d426c80 (diff)
ControlLoop S3P test results
Issue-ID: POLICY-3659 Signed-off-by: Sirisha_Manchikanti <sirisha.manchikanti@est.tech> Change-Id: I1d62b9efe9cb154ce0a8224f32f943ec5d1fffab (cherry picked from commit f9d03ae53dc04dd8ca90e61507e43b23bb3e5a59)
Diffstat (limited to 'docs/development/devtools')
-rw-r--r--docs/development/devtools/clamp-s3p-results/Stability_after_stats.pngbin0 -> 24482 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/Stability_before_stats.pngbin0 -> 24290 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.pngbin0 -> 62523 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/controlloop_stability_jmeter.pngbin0 -> 81751 bytes
-rw-r--r--docs/development/devtools/clamp-s3p-results/controlloop_stability_table.pngbin0 -> 216697 bytes
-rw-r--r--docs/development/devtools/clamp-s3p.rst197
-rw-r--r--docs/development/devtools/devtools.rst1
7 files changed, 198 insertions, 0 deletions
diff --git a/docs/development/devtools/clamp-s3p-results/Stability_after_stats.png b/docs/development/devtools/clamp-s3p-results/Stability_after_stats.png
new file mode 100644
index 00000000..8370f0df
--- /dev/null
+++ b/docs/development/devtools/clamp-s3p-results/Stability_after_stats.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/Stability_before_stats.png b/docs/development/devtools/clamp-s3p-results/Stability_before_stats.png
new file mode 100644
index 00000000..46eaddf6
--- /dev/null
+++ b/docs/development/devtools/clamp-s3p-results/Stability_before_stats.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.png b/docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.png
new file mode 100644
index 00000000..30fc4bba
--- /dev/null
+++ b/docs/development/devtools/clamp-s3p-results/cl-s3p-performance-result-jmeter.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/controlloop_stability_jmeter.png b/docs/development/devtools/clamp-s3p-results/controlloop_stability_jmeter.png
new file mode 100644
index 00000000..058b98ae
--- /dev/null
+++ b/docs/development/devtools/clamp-s3p-results/controlloop_stability_jmeter.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p-results/controlloop_stability_table.png b/docs/development/devtools/clamp-s3p-results/controlloop_stability_table.png
new file mode 100644
index 00000000..0289a289
--- /dev/null
+++ b/docs/development/devtools/clamp-s3p-results/controlloop_stability_table.png
Binary files differ
diff --git a/docs/development/devtools/clamp-s3p.rst b/docs/development/devtools/clamp-s3p.rst
new file mode 100644
index 00000000..e01848da
--- /dev/null
+++ b/docs/development/devtools/clamp-s3p.rst
@@ -0,0 +1,197 @@
+.. This work is licensed under a
+.. Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. _controlloop-s3p-label:
+
+.. toctree::
+ :maxdepth: 2
+
+Policy Clamp Controlloop
+~~~~~~~~~~~~~~~~~~~~~~~~
+
+Both the Performance and the Stability tests were executed by performing requests
+against controlloop components installed as docker images in local environment.
+
+Setup Details
++++++++++++++
+
+- Controlloop runtime component docker image is started and running.
+- Participant docker images policy-clamp-cl-pf-ppnt, policy-clamp-cl-http-ppnt, policy-clamp-cl-k8s-ppnt are started and running.
+- Dmaap simulator for communication between components.
+- mariadb docker container for policy and controlloop database.
+- policy-api for communication between policy participant and policy-framework
+- Both tests were run via jMeter, which was installed on a separate VM.
+
+Stability Test of Controlloop components
+++++++++++++++++++++++++++++++++++++++++
+
+Test Plan
+---------
+The 72 hours stability test ran the following steps sequentially in a single threaded loop.
+
+- **Create Policy defaultDomain** - creates an operational policy using policy/api component
+- **Delete Policy sampleDomain** - deletes the operational policy sampleDomain using policy/api component
+- **Commission Contorlloop definition** - commissions the controlloop definition in runtime
+- **Instantiate controlloop** - Instantiate the controlloop towards participants
+- **Check controlloop state** - check the current state of controlloop
+- **Change State to PASSIVE** - change the state of the controlloop to PASSIVE
+- **Check controlloop state** - check the current state of controlloop
+- **Change State to UNINITIALISED** - change the state of the controloop to UNINITIALISED
+- **Check controlloop state** - check the current state of controlloop
+- **Delete instantiated controlloop** - delete the instantiated controlloop from all participants
+- **Delete ControlLoop Definition** - delete the controlloop definition on runtime
+
+The following steps can be used to configure the parameters of test plan.
+
+- **HTTP Authorization Manager** - used to store user/password authentication details.
+- **HTTP Header Manager** - used to store headers which will be used for making HTTP requests.
+- **User Defined Variables** - used to store following user defined parameters.
+
+=========== ===================================================================
+ **Name** **Description**
+=========== ===================================================================
+ RUNTIME_HOST IP Address or host name of controlloop runtime component
+ RUNTIME_PORT Port number of controlloop runtime components for making REST API calls
+ POLICY_PARTICIPANT_HOST IP Address or host name of policy participant
+ POLICY_PARTICIPANT_HOST_PORT Port number of policy participant
+=========== ===================================================================
+
+The test was run in the background via "nohup", to prevent it from being interrupted:
+
+.. code-block:: bash
+
+ nohup ./jMeter/apache-jmeter-5.2.1/bin/jmeter -n -t stability.jmx -l testresults.jtl
+
+Test Results
+------------
+
+**Summary**
+
+Stability test plan was triggered for 72 hours.
+
+.. Note::
+
+ .. container:: paragraph
+
+ The assertions of state changes are not completely taken care of, as the stability is ran with controlloop componenets
+ alone, and not including complete policy framework deployment, which makes it difficult for actual state changes from
+ PASSIVE to RUNNING etc to happen.
+
+**Test Statistics**
+
+======================= ================= ================== ==================================
+**Total # of requests** **Success %** **Error %** **Average time taken per request**
+======================= ================= ================== ==================================
+99992 100.00 % 0.00 % 192 ms
+======================= ================= ================== ==================================
+
+**Controloop component Setup**
+
+================ ======================= ================== ==========================
+**CONTAINER ID** **IMAGE** **PORTS** **NAMES**
+================ ======================= ================== ================================== ==========================
+ a9cb0cd103cf onap/policy-clamp-cl-runtime:latest 6969/tcp policy-clamp-cl-runtime
+ 886e572b8438 onap/policy-clamp-cl-pf-ppnt:latest 6973/tcp policy-clamp-cl-pf-ppnt
+ 035707b1b95f nexus3.onap.org:10001/onap/policy-api:latest 6969/tcp policy-api
+ d34204f95ff3 onap/policy-clamp-cl-http-ppnt:latest 6971/tcp policy-clamp-cl-http-ppnt
+ 4470e608c9a8 onap/policy-clamp-cl-k8s-ppnt:latest 6972/tcp, 8083/tcp policy-clamp-cl-k8s-ppnt
+ 62229d46b79c nexus3.onap.org:10001/onap/policy-models-simulator:latest 3905/tcp, 6666/tcp, 6668-6670/tcp, 6680/tcp simulator
+ efaf0ca5e1f0 nexus3.onap.org:10001/mariadb:10.5.8 3306/tcp mariadb
+======================= ================= ================== ====================================== ===========================
+
+.. Note::
+
+ .. container:: paragraph
+
+ There were no failures during the 72 hours test.
+
+**JMeter Screenshot**
+
+.. image:: clamp-s3p-results/controlloop_stability_jmeter.PNG
+
+**JMeter Screenshot**
+
+.. image:: clamp-s3p-results/controlloop_stability_table.PNG
+
+**Memory and CPU usage**
+
+The memory and CPU usage can be monitored by running "docker stats" command. A snapshot is taken before and after test execution to monitor the changes in resource utilization.
+
+Memory and CPU usage before test execution:
+
+.. image:: clamp-s3p-results/Stability_before_stats.PNG
+
+Memory and CPU usage after test execution:
+
+.. image:: clamp-s3p-results/Stability_after_stats.PNG
+
+
+Performance Test of Controlloop components
+++++++++++++++++++++++++++++++++++++++++++
+
+Introduction
+------------
+
+Performance test of Controlloop components has the goal of testing the min/avg/max processing time and rest call throughput for all the requests with multiple requests at the same time.
+
+Setup Details
+-------------
+
+The performance test is performed on a similar setup as Stability test. The JMeter VM will be sending a large number of REST requests to the runtime component and collecting the statistics.
+
+
+Test Plan
+---------
+
+Performance test plan is the same as the stability test plan above except for the few differences listed below.
+
+- Increase the number of threads up to 5 (simulating 5 users' behaviours at the same time).
+- Reduce the test time to 2 hours.
+
+Run Test
+--------
+
+Running/Triggering the performance test will be the same as the stability test. That is, launch JMeter pointing to corresponding *.jmx* test plan. The *RUNTIME_HOST*, *RUNTIME_PORT*, *POLICY_PARTICIPANT_HOST*, *POLICY_PARTICIPANT_HOST_PORT* are already set up in *.jmx*
+
+.. code-block:: bash
+
+ nohup ./jMeter/apache-jmeter-5.2.1/bin/jmeter -n -t performance.jmx -l testresults.jtl
+
+Once the test execution is completed, execute the below script to get the statistics:
+
+.. code-block:: bash
+
+ $ cd ./clamp/testsuites/performance/src/main/resources/testplans
+ $ ./results.sh resultTree.log
+
+Test Results
+------------
+
+Test results are shown as below.
+
+**Test Statistics**
+
+======================= ================= ================== ==================================
+**Total # of requests** **Success %** **Error %** **Average time taken per request**
+======================= ================= ================== ==================================
+13809 100 % 0.00 % 206 ms
+======================= ================= ================== ==================================
+
+**Controloop component Setup**
+
+================ ======================= ================== ==========================
+**CONTAINER ID** **IMAGE** **PORTS** **NAMES**
+================ ======================= ================== ================================== ==========================
+ a9cb0cd103cf onap/policy-clamp-cl-runtime:latest 6969/tcp policy-clamp-cl-runtime
+ 886e572b8438 onap/policy-clamp-cl-pf-ppnt:latest 6973/tcp policy-clamp-cl-pf-ppnt
+ 035707b1b95f nexus3.onap.org:10001/onap/policy-api:latest 6969/tcp policy-api
+ d34204f95ff3 onap/policy-clamp-cl-http-ppnt:latest 6971/tcp policy-clamp-cl-http-ppnt
+ 4470e608c9a8 onap/policy-clamp-cl-k8s-ppnt:latest 6972/tcp, 8083/tcp policy-clamp-cl-k8s-ppnt
+ 62229d46b79c nexus3.onap.org:10001/onap/policy-models-simulator:latest 3905/tcp, 6666/tcp, 6668-6670/tcp, 6680/tcp simulator
+ efaf0ca5e1f0 nexus3.onap.org:10001/mariadb:10.5.8 3306/tcp mariadb
+======================= ================= ================== ====================================== ===========================
+
+**JMeter Screenshot**
+
+.. image:: clamp-s3p-results/cl-s3p-performance-result-jmeter.PNG
diff --git a/docs/development/devtools/devtools.rst b/docs/development/devtools/devtools.rst
index 1a001b02..0cf11a4c 100644
--- a/docs/development/devtools/devtools.rst
+++ b/docs/development/devtools/devtools.rst
@@ -312,6 +312,7 @@ familiar with the Policy Framework components and test any local changes.
drools-s3p.rst
xacml-s3p.rst
distribution-s3p.rst
+ clamp-s3p.rst
Running the Pairwise Tests
***********************