Update S3P testing xacml PDP documentation

Issue-ID: POLICY-4873
Change-Id: I9137c4b75c2e43dc68e225ac86bd18a8a7175af2
Signed-off-by: FrancescoFioraEst <francesco.fiora@est.tech>

3 files changed, 126 insertions, 66 deletions

diff --git a/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-perf-xacml.png b/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-perf-xacml.png
index 2c27967f..b5ef5794 100644..100755
--- a/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-perf-xacml.png
+++ b/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-perf-xacml.png
diff --git a/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-stability-xacml.png b/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-stability-xacml.png
new file mode 100755
index 00000000..d45aeee4
--- /dev/null
+++ b/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-stability-xacml.png
diff --git a/docs/development/devtools/testing/s3p/xacml-s3p.rst b/docs/development/devtools/testing/s3p/xacml-s3p.rst
index 5ea2e287..2ca0a533 100644
--- a/docs/development/devtools/testing/s3p/xacml-s3p.rst
+++ b/docs/development/devtools/testing/s3p/xacml-s3p.rst
@@ -9,39 +9,48 @@
 
 ##########################
 
-Performance Test of Policy XACML PDP (Jakarta)
-**********************************************
+Policy XACML PDP S3P Tests
+~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-The Performance test was executed by performing requests
-against the Policy RESTful APIs.
+Both the Performance and the Stability tests were executed by performing requests
+against Policy components installed in Kubernetes environment. These tests were all
+performed on a Ubuntu VM with 32GB of memory, 16 CPU and 100GB of disk space.
 
-A default ONAP installation in the Policy tenant in UNH was used to run the tests.
+Policy XACML PDP Deployment
++++++++++++++++++++++++++++
 
-The Agent VMs in this lab have the following configuration:
+In an effort to allow the execution of the s3p tests to be as close to automatic as possible,
+a script will be executed that will perform the following:
 
-- 16GB RAM
-- 8 VCPU
+- Install of a microk8s kubernetes environment
+- Bring up the policy components
+- Checks that the components are successfully up and running before proceeding
+- Install Java 17
+- Install Jmeter locally and configure it
+- Specify whether you want to run stability or performance tests
 
-Summary
-=======
 
-The Performance test was executed, and the result analysed, via:
+The remainder of this document outlines how to run the tests and the test results
 
-.. code-block:: bash
+Common Setup
+++++++++++++
+The common setup for performance and stability tests is now automated - being carried out by a script in- **testsuites/run-s3p-test.sh**.
 
-    jmeter -Jduration=1200 -Jusers=10 \
-        -Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
-        -Jxacml_port=30111 -Jpap_port=30197 -Japi_port=30664 \
-        -n -t perf.jmx -l testresults.jtl
+Clone the policy-xacml-pdp repo to access the test scripts
 
-Note: the ports listed above correspond to port 6969 of the respective components.
+.. code-block:: bash
 
-The performance tests runs the following, all in parallel:
+    git clone https://gerrit.onap.org/r/policy/xacml-pdp xacml-pdp
 
-- Healthcheck, 10 simultaneous threads
-- Statistics, 10 simultaneous threads
-- Decisions, 10 simultaneous threads, each running the following in sequence:
+Stability Test of Policy XACML PDP
+++++++++++++++++++++++++++++++++++
 
+Test Plan
+---------
+The 24 hours stability test ran the following steps.
+
+- Healthcheck, 2 simultaneous threads
+- Decisions, 2 simultaneous threads, each running the following tasks in sequence:
    - Monitoring Decision
    - Monitoring Decision, abbreviated
    - Naming Decision
@@ -50,62 +59,97 @@ The performance tests runs the following, all in parallel:
    - Frequency Limiter Guard Decision
    - Min/Max Guard Decision
 
-When the script starts up, it uses policy-api to create, and policy-pap to deploy,
-the policies that are needed by the test.  It assumes that the "naming" policy has
-already been created and deployed.  Once the test completes, it undeploys and deletes
-the policies that it previously created.
+This runs for 24 hours. Test results are present in the **testsuites/automated-performance/s3pTestResults.jtl**
+file and in **/tmp/** directory. Logs are present for jmeter in **testsuites/automated-performance/jmeter.log** and
+**testsuites/automated-performance/nohup.out**
 
-Results
-=======
+Run Test
+--------
 
-The test was run for 20 minutes with 10 users (i.e., threads), with the following results:
+The code in the setup section also serves to run the tests. Just one execution needed to do it all.
 
-.. csv-table::
-   :header: "Number of Users", "Throughput (requests/second)", "Average Latency (ms)"
+.. code-block:: bash
 
-   10, 4603, 2
+    bash run-s3p-test.sh run stability
 
-.. image:: xacml-s3p-results/s3p-perf-xacml.png
+Once the test execution is completed, the results are present in the **automate-performance/s3pTestResults.jtl** file.
 
+This file can be imported into the Jmeter GUI for visualization. The below results are tabulated from the GUI.
 
-Stability Test of Policy XACML PDP
-**********************************
+Test Results
+------------
 
-This test was run using jmeter on a default
-ONAP installation in the Policy tenant in UNH.
+**Summary**
 
-The Agent VMs in this lab have the following configuration:
+Stability test plan was triggered for 24 hours.
 
-- 16GB RAM
-- 8 VCPU
+**Test Statistics**
 
-Summary
-=======
+=======================  =================  ========================  =========================
+**Total # of requests**  **Error %**        **Average Latency (ms)**  **Measured requests/sec**
+=======================  =================  ========================  =========================
+  63618103                 0.00 %             5 ms                      736.064 ms
+=======================  =================  ========================  =========================
 
-The stability test was performed on a default ONAP OOM installation in the Policy tenant of the UNH lab.
-JMeter injected the traffic defined in the
-`XACML PDP stability script
-<https://git.onap.org/policy/xacml-pdp/tree/testsuites/stability/src/main/resources/testplans/stability.jmx>`_
-with the following command:
+**JMeter Results**
 
-.. code-block:: bash
+.. image:: xacml-s3p-results/s3p-stability-xacml.png
 
-    jmeter.sh -Jduration=259200 -Jusers=2 -Jxacml_ip=$ip -Jpap_ip=$ip -Japi_ip=$ip \
-        -Jxacml_port=30111 -Jpap_port=30197 -Japi_port=30664 --nongui --testfile stability.jmx
+**Policy component Setup**
 
-Note: the ports listed above correspond to port 6969 of the respective components.
+==============================================  ====================================================================  ===========================================
+**NAME**                                        **IMAGE**                                                             **PORT**
+==============================================  ====================================================================  ===========================================
+mariadb-galera-0                                docker.io/bitnami/mariadb-galera:10.5.8                               3306/TCP
+policy-models-simulator-bcd494d87-cldfr         nexus3.onap.org:10001/onap/policy-models-simulator:3.0.2-SNAPSHOT     30904/TCP
+policy-apex-pdp-0                               nexus3.onap.org:10001/onap/policy-apex-pdp:3.0.2-SNAPSHOT             30001/TCP
+prometheus-f66f97b6-kpg5c                       nexus3.onap.org:10001/prom/prometheus:latest                          30909/TCP
+policy-distribution-86777bdd78-95zjx            nexus3.onap.org:10001/onap/policy-distribution:3.0.2-SNAPSHOT         30006/TCP
+policy-clamp-ac-http-ppnt-7b99cbfbf8-d5zjk      nexus3.onap.org:10001/onap/policy-clamp-ac-http-ppnt:7.0.3-SNAPSHOT
+policy-clamp-ac-pf-ppnt-55c4cb99f4-pp699        nexus3.onap.org:10001/onap/policy-clamp-ac-pf-ppnt:7.0.3-SNAPSHOT     30008/TCP
+policy-clamp-ac-k8s-ppnt-6d854cc8b6-nzlgh       nexus3.onap.org:10001/onap/policy-clamp-ac-k8s-ppnt:7.0.3-SNAPSHOT
+policy-clamp-runtime-acm-5c6d8fbfb-zww5j        nexus3.onap.org:10001/onap/policy-clamp-runtime-acm:7.0.3-SNAPSHOT    30007/TCP
+policy-pap-847d89997d-gljzh                     nexus3.onap.org:10001/onap/policy-pap:3.0.2-SNAPSHOT                  30003/TCP
+policy-xacml-pdp-656d57d578-9h6fx               nexus3.onap.org:10001/onap/policy-xacml-pdp:3.0.2-SNAPSHOT            30004/TCP
+policy-api-58cb45fc9b-9qcx6                     nexus3.onap.org:10001/onap/policy-api:3.0.2-SNAPSHOT                  30002/tcp
+==============================================  ====================================================================  ===========================================
 
-The default log level of the root and org.eclipse.jetty.server.RequestLog loggers in the logback.xml
-of the XACML PDP
-(om/kubernetes/policy/components/policy-xacml-pdp/resources/config/logback.xml)
-was set to WARN since the OOM installation did have log rotation enabled of the
-container logs in the kubernetes worker nodes.
+.. Note::
 
-The stability test, stability.jmx, runs the following, all in parallel:
+              .. container:: paragraph
+
+                  There were no failures during the 24 hours test.
+
+The XACML PDP offered very good performance with JMeter for the traffic mix described above.
+The average transaction time is insignificant.
+
+
+Performance Test of Policy XACML PDP
+++++++++++++++++++++++++++++++++++++
+
+Introduction
+------------
+
+Performance test of acm components has the goal of testing the min/avg/max processing time and rest call throughput for all the requests with multiple requests at the same time.
+
+Setup Details
+-------------
+
+We can setup the environment and execute the tests like this from the **xacml-pdp/testsuites** directory
+
+Test Plan
+---------
+
+Performance test plan is the same as the stability test plan above except for the few differences listed below.
+
+- Increase the number of threads up to 10 (simulating 10 users' behaviours at the same time).
+- Reduce the test time to 20 minutes.
+
+The performance tests runs the following, all in parallel:
+
+- Healthcheck, 10 simultaneous threads
+- Decisions, 10 simultaneous threads, each running the following in sequence:
 
-- Healthcheck, 2 simultaneous threads
-- Statistics, 2 simultaneous threads
-- Decisions, 2 simultaneous threads, each running the following tasks in sequence:
    - Monitoring Decision
    - Monitoring Decision, abbreviated
    - Naming Decision
@@ -119,16 +163,32 @@ the policies that are needed by the test.  It assumes that the "naming" policy h
 already been created and deployed.  Once the test completes, it undeploys and deletes
 the policies that it previously created.
 
-Results
-=======
+Run Test
+--------
 
-The stability summary results were reported by JMeter with the following summary line:
+The code in the setup section also serves to run the tests. Just one execution needed to do it all.
 
 .. code-block:: bash
 
-    summary = 941639699 in 71:59:36 = 3633.2/s Avg:     1 Min:     0 Max:   842 Err:     0 (0.00%)
+    bash run-s3p-test.sh run performance
 
-The XACML PDP offered very good performance with JMeter for the traffic mix described above.
-The average transaction time is insignificant.  The maximum transaction time of 842 ms.
-There was a Drools stability test running in parallel, hence the actual load was higher.
+Once the test execution is completed, the results are present in the **automate-performance/s3pTestResults.jtl** file and in **/tmp/** directory.
+
+This file can be imported into the Jmeter GUI for visualization. The below results are tabulated from the GUI.
+
+Test Results
+------------
+
+**Summary**
 
+The test was run for 20 minutes with 10 users (i.e., threads), with the following results:
+
+**Test Statistics**
+
+=======================  =================  ========================  =========================
+**Total # of requests**  **Error %**        **Average Latency (ms)**  **Measured requests/sec**
+=======================  =================  ========================  =========================
+  912141                   0.00 %             25.5666 ms                741.917 ms
+=======================  =================  ========================  =========================
+
+.. image:: xacml-s3p-results/s3p-perf-xacml.png