summaryrefslogtreecommitdiffstats
path: root/docs/development
diff options
context:
space:
mode:
authoradheli.tavares <>2021-03-30 14:46:30 +0100
committerJim Hahn <jrh3@att.com>2021-03-31 14:44:40 +0000
commit5ad69c59cc28b623cb84d93b00ce966b779b4606 (patch)
treea55b8aaa3f5c9875d52616ad6f1e824909fa8e48 /docs/development
parent4c5f2bf9d4e5cf875154d9e7b035b8f2bb79f4a7 (diff)
Distribution S3P tests results
This review must be updated as it has merge conflicts for some reason. Issue-ID: POLICY-3113 Change-Id: I70dd8cc1751c28240b11e5c4598c7603e0bb0c46 Signed-off-by: Jim Hahn <jrh3@att.com> (cherry picked from commit 6c5adb90242ed106336d36e6449ad0eedc17a1e4)
Diffstat (limited to 'docs/development')
-rw-r--r--docs/development/devtools/distribution-s3p.rst38
-rwxr-xr-xdocs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-monitor.pngbin48913 -> 0 bytes
-rwxr-xr-xdocs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-threads.pngbin64954 -> 0 bytes
-rw-r--r--docs/development/devtools/images/dist_perf_statistics.PNGbin0 -> 48205 bytes
-rw-r--r--docs/development/devtools/images/dist_perf_threshold.PNGbin0 -> 26252 bytes
-rw-r--r--docs/development/devtools/images/dist_stability_monitor.PNGbin0 -> 81517 bytes
-rw-r--r--docs/development/devtools/images/dist_stability_statistics.PNGbin0 -> 53205 bytes
-rw-r--r--docs/development/devtools/images/dist_stability_threads.PNGbin0 -> 77025 bytes
-rw-r--r--docs/development/devtools/images/dist_stability_threshold.PNGbin0 -> 28238 bytes
9 files changed, 14 insertions, 24 deletions
diff --git a/docs/development/devtools/distribution-s3p.rst b/docs/development/devtools/distribution-s3p.rst
index 1db411c6..13c47924 100644
--- a/docs/development/devtools/distribution-s3p.rst
+++ b/docs/development/devtools/distribution-s3p.rst
@@ -121,7 +121,7 @@ Run the setup_components.sh script to start the test support components:
.. code-block:: bash
- ~/distribution/testsuites/stability/src/main/resources/distributionsetup/setup_distribution.sh
+ ~/distribution/testsuites/stability/src/main/resources/simulatorsetup/setup_components.sh
After installation, ensure the following docker containers are up and running:
@@ -182,7 +182,7 @@ Download and install JMeter
VM2 Only: Install & configure visualVM
--------------------------------------
-VisualVM needs to be installed in the virtual machine running Distrbution (VM2). It will be used to monitor CPU, Memory and GC for Distribution while the stability tests are running.
+VisualVM needs to be installed in the virtual machine running Distribution (VM2). It will be used to monitor CPU, Memory and GC for Distribution while the stability tests are running.
.. code-block:: bash
@@ -222,7 +222,7 @@ This will load up the visualVM GUI
Connect to Distribution JMX Port.
1. Right click on "Local" in the left panel of the screen and select "Add JMX Connection"
- 2. Enter the Port 9090. this is the JMX port exposed by the dsitribution container
+ 2. Enter the Port 9090. this is the JMX port exposed by the distribution container
3. Double click on the newly added nodes under "Local" to start monitoring CPU, Memory & GC.
Example Screenshot of visualVM
@@ -236,9 +236,9 @@ Stability Test of Policy Distribution
Introduction
------------
-The 72 hour Stability Test for policy distribution has the goal of introducing a steady flow of transactions initiated from a test client server running JMeter. The policy distribution is configured with a special FileSystemReception plugin to monitor a local directory for newly added csar files to be processed by itself. The input CSAR will be added/removed by the test client(JMeter) and the result will be pulled from the backend(PAP and PolicyAPI) by the test client (JMeter).
+The 72 hour Stability Test for policy distribution has the goal of introducing a steady flow of transactions initiated from a test client server running JMeter. The policy distribution is configured with a special FileSystemReception plugin to monitor a local directory for newly added csar files to be processed by itself. The input CSAR will be added/removed by the test client(JMeter) and the result will be pulled from the backend (PAP and PolicyAPI) by the test client (JMeter).
-The test will be performed in an environment where Jmeter will continuously add/remove a test csar into the special directory where policy distribuion is monitoring and will then get the processed results from PAP and PolicyAPI to verify the successful deployment of the policy. The policy will then be undeployed and the test will loop continuously until 72 hours have elapsed.
+The test will be performed in an environment where Jmeter will continuously add/remove a test csar into the special directory where policy distribution is monitoring and will then get the processed results from PAP and PolicyAPI to verify the successful deployment of the policy. The policy will then be undeployed and the test will loop continuously until 72 hours have elapsed.
Test Plan Sequence
@@ -291,7 +291,7 @@ From the apache JMeter folder run the test for 72h, pointing it towards the stab
.. code-block:: bash
- ~/jmeter/apache-jmeter-5.1.1/bin/jmeter -n -t ~/distribution/testsuites/stability/src/main/resources/testplans/stability.jmx -Jduration=259200 -l ~/20201016-1715-distr-stability.jtl &
+ ~/jmeter/apache-jmeter-5.1.1/bin/jmeter -n -t ~/distribution/testsuites/stability/src/main/resources/testplans/stability.jmx -Jduration=259200 -l ~/distr-stability.jtl &
Test Results
@@ -304,18 +304,13 @@ Test Results
**Test Statistics**
-.. csv-table:: Stability Results - Summary Report
- :file: csv/20201016-1715-distr-stability-summary.csv
- :header-rows: 1
-
-.. csv-table:: Stability Results - Aggregate Report
- :file: csv/20201016-1715-distr-stability-aggregate.csv
- :header-rows: 1
+.. image:: images/dist_stability_statistics.PNG
+.. image:: images/dist_stability_threshold.PNG
**VisualVM Screenshots**
-.. image:: images/20201016-1715-distr-stability-20201018T2040-monitor.png
-.. image:: images/20201016-1715-distr-stability-20201018T2040-threads.png
+.. image:: images/dist_stability_monitor.PNG
+.. image:: images/dist_stability_threads.PNG
Performance Test of Policy Distribution
@@ -342,7 +337,7 @@ Performance test plan is different from the stability test plan.
- Instead of handling one policy csar at a time, multiple csar's are deployed within the watched folder at the exact same time.
- We expect all policies from these csar's to be deployed within 30 seconds.
-- There are also multithreaded tests running towards the healtchcheck and statistics endpoints of the distribution service.
+- There are also multithreaded tests running towards the healthcheck and statistics endpoints of the distribution service.
Running the Test Plan
@@ -359,7 +354,7 @@ From the apache JMeter folder run the test for 4h, pointing it towards the perfo
.. code-block:: bash
- ~/jmeter/apache-jmeter-5.1.1/bin/jmeter -n -t ~/distribution/testsuites/performance/src/main/resources/testplans/performance.jmx -Jduration=14400 -l ~/20201020-1730-distr-performance.jtl &
+ ~/jmeter/apache-jmeter-5.1.1/bin/jmeter -n -t ~/distribution/testsuites/performance/src/main/resources/testplans/performance.jmx -Jduration=14400 -l ~/distr-performance.jtl &
Test Results
------------
@@ -371,13 +366,8 @@ Test Results
**Test Statistics**
-.. csv-table:: Performance Results - Summary Report
- :file: csv/20201020-1730-distr-performance-summary.csv
- :header-rows: 1
-
-.. csv-table:: Performance Results - Aggregate Report
- :file: csv/20201020-1730-distr-performance-aggregate.csv
- :header-rows: 1
+.. image:: images/dist_perf_statistics.PNG
+.. image:: images/dist_perf_threshold.PNG
**VisualVM Screenshots**
diff --git a/docs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-monitor.png b/docs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-monitor.png
deleted file mode 100755
index abdba921..00000000
--- a/docs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-monitor.png
+++ /dev/null
Binary files differ
diff --git a/docs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-threads.png b/docs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-threads.png
deleted file mode 100755
index 2a9745ae..00000000
--- a/docs/development/devtools/images/20201016-1715-distr-stability-20201018T2040-threads.png
+++ /dev/null
Binary files differ
diff --git a/docs/development/devtools/images/dist_perf_statistics.PNG b/docs/development/devtools/images/dist_perf_statistics.PNG
new file mode 100644
index 00000000..eeefeeee
--- /dev/null
+++ b/docs/development/devtools/images/dist_perf_statistics.PNG
Binary files differ
diff --git a/docs/development/devtools/images/dist_perf_threshold.PNG b/docs/development/devtools/images/dist_perf_threshold.PNG
new file mode 100644
index 00000000..58fbffd1
--- /dev/null
+++ b/docs/development/devtools/images/dist_perf_threshold.PNG
Binary files differ
diff --git a/docs/development/devtools/images/dist_stability_monitor.PNG b/docs/development/devtools/images/dist_stability_monitor.PNG
new file mode 100644
index 00000000..83eae8cc
--- /dev/null
+++ b/docs/development/devtools/images/dist_stability_monitor.PNG
Binary files differ
diff --git a/docs/development/devtools/images/dist_stability_statistics.PNG b/docs/development/devtools/images/dist_stability_statistics.PNG
new file mode 100644
index 00000000..dce9b7cc
--- /dev/null
+++ b/docs/development/devtools/images/dist_stability_statistics.PNG
Binary files differ
diff --git a/docs/development/devtools/images/dist_stability_threads.PNG b/docs/development/devtools/images/dist_stability_threads.PNG
new file mode 100644
index 00000000..13e27c99
--- /dev/null
+++ b/docs/development/devtools/images/dist_stability_threads.PNG
Binary files differ
diff --git a/docs/development/devtools/images/dist_stability_threshold.PNG b/docs/development/devtools/images/dist_stability_threshold.PNG
new file mode 100644
index 00000000..d65e8cc3
--- /dev/null
+++ b/docs/development/devtools/images/dist_stability_threshold.PNG
Binary files differ