summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
-rw-r--r--docs/development/devtools/drools-s3p.rst37
-rw-r--r--docs/development/devtools/images/s3p-drools-1.pngbin234824 -> 302657 bytes
-rw-r--r--docs/development/devtools/images/s3p-drools-2.pngbin248426 -> 216610 bytes
-rw-r--r--docs/development/devtools/images/s3p-drools-3.pngbin160364 -> 141505 bytes
-rw-r--r--docs/development/devtools/xacml-s3p.rst22
5 files changed, 24 insertions, 35 deletions
diff --git a/docs/development/devtools/drools-s3p.rst b/docs/development/devtools/drools-s3p.rst
index 571e09a3..bc8b79b3 100644
--- a/docs/development/devtools/drools-s3p.rst
+++ b/docs/development/devtools/drools-s3p.rst
@@ -32,12 +32,11 @@ Other ONAP components exercised during the stability tests were:
- Policy API to create (and delete at the end of the tests) policies for each
scenario under test.
- Policy PAP to deploy (and undeploy at the end of the tests) policies for each scenario under test.
+- XACML PDP Stability test was running at the same time.
The following components are simulated during the tests.
-- SO actor for the vDNS use case.
-- APPC responses for the vCPE and vFW use cases.
-- AAI to answer queries for the use cases under test.
+- SDNR.
Stability Test of Policy PDP-D
******************************
@@ -45,30 +44,21 @@ Stability Test of Policy PDP-D
PDP-D performance
=================
-The tests focused on the following use cases:
+The tests focused on the following use cases running in parallel:
- vCPE
-- vDNS
-- vFirewall
+- SON O1
+- SON A1
-For 72 hours the following 5 scenarios ran in parallel:
-
-- vCPE success scenario
-- vDNS success scenario.
-- vFirewall success scenario.
-- vCPE failure scenario (simulates a failure scenario returned by simulated APPC recipient through DMaaP).
-- vDNS failure scenario (simulates a failure by introducing in the DCAE ONSET a non-existent vserver-name reference).
-
-Five threads ran in parallel, one for each scenario, back to back with no pauses. The transactions were initiated
-by each jmeter thread group. Each thread initiated a transaction, monitored the transaction, and
-as soon as the transaction ending was detected, it initiated the next one.
+Three threads ran in parallel, one for each scenario. The transactions were initiated
+by each jmeter thread group. Each thread initiated a transaction, monitored the transaction, and
+started the next one 250 ms. later.
The results are illustrated on the following graphs:
.. image:: images/s3p-drools-1.png
.. image:: images/s3p-drools-2.png
.. image:: images/s3p-drools-3.png
-.. image:: images/s3p-drools-4.png
Commentary
@@ -79,13 +69,6 @@ final output of jmeter:
.. code-block:: bash
- summary = 37705505 in 72:00:56 = 145.4/s Avg: 30 Min: 0 Max: 20345 Err: 360852 (0.96%)
-
-The 1% errors were found to be related to the nature of the run, where each one of the 5 use case
-threads run without pauses starting one after the other a new round of their assigned control loop.
-It has been found that at times, the release time of the lock (which requires DB operations) outruns
-the initiation of the next control loop (using the same resource), therefore the newly initiated control
-loop fails. In reality, this scenario with the same resource being used back to back in consecutive control
-loop rounds will be unlikely.
-
+ summary = 4751546 in 72:00:37 = 18.3/s Avg: 150 Min: 0 Max: 15087 Err: 47891 (1.01%)
+Sporadic database errors have been observed and seem related to the 1% failure percentage rate.
diff --git a/docs/development/devtools/images/s3p-drools-1.png b/docs/development/devtools/images/s3p-drools-1.png
index 5dc70c57..3c1e06f7 100644
--- a/docs/development/devtools/images/s3p-drools-1.png
+++ b/docs/development/devtools/images/s3p-drools-1.png
Binary files differ
diff --git a/docs/development/devtools/images/s3p-drools-2.png b/docs/development/devtools/images/s3p-drools-2.png
index e985a712..7e124716 100644
--- a/docs/development/devtools/images/s3p-drools-2.png
+++ b/docs/development/devtools/images/s3p-drools-2.png
Binary files differ
diff --git a/docs/development/devtools/images/s3p-drools-3.png b/docs/development/devtools/images/s3p-drools-3.png
index 8f2a1d4c..50f2c148 100644
--- a/docs/development/devtools/images/s3p-drools-3.png
+++ b/docs/development/devtools/images/s3p-drools-3.png
Binary files differ
diff --git a/docs/development/devtools/xacml-s3p.rst b/docs/development/devtools/xacml-s3p.rst
index 57709ad0..c52a21ab 100644
--- a/docs/development/devtools/xacml-s3p.rst
+++ b/docs/development/devtools/xacml-s3p.rst
@@ -9,8 +9,8 @@
##########################
-Performance Test of Policy XACML PDP
-************************************
+Performance Test of Policy XACML PDP (Jakarta)
+**********************************************
The Performance test was executed by performing requests
against the Policy RESTful APIs.
@@ -69,9 +69,15 @@ The test was run for 20 minutes with 10 users (i.e., threads), with the followin
Stability Test of Policy XACML PDP
-************************************
+**********************************
-The stability test were executed in the same lab. The test was run via jmeter.
+This test was run using jmeter on a default
+ONAP installation in the Policy tenant in UNH.
+
+The Agent VMs in this lab have the following configuration:
+
+- 16GB RAM
+- 8 VCPU
Summary
=======
@@ -120,9 +126,9 @@ The stability summary results were reported by JMeter with the following summary
.. code-block:: bash
- summary = 997436933 in 71:59:45 = 3848.4/s Avg: 0 Min: 0 Max: 1480 Err: 0 (0.00%)
+ summary = 941639699 in 71:59:36 = 3633.2/s Avg: 1 Min: 0 Max: 842 Err: 0 (0.00%)
-The XACML PDP offered very good performance with JMeter for the traffic mix described above, using 3848 threads per second
-to inject the traffic load. The average transaction time is insignificant. The maximum transaction time of 1480ms.
-occured in the beginning of the run while the JVM was warming up.
+The XACML PDP offered very good performance with JMeter for the traffic mix described above.
+The average transaction time is insignificant. The maximum transaction time of 842 ms.
+There was a Drools stability test running in parallel, hence the actual load was higher.