diff options
Diffstat (limited to 'docs/development')
| -rw-r--r-- | docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png | bin | 172847 -> 83623 bytes | |||
| -rw-r--r-- | docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png | bin | 180140 -> 106062 bytes | |||
| -rw-r--r-- | docs/development/devtools/testing/s3p/apex-s3p.rst | 26 | ||||
| -rw-r--r-- | docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png | bin | 140061 -> 207292 bytes | |||
| -rw-r--r-- | docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png | bin | 186214 -> 230292 bytes | |||
| -rw-r--r-- | docs/development/devtools/testing/s3p/api-s3p.rst | 4 |
6 files changed, 14 insertions, 16 deletions
diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png Binary files differindex 67aa529f..7ddb0a67 100644 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png +++ b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png Binary files differindex c849204f..8dddd470 100644 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png +++ b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png diff --git a/docs/development/devtools/testing/s3p/apex-s3p.rst b/docs/development/devtools/testing/s3p/apex-s3p.rst index f756e4b5..0bbb0363 100644 --- a/docs/development/devtools/testing/s3p/apex-s3p.rst +++ b/docs/development/devtools/testing/s3p/apex-s3p.rst @@ -19,7 +19,7 @@ Deploying ONAP using OOM ------------------------ APEX-PDP along with all policy components are deployed as part of a full Policy Framework deployment. -At a minimum, the following components are needed: policy, mariadb-galera, prometheus and dmaap. +At a minimum, the following components are needed: policy, mariadb-galera, prometheus and kafka. The S3P tests utilise the ./run-s3p-tests script in the apex component. This will setup the microk8s environment, deploy policy and prometheus, expose the services so they can be reached by JMeter, install JMeter and run the tests based on @@ -28,11 +28,11 @@ the arguments provided. Set up policy-models-simulator ------------------------------ -Policy-models-simulator is deployed to use the DMaaP simulator during policy execution. +Kafka is deployed and is used during policy execution. Simulator configurations used are available in apex-pdp repository: testsuites/apex-pdp-stability/src/main/resources/simulatorConfig/ -The published port 30904 is used in JMeter for the DMaaP simulator. +The published port 29092 is used in JMeter for the Kafka. JMeter Tests ------------ @@ -48,8 +48,8 @@ Both tests were run via jMeter. .. Note:: Policy executions are validated in a stricter fashion during the tests. - There are test cases where up to 80 events are expected on the DMaaP topic. - DMaaP simulator is used to keep it simple and avoid any message pickup timing related issues. + There are test cases where up to 80 events are expected on the Kafka topic. + Kafka is used to keep it simple and avoid any message pickup timing related issues. Stability Test of APEX-PDP ++++++++++++++++++++++++++ @@ -77,20 +77,18 @@ Once the policies are created and deployed to APEX-PDP by the setup thread, five - **Healthcheck** - checks the health status of APEX-PDP - **Prometheus Metrics** - checks that APEX-PDP is exposing prometheus metrics -- **Test Simplecontrolloop policy success case** - Send a trigger event to *unauthenticated.DCAE_CL_OUTPUT* DMaaP topic. +- **Test Simplecontrolloop policy success case** - Send a trigger event to *unauthenticated.DCAE_CL_OUTPUT* Kafka topic. If the policy execution is successful, 3 different notification events are sent to *APEX-CL-MGT* topic by each one of the 5 threads. So, it is checked if 15 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. -- **Test Simplecontrolloop policy failure case** - Send a trigger event with invalid pnfName to *unauthenticated.DCAE_CL_OUTPUT* DMaaP topic. +- **Test Simplecontrolloop policy failure case** - Send a trigger event with invalid pnfName to *unauthenticated.DCAE_CL_OUTPUT* Kafka topic. The policy execution is expected to fail due to AAI failure response. 2 notification events are expected on *APEX-CL-MGT* topic by a thread in this case. It is checked if 10 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. -- **Test Example policy success case** - Send a trigger event to *unauthenticated.DCAE_POLICY_EXAMPLE_OUTPUT* DMaaP topic. +- **Test Example policy success case** - Send a trigger event to *unauthenticated.DCAE_POLICY_EXAMPLE_OUTPUT* Kafka topic. If the policy execution is successful, 4 different notification events are sent to *APEX-CL-MGT* topic by each one of the 5 threads. So, it is checked if 20 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. -- **Test Example policy failure case** - Send a trigger event with invalid vnfName to *unauthenticated.DCAE_POLICY_EXAMPLE_OUTPUT* DMaaP topic. +- **Test Example policy failure case** - Send a trigger event with invalid vnfName to *unauthenticated.DCAE_POLICY_EXAMPLE_OUTPUT* Kafka topic. The policy execution is expected to fail due to AAI failure response. 2 notification events are expected on *APEX-CL-MGT* topic by a thread in this case. So, it is checked if 10 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. -- **Clean up DMaaP notification topic** - DMaaP notification topic which is *APEX-CL-MGT* is cleaned up after each test to make sure that one failure doesn't lead to cascading errors. - Teardown Phase """""""""""""" @@ -119,7 +117,7 @@ The following steps can be used to configure the parameters of test plan. API_PORT Port number of API for making REST API calls such as create/delete of policy APEX_PORT Port number of APEX for making REST API calls such as healthcheck/metrics SIM_HOST IP Address or hostname running policy-models-simulator - DMAAP_PORT Port number of DMaaP simulator for making REST API calls such as reading notification events + KAFKA_PORT Port number of Kafka bootstrap server for sending message events wait Wait time if required after a request (in milliseconds) threads Number of threads to run test cases in parallel threadsTimeOutInMs Synchronization timer for threads running in parallel (in milliseconds) @@ -143,7 +141,7 @@ Stability test plan was triggered for 72 hours. There were no failures during th ======================= ================= ================== ================================== **Total # of requests** **Success %** **Error %** **Average time taken per request** ======================= ================= ================== ================================== -112245 99.47 % 0.53 % 2.309 sec. +312366 100 % 0 % 4148ms ======================= ================= ================== ================================== **JMeter Screenshot** @@ -191,7 +189,7 @@ Test results are shown as below. ======================= ================= ================== ================================== **Total # of requests** **Success %** **Error %** **Average time taken per request** ======================= ================= ================== ================================== -12486 99.32 % 0.68 % 576.64 ms +344624 100 % 0 % 4178 ms ======================= ================= ================== ================================== **JMeter Screenshot** diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png Binary files differindex 4edc144a..8dfbf55f 100644 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png +++ b/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png Binary files differindex 4314809c..68b654c2 100644 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png +++ b/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png diff --git a/docs/development/devtools/testing/s3p/api-s3p.rst b/docs/development/devtools/testing/s3p/api-s3p.rst index c34ec06e..c3bbc9e9 100644 --- a/docs/development/devtools/testing/s3p/api-s3p.rst +++ b/docs/development/devtools/testing/s3p/api-s3p.rst @@ -126,7 +126,7 @@ The load was performed against a non-tweaked Policy docker deployment. ======================= ============= =========== =============================== =============================== =============================== **Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request** ======================= ============= =========== =============================== =============================== =============================== - 730178 100% 2.8 1766 ms 127 ms 107029 ms + 214617 100% 2.8 6028 ms 206 ms 115153 ms ======================= ============= =========== =============================== =============================== =============================== .. image:: api-s3p-results/api-s3p-jm-1_J.png @@ -192,7 +192,7 @@ Running/Triggering performance test will be the same as stability test. That is, ======================= ============= =========== =============================== =============================== =============================== **Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request** ======================= ============= =========== =============================== =============================== =============================== - 56740 100% 6.3 6340 ms 72 ms 65557 ms + 1671 99.7% 6.3 108379 ms 223 ms 1921999 ms ======================= ============= =========== =============================== =============================== =============================== .. image:: api-s3p-results/api-s3p-jm-2_J.png |
