diff options
| author |   waynedunican <wayne.dunican@est.tech> | 2024-11-29 09:54:10 +0000 |
|---|---|---|
| committer | waynedunican <wayne.dunican@est.tech> | 2024-11-29 10:35:12 +0000 |
| commit | 4580ecffc9ebc5a3dc4e0e20e73f5f569eb508fb (patch) | |
| tree | 0a6e18aff94ffce5a111fd25c915e057db9fd1a4 /docs | |
| parent | db14aad1c5ab38c2313e2edfbad44887bb6f980a (diff) | |
Add S3P testing documentation
- Add s3p-test-overview doc
- Remove s3p results from previous release
Issue-ID: POLICY-5179
Change-Id: Ie97ed5037f7a07896708b20d28ed6ab266e69dc2
Signed-off-by: waynedunican <wayne.dunican@est.tech>
Diffstat (limited to 'docs')
53 files changed, 119 insertions, 2596 deletions
diff --git a/docs/development/devtools/devtools.rst b/docs/development/devtools/devtools.rst index 4e63fdbc..de0a6259 100644 --- a/docs/development/devtools/devtools.rst +++ b/docs/development/devtools/devtools.rst @@ -303,14 +303,8 @@ to developers to become familiar with the Policy Framework components and test a .. toctree:: :maxdepth: 2 + testing/s3p/s3p-test-overview.rst testing/s3p/run-s3p.rst - testing/s3p/api-s3p.rst - testing/s3p/pap-s3p.rst - testing/s3p/apex-s3p.rst - testing/s3p/drools-s3p.rst - testing/s3p/xacml-s3p.rst - testing/s3p/distribution-s3p.rst - testing/s3p/clamp-s3p.rst Running the Pairwise Tests diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_metrics_after_72h.txt b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_metrics_after_72h.txt deleted file mode 100644 index 56f13907..00000000 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_metrics_after_72h.txt +++ /dev/null @@ -1,316 +0,0 @@ -# HELP jvm_threads_current Current thread count of a JVM -# TYPE jvm_threads_current gauge -jvm_threads_current 32.0 -# HELP jvm_threads_daemon Daemon thread count of a JVM -# TYPE jvm_threads_daemon gauge -jvm_threads_daemon 17.0 -# HELP jvm_threads_peak Peak thread count of a JVM -# TYPE jvm_threads_peak gauge -jvm_threads_peak 81.0 -# HELP jvm_threads_started_total Started thread count of a JVM -# TYPE jvm_threads_started_total counter -jvm_threads_started_total 423360.0 -# HELP jvm_threads_deadlocked Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers -# TYPE jvm_threads_deadlocked gauge -jvm_threads_deadlocked 0.0 -# HELP jvm_threads_deadlocked_monitor Cycles of JVM-threads that are in deadlock waiting to acquire object monitors -# TYPE jvm_threads_deadlocked_monitor gauge -jvm_threads_deadlocked_monitor 0.0 -# HELP jvm_threads_state Current count of threads by state -# TYPE jvm_threads_state gauge -jvm_threads_state{state="BLOCKED",} 0.0 -jvm_threads_state{state="TIMED_WAITING",} 11.0 -jvm_threads_state{state="NEW",} 0.0 -jvm_threads_state{state="RUNNABLE",} 7.0 -jvm_threads_state{state="TERMINATED",} 0.0 -jvm_threads_state{state="WAITING",} 14.0 -# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds. -# TYPE process_cpu_seconds_total counter -process_cpu_seconds_total 16418.06 -# HELP process_start_time_seconds Start time of the process since unix epoch in seconds. -# TYPE process_start_time_seconds gauge -process_start_time_seconds 1.651077494162E9 -# HELP process_open_fds Number of open file descriptors. -# TYPE process_open_fds gauge -process_open_fds 357.0 -# HELP process_max_fds Maximum number of open file descriptors. -# TYPE process_max_fds gauge -process_max_fds 1048576.0 -# HELP process_virtual_memory_bytes Virtual memory size in bytes. -# TYPE process_virtual_memory_bytes gauge -process_virtual_memory_bytes 1.0165403648E10 -# HELP process_resident_memory_bytes Resident memory size in bytes. -# TYPE process_resident_memory_bytes gauge -process_resident_memory_bytes 5.58034944E8 -# HELP pdpa_engine_event_executions Total number of APEX events processed by the engine. -# TYPE pdpa_engine_event_executions gauge -pdpa_engine_event_executions{engine_instance_id="NSOApexEngine-1:0.0.1",} 30743.0 -pdpa_engine_event_executions{engine_instance_id="NSOApexEngine-4:0.0.1",} 30766.0 -pdpa_engine_event_executions{engine_instance_id="NSOApexEngine-3:0.0.1",} 30722.0 -pdpa_engine_event_executions{engine_instance_id="NSOApexEngine-0:0.0.1",} 30727.0 -pdpa_engine_event_executions{engine_instance_id="NSOApexEngine-2:0.0.1",} 30742.0 -# HELP jvm_buffer_pool_used_bytes Used bytes of a given JVM buffer pool. -# TYPE jvm_buffer_pool_used_bytes gauge -jvm_buffer_pool_used_bytes{pool="mapped",} 0.0 -jvm_buffer_pool_used_bytes{pool="direct",} 3.3833905E7 -# HELP jvm_buffer_pool_capacity_bytes Bytes capacity of a given JVM buffer pool. -# TYPE jvm_buffer_pool_capacity_bytes gauge -jvm_buffer_pool_capacity_bytes{pool="mapped",} 0.0 -jvm_buffer_pool_capacity_bytes{pool="direct",} 3.3833904E7 -# HELP jvm_buffer_pool_used_buffers Used buffers of a given JVM buffer pool. -# TYPE jvm_buffer_pool_used_buffers gauge -jvm_buffer_pool_used_buffers{pool="mapped",} 0.0 -jvm_buffer_pool_used_buffers{pool="direct",} 15.0 -# HELP pdpa_policy_executions_total The total number of TOSCA policy executions. -# TYPE pdpa_policy_executions_total counter -# HELP pdpa_policy_deployments_total The total number of policy deployments. -# TYPE pdpa_policy_deployments_total counter -pdpa_policy_deployments_total{operation="deploy",status="TOTAL",} 5.0 -pdpa_policy_deployments_total{operation="undeploy",status="TOTAL",} 5.0 -pdpa_policy_deployments_total{operation="undeploy",status="SUCCESS",} 5.0 -pdpa_policy_deployments_total{operation="deploy",status="SUCCESS",} 5.0 -# HELP pdpa_engine_average_execution_time_seconds Average time taken to execute an APEX policy in seconds. -# TYPE pdpa_engine_average_execution_time_seconds gauge -pdpa_engine_average_execution_time_seconds{engine_instance_id="NSOApexEngine-1:0.0.1",} 0.00515235988680349 -pdpa_engine_average_execution_time_seconds{engine_instance_id="NSOApexEngine-4:0.0.1",} 0.00521845543782099 -pdpa_engine_average_execution_time_seconds{engine_instance_id="NSOApexEngine-3:0.0.1",} 0.005200800729119198 -pdpa_engine_average_execution_time_seconds{engine_instance_id="NSOApexEngine-0:0.0.1",} 0.005191785725908804 -pdpa_engine_average_execution_time_seconds{engine_instance_id="NSOApexEngine-2:0.0.1",} 0.0051784854596317684 -# HELP pdpa_engine_state State of the APEX engine as integers mapped as - 0:UNDEFINED, 1:STOPPED, 2:READY, 3:EXECUTING, 4:STOPPING -# TYPE pdpa_engine_state gauge -pdpa_engine_state{engine_instance_id="NSOApexEngine-1:0.0.1",} 1.0 -pdpa_engine_state{engine_instance_id="NSOApexEngine-4:0.0.1",} 1.0 -pdpa_engine_state{engine_instance_id="NSOApexEngine-3:0.0.1",} 1.0 -pdpa_engine_state{engine_instance_id="NSOApexEngine-0:0.0.1",} 1.0 -pdpa_engine_state{engine_instance_id="NSOApexEngine-2:0.0.1",} 1.0 -# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds. -# TYPE jvm_gc_collection_seconds summary -jvm_gc_collection_seconds_count{gc="Copy",} 5883.0 -jvm_gc_collection_seconds_sum{gc="Copy",} 97.808 -jvm_gc_collection_seconds_count{gc="MarkSweepCompact",} 3.0 -jvm_gc_collection_seconds_sum{gc="MarkSweepCompact",} 0.357 -# HELP pdpa_engine_last_start_timestamp_epoch Epoch timestamp of the instance when engine was last started. -# TYPE pdpa_engine_last_start_timestamp_epoch gauge -pdpa_engine_last_start_timestamp_epoch{engine_instance_id="NSOApexEngine-1:0.0.1",} 0.0 -pdpa_engine_last_start_timestamp_epoch{engine_instance_id="NSOApexEngine-4:0.0.1",} 0.0 -pdpa_engine_last_start_timestamp_epoch{engine_instance_id="NSOApexEngine-3:0.0.1",} 0.0 -pdpa_engine_last_start_timestamp_epoch{engine_instance_id="NSOApexEngine-0:0.0.1",} 0.0 -pdpa_engine_last_start_timestamp_epoch{engine_instance_id="NSOApexEngine-2:0.0.1",} 0.0 -# HELP jvm_memory_pool_allocated_bytes_total Total bytes allocated in a given JVM memory pool. Only updated after GC, not continuously. -# TYPE jvm_memory_pool_allocated_bytes_total counter -jvm_memory_pool_allocated_bytes_total{pool="Eden Space",} 8.29800936264E11 -jvm_memory_pool_allocated_bytes_total{pool="CodeHeap 'profiled nmethods'",} 4.839232E7 -jvm_memory_pool_allocated_bytes_total{pool="CodeHeap 'non-profiled nmethods'",} 3.5181056E7 -jvm_memory_pool_allocated_bytes_total{pool="Compressed Class Space",} 8194120.0 -jvm_memory_pool_allocated_bytes_total{pool="Metaspace",} 7.7729144E7 -jvm_memory_pool_allocated_bytes_total{pool="Tenured Gen",} 1.41180272E8 -jvm_memory_pool_allocated_bytes_total{pool="Survivor Space",} 4.78761928E8 -jvm_memory_pool_allocated_bytes_total{pool="CodeHeap 'non-nmethods'",} 1392128.0 -# HELP pdpa_engine_uptime Time elapsed since the engine was started. -# TYPE pdpa_engine_uptime gauge -pdpa_engine_uptime{engine_instance_id="NSOApexEngine-1:0.0.1",} 259200.522 -pdpa_engine_uptime{engine_instance_id="NSOApexEngine-4:0.0.1",} 259200.751 -pdpa_engine_uptime{engine_instance_id="NSOApexEngine-3:0.0.1",} 259200.678 -pdpa_engine_uptime{engine_instance_id="NSOApexEngine-0:0.0.1",} 259200.439 -pdpa_engine_uptime{engine_instance_id="NSOApexEngine-2:0.0.1",} 259200.601 -# HELP pdpa_engine_last_execution_time Time taken to execute the last APEX policy in seconds. -# TYPE pdpa_engine_last_execution_time histogram -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.005",} 24726.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.01",} 50195.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.025",} 70836.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.05",} 71947.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.075",} 71996.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.1",} 72001.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.25",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.5",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="0.75",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="1.0",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="2.5",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="5.0",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="7.5",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="10.0",} 72002.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-1:0.0.1",le="+Inf",} 72002.0 -pdpa_engine_last_execution_time_count{engine_instance_id="NSOApexEngine-1:0.0.1",} 72002.0 -pdpa_engine_last_execution_time_sum{engine_instance_id="NSOApexEngine-1:0.0.1",} 609.1939999998591 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.005",} 24512.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.01",} 50115.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.025",} 70746.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.05",} 71918.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.075",} 71966.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.1",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.25",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.5",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="0.75",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="1.0",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="2.5",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="5.0",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="7.5",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="10.0",} 71967.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-4:0.0.1",le="+Inf",} 71967.0 -pdpa_engine_last_execution_time_count{engine_instance_id="NSOApexEngine-4:0.0.1",} 71967.0 -pdpa_engine_last_execution_time_sum{engine_instance_id="NSOApexEngine-4:0.0.1",} 610.3469999998522 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.005",} 24607.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.01",} 50182.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.025",} 70791.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.05",} 71929.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.075",} 71965.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.1",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.25",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.5",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="0.75",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="1.0",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="2.5",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="5.0",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="7.5",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="10.0",} 71970.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-3:0.0.1",le="+Inf",} 71970.0 -pdpa_engine_last_execution_time_count{engine_instance_id="NSOApexEngine-3:0.0.1",} 71970.0 -pdpa_engine_last_execution_time_sum{engine_instance_id="NSOApexEngine-3:0.0.1",} 608.8539999998619 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.005",} 24623.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.01",} 50207.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.025",} 70783.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.05",} 71934.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.075",} 71981.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.1",} 71986.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.25",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.5",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="0.75",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="1.0",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="2.5",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="5.0",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="7.5",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="10.0",} 71988.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-0:0.0.1",le="+Inf",} 71988.0 -pdpa_engine_last_execution_time_count{engine_instance_id="NSOApexEngine-0:0.0.1",} 71988.0 -pdpa_engine_last_execution_time_sum{engine_instance_id="NSOApexEngine-0:0.0.1",} 610.5579999998558 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.005",} 24594.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.01",} 50131.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.025",} 70816.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.05",} 71905.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.075",} 71959.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.1",} 71961.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.25",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.5",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="0.75",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="1.0",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="2.5",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="5.0",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="7.5",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="10.0",} 71962.0 -pdpa_engine_last_execution_time_bucket{engine_instance_id="NSOApexEngine-2:0.0.1",le="+Inf",} 71962.0 -pdpa_engine_last_execution_time_count{engine_instance_id="NSOApexEngine-2:0.0.1",} 71962.0 -pdpa_engine_last_execution_time_sum{engine_instance_id="NSOApexEngine-2:0.0.1",} 608.3549999998555 -# HELP jvm_memory_objects_pending_finalization The number of objects waiting in the finalizer queue. -# TYPE jvm_memory_objects_pending_finalization gauge -jvm_memory_objects_pending_finalization 0.0 -# HELP jvm_memory_bytes_used Used bytes of a given JVM memory area. -# TYPE jvm_memory_bytes_used gauge -jvm_memory_bytes_used{area="heap",} 1.90274552E8 -jvm_memory_bytes_used{area="nonheap",} 1.16193856E8 -# HELP jvm_memory_bytes_committed Committed (bytes) of a given JVM memory area. -# TYPE jvm_memory_bytes_committed gauge -jvm_memory_bytes_committed{area="heap",} 5.10984192E8 -jvm_memory_bytes_committed{area="nonheap",} 1.56127232E8 -# HELP jvm_memory_bytes_max Max (bytes) of a given JVM memory area. -# TYPE jvm_memory_bytes_max gauge -jvm_memory_bytes_max{area="heap",} 8.151564288E9 -jvm_memory_bytes_max{area="nonheap",} -1.0 -# HELP jvm_memory_bytes_init Initial bytes of a given JVM memory area. -# TYPE jvm_memory_bytes_init gauge -jvm_memory_bytes_init{area="heap",} 5.28482304E8 -jvm_memory_bytes_init{area="nonheap",} 7667712.0 -# HELP jvm_memory_pool_bytes_used Used bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_used gauge -jvm_memory_pool_bytes_used{pool="CodeHeap 'non-nmethods'",} 1353600.0 -jvm_memory_pool_bytes_used{pool="Metaspace",} 7.7729144E7 -jvm_memory_pool_bytes_used{pool="Tenured Gen",} 1.41180272E8 -jvm_memory_pool_bytes_used{pool="CodeHeap 'profiled nmethods'",} 4831104.0 -jvm_memory_pool_bytes_used{pool="Eden Space",} 4.5145032E7 -jvm_memory_pool_bytes_used{pool="Survivor Space",} 3949248.0 -jvm_memory_pool_bytes_used{pool="Compressed Class Space",} 8194120.0 -jvm_memory_pool_bytes_used{pool="CodeHeap 'non-profiled nmethods'",} 2.4085888E7 -# HELP jvm_memory_pool_bytes_committed Committed bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_committed gauge -jvm_memory_pool_bytes_committed{pool="CodeHeap 'non-nmethods'",} 2555904.0 -jvm_memory_pool_bytes_committed{pool="Metaspace",} 8.5348352E7 -jvm_memory_pool_bytes_committed{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_bytes_committed{pool="CodeHeap 'profiled nmethods'",} 3.3030144E7 -jvm_memory_pool_bytes_committed{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_bytes_committed{pool="Survivor Space",} 1.7629184E7 -jvm_memory_pool_bytes_committed{pool="Compressed Class Space",} 9175040.0 -jvm_memory_pool_bytes_committed{pool="CodeHeap 'non-profiled nmethods'",} 2.6017792E7 -# HELP jvm_memory_pool_bytes_max Max bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_max gauge -jvm_memory_pool_bytes_max{pool="CodeHeap 'non-nmethods'",} 5828608.0 -jvm_memory_pool_bytes_max{pool="Metaspace",} -1.0 -jvm_memory_pool_bytes_max{pool="Tenured Gen",} 5.621809152E9 -jvm_memory_pool_bytes_max{pool="CodeHeap 'profiled nmethods'",} 1.22912768E8 -jvm_memory_pool_bytes_max{pool="Eden Space",} 2.248671232E9 -jvm_memory_pool_bytes_max{pool="Survivor Space",} 2.81083904E8 -jvm_memory_pool_bytes_max{pool="Compressed Class Space",} 1.073741824E9 -jvm_memory_pool_bytes_max{pool="CodeHeap 'non-profiled nmethods'",} 1.22916864E8 -# HELP jvm_memory_pool_bytes_init Initial bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_init gauge -jvm_memory_pool_bytes_init{pool="CodeHeap 'non-nmethods'",} 2555904.0 -jvm_memory_pool_bytes_init{pool="Metaspace",} 0.0 -jvm_memory_pool_bytes_init{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_bytes_init{pool="CodeHeap 'profiled nmethods'",} 2555904.0 -jvm_memory_pool_bytes_init{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_bytes_init{pool="Survivor Space",} 1.7563648E7 -jvm_memory_pool_bytes_init{pool="Compressed Class Space",} 0.0 -jvm_memory_pool_bytes_init{pool="CodeHeap 'non-profiled nmethods'",} 2555904.0 -# HELP jvm_memory_pool_collection_used_bytes Used bytes after last collection of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_used_bytes gauge -jvm_memory_pool_collection_used_bytes{pool="Tenured Gen",} 3.853812E7 -jvm_memory_pool_collection_used_bytes{pool="Eden Space",} 0.0 -jvm_memory_pool_collection_used_bytes{pool="Survivor Space",} 3949248.0 -# HELP jvm_memory_pool_collection_committed_bytes Committed after last collection bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_committed_bytes gauge -jvm_memory_pool_collection_committed_bytes{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_collection_committed_bytes{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_collection_committed_bytes{pool="Survivor Space",} 1.7629184E7 -# HELP jvm_memory_pool_collection_max_bytes Max bytes after last collection of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_max_bytes gauge -jvm_memory_pool_collection_max_bytes{pool="Tenured Gen",} 5.621809152E9 -jvm_memory_pool_collection_max_bytes{pool="Eden Space",} 2.248671232E9 -jvm_memory_pool_collection_max_bytes{pool="Survivor Space",} 2.81083904E8 -# HELP jvm_memory_pool_collection_init_bytes Initial after last collection bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_init_bytes gauge -jvm_memory_pool_collection_init_bytes{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_collection_init_bytes{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_collection_init_bytes{pool="Survivor Space",} 1.7563648E7 -# HELP jvm_classes_loaded The number of classes that are currently loaded in the JVM -# TYPE jvm_classes_loaded gauge -jvm_classes_loaded 11386.0 -# HELP jvm_classes_loaded_total The total number of classes that have been loaded since the JVM has started execution -# TYPE jvm_classes_loaded_total counter -jvm_classes_loaded_total 11448.0 -# HELP jvm_classes_unloaded_total The total number of classes that have been unloaded since the JVM has started execution -# TYPE jvm_classes_unloaded_total counter -jvm_classes_unloaded_total 62.0 -# HELP jvm_info VM version info -# TYPE jvm_info gauge -jvm_info{runtime="OpenJDK Runtime Environment",vendor="Alpine",version="11.0.9+11-alpine-r1",} 1.0 -# HELP jvm_memory_pool_allocated_bytes_created Total bytes allocated in a given JVM memory pool. Only updated after GC, not continuously. -# TYPE jvm_memory_pool_allocated_bytes_created gauge -jvm_memory_pool_allocated_bytes_created{pool="Eden Space",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="CodeHeap 'profiled nmethods'",} 1.651077501657E9 -jvm_memory_pool_allocated_bytes_created{pool="CodeHeap 'non-profiled nmethods'",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Compressed Class Space",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Metaspace",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Tenured Gen",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Survivor Space",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="CodeHeap 'non-nmethods'",} 1.651077501662E9 -# HELP pdpa_engine_last_execution_time_created Time taken to execute the last APEX policy in seconds. -# TYPE pdpa_engine_last_execution_time_created gauge -pdpa_engine_last_execution_time_created{engine_instance_id="NSOApexEngine-1:0.0.1",} 1.651080501294E9 -pdpa_engine_last_execution_time_created{engine_instance_id="NSOApexEngine-4:0.0.1",} 1.651080501295E9 -pdpa_engine_last_execution_time_created{engine_instance_id="NSOApexEngine-3:0.0.1",} 1.651080501295E9 -pdpa_engine_last_execution_time_created{engine_instance_id="NSOApexEngine-0:0.0.1",} 1.651080501294E9 -pdpa_engine_last_execution_time_created{engine_instance_id="NSOApexEngine-2:0.0.1",} 1.651080501294E9 -# HELP pdpa_policy_deployments_created The total number of policy deployments. -# TYPE pdpa_policy_deployments_created gauge -pdpa_policy_deployments_created{operation="deploy",status="TOTAL",} 1.651080501289E9 -pdpa_policy_deployments_created{operation="undeploy",status="TOTAL",} 1.651081148331E9 -pdpa_policy_deployments_created{operation="undeploy",status="SUCCESS",} 1.651081148331E9 -pdpa_policy_deployments_created{operation="deploy",status="SUCCESS",} 1.651080501289E9 diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_metrics_before_72h.txt b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_metrics_before_72h.txt deleted file mode 100644 index 4a3d8835..00000000 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_metrics_before_72h.txt +++ /dev/null @@ -1,175 +0,0 @@ -# HELP jvm_threads_current Current thread count of a JVM -# TYPE jvm_threads_current gauge -jvm_threads_current 31.0 -# HELP jvm_threads_daemon Daemon thread count of a JVM -# TYPE jvm_threads_daemon gauge -jvm_threads_daemon 16.0 -# HELP jvm_threads_peak Peak thread count of a JVM -# TYPE jvm_threads_peak gauge -jvm_threads_peak 31.0 -# HELP jvm_threads_started_total Started thread count of a JVM -# TYPE jvm_threads_started_total counter -jvm_threads_started_total 32.0 -# HELP jvm_threads_deadlocked Cycles of JVM-threads that are in deadlock waiting to acquire object monitors or ownable synchronizers -# TYPE jvm_threads_deadlocked gauge -jvm_threads_deadlocked 0.0 -# HELP jvm_threads_deadlocked_monitor Cycles of JVM-threads that are in deadlock waiting to acquire object monitors -# TYPE jvm_threads_deadlocked_monitor gauge -jvm_threads_deadlocked_monitor 0.0 -# HELP jvm_threads_state Current count of threads by state -# TYPE jvm_threads_state gauge -jvm_threads_state{state="BLOCKED",} 0.0 -jvm_threads_state{state="TIMED_WAITING",} 11.0 -jvm_threads_state{state="NEW",} 0.0 -jvm_threads_state{state="RUNNABLE",} 7.0 -jvm_threads_state{state="TERMINATED",} 0.0 -jvm_threads_state{state="WAITING",} 13.0 -# HELP jvm_gc_collection_seconds Time spent in a given JVM garbage collector in seconds. -# TYPE jvm_gc_collection_seconds summary -jvm_gc_collection_seconds_count{gc="Copy",} 2.0 -jvm_gc_collection_seconds_sum{gc="Copy",} 0.059 -jvm_gc_collection_seconds_count{gc="MarkSweepCompact",} 2.0 -jvm_gc_collection_seconds_sum{gc="MarkSweepCompact",} 0.185 -# HELP process_cpu_seconds_total Total user and system CPU time spent in seconds. -# TYPE process_cpu_seconds_total counter -process_cpu_seconds_total 38.14 -# HELP process_start_time_seconds Start time of the process since unix epoch in seconds. -# TYPE process_start_time_seconds gauge -process_start_time_seconds 1.651077494162E9 -# HELP process_open_fds Number of open file descriptors. -# TYPE process_open_fds gauge -process_open_fds 355.0 -# HELP process_max_fds Maximum number of open file descriptors. -# TYPE process_max_fds gauge -process_max_fds 1048576.0 -# HELP process_virtual_memory_bytes Virtual memory size in bytes. -# TYPE process_virtual_memory_bytes gauge -process_virtual_memory_bytes 1.0070171648E10 -# HELP process_resident_memory_bytes Resident memory size in bytes. -# TYPE process_resident_memory_bytes gauge -process_resident_memory_bytes 2.9052928E8 -# HELP jvm_buffer_pool_used_bytes Used bytes of a given JVM buffer pool. -# TYPE jvm_buffer_pool_used_bytes gauge -jvm_buffer_pool_used_bytes{pool="mapped",} 0.0 -jvm_buffer_pool_used_bytes{pool="direct",} 187432.0 -# HELP jvm_buffer_pool_capacity_bytes Bytes capacity of a given JVM buffer pool. -# TYPE jvm_buffer_pool_capacity_bytes gauge -jvm_buffer_pool_capacity_bytes{pool="mapped",} 0.0 -jvm_buffer_pool_capacity_bytes{pool="direct",} 187432.0 -# HELP jvm_buffer_pool_used_buffers Used buffers of a given JVM buffer pool. -# TYPE jvm_buffer_pool_used_buffers gauge -jvm_buffer_pool_used_buffers{pool="mapped",} 0.0 -jvm_buffer_pool_used_buffers{pool="direct",} 9.0 -# HELP jvm_memory_pool_allocated_bytes_total Total bytes allocated in a given JVM memory pool. Only updated after GC, not continuously. -# TYPE jvm_memory_pool_allocated_bytes_total counter -jvm_memory_pool_allocated_bytes_total{pool="Eden Space",} 3.035482E8 -jvm_memory_pool_allocated_bytes_total{pool="CodeHeap 'profiled nmethods'",} 9772800.0 -jvm_memory_pool_allocated_bytes_total{pool="CodeHeap 'non-profiled nmethods'",} 2152064.0 -jvm_memory_pool_allocated_bytes_total{pool="Compressed Class Space",} 4912232.0 -jvm_memory_pool_allocated_bytes_total{pool="Metaspace",} 4.1337744E7 -jvm_memory_pool_allocated_bytes_total{pool="Tenured Gen",} 2.8136056E7 -jvm_memory_pool_allocated_bytes_total{pool="Survivor Space",} 6813240.0 -jvm_memory_pool_allocated_bytes_total{pool="CodeHeap 'non-nmethods'",} 1272320.0 -# HELP pdpa_policy_deployments_total The total number of policy deployments. -# TYPE pdpa_policy_deployments_total counter -# HELP jvm_memory_objects_pending_finalization The number of objects waiting in the finalizer queue. -# TYPE jvm_memory_objects_pending_finalization gauge -jvm_memory_objects_pending_finalization 0.0 -# HELP jvm_memory_bytes_used Used bytes of a given JVM memory area. -# TYPE jvm_memory_bytes_used gauge -jvm_memory_bytes_used{area="heap",} 9.5900224E7 -jvm_memory_bytes_used{area="nonheap",} 6.0285288E7 -# HELP jvm_memory_bytes_committed Committed (bytes) of a given JVM memory area. -# TYPE jvm_memory_bytes_committed gauge -jvm_memory_bytes_committed{area="heap",} 5.10984192E8 -jvm_memory_bytes_committed{area="nonheap",} 6.3922176E7 -# HELP jvm_memory_bytes_max Max (bytes) of a given JVM memory area. -# TYPE jvm_memory_bytes_max gauge -jvm_memory_bytes_max{area="heap",} 8.151564288E9 -jvm_memory_bytes_max{area="nonheap",} -1.0 -# HELP jvm_memory_bytes_init Initial bytes of a given JVM memory area. -# TYPE jvm_memory_bytes_init gauge -jvm_memory_bytes_init{area="heap",} 5.28482304E8 -jvm_memory_bytes_init{area="nonheap",} 7667712.0 -# HELP jvm_memory_pool_bytes_used Used bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_used gauge -jvm_memory_pool_bytes_used{pool="CodeHeap 'non-nmethods'",} 1272320.0 -jvm_memory_pool_bytes_used{pool="Metaspace",} 4.1681312E7 -jvm_memory_pool_bytes_used{pool="Tenured Gen",} 2.8136056E7 -jvm_memory_pool_bytes_used{pool="CodeHeap 'profiled nmethods'",} 1.0006912E7 -jvm_memory_pool_bytes_used{pool="Eden Space",} 6.5005376E7 -jvm_memory_pool_bytes_used{pool="Survivor Space",} 2758792.0 -jvm_memory_pool_bytes_used{pool="Compressed Class Space",} 4913352.0 -jvm_memory_pool_bytes_used{pool="CodeHeap 'non-profiled nmethods'",} 2411392.0 -# HELP jvm_memory_pool_bytes_committed Committed bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_committed gauge -jvm_memory_pool_bytes_committed{pool="CodeHeap 'non-nmethods'",} 2555904.0 -jvm_memory_pool_bytes_committed{pool="Metaspace",} 4.32128E7 -jvm_memory_pool_bytes_committed{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_bytes_committed{pool="CodeHeap 'profiled nmethods'",} 1.0092544E7 -jvm_memory_pool_bytes_committed{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_bytes_committed{pool="Survivor Space",} 1.7629184E7 -jvm_memory_pool_bytes_committed{pool="Compressed Class Space",} 5505024.0 -jvm_memory_pool_bytes_committed{pool="CodeHeap 'non-profiled nmethods'",} 2555904.0 -# HELP jvm_memory_pool_bytes_max Max bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_max gauge -jvm_memory_pool_bytes_max{pool="CodeHeap 'non-nmethods'",} 5828608.0 -jvm_memory_pool_bytes_max{pool="Metaspace",} -1.0 -jvm_memory_pool_bytes_max{pool="Tenured Gen",} 5.621809152E9 -jvm_memory_pool_bytes_max{pool="CodeHeap 'profiled nmethods'",} 1.22912768E8 -jvm_memory_pool_bytes_max{pool="Eden Space",} 2.248671232E9 -jvm_memory_pool_bytes_max{pool="Survivor Space",} 2.81083904E8 -jvm_memory_pool_bytes_max{pool="Compressed Class Space",} 1.073741824E9 -jvm_memory_pool_bytes_max{pool="CodeHeap 'non-profiled nmethods'",} 1.22916864E8 -# HELP jvm_memory_pool_bytes_init Initial bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_bytes_init gauge -jvm_memory_pool_bytes_init{pool="CodeHeap 'non-nmethods'",} 2555904.0 -jvm_memory_pool_bytes_init{pool="Metaspace",} 0.0 -jvm_memory_pool_bytes_init{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_bytes_init{pool="CodeHeap 'profiled nmethods'",} 2555904.0 -jvm_memory_pool_bytes_init{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_bytes_init{pool="Survivor Space",} 1.7563648E7 -jvm_memory_pool_bytes_init{pool="Compressed Class Space",} 0.0 -jvm_memory_pool_bytes_init{pool="CodeHeap 'non-profiled nmethods'",} 2555904.0 -# HELP jvm_memory_pool_collection_used_bytes Used bytes after last collection of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_used_bytes gauge -jvm_memory_pool_collection_used_bytes{pool="Tenured Gen",} 2.8136056E7 -jvm_memory_pool_collection_used_bytes{pool="Eden Space",} 0.0 -jvm_memory_pool_collection_used_bytes{pool="Survivor Space",} 2758792.0 -# HELP jvm_memory_pool_collection_committed_bytes Committed after last collection bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_committed_bytes gauge -jvm_memory_pool_collection_committed_bytes{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_collection_committed_bytes{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_collection_committed_bytes{pool="Survivor Space",} 1.7629184E7 -# HELP jvm_memory_pool_collection_max_bytes Max bytes after last collection of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_max_bytes gauge -jvm_memory_pool_collection_max_bytes{pool="Tenured Gen",} 5.621809152E9 -jvm_memory_pool_collection_max_bytes{pool="Eden Space",} 2.248671232E9 -jvm_memory_pool_collection_max_bytes{pool="Survivor Space",} 2.81083904E8 -# HELP jvm_memory_pool_collection_init_bytes Initial after last collection bytes of a given JVM memory pool. -# TYPE jvm_memory_pool_collection_init_bytes gauge -jvm_memory_pool_collection_init_bytes{pool="Tenured Gen",} 3.52321536E8 -jvm_memory_pool_collection_init_bytes{pool="Eden Space",} 1.41033472E8 -jvm_memory_pool_collection_init_bytes{pool="Survivor Space",} 1.7563648E7 -# HELP jvm_classes_loaded The number of classes that are currently loaded in the JVM -# TYPE jvm_classes_loaded gauge -jvm_classes_loaded 7378.0 -# HELP jvm_classes_loaded_total The total number of classes that have been loaded since the JVM has started execution -# TYPE jvm_classes_loaded_total counter -jvm_classes_loaded_total 7378.0 -# HELP jvm_classes_unloaded_total The total number of classes that have been unloaded since the JVM has started execution -# TYPE jvm_classes_unloaded_total counter -jvm_classes_unloaded_total 0.0 -# HELP jvm_info VM version info -# TYPE jvm_info gauge -jvm_info{runtime="OpenJDK Runtime Environment",vendor="Alpine",version="11.0.9+11-alpine-r1",} 1.0 -# HELP jvm_memory_pool_allocated_bytes_created Total bytes allocated in a given JVM memory pool. Only updated after GC, not continuously. -# TYPE jvm_memory_pool_allocated_bytes_created gauge -jvm_memory_pool_allocated_bytes_created{pool="Eden Space",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="CodeHeap 'profiled nmethods'",} 1.651077501657E9 -jvm_memory_pool_allocated_bytes_created{pool="CodeHeap 'non-profiled nmethods'",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Compressed Class Space",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Metaspace",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Tenured Gen",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="Survivor Space",} 1.651077501662E9 -jvm_memory_pool_allocated_bytes_created{pool="CodeHeap 'non-nmethods'",} 1.651077501662E9 diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png Binary files differdeleted file mode 100644 index 7ddb0a67..00000000 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_performance_results.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png Binary files differdeleted file mode 100644 index 8dddd470..00000000 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_stability_results.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_top_after_72h.png b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_top_after_72h.png Binary files differdeleted file mode 100644 index dafc7002..00000000 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_top_after_72h.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_top_before_72h.png b/docs/development/devtools/testing/s3p/apex-s3p-results/apex_top_before_72h.png Binary files differdeleted file mode 100644 index 2e2e7574..00000000 --- a/docs/development/devtools/testing/s3p/apex-s3p-results/apex_top_before_72h.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/apex-s3p.rst b/docs/development/devtools/testing/s3p/apex-s3p.rst deleted file mode 100644 index 0bbb0363..00000000 --- a/docs/development/devtools/testing/s3p/apex-s3p.rst +++ /dev/null @@ -1,207 +0,0 @@ -.. This work is licensed under a -.. Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _apex-s3p-label: - -.. toctree:: - :maxdepth: 2 - -Policy APEX PDP component -~~~~~~~~~~~~~~~~~~~~~~~~~ - -Both the Stability and the Performance tests were executed in a full Policy Framework deployment in a VM. - -Setup Details -+++++++++++++ - -Deploying ONAP using OOM ------------------------- - -APEX-PDP along with all policy components are deployed as part of a full Policy Framework deployment. -At a minimum, the following components are needed: policy, mariadb-galera, prometheus and kafka. - -The S3P tests utilise the ./run-s3p-tests script in the apex component. This will setup the microk8s environment, deploy -policy and prometheus, expose the services so they can be reached by JMeter, install JMeter and run the tests based on -the arguments provided. - -Set up policy-models-simulator ------------------------------- - -Kafka is deployed and is used during policy execution. - Simulator configurations used are available in apex-pdp repository: - testsuites/apex-pdp-stability/src/main/resources/simulatorConfig/ - -The published port 29092 is used in JMeter for the Kafka. - -JMeter Tests ------------- - -Two APEX policies are executed in the APEX-PDP engine, and are triggered by multiple threads during the tests. -Both tests were run via jMeter. - - Stability test script is available in apex-pdp repository: - testsuites/apex-pdp-stability/src/main/resources/apexPdpStabilityTestPlan.jmx - - Performance test script is available in apex-pdp repository: - testsuites/performance/performance-benchmark-test/src/main/resources/apexPdpPerformanceTestPlan.jmx - -.. Note:: - Policy executions are validated in a stricter fashion during the tests. - There are test cases where up to 80 events are expected on the Kafka topic. - Kafka is used to keep it simple and avoid any message pickup timing related issues. - -Stability Test of APEX-PDP -++++++++++++++++++++++++++ - -Test Plan ---------- - -The 72 hours stability test ran the following steps. - -Setup Phase -""""""""""" - -Policies are created and deployed to APEX-PDP during this phase. Only one thread is in action and this step is done only once. - -- **Create Policy onap.policies.apex.Simplecontrolloop** - creates the first APEX policy using policy/api component. - This is a sample policy used for PNF testing. -- **Create Policy onap.policies.apex.Example** - creates the second APEX policy using policy/api component. - This is a sample policy used for VNF testing. -- **Deploy Policies** - Deploy both the policies created to APEX-PDP using policy/pap component - -Main Phase -"""""""""" - -Once the policies are created and deployed to APEX-PDP by the setup thread, five threads execute the below tests for 72 hours. - -- **Healthcheck** - checks the health status of APEX-PDP -- **Prometheus Metrics** - checks that APEX-PDP is exposing prometheus metrics -- **Test Simplecontrolloop policy success case** - Send a trigger event to *unauthenticated.DCAE_CL_OUTPUT* Kafka topic. - If the policy execution is successful, 3 different notification events are sent to *APEX-CL-MGT* topic by each one of the 5 threads. - So, it is checked if 15 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. -- **Test Simplecontrolloop policy failure case** - Send a trigger event with invalid pnfName to *unauthenticated.DCAE_CL_OUTPUT* Kafka topic. - The policy execution is expected to fail due to AAI failure response. 2 notification events are expected on *APEX-CL-MGT* topic by a thread in this case. - It is checked if 10 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. -- **Test Example policy success case** - Send a trigger event to *unauthenticated.DCAE_POLICY_EXAMPLE_OUTPUT* Kafka topic. - If the policy execution is successful, 4 different notification events are sent to *APEX-CL-MGT* topic by each one of the 5 threads. - So, it is checked if 20 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. -- **Test Example policy failure case** - Send a trigger event with invalid vnfName to *unauthenticated.DCAE_POLICY_EXAMPLE_OUTPUT* Kafka topic. - The policy execution is expected to fail due to AAI failure response. 2 notification events are expected on *APEX-CL-MGT* topic by a thread in this case. - So, it is checked if 10 notification messages are received in total on *APEX-CL-MGT* topic with the relevant messages. - -Teardown Phase -"""""""""""""" - -Policies are undeployed from APEX-PDP and deleted during this phase. -Only one thread is in action and this step is done only once after the Main phase is complete. - -- **Undeploy Policies** - Undeploy both the policies from APEX-PDP using policy/pap component -- **Delete Policy onap.policies.apex.Simplecontrolloop** - delete the first APEX policy using policy/api component. -- **Delete Policy onap.policies.apex.Example** - delete the second APEX policy also using policy/api component. - -Test Configuration ------------------- - -The following steps can be used to configure the parameters of test plan. - -- **HTTP Authorization Manager** - used to store user/password authentication details. -- **HTTP Header Manager** - used to store headers which will be used for making HTTP requests. -- **User Defined Variables** - used to store following user defined parameters. - -=================== =============================================================================== - **Name** **Description** -=================== =============================================================================== - HOSTNAME IP Address or host name to access the components - PAP_PORT Port number of PAP for making REST API calls such as deploy/undeploy of policy - API_PORT Port number of API for making REST API calls such as create/delete of policy - APEX_PORT Port number of APEX for making REST API calls such as healthcheck/metrics - SIM_HOST IP Address or hostname running policy-models-simulator - KAFKA_PORT Port number of Kafka bootstrap server for sending message events - wait Wait time if required after a request (in milliseconds) - threads Number of threads to run test cases in parallel - threadsTimeOutInMs Synchronization timer for threads running in parallel (in milliseconds) -=================== =============================================================================== - -Run Test --------- - -The test was run in the background via "nohup", to prevent it from being interrupted. - -Test Results ------------- - -**Summary** - -Stability test plan was triggered for 72 hours. There were no failures during the 72 hours test. - - -**Test Statistics** - -======================= ================= ================== ================================== -**Total # of requests** **Success %** **Error %** **Average time taken per request** -======================= ================= ================== ================================== -312366 100 % 0 % 4148ms -======================= ================= ================== ================================== - -**JMeter Screenshot** - -.. image:: apex-s3p-results/apex_stability_results.png - -.. Note:: - These results show a huge dip in the number of requests compared to the previous release of Apex-PDP> - Further investigation and improvement is needed in the coming release. - -Performance Test of APEX-PDP -++++++++++++++++++++++++++++ - -Introduction ------------- - -Performance test of APEX-PDP is done similar to the stability test, but in a more extreme manner using higher thread count. - -Setup Details -------------- - -The performance test is performed on a similar setup as Stability test. - - -Test Plan ---------- - -Performance test plan is the same as the stability test plan above except for the few differences listed below. - -- Increase the number of threads used in the Main Phase from 5 to 20. -- Reduce the test time to 2 hours. - -Run Test --------- - -The test was run in the background via "nohup", to prevent it from being interrupted. - -Test Results ------------- - -Test results are shown as below. - -**Test Statistics** - -======================= ================= ================== ================================== -**Total # of requests** **Success %** **Error %** **Average time taken per request** -======================= ================= ================== ================================== -344624 100 % 0 % 4178 ms -======================= ================= ================== ================================== - -**JMeter Screenshot** - -.. image:: apex-s3p-results/apex_performance_results.png - -.. Note:: - These results show a huge dip in the number of requests compared to the previous release of Apex-PDP> - Further investigation and improvement is needed in the coming release. - -Summary -+++++++ - -Multiple policies were executed in a multi-threaded fashion for both stability and performance tests. -Both tests showed a dip in performance and stability. diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-distribution_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-distribution_J.png Binary files differdeleted file mode 100644 index 6d6033ae..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-distribution_J.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-distribution_performance_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-distribution_performance_J.png Binary files differdeleted file mode 100644 index aa2fd621..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-distribution_performance_J.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-overtime_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-overtime_J.png Binary files differdeleted file mode 100644 index aa40dd94..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-overtime_J.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-overtime_performance_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-overtime_performance_J.png Binary files differdeleted file mode 100644 index 4ba5dd75..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-response-time-overtime_performance_J.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png Binary files differdeleted file mode 100644 index 8dfbf55f..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-1_J.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png b/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png Binary files differdeleted file mode 100644 index 68b654c2..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api-s3p-jm-2_J.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api_stat_after_72h.png b/docs/development/devtools/testing/s3p/api-s3p-results/api_stat_after_72h.png Binary files differdeleted file mode 100644 index 3ecef541..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api_stat_after_72h.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p-results/api_stat_before_72h.png b/docs/development/devtools/testing/s3p/api-s3p-results/api_stat_before_72h.png Binary files differdeleted file mode 100644 index 927ab6a1..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p-results/api_stat_before_72h.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/api-s3p.rst b/docs/development/devtools/testing/s3p/api-s3p.rst deleted file mode 100644 index c3bbc9e9..00000000 --- a/docs/development/devtools/testing/s3p/api-s3p.rst +++ /dev/null @@ -1,210 +0,0 @@ -.. This work is licensed under a -.. Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _api-s3p-label: - -.. toctree:: - :maxdepth: 2 - -Policy API S3P Tests -#################### - - -72 Hours Stability Test of Policy API -+++++++++++++++++++++++++++++++++++++ - -Introduction ------------- - -The 72 hour stability test of policy API has the goal of verifying the stability of running policy design API REST -service by ingesting a steady flow of transactions in a multi-threaded fashion to -simulate multiple clients' behaviours. -All the transaction flows are initiated from a test client server running JMeter for the duration of 72 hours. - -Setup Details -------------- - -The stability test was performed on a default Policy docker installation in the Nordix Lab environment. -JMeter was installed on a separate VM to inject the traffic defined in the -`API stability script -<https://git.onap.org/policy/api/tree/testsuites/stability/src/main/resources/testplans/policy_api_stability.jmx>`_ -with the following command: - -.. code-block:: bash - - nohup apache-jmeter-5.6.2/bin/jmeter -n -t policy_api_stability.jmx -l stabilityTestResultsPolicyApi.jtl & - -The test was run in the background via “nohup” and “&”, to prevent it from being interrupted. - -Test Plan ---------- - -The 72+ hours stability test will be running the following steps sequentially -in multi-threaded loops. Thread number is set to 5 to simulate 5 API clients' -behaviours (they can be calling the same policy CRUD API simultaneously). -Each thread creates a different version of the policy types and policies to not -interfere with one another while operating simultaneously. The point version -of each entity is set to the running thread number. - -**Setup Thread (will be running only once)** - -- Get policy-api Healthcheck -- Get Preloaded Policy Types - -**API Test Flow (5 threads running the same steps in the same loop)** - -- Create a new Monitoring Policy Type with Version 6.0.# -- Create a new Monitoring Policy Type with Version 7.0.# -- Create a new Optimization Policy Type with Version 6.0.# -- Create a new Guard Policy Type with Version 6.0.# -- Create a new Native APEX Policy Type with Version 6.0.# -- Create a new Native Drools Policy Type with Version 6.0.# -- Create a new Native XACML Policy Type with Version 6.0.# -- Get All Policy Types -- Get All Versions of the new Monitoring Policy Type -- Get Version 6.0.# of the new Monitoring Policy Type -- Get Version 6.0.# of the new Optimization Policy Type -- Get Version 6.0.# of the new Guard Policy Type -- Get Version 6.0.# of the new Native APEX Policy Type -- Get Version 6.0.# of the new Native Drools Policy Type -- Get Version 6.0.# of the new Native XACML Policy Type -- Get the Latest Version of the New Monitoring Policy Type -- Create Version 6.0.# of Node Template -- Create Monitoring Policy Ver 6.0.# w/Monitoring Policy Type Ver 6.0.# -- Create Monitoring Policy Ver 7.0.# w/Monitoring Policy Type Ver 7.0.# -- Create Optimization Policy Ver 6.0.# w/Optimization Policy Type Ver 6.0.# -- Create Guard Policy Ver 6.0.# w/Guard Policy Type Ver 6.0.# -- Create Native APEX Policy Ver 6.0.# w/Native APEX Policy Type Ver 6.0.# -- Create Native Drools Policy Ver 6.0.# w/Native Drools Policy Type Ver 6.0.# -- Create Native XACML Policy Ver 6.0.# w/Native XACML Policy Type Ver 6.0.# -- Create Version 6.0.# of PNF Example Policy with Metadata -- Get Node Template -- Get All TCA Policies -- Get All Versions of Monitoring Policy Type -- Get Version 6.0.# of the new Monitoring Policy -- Get Version 6.0.# of the new Optimization Policy -- Get Version 6.0.# of the new Guard Policy -- Get Version 6.0.# of the new Native APEX Policy -- Get Version 6.0.# of the new Native Drools Policy -- Get Version 6.0.# of the new Native XACML Policy -- Get the Latest Version of the new Monitoring Policy -- Delete Version 6.0.# of the new Monitoring Policy -- Delete Version 7.0.# of the new Monitoring Policy -- Delete Version 6.0.# of the new OptimizationPolicy -- Delete Version 6.0.# of the new Guard Policy -- Delete Version 6.0.# of the new Native APEX Policy -- Delete Version 6.0.# of PNF Example Policy having Metadata -- Delete Version 6.0.# of the new Native Drools Policy -- Delete Version 6.0.# of the new Native XACML Policy -- Delete Monitoring Policy Type with Version 6.0.# -- Delete Monitoring Policy Type with Version 7.0.# -- Delete Optimization Policy Type with Version 6.0.# -- Delete Guard Policy Type with Version 6.0.# -- Delete Native APEX Policy Type with Version 6.0.# -- Delete Native Drools Policy Type with Version 6.0.# -- Delete Native XACML Policy Type with Version 6.0.# -- Delete Node Template -- Get Policy Metrics - -**TearDown Thread (will only be running after API Test Flow is completed)** - -- Get policy-api Healthcheck -- Get Preloaded Policy Types - - -Test Results ------------- - -**Summary** - -No errors were found during the 72 hours of the Policy API stability run. -The load was performed against a non-tweaked Policy docker deployment. - -**Test Statistics** - -======================= ============= =========== =============================== =============================== =============================== -**Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request** -======================= ============= =========== =============================== =============================== =============================== - 214617 100% 2.8 6028 ms 206 ms 115153 ms -======================= ============= =========== =============================== =============================== =============================== - -.. image:: api-s3p-results/api-s3p-jm-1_J.png - -**JMeter Results** - -The following graphs show the response time distributions. The "Get Policy Types" API calls are the most expensive calls that -average a 8.6 seconds plus response time. - -.. image:: api-s3p-results/api-response-time-distribution_J.png -.. image:: api-s3p-results/api-response-time-overtime_J.png - -**Memory and CPU usage** - -The memory and CPU usage can be monitored by running "docker stats" command in the policy-api container. -A snapshot is taken before and after test execution to monitor the changes in resource utilization. - -Memory and CPU usage before test execution: - -.. image:: api-s3p-results/api_stat_before_72h.png - -Memory and CPU usage after test execution: - -.. image:: api-s3p-results/api_stat_after_72h.png - - -Performance Test of Policy API -++++++++++++++++++++++++++++++ - -Introduction ------------- - -Performance test of policy-api has the goal of testing the min/avg/max processing time and rest call throughput for all the requests when the number of requests are large enough to saturate the resource and find the bottleneck. - -Setup Details -------------- - -The performance test was performed on a default Policy docker installation in the Nordix Lab environment. -JMeter was installed on a separate VM to inject the traffic defined in the -`API performance script -<https://git.onap.org/policy/api/tree/testsuites/performance/src/main/resources/testplans/policy_api_performance.jmx>`_ -with the following command: - -.. code-block:: bash - - nohup apache-jmeter-5.6.2/bin/jmeter -n -t policy_api_performance.jmx -l performanceTestResultsPolicyApi.jtl & - -The test was run in the background via “nohup” and “&”, to prevent it from being interrupted. - -Test Plan ---------- - -Performance test plan is the same as stability test plan above. -Only differences are, in performance test, we increase the number of threads up to 20 (simulating 20 users' behaviours at the same time) whereas reducing the test time down to 2.5 hours. - -Run Test --------- - -Running/Triggering performance test will be the same as stability test. That is, launch JMeter pointing to corresponding *.jmx* test plan. The *API_HOST* and *API_PORT* are already set up in *.jmx*. - -**Test Statistics** - -======================= ============= =========== =============================== =============================== =============================== -**Total # of requests** **Success %** **TPS** **Avg. time taken per request** **Min. time taken per request** **Max. time taken per request** -======================= ============= =========== =============================== =============================== =============================== - 1671 99.7% 6.3 108379 ms 223 ms 1921999 ms -======================= ============= =========== =============================== =============================== =============================== - -.. image:: api-s3p-results/api-s3p-jm-2_J.png - -Test Results ------------- - -The following graphs show the response time distributions. - -.. image:: api-s3p-results/api-response-time-distribution_performance_J.png -.. image:: api-s3p-results/api-response-time-overtime_performance_J.png - - - - diff --git a/docs/development/devtools/testing/s3p/clamp-s3p-results/Stability_after_stats.png b/docs/development/devtools/testing/s3p/clamp-s3p-results/Stability_after_stats.png Binary files differdeleted file mode 100644 index e53641f5..00000000 --- a/docs/development/devtools/testing/s3p/clamp-s3p-results/Stability_after_stats.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_performance_jmeter.png b/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_performance_jmeter.png Binary files differdeleted file mode 100644 index 38b6c000..00000000 --- a/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_performance_jmeter.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_stability_jmeter.png b/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_stability_jmeter.png Binary files differdeleted file mode 100644 index bd9d0e84..00000000 --- a/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_stability_jmeter.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_stability_table.png b/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_stability_table.png Binary files differdeleted file mode 100644 index 94402c8f..00000000 --- a/docs/development/devtools/testing/s3p/clamp-s3p-results/acm_stability_table.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/clamp-s3p.rst b/docs/development/devtools/testing/s3p/clamp-s3p.rst deleted file mode 100644 index 2cf3e236..00000000 --- a/docs/development/devtools/testing/s3p/clamp-s3p.rst +++ /dev/null @@ -1,224 +0,0 @@ -.. This work is licensed under a -.. Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _acm-s3p-label: - -.. toctree:: - :maxdepth: 2 - -Policy Clamp Automation Composition -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Both the Performance and the Stability tests were executed by performing requests -against acm components installed as docker images in local environment. These tests we all -performed on a Ubuntu VM with 32GB of memory, 16 CPU and 50GB of disk space. - - -ACM Deployment -++++++++++++++ - -In an effort to allow the execution of the s3p tests to be as close to automatic as possible, -a script will be executed that will perform the following: - -- Install of a microk8s kubernetes environment -- Bring up the policy components -- Checks that the components are successfully up and running before proceeding -- Install Java 17 -- Install Jmeter locally and configure it -- Specify whether you want to run stability or performance tests - - -The remainder of this document outlines how to run the tests and the test results - -Common Setup -++++++++++++ -The common setup for performance and stability tests is now automated - being carried out by a script in- **testsuites/run-s3p-test.sh**. - -Clone the policy-clamp repo to access the test scripts - -.. code-block:: sh - - git clone https://gerrit.onap.org/r/policy/clamp - -Stability Test of acm components -++++++++++++++++++++++++++++++++ - -Test Plan ---------- -The 72 hours stability test ran the following steps sequentially in a single threaded loop. - -- **Commission Automation Composition Definitions** - Commissions the ACM Definitions -- **Register Participants** - Registers the presence of participants in the acm database -- **Prime AC definition** - Primes the AC Definition to the participants -- **Instantiate acm** - Instantiate the acm instance -- **DEPLOY the ACM instance** - change the state of the acm to DEPLOYED -- **Check instance state** - check the current state of instance and that it is DEPLOYED -- **UNDEPLOY the ACM instance** - change the state of the ACM to UNDEPLOYED -- **Check instance state** - check the current state of instance and that it is UNDEPLOYED -- **Delete instance** - delete the instance from all participants and ACM db -- **DEPRIME ACM definitions** - DEPRIME ACM definitions from participants -- **Delete ACM Definition** - delete the ACM definition on runtime - -This runs for 72 hours. Test results are present in the **testsuites/automated-performance/s3pTestResults.jtl** -directory. Logs are present for jmeter in **testsuites/automated-performance/jmeter.log** and -**testsuites/automated-performance/nohup.out** - -Run Test --------- - -The code in the setup section also serves to run the tests. Just one execution needed to do it all. - -.. code-block:: sh - - ./run-s3p-test.sh run stability - -Once the test execution is completed, the results are present in the **automate-performance/s3pTestResults.jtl** file. - -This file can be imported into the Jmeter GUI for visualization. The below results are tabulated from the GUI. - -Test Results ------------- - -**Summary** - -Stability test plan was triggered for 72 hours. - -**Test Statistics** - -======================= ================= ================== ================================== -**Total # of requests** **Success %** **Error %** **Average time taken per request** -======================= ================= ================== ================================== -261852 100.00 % 0.00 % 387.126 ms -======================= ================= ================== ================================== - -**ACM component Setup** - -============================================== ================================================================== ==================== -**NAME** **IMAGE** **PORT** -============================================== ================================================================== ==================== - zookeeper-deployment-7ff87c7fcc-ptkwv confluentinc/cp-zookeeper:latest 2181/TCP - kafka-deployment-5c87d497b-2jv27 confluentinc/cp-kafka:latest 9092/TCP - policy-models-simulator-6947667bdc-v4czs nexus3.onap.org:10001/onap/policy-models-simulator:latest 3904:30904/TCP - prometheus-f66f97b6-rknvp nexus3.onap.org:10001/prom/prometheus:latest 9090:30909/TCP - mariadb-galera-0 nexus3.onap.org:10001/bitnami/mariadb-galera:10.5.8 3306/TCP - policy-apex-pdp-0 nexus3.onap.org:10001/onap/policy-apex-pdp:3.1.3-SNAPSHOT 6969:30001/TCP - policy-clamp-ac-http-ppnt-7d747b5d98-4phjf nexus3.onap.org:10001/onap/policy-clamp-ac-http-ppnt7.1.3-SNAPSHOT 8084/TCP - policy-clamp-ac-sim-ppnt-97f487577-4p7ks nexus3.onap.org:10001/onap/policy-clamp-ac-sim-ppnt7.1.3-SNAPSHOT 6969/TCP - policy-clamp-ac-k8s-ppnt-6bbd86bbc6-csknn nexus3.onap.org:10001/onap/policy-clamp-ac-k8s-ppnt7.1.3-SNAPSHOT 8083:30443/TCP - policy-clamp-ac-pf-ppnt-5fcbbcdb6c-twkxw nexus3.onap.org:10001/onap/policy-clamp-ac-pf-ppnt7.1.3-SNAPSHOT 6969:30008/TCP - policy-clamp-runtime-acm-66b5d6b64-4gnth nexus3.onap.org:10001/onap/policy-clamp-runtime-acm7.1.3-SNAPSHOT 6969:30007/TCP - policy-pap-f7899d4cd-7m898 nexus3.onap.org:10001/onap/policy-pap:3.1.3-SNAPSHOT 6969:30003/TCP - policy-api-7f7d995b4-ckb84 nexus3.onap.org:10001/onap/policy-api:3.1.3-SNAPSHOT 6969:30002/TCP -============================================== ================================================================== ==================== - - - -.. Note:: - - .. container:: paragraph - - There were no failures during the 72 hours test. - -**JMeter Screenshot** - -.. image:: clamp-s3p-results/acm_stability_jmeter.png - -**JMeter Screenshot** - -.. image:: clamp-s3p-results/acm_stability_table.png - - -Performance Test of acm components -++++++++++++++++++++++++++++++++++ - -Introduction ------------- - -Performance test of acm components has the goal of testing the min/avg/max processing time and rest call throughput for all the requests with multiple requests at the same time. - -Setup Details -------------- - -We can setup the environment and execute the tests like this from the **clamp/testsuites** directory - -.. code-block:: sh - - ./run-s3p-test.sh run performance - -Test results are present in the **testsuites/automate-performance/s3pTestResults.jtl** -directory. Logs are present for jmeter in **testsuites/automate-performance/jmeter.log** and -**testsuites/automated-performance/nohup.out** - -Test Plan ---------- - -The Performance test ran the following steps sequentially by 5 threaded users. Any user will create 100 compositions/instances. - -- **SetUp** - SetUp Thread Group - - **Register Participants** - Registers the presence of participants in the acm database -- **AutomationComposition Test Flow** - flow by 5 threaded users. - - **Creation and Deploy** - Creates 100 Compositions and Instances - - **Commission Automation Composition Definitions** - Commissions the ACM Definitions - - **Prime AC definition** - Primes the AC Definition to the participants - - **Instantiate acm** - Instantiate the acm instance - - **DEPLOY the ACM instance** - change the state of the acm to DEPLOYED - - **Check instance state** - check the current state of instance and that it is DEPLOYED - - **Get participants** - fetch all participants - - **Get compositions** - fetch all compositions - - **Undeploy and Delete** - Deletes instances and Compositions created before - - **UNDEPLOY the ACM instance** - change the state of the ACM to UNDEPLOYED - - **Check instance state** - check the current state of instance and that it is UNDEPLOYED - - **Delete instance** - delete the instance from all participants and ACM db - - **DEPRIME ACM definitions** - DEPRIME ACM definitions from participants - - **Delete ACM Definition** - delete the ACM definition on runtime - -Run Test --------- - -The code in the setup section also serves to run the tests. Just one execution needed to do it all. - -.. code-block:: sh - - ./run-s3p-test.sh run performance - -Once the test execution is completed, the results are present in the **automate-performance/s3pTestResults.jtl** file. - -This file can be imported into the Jmeter GUI for visualization. The below results are tabulated from the Jmeter GUI. - -Test Results ------------- - -Test results are shown as below. - -**Test Statistics** - -======================= ================= ================== ================================== -**Total # of requests** **Success %** **Error %** **Average time taken per request** -======================= ================= ================== ================================== -8624 100 % 0.00 % 1296.8 ms -======================= ================= ================== ================================== - -**ACM component Setup** - -============================================== ================================================================== ==================== -**NAME** **IMAGE** **PORT** -============================================== ================================================================== ==================== - zookeeper-deployment-7ff87c7fcc-5svgw confluentinc/cp-zookeeper:latest 2181/TCP - kafka-deployment-5c87d497b-hmbhc confluentinc/cp-kafka:latest 9092/TCP - policy-models-simulator-6947667bdc-crcwq nexus3.onap.org:10001/onap/policy-models-simulator:latest 3904:30904/TCP - prometheus-f66f97b6-24dvx nexus3.onap.org:10001/prom/prometheus:latest 9090:30909/TCP - mariadb-galera-0 nexus3.onap.org:10001/bitnami/mariadb-galera:10.5.8 3306/TCP - policy-apex-pdp-0 nexus3.onap.org:10001/onap/policy-apex-pdp:3.1.3-SNAPSHOT 6969:30001/TCP - policy-clamp-ac-sim-ppnt-97f487577-pn56t nexus3.onap.org:10001/onap/policy-clamp-ac-sim-ppnt7.1.3-SNAPSHOT 6969/TCP - policy-clamp-ac-http-ppnt-7d747b5d98-qjjlv nexus3.onap.org:10001/onap/policy-clamp-ac-http-ppnt7.1.3-SNAPSHOT 8084/TCP - policy-clamp-ac-k8s-ppnt-6bbd86bbc6-ffbz2 nexus3.onap.org:10001/onap/policy-clamp-ac-k8s-ppnt7.1.3-SNAPSHOT 8083:30443/TCP - policy-clamp-ac-pf-ppnt-5fcbbcdb6c-vmsnv nexus3.onap.org:10001/onap/policy-clamp-ac-pf-ppnt7.1.3-SNAPSHOT 6969:30008/TCP - policy-clamp-runtime-acm-66b5d6b64-6vjl5 nexus3.onap.org:10001/onap/policy-clamp-runtime-acm7.1.3-SNAPSHOT 6969:30007/TCP - policy-pap-f7899d4cd-8sjk9 nexus3.onap.org:10001/onap/policy-pap:3.1.3-SNAPSHOT 6969:30003/TCP - policy-api-7f7d995b4-dktdw nexus3.onap.org:10001/onap/policy-api:3.1.3-SNAPSHOT 6969:30002/TCP -============================================== ================================================================== ==================== - -**JMeter Screenshot** - -.. image:: clamp-s3p-results/acm_performance_jmeter.png diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/distribution-jmeter-testcases.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/distribution-jmeter-testcases.png Binary files differdeleted file mode 100644 index 1159bca3..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/distribution-jmeter-testcases.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/distribution-visualvm-snapshot.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/distribution-visualvm-snapshot.png Binary files differdeleted file mode 100644 index b9b175a4..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/distribution-visualvm-snapshot.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-monitor.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-monitor.png Binary files differdeleted file mode 100644 index d535c4aa..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-monitor.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-statistics.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-statistics.png Binary files differdeleted file mode 100644 index a2caab4e..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-statistics.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-threads.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-threads.png Binary files differdeleted file mode 100644 index 9b6c3d23..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-threads.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-threshold.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-threshold.png Binary files differdeleted file mode 100644 index 6a26a09e..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/performance-threshold.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-monitor.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-monitor.png Binary files differdeleted file mode 100644 index cbb675ba..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-monitor.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-statistics.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-statistics.png Binary files differdeleted file mode 100644 index ae1853f9..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-statistics.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-threads.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-threads.png Binary files differdeleted file mode 100644 index 67da4a62..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-threads.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-threshold.png b/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-threshold.png Binary files differdeleted file mode 100644 index 5aa6cc64..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p-results/stability-threshold.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/distribution-s3p.rst b/docs/development/devtools/testing/s3p/distribution-s3p.rst deleted file mode 100644 index 40ade31c..00000000 --- a/docs/development/devtools/testing/s3p/distribution-s3p.rst +++ /dev/null @@ -1,238 +0,0 @@ -.. This work is licensed under a -.. Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _distribution-s3p-label: - -Policy Distribution component -############################# - -72h Stability and 4h Performance Tests of Distribution -++++++++++++++++++++++++++++++++++++++++++++++++++++++ - -Common Setup ------------- - -The common setup for performance and stability tests is now automated - being carried out by a script in- **testsuites/run-sc3-test.sh**. - -Clone the policy-distribution repo to access the test scripts - -.. code-block:: bash - - git clone https://gerrit.onap.org/r/policy/distribution - -**The following common steps are carried out by the scripts** - -* Updates the repo package lists for apt -* Installs Java 17 open jdk -* Installs docker -* Installs docker-compose -* Retrieves version information in environment variables from th release info file -* Builds relevant images including the pdp simulator -* Triggers docker compose to bring up containers required for the testing -* Installs jmeter -* Installs visualvm (and starts it in a GUI environment) -* Configures permissions for monitoring -* Starts jstatd -* Waits for containers to come up -* Runs either stability or performance tests for a specified duration depending on the arguments specified - -For example, the below runs performance tests for 2 hours. Start from the root directory of policy distribution - -.. code-block:: bash - - cd testsuites - ./run-s3p-test.sh performance 7200 - -.. note:: - The containers on this docker-compose are running with HTTP configuration. For HTTPS, ports - and configurations will need to be changed, as well certificates and keys must be generated - for security. - -The script will load up the visualvm GUI on your virtual machine. You will need to manually connect -it to the distribution JMX port. - -Connect to Distribution JMX Port. - - 1. On the visualvm toolbar, click on "Add JMX Connection" - 2. Enter localhost as the IP address and Port 9090. This is the JMX port exposed by the - distribution container - 3. Double click on the newly added nodes under "Local" to start monitoring CPU, Memory & GC. - -Example Screenshot of visualVM - -.. image:: distribution-s3p-results/distribution-visualvm-snapshot.png - -Teardown Docker - -Once the testing is finished, you can tear down the docker setup from **./testsuites** with: - -.. code-block:: bash - - docker-compose -f stability/src/main/resources/setup/docker-compose.yml down - -Stability Test of Policy Distribution -+++++++++++++++++++++++++++++++++++++ - -Introduction ------------- - -The 72 hour Stability Test for policy distribution has the goal of introducing a steady flow of -transactions initiated from a test client server running JMeter. The policy distribution is -configured with a special FileSystemReception plugin to monitor a local directory for newly added -csar files to be processed by itself. The input CSAR will be added/removed by the test client -(JMeter) and the result will be pulled from the backend (PAP and PolicyAPI) by the test client -(JMeter). - -The test will be performed in an environment where Jmeter will continuously add/remove a test csar -into the special directory where policy distribution is monitoring and will then get the processed -results from PAP and PolicyAPI to verify the successful deployment of the policy. The policy will -then be undeployed and the test will loop continuously until 72 hours have elapsed. - - -Test Plan Sequence ------------------- - -The 72h stability test will run the following steps sequentially in a single threaded loop. - -- **Delete Old CSAR** - Checks if CSAR already exists in the watched directory, if so it deletes it -- **Add CSAR** - Adds CSAR to the directory that distribution is watching -- **Get Healthcheck** - Ensures Healthcheck is returning 200 OK -- **Get Metrics** - Ensures Metrics is returning 200 OK -- **Assert PDP Group Query** - Checks that PDPGroupQuery contains the deployed policy -- **Assert PoliciesDeployed** - Checks that the policy is deployed -- **Undeploy/Delete Policy** - Undeploys and deletes the Policy for the next loop -- **Assert PDP Group Query for Deleted Policy** - Ensures the policy has been removed and does not exist - -The following steps can be used to configure the parameters of the test plan. - -- **HTTP Authorization Manager** - used to store user/password authentication details. -- **HTTP Header Manager** - used to store headers which will be used for making HTTP requests. -- **User Defined Variables** - used to store following user defined parameters. - -========== =============================================== - **Name** **Description** -========== =============================================== - PAP_HOST IP Address or host name of PAP component - PAP_PORT Port number of PAP for making REST API calls - API_HOST IP Address or host name of API component - API_PORT Port number of API for making REST API calls - DURATION Duration of Test -========== =============================================== - -Screenshot of Distribution stability test plan - -.. image:: distribution-s3p-results/distribution-jmeter-testcases.png - - -Running the Test Plan ---------------------- - -The main script takes care of everything. To run the 72 hour stability tests do as follows - -.. code-block:: bash - - cd testsuites - ./run-s3p-test.sh stability 259200 - -* visualvm produces the monitor and threads - we can screenshot those and add them to the test results -* A jmeter .jtl file is produced by the run - it is called distribution-stability.jtl -* The file can be imported into the jmeter GUI to view statistics -* The application performance index table can be produced with jmeter on the cli as below:ls - -.. code-block:: bash - - jmeter -n -t your_test_plan.jmx -l test_results.jtl -e -o report_directory - -Test Results ------------- - -**Summary** - -- Stability test plan was triggered for 72 hours. -- No errors were reported - -**Test Statistics** - -.. image:: distribution-s3p-results/stability-statistics.png -.. image:: distribution-s3p-results/stability-threshold.png - -**VisualVM Screenshots** - -.. image:: distribution-s3p-results/stability-monitor.png -.. image:: distribution-s3p-results/stability-threads.png - - -Performance Test of Policy Distribution -+++++++++++++++++++++++++++++++++++++++ - -Introduction ------------- - -The 4h Performance Test of Policy Distribution has the goal of testing the min/avg/max processing -time and rest call throughput for all the requests when the number of requests are large enough to -saturate the resource and find the bottleneck. - -It also tests that distribution can handle multiple policy CSARs and that these are deployed within -60 seconds consistently. - - -Setup Details -------------- - -The performance test is based on the same setup as the distribution stability tests. This setup is done by the main -**run-s3p-test.sh** script - - -Test Plan Sequence ------------------- - -Performance test plan is different from the stability test plan. - -- Instead of handling one policy csar at a time, multiple csar's are deployed within the watched - folder at the exact same time. -- We expect all policies from these csar's to be deployed within 60 seconds. -- There are also multithreaded tests running towards the healthcheck and statistics endpoints of - the distribution service. - - -Running the Test Plan ---------------------- - -The main script takes care of everything. To run the 4 hour performance tests do as follows - -.. code-block:: bash - - cd testsuites - ./run-s3p-test.sh performance 14400 - -* visualvm produces the monitor and threads - we can screenshot those and add them to the test results -* A jmeter .jtl file is produced by the run - it is called distribution-performance.jtl -* The file can be imported into the jmeter GUI to view statistics -* The application performance index table can be produced with jmeter on the cli as below: - -.. code-block:: bash - - jmeter -n -t your_test_plan.jmx -l test_results.jtl -e -o report_directory - -This produced html pages where statistics tables can be seen and added to the results. - -Test Results ------------- - -**Summary** - -- Performance test plan was triggered for 4 hours. -- No errors were reported - -**Test Statistics** - -.. image:: distribution-s3p-results/performance-statistics.png -.. image:: distribution-s3p-results/performance-threshold.png - -**VisualVM Screenshots** - -.. image:: distribution-s3p-results/performance-monitor.png -.. image:: distribution-s3p-results/performance-threads.png - -End of document diff --git a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-1.png b/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-1.png Binary files differdeleted file mode 100644 index 3c1e06f7..00000000 --- a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-1.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-2.png b/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-2.png Binary files differdeleted file mode 100644 index 7e124716..00000000 --- a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-2.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-3.png b/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-3.png Binary files differdeleted file mode 100644 index 50f2c148..00000000 --- a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-3.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-4.png b/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-4.png Binary files differdeleted file mode 100644 index 369d1f33..00000000 --- a/docs/development/devtools/testing/s3p/drools-s3p-results/s3p-drools-4.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/drools-s3p.rst b/docs/development/devtools/testing/s3p/drools-s3p.rst deleted file mode 100644 index 88f601bd..00000000 --- a/docs/development/devtools/testing/s3p/drools-s3p.rst +++ /dev/null @@ -1,74 +0,0 @@ -.. This work is licensed under a -.. Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _drools-s3p-label: - -.. toctree:: - :maxdepth: 2 - -Policy Drools PDP component -~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Both the Performance and the Stability tests were executed against an ONAP installation in the Policy tenant -in the UNH lab, from the admin VM running the jmeter tool to inject the load. - -General Setup -************* - -Agent VMs in this lab have the following configuration: - -- 16GB RAM -- 8 VCPU - -Jmeter is run from the admin VM. - -The drools-pdp container uses the JVM memory and CPU settings from the default OOM installation. - -Other ONAP components exercised during the stability tests were: - -- Policy XACML PDP to process guard queries for each transaction. -- DMaaP to carry PDP-D and jmeter initiated traffic to complete transactions. -- Policy API to create (and delete at the end of the tests) policies for each - scenario under test. -- Policy PAP to deploy (and undeploy at the end of the tests) policies for each scenario under test. -- XACML PDP Stability test was running at the same time. - -The following components are simulated during the tests. - -- SDNR. - -Stability Test of Policy PDP-D -****************************** - -PDP-D performance -================= - -The tests focused on the following use cases running in parallel: - -- vCPE -- SON O1 -- SON A1 - -Three threads ran in parallel, one for each scenario. The transactions were initiated -by each jmeter thread group. Each thread initiated a transaction, monitored the transaction, and -started the next one 250 ms. later. - -The results are illustrated on the following graphs: - -.. image:: drools-s3p-results/s3p-drools-1.png -.. image:: drools-s3p-results/s3p-drools-2.png -.. image:: drools-s3p-results/s3p-drools-3.png - - -Commentary -========== - -There is around 1% unexpected failures during the 72-hour run. This can also be seen in the -final output of jmeter: - -.. code-block:: bash - - summary = 4751546 in 72:00:37 = 18.3/s Avg: 150 Min: 0 Max: 15087 Err: 47891 (1.01%) - -Sporadic database errors have been observed and seem related to the 1% failure percentage rate. diff --git a/docs/development/devtools/testing/s3p/images/workflow-results.png b/docs/development/devtools/testing/s3p/images/workflow-results.png Binary files differnew file mode 100644 index 00000000..d287754a --- /dev/null +++ b/docs/development/devtools/testing/s3p/images/workflow-results.png diff --git a/docs/development/devtools/testing/s3p/images/workflow-test-result.png b/docs/development/devtools/testing/s3p/images/workflow-test-result.png Binary files differnew file mode 100644 index 00000000..d192205d --- /dev/null +++ b/docs/development/devtools/testing/s3p/images/workflow-test-result.png diff --git a/docs/development/devtools/testing/s3p/images/workflows.png b/docs/development/devtools/testing/s3p/images/workflows.png Binary files differnew file mode 100644 index 00000000..7b05e22d --- /dev/null +++ b/docs/development/devtools/testing/s3p/images/workflows.png diff --git a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_metrics_after_72h.txt b/docs/development/devtools/testing/s3p/pap-s3p-results/pap_metrics_after_72h.txt deleted file mode 100644 index 1851bf63..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_metrics_after_72h.txt +++ /dev/null @@ -1,521 +0,0 @@ -# HELP hikaricp_connections_min Min connections -# TYPE hikaricp_connections_min gauge -hikaricp_connections_min{pool="HikariPool-1",} 10.0 -# HELP tomcat_sessions_created_sessions_total -# TYPE tomcat_sessions_created_sessions_total counter -tomcat_sessions_created_sessions_total 3.0 -# HELP disk_total_bytes Total space for path -# TYPE disk_total_bytes gauge -disk_total_bytes{path="/opt/app/policy/pap/bin/.",} 1.0386530304E11 -# HELP jvm_classes_loaded_classes The number of classes that are currently loaded in the Java virtual machine -# TYPE jvm_classes_loaded_classes gauge -jvm_classes_loaded_classes 20615.0 -# HELP hikaricp_connections_usage_seconds Connection usage time -# TYPE hikaricp_connections_usage_seconds summary -hikaricp_connections_usage_seconds_count{pool="HikariPool-1",} 321133.0 -hikaricp_connections_usage_seconds_sum{pool="HikariPool-1",} 45213.218 -# HELP hikaricp_connections_usage_seconds_max Connection usage time -# TYPE hikaricp_connections_usage_seconds_max gauge -hikaricp_connections_usage_seconds_max{pool="HikariPool-1",} 0.027 -# HELP hikaricp_connections_active Active connections -# TYPE hikaricp_connections_active gauge -hikaricp_connections_active{pool="HikariPool-1",} 0.0 -# HELP process_start_time_seconds Start time of the process since unix epoch. -# TYPE process_start_time_seconds gauge -process_start_time_seconds 1.700139959198E9 -# HELP jvm_memory_used_bytes The amount of used memory -# TYPE jvm_memory_used_bytes gauge -jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'profiled nmethods'",} 3.2981376E7 -jvm_memory_used_bytes{area="heap",id="G1 Survivor Space",} 494864.0 -jvm_memory_used_bytes{area="heap",id="G1 Old Gen",} 2.1805824E8 -jvm_memory_used_bytes{area="nonheap",id="Metaspace",} 1.13110752E8 -jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'non-nmethods'",} 1472896.0 -jvm_memory_used_bytes{area="heap",id="G1 Eden Space",} 3.7748736E7 -jvm_memory_used_bytes{area="nonheap",id="Compressed Class Space",} 1.4127568E7 -jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'non-profiled nmethods'",} 3.4159744E7 -# HELP jvm_gc_memory_promoted_bytes_total Count of positive increases in the size of the old generation memory pool before GC to after GC -# TYPE jvm_gc_memory_promoted_bytes_total counter -jvm_gc_memory_promoted_bytes_total 1.78894336E8 -# HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset -# TYPE jvm_threads_peak_threads gauge -jvm_threads_peak_threads 43.0 -# HELP hikaricp_connections_creation_seconds_max Connection creation time -# TYPE hikaricp_connections_creation_seconds_max gauge -hikaricp_connections_creation_seconds_max{pool="HikariPool-1",} 0.0 -# HELP hikaricp_connections_creation_seconds Connection creation time -# TYPE hikaricp_connections_creation_seconds summary -hikaricp_connections_creation_seconds_count{pool="HikariPool-1",} 2131.0 -hikaricp_connections_creation_seconds_sum{pool="HikariPool-1",} 17.144 -# HELP system_cpu_count The number of processors available to the Java virtual machine -# TYPE system_cpu_count gauge -system_cpu_count 16.0 -# HELP spring_security_filterchains_session_url_encoding_after_total -# TYPE spring_security_filterchains_session_url_encoding_after_total counter -spring_security_filterchains_session_url_encoding_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP executor_queue_remaining_tasks The number of additional elements that this queue can ideally accept without blocking -# TYPE executor_queue_remaining_tasks gauge -executor_queue_remaining_tasks{name="applicationTaskExecutor",} 2.147483647E9 -# HELP hikaricp_connections Total connections -# TYPE hikaricp_connections gauge -hikaricp_connections{pool="HikariPool-1",} 10.0 -# HELP tomcat_sessions_expired_sessions_total -# TYPE tomcat_sessions_expired_sessions_total counter -tomcat_sessions_expired_sessions_total 2.0 -# HELP tomcat_sessions_active_current_sessions -# TYPE tomcat_sessions_active_current_sessions gauge -tomcat_sessions_active_current_sessions 1.0 -# HELP hikaricp_connections_timeout_total Connection timeout total count -# TYPE hikaricp_connections_timeout_total counter -hikaricp_connections_timeout_total{pool="HikariPool-1",} 0.0 -# HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads -# TYPE jvm_threads_live_threads gauge -jvm_threads_live_threads 38.0 -# HELP spring_security_filterchains_active_seconds_max -# TYPE spring_security_filterchains_active_seconds_max gauge -spring_security_filterchains_active_seconds_max{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_max{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -# HELP spring_security_filterchains_active_seconds -# TYPE spring_security_filterchains_active_seconds summary -spring_security_filterchains_active_seconds_active_count{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_duration_sum{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_active_count{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_duration_sum{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -# HELP spring_security_filterchains_logout_after_total -# TYPE spring_security_filterchains_logout_after_total counter -spring_security_filterchains_logout_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP jvm_info JVM version info -# TYPE jvm_info gauge -jvm_info{runtime="OpenJDK Runtime Environment",vendor="Alpine",version="17.0.9+8-alpine-r0",} 1.0 -# HELP disk_free_bytes Usable space for path -# TYPE disk_free_bytes gauge -disk_free_bytes{path="/opt/app/policy/pap/bin/.",} 8.5940973568E10 -# HELP spring_security_authentications_active_seconds -# TYPE spring_security_authentications_active_seconds summary -spring_security_authentications_active_seconds_active_count{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="n/a",} 0.0 -spring_security_authentications_active_seconds_duration_sum{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="n/a",} 0.0 -# HELP spring_security_authentications_active_seconds_max -# TYPE spring_security_authentications_active_seconds_max gauge -spring_security_authentications_active_seconds_max{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="n/a",} 0.0 -# HELP jvm_threads_daemon_threads The current number of live daemon threads -# TYPE jvm_threads_daemon_threads gauge -jvm_threads_daemon_threads 28.0 -# HELP spring_security_filterchains_context_holder_before_total -# TYPE spring_security_filterchains_context_holder_before_total counter -spring_security_filterchains_context_holder_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP spring_security_filterchains_context_holder_after_total -# TYPE spring_security_filterchains_context_holder_after_total counter -spring_security_filterchains_context_holder_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP jvm_gc_memory_allocated_bytes_total Incremented for an increase in the size of the (young) heap memory pool after one GC to before the next -# TYPE jvm_gc_memory_allocated_bytes_total counter -jvm_gc_memory_allocated_bytes_total 2.70538060492E12 -# HELP executor_pool_core_threads The core number of threads for the pool -# TYPE executor_pool_core_threads gauge -executor_pool_core_threads{name="applicationTaskExecutor",} 8.0 -# HELP spring_security_filterchains_authentication_anonymous_before_total -# TYPE spring_security_filterchains_authentication_anonymous_before_total counter -spring_security_filterchains_authentication_anonymous_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP jdbc_connections_active Current number of active connections that have been allocated from the data source. -# TYPE jdbc_connections_active gauge -jdbc_connections_active{name="dataSource",} 0.0 -# HELP http_server_requests_seconds -# TYPE http_server_requests_seconds summary -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/metrics",} 16898.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/metrics",} 3967.357676154 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/status",} 16898.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/status",} 3952.559792217 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/healthcheck",} 8449.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/healthcheck",} 1962.407770331 -http_server_requests_seconds_count{error="none",exception="none",method="DELETE",outcome="SUCCESS",status="200",uri="/pdps/groups/{name}",} 1.0 -http_server_requests_seconds_sum{error="none",exception="none",method="DELETE",outcome="SUCCESS",status="200",uri="/pdps/groups/{name}",} 1.13003001 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/pdps",} 33794.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/pdps",} 8534.756994317 -http_server_requests_seconds_count{error="none",exception="none",method="POST",outcome="SUCCESS",status="202",uri="/pdps/policies",} 8449.0 -http_server_requests_seconds_sum{error="none",exception="none",method="POST",outcome="SUCCESS",status="202",uri="/pdps/policies",} 9029.386618813 -http_server_requests_seconds_count{error="none",exception="none",method="DELETE",outcome="SUCCESS",status="202",uri="/pdps/policies/{name}",} 8448.0 -http_server_requests_seconds_sum{error="none",exception="none",method="DELETE",outcome="SUCCESS",status="202",uri="/pdps/policies/{name}",} 9292.095374281 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="CLIENT_ERROR",status="401",uri="UNKNOWN",} 3.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="CLIENT_ERROR",status="401",uri="UNKNOWN",} 0.146722928 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/deployed",} 8448.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/deployed",} 1963.048694006 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/audit/{pdpGroupName}",} 8448.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/audit/{pdpGroupName}",} 2081.277984093 -http_server_requests_seconds_count{error="none",exception="none",method="POST",outcome="SUCCESS",status="202",uri="/pdps/deployments/batch",} 16896.0 -http_server_requests_seconds_sum{error="none",exception="none",method="POST",outcome="SUCCESS",status="202",uri="/pdps/deployments/batch",} 18067.385431232 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/status/{pdpGroupName}/{policyName}/{policyVersion}",} 16915.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/status/{pdpGroupName}/{policyName}/{policyVersion}",} 4012.92045444 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/audit/{pdpGroupName}/{policyName}/{policyVersion}",} 16896.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/audit/{pdpGroupName}/{policyName}/{policyVersion}",} 4284.22784792 -http_server_requests_seconds_count{error="none",exception="none",method="PUT",outcome="SUCCESS",status="200",uri="/pdps/groups/{name}",} 3.0 -http_server_requests_seconds_sum{error="none",exception="none",method="PUT",outcome="SUCCESS",status="200",uri="/pdps/groups/{name}",} 1.687419501 -http_server_requests_seconds_count{error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/pdps/groups/batch",} 1.0 -http_server_requests_seconds_sum{error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/pdps/groups/batch",} 1.716173275 -http_server_requests_seconds_count{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/components/healthcheck",} 8448.0 -http_server_requests_seconds_sum{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/components/healthcheck",} 4213.059172045 -# HELP http_server_requests_seconds_max -# TYPE http_server_requests_seconds_max gauge -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/metrics",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/status",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/healthcheck",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="DELETE",outcome="SUCCESS",status="200",uri="/pdps/groups/{name}",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/pdps",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="POST",outcome="SUCCESS",status="202",uri="/pdps/policies",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="DELETE",outcome="SUCCESS",status="202",uri="/pdps/policies/{name}",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="CLIENT_ERROR",status="401",uri="UNKNOWN",} 0.051127942 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/deployed",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/audit/{pdpGroupName}",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="POST",outcome="SUCCESS",status="202",uri="/pdps/deployments/batch",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/status/{pdpGroupName}/{policyName}/{policyVersion}",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/policies/audit/{pdpGroupName}/{policyName}/{policyVersion}",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="PUT",outcome="SUCCESS",status="200",uri="/pdps/groups/{name}",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="POST",outcome="SUCCESS",status="200",uri="/pdps/groups/batch",} 0.0 -http_server_requests_seconds_max{error="none",exception="none",method="GET",outcome="SUCCESS",status="200",uri="/components/healthcheck",} 0.0 -# HELP spring_security_filterchains_authentication_basic_before_total -# TYPE spring_security_filterchains_authentication_basic_before_total counter -spring_security_filterchains_authentication_basic_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP pap_policy_deployments_total -# TYPE pap_policy_deployments_total counter -pap_policy_deployments_total{operation="deploy",status="FAILURE",} 0.0 -pap_policy_deployments_total{operation="undeploy",status="SUCCESS",} 16896.0 -pap_policy_deployments_total{operation="deploy",status="SUCCESS",} 16897.0 -pap_policy_deployments_total{operation="undeploy",status="FAILURE",} 0.0 -# HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool -# TYPE jvm_buffer_total_capacity_bytes gauge -jvm_buffer_total_capacity_bytes{id="mapped - 'non-volatile memory'",} 0.0 -jvm_buffer_total_capacity_bytes{id="mapped",} 0.0 -jvm_buffer_total_capacity_bytes{id="direct",} 1544596.0 -# HELP jvm_gc_live_data_size_bytes Size of long-lived heap memory pool after reclamation -# TYPE jvm_gc_live_data_size_bytes gauge -jvm_gc_live_data_size_bytes 5.4770176E7 -# HELP process_files_max_files The maximum file descriptor count -# TYPE process_files_max_files gauge -process_files_max_files 1048576.0 -# HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use -# TYPE jvm_memory_committed_bytes gauge -jvm_memory_committed_bytes{area="nonheap",id="CodeHeap 'profiled nmethods'",} 4.1222144E7 -jvm_memory_committed_bytes{area="heap",id="G1 Survivor Space",} 4194304.0 -jvm_memory_committed_bytes{area="heap",id="G1 Old Gen",} 2.60046848E8 -jvm_memory_committed_bytes{area="nonheap",id="Metaspace",} 1.13967104E8 -jvm_memory_committed_bytes{area="nonheap",id="CodeHeap 'non-nmethods'",} 3342336.0 -jvm_memory_committed_bytes{area="heap",id="G1 Eden Space",} 9.6468992E7 -jvm_memory_committed_bytes{area="nonheap",id="Compressed Class Space",} 1.4548992E7 -jvm_memory_committed_bytes{area="nonheap",id="CodeHeap 'non-profiled nmethods'",} 3.5454976E7 -# HELP spring_data_repository_invocations_seconds_max Duration of repository invocations -# TYPE spring_data_repository_invocations_seconds_max gauge -spring_data_repository_invocations_seconds_max{exception="None",method="findByKeyParentKeyNameAndKeyParentKeyVersion",repository="PolicyStatusRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="save",repository="PdpGroupRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="saveAll",repository="PolicyStatusRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findByPdpGroup",repository="PolicyAuditRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findById",repository="ToscaServiceTemplateRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findByPdpGroup",repository="PolicyStatusRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findAll",repository="PdpGroupRepository",state="SUCCESS",} 0.010393136 -spring_data_repository_invocations_seconds_max{exception="None",method="deleteById",repository="PdpGroupRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findAll",repository="PolicyStatusRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="saveAll",repository="PolicyAuditRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findByPdpGroupAndNameAndVersion",repository="PolicyAuditRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findById",repository="ToscaNodeTemplateRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="existsById",repository="PdpGroupRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="findByKeyName",repository="PdpGroupRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="save",repository="PdpSubGroupRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="save",repository="PdpRepository",state="SUCCESS",} 0.0 -spring_data_repository_invocations_seconds_max{exception="None",method="deleteAll",repository="PolicyStatusRepository",state="SUCCESS",} 0.0 -# HELP spring_data_repository_invocations_seconds Duration of repository invocations -# TYPE spring_data_repository_invocations_seconds summary -spring_data_repository_invocations_seconds_count{exception="None",method="findByKeyParentKeyNameAndKeyParentKeyVersion",repository="PolicyStatusRepository",state="SUCCESS",} 16915.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findByKeyParentKeyNameAndKeyParentKeyVersion",repository="PolicyStatusRepository",state="SUCCESS",} 34.05336667 -spring_data_repository_invocations_seconds_count{exception="None",method="save",repository="PdpGroupRepository",state="SUCCESS",} 33797.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="save",repository="PdpGroupRepository",state="SUCCESS",} 14.614549552 -spring_data_repository_invocations_seconds_count{exception="None",method="saveAll",repository="PolicyStatusRepository",state="SUCCESS",} 135740.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="saveAll",repository="PolicyStatusRepository",state="SUCCESS",} 60.741361443 -spring_data_repository_invocations_seconds_count{exception="None",method="findByPdpGroup",repository="PolicyAuditRepository",state="SUCCESS",} 8448.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findByPdpGroup",repository="PolicyAuditRepository",state="SUCCESS",} 140.930950983 -spring_data_repository_invocations_seconds_count{exception="None",method="findById",repository="ToscaServiceTemplateRepository",state="SUCCESS",} 52516.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findById",repository="ToscaServiceTemplateRepository",state="SUCCESS",} 125.813080008 -spring_data_repository_invocations_seconds_count{exception="None",method="findByPdpGroup",repository="PolicyStatusRepository",state="SUCCESS",} 50972.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findByPdpGroup",repository="PolicyStatusRepository",state="SUCCESS",} 106.770108329 -spring_data_repository_invocations_seconds_count{exception="None",method="findAll",repository="PdpGroupRepository",state="SUCCESS",} 102439.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findAll",repository="PdpGroupRepository",state="SUCCESS",} 1036.574172723 -spring_data_repository_invocations_seconds_count{exception="None",method="deleteById",repository="PdpGroupRepository",state="SUCCESS",} 1.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="deleteById",repository="PdpGroupRepository",state="SUCCESS",} 0.007675311 -spring_data_repository_invocations_seconds_count{exception="None",method="findAll",repository="PolicyStatusRepository",state="SUCCESS",} 33795.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findAll",repository="PolicyStatusRepository",state="SUCCESS",} 53.729542707 -spring_data_repository_invocations_seconds_count{exception="None",method="saveAll",repository="PolicyAuditRepository",state="SUCCESS",} 33793.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="saveAll",repository="PolicyAuditRepository",state="SUCCESS",} 41.990371471 -spring_data_repository_invocations_seconds_count{exception="None",method="findByPdpGroupAndNameAndVersion",repository="PolicyAuditRepository",state="SUCCESS",} 16896.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findByPdpGroupAndNameAndVersion",repository="PolicyAuditRepository",state="SUCCESS",} 371.016745717 -spring_data_repository_invocations_seconds_count{exception="None",method="findById",repository="ToscaNodeTemplateRepository",state="SUCCESS",} 25663.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findById",repository="ToscaNodeTemplateRepository",state="SUCCESS",} 2.215574842 -spring_data_repository_invocations_seconds_count{exception="None",method="existsById",repository="PdpGroupRepository",state="SUCCESS",} 1.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="existsById",repository="PdpGroupRepository",state="SUCCESS",} 0.843078054 -spring_data_repository_invocations_seconds_count{exception="None",method="findByKeyName",repository="PdpGroupRepository",state="SUCCESS",} 16902.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="findByKeyName",repository="PdpGroupRepository",state="SUCCESS",} 80.237779619 -spring_data_repository_invocations_seconds_count{exception="None",method="save",repository="PdpSubGroupRepository",state="SUCCESS",} 2.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="save",repository="PdpSubGroupRepository",state="SUCCESS",} 0.03577736 -spring_data_repository_invocations_seconds_count{exception="None",method="save",repository="PdpRepository",state="SUCCESS",} 38194.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="save",repository="PdpRepository",state="SUCCESS",} 52.220218057 -spring_data_repository_invocations_seconds_count{exception="None",method="deleteAll",repository="PolicyStatusRepository",state="SUCCESS",} 67870.0 -spring_data_repository_invocations_seconds_sum{exception="None",method="deleteAll",repository="PolicyStatusRepository",state="SUCCESS",} 24.905966529 -# HELP system_load_average_1m The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time -# TYPE system_load_average_1m gauge -system_load_average_1m 0.34375 -# HELP spring_security_filterchains_requestcache_before_total -# TYPE spring_security_filterchains_requestcache_before_total counter -spring_security_filterchains_requestcache_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP tomcat_sessions_alive_max_seconds -# TYPE tomcat_sessions_alive_max_seconds gauge -tomcat_sessions_alive_max_seconds 1853.0 -# HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management -# TYPE jvm_memory_max_bytes gauge -jvm_memory_max_bytes{area="nonheap",id="CodeHeap 'profiled nmethods'",} 1.22023936E8 -jvm_memory_max_bytes{area="heap",id="G1 Survivor Space",} -1.0 -jvm_memory_max_bytes{area="heap",id="G1 Old Gen",} 8.434745344E9 -jvm_memory_max_bytes{area="nonheap",id="Metaspace",} -1.0 -jvm_memory_max_bytes{area="nonheap",id="CodeHeap 'non-nmethods'",} 7606272.0 -jvm_memory_max_bytes{area="heap",id="G1 Eden Space",} -1.0 -jvm_memory_max_bytes{area="nonheap",id="Compressed Class Space",} 1.073741824E9 -jvm_memory_max_bytes{area="nonheap",id="CodeHeap 'non-profiled nmethods'",} 1.22028032E8 -# HELP hikaricp_connections_acquire_seconds Connection acquire time -# TYPE hikaricp_connections_acquire_seconds summary -hikaricp_connections_acquire_seconds_count{pool="HikariPool-1",} 321133.0 -hikaricp_connections_acquire_seconds_sum{pool="HikariPool-1",} 68.182780148 -# HELP hikaricp_connections_acquire_seconds_max Connection acquire time -# TYPE hikaricp_connections_acquire_seconds_max gauge -hikaricp_connections_acquire_seconds_max{pool="HikariPool-1",} 0.001688513 -# HELP hikaricp_connections_idle Idle connections -# TYPE hikaricp_connections_idle gauge -hikaricp_connections_idle{pool="HikariPool-1",} 10.0 -# HELP jvm_gc_pause_seconds Time spent in GC pause -# TYPE jvm_gc_pause_seconds summary -jvm_gc_pause_seconds_count{action="end of minor GC",cause="Metadata GC Threshold",gc="G1 Young Generation",} 1.0 -jvm_gc_pause_seconds_sum{action="end of minor GC",cause="Metadata GC Threshold",gc="G1 Young Generation",} 0.03 -jvm_gc_pause_seconds_count{action="end of minor GC",cause="GCLocker Initiated GC",gc="G1 Young Generation",} 5.0 -jvm_gc_pause_seconds_sum{action="end of minor GC",cause="GCLocker Initiated GC",gc="G1 Young Generation",} 0.032 -jvm_gc_pause_seconds_count{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",} 29819.0 -jvm_gc_pause_seconds_sum{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",} 205.153 -# HELP jvm_gc_pause_seconds_max Time spent in GC pause -# TYPE jvm_gc_pause_seconds_max gauge -jvm_gc_pause_seconds_max{action="end of minor GC",cause="Metadata GC Threshold",gc="G1 Young Generation",} 0.0 -jvm_gc_pause_seconds_max{action="end of minor GC",cause="GCLocker Initiated GC",gc="G1 Young Generation",} 0.0 -jvm_gc_pause_seconds_max{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",} 0.0 -# HELP spring_security_authentications_seconds_max -# TYPE spring_security_authentications_seconds_max gauge -spring_security_authentications_seconds_max{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="UsernamePasswordAuthenticationToken",error="none",} 0.269684484 -# HELP spring_security_authentications_seconds -# TYPE spring_security_authentications_seconds summary -spring_security_authentications_seconds_count{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="UsernamePasswordAuthenticationToken",error="none",} 168993.0 -spring_security_authentications_seconds_sum{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="UsernamePasswordAuthenticationToken",error="none",} 38517.298249707 -# HELP tomcat_sessions_rejected_sessions_total -# TYPE tomcat_sessions_rejected_sessions_total counter -tomcat_sessions_rejected_sessions_total 0.0 -# HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution -# TYPE jvm_classes_unloaded_classes_total counter -jvm_classes_unloaded_classes_total 268.0 -# HELP spring_security_http_secured_requests_seconds -# TYPE spring_security_http_secured_requests_seconds summary -spring_security_http_secured_requests_seconds_count{error="none",} 168992.0 -spring_security_http_secured_requests_seconds_sum{error="none",} 32721.168866206 -# HELP spring_security_http_secured_requests_seconds_max -# TYPE spring_security_http_secured_requests_seconds_max gauge -spring_security_http_secured_requests_seconds_max{error="none",} 0.0 -# HELP spring_security_filterchains_context_async_before_total -# TYPE spring_security_filterchains_context_async_before_total counter -spring_security_filterchains_context_async_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP hikaricp_connections_max Max connections -# TYPE hikaricp_connections_max gauge -hikaricp_connections_max{pool="HikariPool-1",} 10.0 -# HELP spring_security_http_secured_requests_active_seconds -# TYPE spring_security_http_secured_requests_active_seconds summary -spring_security_http_secured_requests_active_seconds_active_count 1.0 -spring_security_http_secured_requests_active_seconds_duration_sum 0.011941797 -# HELP spring_security_http_secured_requests_active_seconds_max -# TYPE spring_security_http_secured_requests_active_seconds_max gauge -spring_security_http_secured_requests_active_seconds_max 0.011942844 -# HELP spring_security_filterchains_authentication_basic_after_total -# TYPE spring_security_filterchains_authentication_basic_after_total counter -spring_security_filterchains_authentication_basic_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP process_cpu_usage The "recent cpu usage" for the Java Virtual Machine process -# TYPE process_cpu_usage gauge -process_cpu_usage 3.61387926826283E-4 -# HELP executor_completed_tasks_total The approximate total number of tasks that have completed execution -# TYPE executor_completed_tasks_total counter -executor_completed_tasks_total{name="applicationTaskExecutor",} 0.0 -# HELP jvm_threads_started_threads_total The total number of application threads started in the JVM -# TYPE jvm_threads_started_threads_total counter -jvm_threads_started_threads_total 4650.0 -# HELP process_uptime_seconds The uptime of the Java virtual machine -# TYPE process_uptime_seconds gauge -process_uptime_seconds 380261.777 -# HELP pap_policy_deployments_seconds Timer for HTTP request to deploy/undeploy a policy -# TYPE pap_policy_deployments_seconds summary -pap_policy_deployments_seconds_count{operation="deploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_sum{operation="deploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_count{operation="undeploy",status="SUCCESS",} 8448.0 -pap_policy_deployments_seconds_sum{operation="undeploy",status="SUCCESS",} 7322.301986411 -pap_policy_deployments_seconds_count{operation="deploy",status="SUCCESS",} 25345.0 -pap_policy_deployments_seconds_sum{operation="deploy",status="SUCCESS",} 21200.125523501 -pap_policy_deployments_seconds_count{operation="undeploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_sum{operation="undeploy",status="FAILURE",} 0.0 -# HELP pap_policy_deployments_seconds_max Timer for HTTP request to deploy/undeploy a policy -# TYPE pap_policy_deployments_seconds_max gauge -pap_policy_deployments_seconds_max{operation="deploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_max{operation="undeploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_max{operation="deploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_max{operation="undeploy",status="FAILURE",} 0.0 -# HELP jvm_gc_overhead_percent An approximation of the percent of CPU time used by GC activities over the last lookback period or since monitoring began, whichever is shorter, in the range [0..1] -# TYPE jvm_gc_overhead_percent gauge -jvm_gc_overhead_percent 0.0 -# HELP jvm_buffer_memory_used_bytes An estimate of the memory that the Java virtual machine is using for this buffer pool -# TYPE jvm_buffer_memory_used_bytes gauge -jvm_buffer_memory_used_bytes{id="mapped - 'non-volatile memory'",} 0.0 -jvm_buffer_memory_used_bytes{id="mapped",} 0.0 -jvm_buffer_memory_used_bytes{id="direct",} 1544596.0 -# HELP spring_security_filterchains_header_after_total -# TYPE spring_security_filterchains_header_after_total counter -spring_security_filterchains_header_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP jvm_gc_max_data_size_bytes Max size of long-lived heap memory pool -# TYPE jvm_gc_max_data_size_bytes gauge -jvm_gc_max_data_size_bytes 8.434745344E9 -# HELP spring_security_filterchains_authorization_before_total -# TYPE spring_security_filterchains_authorization_before_total counter -spring_security_filterchains_authorization_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP jvm_compilation_time_ms_total The approximate accumulated elapsed time spent in compilation -# TYPE jvm_compilation_time_ms_total counter -jvm_compilation_time_ms_total{compiler="HotSpot 64-Bit Tiered Compilers",} 425964.0 -# HELP application_started_time_seconds Time taken to start the application -# TYPE application_started_time_seconds gauge -application_started_time_seconds{main_application_class="org.onap.policy.pap.main.PolicyPapApplication",} 32.135 -# HELP jdbc_connections_min Minimum number of idle connections in the pool. -# TYPE jdbc_connections_min gauge -jdbc_connections_min{name="dataSource",} 10.0 -# HELP spring_security_filterchains_context_servlet_before_total -# TYPE spring_security_filterchains_context_servlet_before_total counter -spring_security_filterchains_context_servlet_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP hikaricp_connections_pending Pending threads -# TYPE hikaricp_connections_pending gauge -hikaricp_connections_pending{pool="HikariPool-1",} 0.0 -# HELP spring_security_filterchains_logout_before_total -# TYPE spring_security_filterchains_logout_before_total counter -spring_security_filterchains_logout_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP executor_pool_size_threads The current number of threads in the pool -# TYPE executor_pool_size_threads gauge -executor_pool_size_threads{name="applicationTaskExecutor",} 0.0 -# HELP spring_security_filterchains_context_async_after_total -# TYPE spring_security_filterchains_context_async_after_total counter -spring_security_filterchains_context_async_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP system_cpu_usage The "recent cpu usage" of the system the application is running in -# TYPE system_cpu_usage gauge -system_cpu_usage 0.02665967384267763 -# HELP spring_security_authorizations_active_seconds_max -# TYPE spring_security_authorizations_active_seconds_max gauge -spring_security_authorizations_active_seconds_max{spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request",} 0.0 -# HELP spring_security_authorizations_active_seconds -# TYPE spring_security_authorizations_active_seconds summary -spring_security_authorizations_active_seconds_active_count{spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request",} 0.0 -spring_security_authorizations_active_seconds_duration_sum{spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request",} 0.0 -# HELP jdbc_connections_idle Number of established but idle connections. -# TYPE jdbc_connections_idle gauge -jdbc_connections_idle{name="dataSource",} 10.0 -# HELP jdbc_connections_max Maximum number of active connections that can be allocated at the same time. -# TYPE jdbc_connections_max gauge -jdbc_connections_max{name="dataSource",} 10.0 -# HELP tomcat_sessions_active_max_sessions -# TYPE tomcat_sessions_active_max_sessions gauge -tomcat_sessions_active_max_sessions 2.0 -# HELP spring_security_filterchains_access_exceptions_after_total -# TYPE spring_security_filterchains_access_exceptions_after_total counter -spring_security_filterchains_access_exceptions_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP process_files_open_files The open file descriptor count -# TYPE process_files_open_files gauge -process_files_open_files 30.0 -# HELP spring_security_filterchains_authentication_anonymous_after_total -# TYPE spring_security_filterchains_authentication_anonymous_after_total counter -spring_security_filterchains_authentication_anonymous_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP executor_active_threads The approximate number of threads that are actively executing tasks -# TYPE executor_active_threads gauge -executor_active_threads{name="applicationTaskExecutor",} 0.0 -# HELP jvm_threads_states_threads The current number of threads -# TYPE jvm_threads_states_threads gauge -jvm_threads_states_threads{state="runnable",} 9.0 -jvm_threads_states_threads{state="blocked",} 0.0 -jvm_threads_states_threads{state="waiting",} 21.0 -jvm_threads_states_threads{state="timed-waiting",} 8.0 -jvm_threads_states_threads{state="new",} 0.0 -jvm_threads_states_threads{state="terminated",} 0.0 -# HELP logback_events_total Number of log events that were enabled by the effective log level -# TYPE logback_events_total counter -logback_events_total{level="warn",} 0.0 -logback_events_total{level="debug",} 0.0 -logback_events_total{level="error",} 76.0 -logback_events_total{level="trace",} 0.0 -logback_events_total{level="info",} 1846777.0 -# HELP executor_pool_max_threads The maximum allowed number of threads in the pool -# TYPE executor_pool_max_threads gauge -executor_pool_max_threads{name="applicationTaskExecutor",} 2.147483647E9 -# HELP spring_security_filterchains_requestcache_after_total -# TYPE spring_security_filterchains_requestcache_after_total counter -spring_security_filterchains_requestcache_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP spring_security_filterchains_context_servlet_after_total -# TYPE spring_security_filterchains_context_servlet_after_total counter -spring_security_filterchains_context_servlet_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168998.0 -# HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool -# TYPE jvm_buffer_count_buffers gauge -jvm_buffer_count_buffers{id="mapped - 'non-volatile memory'",} 0.0 -jvm_buffer_count_buffers{id="mapped",} 0.0 -jvm_buffer_count_buffers{id="direct",} 16.0 -# HELP jvm_memory_usage_after_gc_percent The percentage of long-lived heap pool used after the last GC event, in the range [0..1] -# TYPE jvm_memory_usage_after_gc_percent gauge -jvm_memory_usage_after_gc_percent{area="heap",pool="long-lived",} 0.02585237978229115 -# HELP application_ready_time_seconds Time taken for the application to be ready to service requests -# TYPE application_ready_time_seconds gauge -application_ready_time_seconds{main_application_class="org.onap.policy.pap.main.PolicyPapApplication",} 32.272 -# HELP http_server_requests_active_seconds_max -# TYPE http_server_requests_active_seconds_max gauge -http_server_requests_active_seconds_max{exception="none",method="GET",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.293631789 -http_server_requests_active_seconds_max{exception="none",method="PUT",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -http_server_requests_active_seconds_max{exception="none",method="DELETE",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -http_server_requests_active_seconds_max{exception="none",method="POST",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -# HELP http_server_requests_active_seconds -# TYPE http_server_requests_active_seconds summary -http_server_requests_active_seconds_active_count{exception="none",method="GET",outcome="SUCCESS",status="200",uri="UNKNOWN",} 1.0 -http_server_requests_active_seconds_duration_sum{exception="none",method="GET",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.293630483 -http_server_requests_active_seconds_active_count{exception="none",method="PUT",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -http_server_requests_active_seconds_duration_sum{exception="none",method="PUT",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -http_server_requests_active_seconds_active_count{exception="none",method="DELETE",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -http_server_requests_active_seconds_duration_sum{exception="none",method="DELETE",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -http_server_requests_active_seconds_active_count{exception="none",method="POST",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -http_server_requests_active_seconds_duration_sum{exception="none",method="POST",outcome="SUCCESS",status="200",uri="UNKNOWN",} 0.0 -# HELP spring_security_filterchains_seconds_max -# TYPE spring_security_filterchains_seconds_max gauge -spring_security_filterchains_seconds_max{error="none",security_security_reached_filter_section="before",spring_security_filterchain_position="11",spring_security_filterchain_size="11",spring_security_reached_filter_name="AuthorizationFilter",} 0.272513877 -spring_security_filterchains_seconds_max{error="none",security_security_reached_filter_section="after",spring_security_filterchain_position="11",spring_security_filterchain_size="11",spring_security_reached_filter_name="DisableEncodeUrlFilter",} 0.001009437 -# HELP spring_security_filterchains_seconds -# TYPE spring_security_filterchains_seconds summary -spring_security_filterchains_seconds_count{error="none",security_security_reached_filter_section="before",spring_security_filterchain_position="11",spring_security_filterchain_size="11",spring_security_reached_filter_name="AuthorizationFilter",} 168999.0 -spring_security_filterchains_seconds_sum{error="none",security_security_reached_filter_section="before",spring_security_filterchain_position="11",spring_security_filterchain_size="11",spring_security_reached_filter_name="AuthorizationFilter",} 38579.546360899 -spring_security_filterchains_seconds_count{error="none",security_security_reached_filter_section="after",spring_security_filterchain_position="11",spring_security_filterchain_size="11",spring_security_reached_filter_name="DisableEncodeUrlFilter",} 168998.0 -spring_security_filterchains_seconds_sum{error="none",security_security_reached_filter_section="after",spring_security_filterchain_position="11",spring_security_filterchain_size="11",spring_security_reached_filter_name="DisableEncodeUrlFilter",} 17.300671502 -# HELP spring_security_filterchains_authorization_after_total -# TYPE spring_security_filterchains_authorization_after_total counter -spring_security_filterchains_authorization_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168992.0 -# HELP spring_security_filterchains_access_exceptions_before_total -# TYPE spring_security_filterchains_access_exceptions_before_total counter -spring_security_filterchains_access_exceptions_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP spring_security_authorizations_seconds -# TYPE spring_security_authorizations_seconds summary -spring_security_authorizations_seconds_count{error="AccessDeniedException",spring_security_authentication_type="AnonymousAuthenticationToken",spring_security_authorization_decision="false",spring_security_object="request",} 6.0 -spring_security_authorizations_seconds_sum{error="AccessDeniedException",spring_security_authentication_type="AnonymousAuthenticationToken",spring_security_authorization_decision="false",spring_security_object="request",} 0.020998153 -spring_security_authorizations_seconds_count{error="none",spring_security_authentication_type="UsernamePasswordAuthenticationToken",spring_security_authorization_decision="true",spring_security_object="request",} 168993.0 -spring_security_authorizations_seconds_sum{error="none",spring_security_authentication_type="UsernamePasswordAuthenticationToken",spring_security_authorization_decision="true",spring_security_object="request",} 4.092135265 -# HELP spring_security_authorizations_seconds_max -# TYPE spring_security_authorizations_seconds_max gauge -spring_security_authorizations_seconds_max{error="AccessDeniedException",spring_security_authentication_type="AnonymousAuthenticationToken",spring_security_authorization_decision="false",spring_security_object="request",} 0.012322361 -spring_security_authorizations_seconds_max{error="none",spring_security_authentication_type="UsernamePasswordAuthenticationToken",spring_security_authorization_decision="true",spring_security_object="request",} 2.03312E-4 -# HELP spring_security_filterchains_session_url_encoding_before_total -# TYPE spring_security_filterchains_session_url_encoding_before_total counter -spring_security_filterchains_session_url_encoding_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP spring_security_filterchains_header_before_total -# TYPE spring_security_filterchains_header_before_total counter -spring_security_filterchains_header_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 168999.0 -# HELP executor_queued_tasks The approximate number of tasks that are queued for execution -# TYPE executor_queued_tasks gauge -executor_queued_tasks{name="applicationTaskExecutor",} 0.0 - diff --git a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_metrics_before_72h.txt b/docs/development/devtools/testing/s3p/pap-s3p-results/pap_metrics_before_72h.txt deleted file mode 100644 index df6df25c..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_metrics_before_72h.txt +++ /dev/null @@ -1,228 +0,0 @@ -# HELP hikaricp_connections_acquire_seconds Connection acquire time -# TYPE hikaricp_connections_acquire_seconds summary -hikaricp_connections_acquire_seconds_count{pool="HikariPool-1",} 39.0 -hikaricp_connections_acquire_seconds_sum{pool="HikariPool-1",} 0.033820135 -# HELP hikaricp_connections_acquire_seconds_max Connection acquire time -# TYPE hikaricp_connections_acquire_seconds_max gauge -hikaricp_connections_acquire_seconds_max{pool="HikariPool-1",} 0.001545051 -# HELP hikaricp_connections_idle Idle connections -# TYPE hikaricp_connections_idle gauge -hikaricp_connections_idle{pool="HikariPool-1",} 10.0 -# HELP hikaricp_connections_min Min connections -# TYPE hikaricp_connections_min gauge -hikaricp_connections_min{pool="HikariPool-1",} 10.0 -# HELP jvm_gc_pause_seconds Time spent in GC pause -# TYPE jvm_gc_pause_seconds summary -jvm_gc_pause_seconds_count{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",} 1.0 -jvm_gc_pause_seconds_sum{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",} 0.037 -# HELP jvm_gc_pause_seconds_max Time spent in GC pause -# TYPE jvm_gc_pause_seconds_max gauge -jvm_gc_pause_seconds_max{action="end of minor GC",cause="G1 Evacuation Pause",gc="G1 Young Generation",} 0.037 -# HELP spring_security_authentications_seconds_max -# TYPE spring_security_authentications_seconds_max gauge -spring_security_authentications_seconds_max{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="UsernamePasswordAuthenticationToken",error="none",} 0.320533592 -# HELP spring_security_authentications_seconds -# TYPE spring_security_authentications_seconds summary -spring_security_authentications_seconds_count{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="UsernamePasswordAuthenticationToken",error="none",} 1.0 -spring_security_authentications_seconds_sum{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="UsernamePasswordAuthenticationToken",error="none",} 0.320533592 -# HELP tomcat_sessions_created_sessions_total -# TYPE tomcat_sessions_created_sessions_total counter -tomcat_sessions_created_sessions_total 2.0 -# HELP disk_total_bytes Total space for path -# TYPE disk_total_bytes gauge -disk_total_bytes{path="/opt/app/policy/pap/bin/.",} 1.0386530304E11 -# HELP tomcat_sessions_rejected_sessions_total -# TYPE tomcat_sessions_rejected_sessions_total counter -tomcat_sessions_rejected_sessions_total 0.0 -# HELP jvm_classes_loaded_classes The number of classes that are currently loaded in the Java virtual machine -# TYPE jvm_classes_loaded_classes gauge -jvm_classes_loaded_classes 18927.0 -# HELP hikaricp_connections_usage_seconds Connection usage time -# TYPE hikaricp_connections_usage_seconds summary -hikaricp_connections_usage_seconds_count{pool="HikariPool-1",} 39.0 -hikaricp_connections_usage_seconds_sum{pool="HikariPool-1",} 9.34 -# HELP hikaricp_connections_usage_seconds_max Connection usage time -# TYPE hikaricp_connections_usage_seconds_max gauge -hikaricp_connections_usage_seconds_max{pool="HikariPool-1",} 0.052 -# HELP jvm_classes_unloaded_classes_total The total number of classes unloaded since the Java virtual machine has started execution -# TYPE jvm_classes_unloaded_classes_total counter -jvm_classes_unloaded_classes_total 0.0 -# HELP hikaricp_connections_active Active connections -# TYPE hikaricp_connections_active gauge -hikaricp_connections_active{pool="HikariPool-1",} 0.0 -# HELP spring_security_filterchains_context_async_before_total -# TYPE spring_security_filterchains_context_async_before_total counter -spring_security_filterchains_context_async_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 5.0 -# HELP process_start_time_seconds Start time of the process since unix epoch. -# TYPE process_start_time_seconds gauge -process_start_time_seconds 1.700139959198E9 -# HELP hikaricp_connections_max Max connections -# TYPE hikaricp_connections_max gauge -hikaricp_connections_max{pool="HikariPool-1",} 10.0 -# HELP spring_security_http_secured_requests_active_seconds -# TYPE spring_security_http_secured_requests_active_seconds summary -spring_security_http_secured_requests_active_seconds_active_count 1.0 -spring_security_http_secured_requests_active_seconds_duration_sum 0.199193291 -# HELP spring_security_http_secured_requests_active_seconds_max -# TYPE spring_security_http_secured_requests_active_seconds_max gauge -spring_security_http_secured_requests_active_seconds_max 0.1992777 -# HELP jvm_memory_used_bytes The amount of used memory -# TYPE jvm_memory_used_bytes gauge -jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'profiled nmethods'",} 2.1837696E7 -jvm_memory_used_bytes{area="heap",id="G1 Survivor Space",} 1.2036896E7 -jvm_memory_used_bytes{area="heap",id="G1 Old Gen",} 4.231168E7 -jvm_memory_used_bytes{area="nonheap",id="Metaspace",} 9.6942648E7 -jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'non-nmethods'",} 1444224.0 -jvm_memory_used_bytes{area="heap",id="G1 Eden Space",} 3.7748736E7 -jvm_memory_used_bytes{area="nonheap",id="Compressed Class Space",} 1.2827304E7 -jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'non-profiled nmethods'",} 9169024.0 -# HELP spring_security_filterchains_authentication_basic_after_total -# TYPE spring_security_filterchains_authentication_basic_after_total counter -spring_security_filterchains_authentication_basic_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 4.0 -# HELP jvm_gc_memory_promoted_bytes_total Count of positive increases in the size of the old generation memory pool before GC to after GC -# TYPE jvm_gc_memory_promoted_bytes_total counter -jvm_gc_memory_promoted_bytes_total 2964480.0 -# HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset -# TYPE jvm_threads_peak_threads gauge -jvm_threads_peak_threads 37.0 -# HELP process_cpu_usage The "recent cpu usage" for the Java Virtual Machine process -# TYPE process_cpu_usage gauge -process_cpu_usage 0.0 -# HELP executor_completed_tasks_total The approximate total number of tasks that have completed execution -# TYPE executor_completed_tasks_total counter -executor_completed_tasks_total{name="applicationTaskExecutor",} 0.0 -# HELP hikaricp_connections_creation_seconds_max Connection creation time -# TYPE hikaricp_connections_creation_seconds_max gauge -hikaricp_connections_creation_seconds_max{pool="HikariPool-1",} 0.0 -# HELP hikaricp_connections_creation_seconds Connection creation time -# TYPE hikaricp_connections_creation_seconds summary -hikaricp_connections_creation_seconds_count{pool="HikariPool-1",} 0.0 -hikaricp_connections_creation_seconds_sum{pool="HikariPool-1",} 0.0 -# HELP jvm_threads_started_threads_total The total number of application threads started in the JVM -# TYPE jvm_threads_started_threads_total counter -jvm_threads_started_threads_total 41.0 -# HELP system_cpu_count The number of processors available to the Java virtual machine -# TYPE system_cpu_count gauge -system_cpu_count 16.0 -# HELP spring_security_filterchains_session_url_encoding_after_total -# TYPE spring_security_filterchains_session_url_encoding_after_total counter -spring_security_filterchains_session_url_encoding_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 4.0 -# HELP process_uptime_seconds The uptime of the Java virtual machine -# TYPE process_uptime_seconds gauge -process_uptime_seconds 824.109 -# HELP pap_policy_deployments_seconds Timer for HTTP request to deploy/undeploy a policy -# TYPE pap_policy_deployments_seconds summary -pap_policy_deployments_seconds_count{operation="deploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_sum{operation="deploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_count{operation="undeploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_sum{operation="undeploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_count{operation="deploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_sum{operation="deploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_count{operation="undeploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_sum{operation="undeploy",status="FAILURE",} 0.0 -# HELP pap_policy_deployments_seconds_max Timer for HTTP request to deploy/undeploy a policy -# TYPE pap_policy_deployments_seconds_max gauge -pap_policy_deployments_seconds_max{operation="deploy",status="FAILURE",} 0.0 -pap_policy_deployments_seconds_max{operation="undeploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_max{operation="deploy",status="SUCCESS",} 0.0 -pap_policy_deployments_seconds_max{operation="undeploy",status="FAILURE",} 0.0 -# HELP jvm_gc_overhead_percent An approximation of the percent of CPU time used by GC activities over the last lookback period or since monitoring began, whichever is shorter, in the range [0..1] -# TYPE jvm_gc_overhead_percent gauge -jvm_gc_overhead_percent 0.0 -# HELP jvm_buffer_memory_used_bytes An estimate of the memory that the Java virtual machine is using for this buffer pool -# TYPE jvm_buffer_memory_used_bytes gauge -jvm_buffer_memory_used_bytes{id="mapped - 'non-volatile memory'",} 0.0 -jvm_buffer_memory_used_bytes{id="mapped",} 0.0 -jvm_buffer_memory_used_bytes{id="direct",} 114688.0 -# HELP executor_queue_remaining_tasks The number of additional elements that this queue can ideally accept without blocking -# TYPE executor_queue_remaining_tasks gauge -executor_queue_remaining_tasks{name="applicationTaskExecutor",} 2.147483647E9 -# HELP hikaricp_connections Total connections -# TYPE hikaricp_connections gauge -hikaricp_connections{pool="HikariPool-1",} 10.0 -# HELP spring_security_filterchains_header_after_total -# TYPE spring_security_filterchains_header_after_total counter -spring_security_filterchains_header_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 4.0 -# HELP tomcat_sessions_expired_sessions_total -# TYPE tomcat_sessions_expired_sessions_total counter -tomcat_sessions_expired_sessions_total 0.0 -# HELP jvm_gc_max_data_size_bytes Max size of long-lived heap memory pool -# TYPE jvm_gc_max_data_size_bytes gauge -jvm_gc_max_data_size_bytes 8.434745344E9 -# HELP tomcat_sessions_active_current_sessions -# TYPE tomcat_sessions_active_current_sessions gauge -tomcat_sessions_active_current_sessions 2.0 -# HELP spring_security_filterchains_authorization_before_total -# TYPE spring_security_filterchains_authorization_before_total counter -spring_security_filterchains_authorization_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 5.0 -# HELP jvm_compilation_time_ms_total The approximate accumulated elapsed time spent in compilation -# TYPE jvm_compilation_time_ms_total counter -jvm_compilation_time_ms_total{compiler="HotSpot 64-Bit Tiered Compilers",} 136782.0 -# HELP hikaricp_connections_timeout_total Connection timeout total count -# TYPE hikaricp_connections_timeout_total counter -hikaricp_connections_timeout_total{pool="HikariPool-1",} 0.0 -# HELP application_started_time_seconds Time taken to start the application -# TYPE application_started_time_seconds gauge -application_started_time_seconds{main_application_class="org.onap.policy.pap.main.PolicyPapApplication",} 32.135 -# HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads -# TYPE jvm_threads_live_threads gauge -jvm_threads_live_threads 37.0 -# HELP spring_security_filterchains_active_seconds_max -# TYPE spring_security_filterchains_active_seconds_max gauge -spring_security_filterchains_active_seconds_max{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_max{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -# HELP spring_security_filterchains_active_seconds -# TYPE spring_security_filterchains_active_seconds summary -spring_security_filterchains_active_seconds_active_count{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_duration_sum{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_active_count{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -spring_security_filterchains_active_seconds_duration_sum{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 0.0 -# HELP jdbc_connections_min Minimum number of idle connections in the pool. -# TYPE jdbc_connections_min gauge -jdbc_connections_min{name="dataSource",} 10.0 -# HELP spring_security_filterchains_context_servlet_before_total -# TYPE spring_security_filterchains_context_servlet_before_total counter -spring_security_filterchains_context_servlet_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 5.0 -# HELP hikaricp_connections_pending Pending threads -# TYPE hikaricp_connections_pending gauge -hikaricp_connections_pending{pool="HikariPool-1",} 0.0 -# HELP spring_security_filterchains_logout_after_total -# TYPE spring_security_filterchains_logout_after_total counter -spring_security_filterchains_logout_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 4.0 -# HELP spring_security_filterchains_logout_before_total -# TYPE spring_security_filterchains_logout_before_total counter -spring_security_filterchains_logout_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 5.0 -# HELP jvm_info JVM version info -# TYPE jvm_info gauge -jvm_info{runtime="OpenJDK Runtime Environment",vendor="Alpine",version="17.0.9+8-alpine-r0",} 1.0 -# HELP disk_free_bytes Usable space for path -# TYPE disk_free_bytes gauge -disk_free_bytes{path="/opt/app/policy/pap/bin/.",} 9.1789115392E10 -# HELP spring_security_authentications_active_seconds -# TYPE spring_security_authentications_active_seconds summary -spring_security_authentications_active_seconds_active_count{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="n/a",} 0.0 -spring_security_authentications_active_seconds_duration_sum{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="n/a",} 0.0 -# HELP spring_security_authentications_active_seconds_max -# TYPE spring_security_authentications_active_seconds_max gauge -spring_security_authentications_active_seconds_max{authentication_failure_type="n/a",authentication_method="ProviderManager",authentication_request_type="UsernamePasswordAuthenticationToken",authentication_result_type="n/a",} 0.0 -# HELP jvm_threads_daemon_threads The current number of live daemon threads -# TYPE jvm_threads_daemon_threads gauge -jvm_threads_daemon_threads 28.0 -# HELP executor_pool_size_threads The current number of threads in the pool -# TYPE executor_pool_size_threads gauge -executor_pool_size_threads{name="applicationTaskExecutor",} 0.0 -# HELP spring_security_filterchains_context_async_after_total -# TYPE spring_security_filterchains_context_async_after_total counter -spring_security_filterchains_context_async_after_total{security_security_reached_filter_section="after",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 4.0 -# HELP system_cpu_usage The "recent cpu usage" of the system the application is running in -# TYPE system_cpu_usage gauge -system_cpu_usage 0.1111111111111111 -# HELP spring_security_filterchains_context_holder_before_total -# TYPE spring_security_filterchains_context_holder_before_total counter -spring_security_filterchains_context_holder_before_total{security_security_reached_filter_section="before",spring_security_filterchain_position="0",spring_security_filterchain_size="0",spring_security_reached_filter_name="none",} 5.0 -# HELP spring_security_authorizations_active_seconds_max -# TYPE spring_security_authorizations_active_seconds_max gauge -spring_security_authorizations_active_seconds_max{spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request",} 0.0 -# HELP spring_security_authorizations_active_seconds -# TYPE spring_security_authorizations_active_seconds summary -spring_security_authorizations_active_seconds_active_count{spring_security_authentication_type="n/a",spring_security_authorization_decision="unknown",spring_security_object="request",} 0.0 diff --git a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_performance_jmeter_results.png b/docs/development/devtools/testing/s3p/pap-s3p-results/pap_performance_jmeter_results.png Binary files differdeleted file mode 100644 index e061ba47..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_performance_jmeter_results.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stability_jmeter_results.png b/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stability_jmeter_results.png Binary files differdeleted file mode 100644 index c1c04f92..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stability_jmeter_results.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_after_72h.png b/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_after_72h.png Binary files differdeleted file mode 100644 index 7c56f74a..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_after_72h.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_before_72h.png b/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_before_72h.png Binary files differdeleted file mode 100644 index 0984521f..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_before_72h.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_during_72h.png b/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_during_72h.png Binary files differdeleted file mode 100644 index 1d86b175..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p-results/pap_stats_during_72h.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/pap-s3p.rst b/docs/development/devtools/testing/s3p/pap-s3p.rst deleted file mode 100644 index c658cbc5..00000000 --- a/docs/development/devtools/testing/s3p/pap-s3p.rst +++ /dev/null @@ -1,198 +0,0 @@ -.. This work is licensed under a -.. Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _pap-s3p-label: - -.. toctree:: - :maxdepth: 2 - -Policy PAP component -~~~~~~~~~~~~~~~~~~~~ - -Both the Performance and the Stability tests were executed by performing requests -against Policy components installed as part of a full ONAP OOM deployment or a docker deployment in Nordix lab. - -Setup Details -+++++++++++++ - -- Policy-PAP along with all policy components deployed as part of a Policy docker deployment. -- A second instance of APEX-PDP is spun up in the setup. Update the configuration file (OnapPfConfig.json) such that the PDP can register to the new group created by PAP in the tests. -- Both tests were run via jMeter. - -Stability Test of PAP -+++++++++++++++++++++ - -Test Plan ---------- -The 72 hours stability test ran the following steps sequentially in a single threaded loop. - -Setup Phase (steps running only once) -""""""""""""""""""""""""""""""""""""" - -- **Create Policy for defaultGroup** - creates an operational policy using policy/api component -- **Create NodeTemplate metadata for sampleGroup policy** - creates a node template containing metadata using policy/api component -- **Create Policy for sampleGroup** - creates an operational policy that refers to the metadata created above using policy/api component -- **Change defaultGroup state to ACTIVE** - changes the state of defaultGroup PdpGroup to ACTIVE -- **Create/Update PDP Group** - creates a new PDPGroup named sampleGroup. - A second instance of the PDP that is already spun up gets registered to this new group -- **Check PdpGroup Query** - makes a PdpGroup query request and verifies that both PdpGroups are in ACTIVE state. - -PAP Test Flow (steps running in a loop for 72 hours) -"""""""""""""""""""""""""""""""""""""""""""""""""""" - -- **Check Health** - checks the health status of pap -- **PAP Metrics** - Fetch prometheus metrics before the deployment/undeployment cycle - Save different counters such as deploy/undeploy-success/failure counters at API and engine level. -- **Check PdpGroup Query** - makes a PdpGroup query request and verifies that both PdpGroups are in the ACTIVE state. -- **Deploy Policy for defaultGroup** - deploys the policy defaultDomain to defaultGroup -- **Check status of defaultGroup policy** - checks the status of defaultGroup PdpGroup with the defaultDomain policy 1.0.0. -- **Check PdpGroup Audit defaultGroup** - checks the audit information for the defaultGroup PdpGroup. -- **Check PdpGroup Audit Policy (defaultGroup)** - checks the audit information for the defaultGroup PdpGroup with the defaultDomain policy 1.0.0. -- **Check PdpGroup Query** - makes a PdpGroup query request and verifies that 2 PdpGroups are in the ACTIVE state and defaultGroup has a policy deployed on it. -- **Deployment Update for sampleGroup policy** - deploys the policy sampleDomain in sampleGroup PdpGroup using pap api -- **Check status of sampleGroup** - checks the status of the sampleGroup PdpGroup. -- **Check status of PdpGroups** - checks the status of both PdpGroups. -- **Check PdpGroup Query** - makes a PdpGroup query request and verifies that the defaultGroup has a policy defaultDomain deployed on it and sampleGroup has policy sampleDomain deployed on it. -- **Check Audit** - checks the audit information for all PdpGroups. -- **Check Consolidated Health** - checks the consolidated health status of all policy components. -- **Check Deployed Policies** - checks for all the deployed policies using pap api. -- **Undeploy policy in sampleGroup** - undeploys the policy sampleDomain from sampleGroup PdpGroup using pap api -- **Undeploy policy in defaultGroup** - undeploys the policy defaultDomain from PdpGroup -- **Check status of policies** - checks the status of all policies and make sure both the policies are undeployed -- **Check PdpGroup Query** - makes a PdpGroup query request and verifies that PdpGroup is in the PASSIVE state. -- **PAP Metrics after deployments** - Fetch prometheus metrics after the deployment/undeployment cycle - Save the new counter values such as deploy/undeploy-success/failure counters at API and engine level, and check that the deploySuccess and undeploySuccess counters are increased by 2. - -.. Note:: - To avoid putting a large Constant Timer value after every deployment/undeployment, the status API is polled until the deployment/undeployment - is successfully completed, or until a timeout. This is to make sure that the operation is completed successfully and the PDPs gets enough time to respond back. - Otherwise, before the deployment is marked successful by PAP, an undeployment could be triggered as part of other tests, - and the operation's corresponding prometheus counter at engine level will not get updated. - -Teardown Phase (steps running only once after PAP Test Flow is completed) -""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""""" - -- **Change state to PASSIVE(sampleGroup)** - changes the state of sampleGroup PdpGroup to PASSIVE -- **Delete PdpGroup sampleGroup** - delete the sampleGroup PdpGroup using pap api -- **Change State to PASSIVE(defaultGroup)** - changes the state of defaultGroup PdpGroup to PASSIVE -- **Delete policy created for defaultGroup** - deletes the operational policy defaultDomain using policy/api component -- **Delete Policy created for sampleGroup** - deletes the operational policy sampleDomain using policy/api component -- **Delete Nodetemplate metadata for sampleGroup policy** - deleted the nodetemplate containing metadata for sampleGroup policy - -The following steps can be used to configure the parameters of test plan. - -- **HTTP Authorization Manager** - used to store user/password authentication details. -- **HTTP Header Manager** - used to store headers which will be used for making HTTP requests. -- **User Defined Variables** - used to store following user defined parameters. - -=========== =================================================================== - **Name** **Description** -=========== =================================================================== - PAP_HOST IP Address or host name of PAP component - PAP_PORT Port number of PAP for making REST API calls - API_HOST IP Address or host name of API component - API_PORT Port number of API for making REST API calls -=========== =================================================================== - -The test was run in the background via "nohup", to prevent it from being interrupted: - -.. code-block:: bash - - nohup apache-jmeter-5.6.2/bin/jmeter -n -t stability.jmx -l stabilityTestResults.jtl & - -Test Results ------------- - -**Summary** - -Stability test plan was triggered for 72 hours. There were no failures during the 72 hours test. - - -**Test Statistics** - -======================= ================= ================== ================================== -**Total # of requests** **Success %** **Error %** **Average time taken per request** -======================= ================= ================== ================================== - 170212 100 % 0.00 % 419 ms -======================= ================= ================== ================================== - - -**JMeter Screenshot** - -.. image:: pap-s3p-results/pap_stability_jmeter_results.png - -**Memory and CPU usage** - -The memory and CPU usage can be monitored by running "docker stats" command in the PAP container. -A snapshot is taken before, during and after test execution to monitor the changes in resource utilization. -Prometheus metrics is also collected before and after the test execution. - -Memory and CPU usage before test execution: - -.. image:: pap-s3p-results/pap_stats_before_72h.png - -:download:`Prometheus metrics before 72h test <pap-s3p-results/pap_metrics_before_72h.txt>` - -Memory and CPU usage during test execution: - -.. image:: pap-s3p-results/pap_stats_during_72h.png - -Memory and CPU usage after test execution: - -.. image:: pap-s3p-results/pap_stats_after_72h.png - -:download:`Prometheus metrics after 72h test <pap-s3p-results/pap_metrics_after_72h.txt>` - -Performance Test of PAP -++++++++++++++++++++++++ - -Introduction ------------- - -Performance test of PAP has the goal of testing the min/avg/max processing time and rest call throughput for all the requests with multiple requests at the same time. - -Setup Details -------------- - -The performance test is performed on a similar setup as Stability test. The JMeter VM will be sending a large number of REST requests to the PAP component and collecting the statistics. - - -Test Plan ---------- - -Performance test plan is the same as the stability test plan above except for the few differences listed below. - -- Increase the number of threads up to 10 (simulating 10 users' behaviours at the same time). -- Reduce the test time to 2 hours. -- Usage of counters (simulating each user) to create different pdpGroups, update their state and later delete them. -- Removed the tests to deploy policies to newly created groups as this will need a larger setup with multiple pdps registered to each group, which will also slow down the performance test with the time needed for registration process etc. -- Usage of counters (simulating each user) to create different drools policies and deploy them to defaultGroup. - In the test, a thread count of 10 is used resulting in 10 different drools policies getting deployed and undeployed continuously for 2 hours. - Other standard operations like checking the deployment status of policies, checking the metrics, health etc remains. - -Run Test --------- - -Running/Triggering the performance test will be the same as the stability test. That is, launch JMeter pointing to corresponding *.jmx* test plan. The *API_HOST* , *API_PORT* , *PAP_HOST* , *PAP_PORT* are already set up in *.jmx*. - -.. code-block:: bash - - nohup apache-jmeter-5.6.2/bin/jmeter -n -t performance.jmx -l performanceTestResults.jtl & - -Test Results ------------- - -Test results are shown as below. - -**Test Statistics** - -======================= ================= ================== ================================== -**Total # of requests** **Success %** **Error %** **Average time taken per request** -======================= ================= ================== ================================== -48093 100 % 0.00 % 1116 ms -======================= ================= ================== ================================== - -**JMeter Screenshot** - -.. image:: pap-s3p-results/pap_performance_jmeter_results.png diff --git a/docs/development/devtools/testing/s3p/s3p-test-overview.rst b/docs/development/devtools/testing/s3p/s3p-test-overview.rst new file mode 100644 index 00000000..f79ba921 --- /dev/null +++ b/docs/development/devtools/testing/s3p/s3p-test-overview.rst @@ -0,0 +1,118 @@ +.. This work is licensed under a +.. Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Policy Framework S3P Tests Overview +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. contents:: + :depth: 2 + +Starting with the Oslo release of the Policy Framework, S3P tests are now triggered differently. + +Our S3P tests for each component run automatically every Monday. This includes both performance and stability tests. These tests are triggered in a GitHub Actions environment, and the results for each test can be found under the "Actions" tab in the GitHub repository for each component. + +Stability and Performance Test Workflows +---------------------------------------- + +Each component of the Policy Framework contains two workflow files in the `.github/workflows` directory: +- `gerrit-{componentName}-performance.yaml` +- `gerrit-{componentName}-stability.yaml` + +.. image:: images/workflows.png + +An example of the configuration for one of these files is shown below: + +.. code-block:: yaml + + name: policy-api-stability-test + + on: + workflow_dispatch: + inputs: + GERRIT_BRANCH: + description: 'Branch that the change is against' + required: true + type: string + GERRIT_CHANGE_ID: + description: 'The ID for the change' + required: true + type: string + GERRIT_CHANGE_NUMBER: + description: 'The Gerrit change number' + required: true + type: string + GERRIT_CHANGE_URL: + description: 'URL of the change' + required: true + type: string + GERRIT_EVENT_TYPE: + description: 'The type of Gerrit event' + required: true + type: string + GERRIT_PATCHSET_NUMBER: + description: 'The patch number for the change' + required: true + type: string + GERRIT_PATCHSET_REVISION: + description: 'The SHA of the revision' + required: true + type: string + GERRIT_PROJECT: + description: 'The project in Gerrit' + required: true + type: string + GERRIT_REFSPEC: + description: 'The Gerrit refspec for the change' + required: true + type: string + branch_protection_rule: + # Ensures that the "Maintained" check is occasionally updated. + # See https://github.com/ossf/scorecard/blob/main/docs/checks.md#maintained + + # Runs every Monday at 16:30 UTC + schedule: + - cron: '30 16 * * 1' + + jobs: + run-s3p-tests: + runs-on: ubuntu-22.04 + + steps: + - uses: actions/checkout@v4 + + - name: Run S3P script + working-directory: ${{ github.workspace }}/testsuites + run: sudo bash ./run-s3p-test.sh run stability + + - name: Archive result .jtl + uses: actions/upload-artifact@v4 + with: + name: policy-api-s3p-results + path: ${{ github.workspace }}/testsuites/automate-performance/s3pTestResults.jtl + + - name: Archive JMeter logs + uses: actions/upload-artifact@v4 + with: + name: policy-api-s3p-jmeter-log + path: ${{ github.workspace }}/testsuites/automate-performance/jmeter.log + +Analyzing the Results +##################### + +The results of each workflow run can be found under the "Actions" tab. + +.. image:: images/workflow-results.png + +To investigate the results further, click on a completed test run. You will see details about: +- The test that was executed +- The test's status (indicated by a green checkmark or a red "X") +- The artifacts produced during the test + +The artifacts include: +- A test result file in `.jtl` format +- JMeter logs, which can assist in debugging test failures + +.. image:: images/workflow-test-result.png + +Both the stability and performance tests run for two hours each in the GitHub Actions environment. Since these tests are conducted weekly and closely monitored by the Policy Framework team, the previous practice of running stability tests for 72 hours has been deemed unnecessary. diff --git a/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-perf-xacml.png b/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-perf-xacml.png Binary files differdeleted file mode 100644 index 6f30f143..00000000 --- a/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-perf-xacml.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-stability-xacml.png b/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-stability-xacml.png Binary files differdeleted file mode 100644 index 842ec9dd..00000000 --- a/docs/development/devtools/testing/s3p/xacml-s3p-results/s3p-stability-xacml.png +++ /dev/null diff --git a/docs/development/devtools/testing/s3p/xacml-s3p.rst b/docs/development/devtools/testing/s3p/xacml-s3p.rst deleted file mode 100644 index 3b81406b..00000000 --- a/docs/development/devtools/testing/s3p/xacml-s3p.rst +++ /dev/null @@ -1,198 +0,0 @@ -.. This work is licensed under a -.. Creative Commons Attribution 4.0 International License. -.. http://creativecommons.org/licenses/by/4.0 - -.. _xacml-s3p-label: - -.. toctree:: - :maxdepth: 2 - -########################## - -Policy XACML PDP S3P Tests -~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Both the Performance and the Stability tests were executed by performing requests -against Policy components installed in Kubernetes environment. These tests were all -performed on a Ubuntu VM with 32GB of memory, 16 CPU and 100GB of disk space. - -Policy XACML PDP Deployment -+++++++++++++++++++++++++++ - -In an effort to allow the execution of the s3p tests to be as close to automatic as possible, -a script will be executed that will perform the following: - -- Install of a microk8s kubernetes environment -- Bring up the policy components -- Checks that the components are successfully up and running before proceeding -- Install Java 17 -- Install Jmeter locally and configure it -- Specify whether you want to run stability or performance tests - - -The remainder of this document outlines how to run the tests and the test results - -Common Setup -++++++++++++ -The common setup for performance and stability tests is now automated - being carried out by a script in- **testsuites/run-s3p-test.sh**. - -Clone the policy-xacml-pdp repo to access the test scripts - -.. code-block:: bash - - git clone https://gerrit.onap.org/r/policy/xacml-pdp xacml-pdp - -Stability Test of Policy XACML PDP -++++++++++++++++++++++++++++++++++ - -Test Plan ---------- -The 24 hours stability test ran the following steps. - -- Healthcheck, 2 simultaneous threads -- Decisions, 2 simultaneous threads, each running the following tasks in sequence: - - Monitoring Decision - - Monitoring Decision, abbreviated - - Naming Decision - - Optimization Decision - - Default Guard Decision (always "Permit") - - Frequency Limiter Guard Decision - - Min/Max Guard Decision - -This runs for 24 hours. Test results are present in the **testsuites/automated-performance/s3pTestResults.jtl** -file and in **/tmp/** directory. Logs are present for jmeter in **testsuites/automated-performance/jmeter.log** and -**testsuites/automated-performance/nohup.out** - -Run Test --------- - -The code in the setup section also serves to run the tests. Just one execution needed to do it all. - -.. code-block:: bash - - bash run-s3p-test.sh run stability - -Once the test execution is completed, the results are present in the **automate-performance/s3pTestResults.jtl** file. - -This file can be imported into the Jmeter GUI for visualization. The below results are tabulated from the GUI. - -Test Results ------------- - -**Summary** - -Stability test plan was triggered for 24 hours. - -**Test Statistics** - -======================= ================= ======================== ========================= -**Total # of requests** **Error %** **Average Latency (ms)** **Measured requests/sec** -======================= ================= ======================== ========================= - 54472562 0.00 % 5 ms 630.1 ms -======================= ================= ======================== ========================= - -**JMeter Results** - -.. image:: xacml-s3p-results/s3p-stability-xacml.png - -**Policy component Setup** - -============================================== ==================================================================== ============================================= -**NAME** **IMAGE** **PORT** -============================================== ==================================================================== ============================================= -zookeeper-deployment-7ff87c7fcc-fbsfb confluentinc/cp-zookeeper:latest 2181/TCP -kafka-deployment-5c87d497b-m8s2g confluentinc/cp-kafka:latest 9092/TCP -policy-drools-pdp-0 nexus3.onap.org:10001/onap/policy-pdpd-cl:2.1.3-SNAPSHOT 6969/TCP 9696/TCP -policy-apex-pdp-0 nexus3.onap.org:10001/onap/policy-apex-pdp:3.1.3-SNAPSHOT 6969/TCP -policy-distribution-f48bff778-48pm2 nexus3.onap.org:10001/onap/policy-distribution:3.1.3-SNAPSHOT 6969/TCP -policy-models-simulator-6947667bdc-wcd9r nexus3.onap.org:10001/onap/policy-models-simulator:3.1.3-SNAPSHOT 6666/TCP 6680/TCP 6668/TCP 6669/TCP 6670/TCP -policy-clamp-ac-http-ppnt-7d747b5d98-wmr5n nexus3.onap.org:10001/onap/policy-clamp-ac-http-ppnt:7.1.3-SNAPSHOT 8084/TCP -policy-clamp-ac-k8s-ppnt-6bbd86bbc6-vnvx6 nexus3.onap.org:10001/onap/policy-clamp-ac-k8s-ppnt:7.1.3-SNAPSHOT 8083/TCP -policy-clamp-ac-pf-ppnt-5fcbbcdb6c-k2cbk nexus3.onap.org:10001/onap/policy-clamp-ac-pf-ppnt:7.1.3-SNAPSHOT 6969/TCP -policy-clamp-ac-sim-ppnt-97f487577-m2zjr nexus3.onap.org:10001/onap/policy-clamp-ac-sim-ppnt:7.1.3-SNAPSHOT 6969/TCP -policy-clamp-runtime-acm-66b5d6b64-l6dpq nexus3.onap.org:10001/onap/policy-clamp-runtime-acm:7.1.3-SNAPSHOT 6969/TCP -mariadb-galera-0 docker.io/bitnami/mariadb-galera:10.5.8 3306/TCP -prometheus-f66f97b6-kkmpq nexus3.onap.org:10001/prom/prometheus:latest 9090/TCP -policy-api-7f7d995b4-2zhnw nexus3.onap.org:10001/onap/policy-api:3.1.3-SNAPSHOT 6969/TCP -policy-pap-f7899d4cd-mfrtp nexus3.onap.org:10001/onap/policy-pap:3.1.3-SNAPSHOT 6969/TCP -policy-xacml-pdp-6c86f85ff6-6qzgf nexus3.onap.org:10001/onap/policy-xacml-pdp:3.1.2 6969/TCP -============================================== ==================================================================== ============================================= - -.. Note:: - - .. container:: paragraph - - There were no failures during the 24 hours test. - -The XACML PDP offered very good performance with JMeter for the traffic mix described above. -The average transaction time is insignificant. - - -Performance Test of Policy XACML PDP -++++++++++++++++++++++++++++++++++++ - -Introduction ------------- - -Performance test of acm components has the goal of testing the min/avg/max processing time and rest call throughput for all the requests with multiple requests at the same time. - -Setup Details -------------- - -We can setup the environment and execute the tests like this from the **xacml-pdp/testsuites** directory - -Test Plan ---------- - -Performance test plan is the same as the stability test plan above except for the few differences listed below. - -- Increase the number of threads up to 10 (simulating 10 users' behaviours at the same time). -- Reduce the test time to 20 minutes. - -The performance tests runs the following, all in parallel: - -- Healthcheck, 10 simultaneous threads -- Decisions, 10 simultaneous threads, each running the following in sequence: - - - Monitoring Decision - - Monitoring Decision, abbreviated - - Naming Decision - - Optimization Decision - - Default Guard Decision (always "Permit") - - Frequency Limiter Guard Decision - - Min/Max Guard Decision - -When the script starts up, it uses policy-api to create, and policy-pap to deploy -the policies that are needed by the test. It assumes that the "naming" policy has -already been created and deployed. Once the test completes, it undeploys and deletes -the policies that it previously created. - -Run Test --------- - -The code in the setup section also serves to run the tests. Just one execution needed to do it all. - -.. code-block:: bash - - bash run-s3p-test.sh run performance - -Once the test execution is completed, the results are present in the **automate-performance/s3pTestResults.jtl** file and in **/tmp/** directory. - -This file can be imported into the Jmeter GUI for visualization. The below results are tabulated from the GUI. - -Test Results ------------- - -**Summary** - -The test was run for 20 minutes with 10 users (i.e., threads), with the following results: - -**Test Statistics** - -======================= ================= ======================== ========================= -**Total # of requests** **Error %** **Average Latency (ms)** **Measured requests/sec** -======================= ================= ======================== ========================= - 888047 0.00 % 25 ms 723.2 ms -======================= ================= ======================== ========================= - -.. image:: xacml-s3p-results/s3p-perf-xacml.png |
