aboutsummaryrefslogtreecommitdiffstats
path: root/docker-compose
AgeCommit message (Collapse)AuthorFilesLines
11 daysAdd healthchecks for docker-compose for k6 and CSITdanielhanrahan3-14/+93
To improve reliability and consistency of healthchecks in k6 and CSIT tests, they are implemented in docker-compose. This commit adds fail-fast logic, where k6 and CSITs will abort immediately if the containers won't start. Implementation: - Add healthchecks for docker containers used in tests. - Change k6 & CSIT tests to use docker healthchecks. - Tests will abort if containers are not healthy. - Start-up timeout for CPS containers is 90 seconds - Start-up timeout for other containers is 60 seconds Other Improvements: - Add --quiet-pull option to suppress junk output in Jenkins logs. - Add kpi.env file containing environment variables for KPI pipeline, just like endurance.env. This allows same code to run either suite. - Changed from port range to port number for Postgres exporter, since only a single instance runs, to be consistent with other containers Issue-ID: CPS-2630 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I50929ca4061bb844fca87a0d6c3103aaa2c45e0b
2025-02-07Disallow docker-compose to use swap spacehalil.cakal1-0/+1
- if swap memory is active on any host, the docker-compose will add it on top of JVM process memory. This will exceed the total memory reservation of the container, eventually cause restart. - swap limit can not be lower than the total memory limit, thus given 3G. this will disallow docker-compose to use swap space Issue-ID: CPS-2628 Change-Id: I01bd4a1fd3f4e894fe8236bbc38cd7718fb0b6fb Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-02-06Merge "Remove the profile from postgres-exporter service"Daniel Hanrahan1-2/+0
2025-02-06Remove the profile from postgres-exporter servicehalil.cakal1-2/+0
- The profile 'monitoring' prevents postgres-exporter service to run for both kpi and endurance pipelines. As a result, the postgres dashboard does not work on Nordix Grafana. Issue-ID: CPS-2585 Change-Id: I272ab609c60094bb347ef405787679860cc43595 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-02-05Add Hikari dashboard to Grafanahalil.cakal6-657/+1029
- hikari dashboard will insight into db connection pool such as connection count e.g. min, max, and idle. - change coloring for some panel from JVM dashboard - add tags to the dashboards Issue-ID: CPS-2592 Change-Id: Ibfb49ab5c1e338d04a7c8f18a413b5db7b8d5ad7 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-02-04Merge "Fix deprecation issue related to KafkaContainer"Daniel Hanrahan1-2/+2
2025-02-04Merge "Enable filtering by instance"Lee Anjella Macabuhay1-39/+39
2025-02-04Increase initial module processing delay to 3 minsdanielhanrahan1-1/+1
Issue-ID: CPS-2616 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie179c6e16cb9c49944b64665aba89c85029f4268
2025-01-31Enable filtering by instancehalil.cakal1-39/+39
- if kpi and endurance suits runing together at the same time, instance flag will help to filter either of them. otherwise the stats for both suits will be visible on a single plot together. Issue-ID: CPS-2585 Change-Id: I422adb0afd3a618c477989a9396636f63808c275 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-01-30Fix deprecation issue related to KafkaContainermpriyank1-2/+2
- Replaced KafkaContainer with ConfluentKafkaContainer to fix the deprecation issue - Bumped the version of cp-kafka and cp-zookeeper to 7.8.0 - Removing reference of registry.nordix as the NCMP internal pipeline is no longer used Issue-ID: CPS-2608 Change-Id: Ic1f3edbc06d63c5f00c1ecb901e07a6bc17ae6eb Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2025-01-30Add new dashboard to monitor some of the stats of PostgreSQLhalil.cakal4-1/+2266
- add postgres-exporter service to expose postgresql db metrics to prometheus - add new job scraper to prometheus for both kpi and endurance jobs - add new dashboard: postgresql-statistics to the grafana Issue-ID: CPS-2585 Change-Id: Ia7ede41d00364918b1054d9b5a38e2feea4fc8e5 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-01-30Merge "Add Data dashboard to Grafana"Daniel Hanrahan2-0/+277
2025-01-24Disable JVM process metrics by defaultdanielhanrahan1-0/+1
Issue-ID: CPS-2583 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I9c7a543201a63c1f128737bb0995843af6075a6b
2025-01-24Add Data dashboard to Grafanahalil.cakal2-0/+277
- data dashboard for NCMP Data operations (NetworkCmProxyController.java) Issue-ID: CPS-2537 Change-Id: Ic518e9169057691702d0d760468e114f5a9fbd7d Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-01-24Merge "Add dashboard for Inventory REST interfaces"Daniel Hanrahan2-217/+25
2025-01-24[k6] 50000 CM-handlesdanielhanrahan1-2/+2
- Increase CM-handles to 50000 in k6 tests - Increase container memory to 3GB (crashes with 2GB) - Decrease max heap percentage to 70% to ensure container has at least 200MB free memory, to prevent JVM from crashing at higher peak loads. Issue-ID: CPS-2584 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8daabff58723114bdc3838b379e83c3ffbbb0ba9
2025-01-21Add dashboard for Inventory REST interfaceshalil.cakal2-217/+25
- rename lcm-state dashboard to inventory - remove panels holding a single state of cm handle - add panel for inventory APIs calls by URI and the STATUS Issue-ID: CPS-2567 Change-Id: I374fa18b53d456519142cf059f74b2a070485355 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-01-20Increase JVM heap to 75% againdanielhanrahan1-1/+1
Heap percent was commented out by mistake leading to OutOfMemoryError in endurance pipeline, due to only 25% of memory being used for heap. Issue-ID: CPS-2570 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ifbffe7222e38f017105cfac8bc8307255d28d11a
2025-01-17Merge "One SchemaSet per moduleSetTag"Sourabh Sourabh1-6/+5
2025-01-16Add LCM State dashboard to Grafanahalil.cakal2-0/+563
- adding a grafana dashboard for lcm state changes of cm handles Issue-ID: CPS-2558 Change-Id: Ie00884736ffa5c08e1be150c71cf8ac603f2ab60 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-01-16One SchemaSet per moduleSetTagToineSiebelink1-6/+5
- Registration: create and upgrade cases. - Handle moduleSetTag deletion (all orphans) for testware - Unit tests updated - additional logging of details for upgrade scenarios - Integration Tests updated - Remove cache for module sets being processed - Removed DbCleaner (startup) - Removed redundant methods in NCMP Inventory for deleting schema set(s) - Removed validation check for all schema set interactions - Updated some schema set tests to use special characters previously not allowed - Checked integration test scenarios for upgrades with and without tags: all scenarios covered! TODO - REST endpoint to remove orphaned schema set data, separate story: CPS-2554 - Investigate exception handling regarding DuplicateYangResourceException: CPS-2555 Issue-ID: CPS-2540 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Iaa59cbdb86b7a4a8044624829bc002506ff40cc7
2025-01-13Change image of Grafana in local docker-composehalil.cakal1-1/+1
- change image of grafana to grafana/grafana from grafana/grafana-oss in local docker-compose file Issue-ID: CPS-2559 Change-Id: I03bf895c10a108c9aeae2db6b2dc6b39db505df5 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-12-04Add persistent storage to prometheushalil.cakal2-3/+41
- add healthcheck to ensure prometheus service is running correctly and restarts it if it becomes unhealthy - set the data retention time to 30d - update readme Issue-ID: CPS-2466 Change-Id: I0f6186f8577e8e0113831ed040298b823e33d624 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-12-04Plot metrics for each cps-and-ncmp instance separately in Grafanahalil.cakal3-2/+14
- assing port range to cps-and-ncmp service to plot metrics separately Issue-ID: CPS-2536 Change-Id: Ic097017bbcca25f28341e8e77f658a0a4e8964d7 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-11-27[CPS] Reverted ncmp-dmi-plugin version to latestsourabh_sourabh1-1/+1
- Reverted DMI_VERSION to latest Ncmp-dmi-Plugin 1.6.0-SNAPSHOT-latest for CSIT. - Corrected dmi-stub docker image name from "dmi-plugin-demo-and-csit-stub" to "dmi-stub" Issue-ID: CPS-2499 Change-Id: I99087b7230a40e958b3b789549a4a8e4c9dd0341 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-11-01Add docker compose deployment for endurance testshalil.cakal2-18/+55
- add new config file containing env variables for endurance test - add notes to describe how to run and stop it Note 1: Existing docker-compose deployment won't be affected by this changes Note 2: Both docker-compose can be deployed simultaneously without any issue Issue-ID: CPS-2463 Change-Id: I15936718dda2f31a5077b9c36eded21c1a85dfcd Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-10-18Enable 2 minute initial module processing delaydanielhanrahan1-1/+1
Issue-ID: CPS-2403 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I614313461bd36ca351a8d42d9683b88460090b50
2024-10-10Merge "Autodiscover cps-and-ncmp containers when load balancing"Daniel Hanrahan2-10/+10
2024-10-09Autodiscover cps-and-ncmp containers when load balancinghalil.cakal2-10/+10
- remove hard-coded container names from load balancer(nginx) and prometheus Issue-ID: CPS-2437 Change-Id: Ia64b949c2eb3e4d67e123b046e66d5bfdda2c801 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-10-09Fine tune JVM parameters for cps-and-ncmphalil.cakal1-4/+2
- limit the total memory with 2G - set heap memory 75% of the total RAM Issue-ID: CPS-2180 Change-Id: Ia8a6c2bd40866052caba4f332ac563e1ed829212 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-10-03Policy Executor: handle errors, part 2ToineSiebelink2-0/+9
(fighting between IntelliJ and Checkstyle best practices) - non-2xx responses are processed using web client exceptions - handle unknown host exception - upgraded spotbugs (checkstyle and related mvn plugin) - fixed some small spotbugs due to upgrade - added commented instructions in docker compose to enable debugging - added some environment variables for policy executor configuration - extract out Sleeper in stub service to achieve 100% coverage - added cause to Policy Executor exceptions where applicable - ignored (new) spotbug rule about catch NPE because of issue in 3pp - ignored (new) spotbug rule about \n in string due to multiline string block Issue-ID: CPS-2412 Change-Id: I6835a73320c436cbeea12cc7a06f15899eec7bf1 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2024-09-19Make nginx load balance using least-connectionsdanielhanrahan1-1/+3
By default, nginx uses round robin for load balancing. Round robin works well when all requests take same amount of time, but is not optimal for dynamic loads. With least-connected load balancing, nginx will try not to overload a busy server with excessive requests, sending requests to a less busy server instead. Issue-ID: CPS-2415 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I0e7d6b36a79aa6541f2f5f667746f2792d55e54b
2024-09-17Policy Executor API: Add Prefix in YamlToineSiebelink1-1/+3
- Remove servers from yaml. - Add prefix to path in yaml. - Remove basepath prefix from Controller Impl. - Added policy server port to 8093. - Exposed docker env. vars for policy services. Issue-ID: CPS-2291 Change-Id: Idcc9e23f9d63bad2480537dc8d39523fe82b4b83 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-09-04K6 Performance Graph fails frequentlysourabh_sourabh1-0/+1
- Overrides initial processing delay of every new module set tag for DMI plugin stub from 2 min (default) to 0 as a temp. solution. Issue-ID: CPS-2395 Change-Id: Ic875262314406ec55457e31a69c00d55293c1b74 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-08-23Support 2 CPS instances in Prometheus/Grafanadanielhanrahan8-8/+3642
- Specify both cps-and-ncmp instances in Prometheus config - Add default JVM Micrometer dashboard to Grafana, taken from https://grafana.com/grafana/dashboards/4701-jvm-micrometer/ Issue-ID: CPS-2314 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I24a42b8b4771a7d9341e920fbd85fa72e1a3518e
2024-07-31Effect changes on scheduled tasks (don't want scheduled tasks to be monitored)sourabh_sourabh1-0/+13
- Applied a filter on tasks.scheduled.execution. - It is configurable from application.ymal into cps.scheduled-task-names: - tasks.scheduled.execution by default. - Added a new property configuration bean for open telemetry. - Introduced new propery attribute that takes list of task names to be filtered. Issue-ID: CPS-2250 Change-Id: I8e5334cd166343181df6dd1985471f5465f14f0e Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-07-31Add k6 for legacy async batch passthrough read operationhalil.cakal1-0/+12
- add kafka support to k6 codebase - add two new tests: data operation batch passthrough read and data operation async batch read (from kafka) - calculate the events/second via Trend - add kafka ui support to docker-compose Note: Before this commit being merged, the k6 executable should have been compiled with the kafka extension of xk6-kafka Issue-ID: CPS-2268 Change-Id: Ib7777b7bc9f15b210ea36d3541cba0e0c943f883 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-07-16Separate dmi delay for passthrough operationshalil.cakal1-1/+2
- set dmi delay to 670 ms for passthrough write (CUD) - set dmi delay to 300 ms for passthrough read Issue-ID:CPS-2285 Change-Id: I7a4294d3ed04c89aeb5988a9975516135317af12 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-07-10Remove nginx logs that not being used anywherehalil.cakal1-1/+0
Issue-ID: CPS-2038 Change-Id: Iba8950db4f3fcfc13f32bf915189f5fcb2ccee90 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-07-09Merge "Create PolicyExecutor Stub"Sourabh Sourabh1-0/+9
2024-07-08Create PolicyExecutor StubToineSiebelink1-0/+9
- Generate interface from OpenApi in RTD docs - Fixed Content Type in OpenAPI - Fixed Paths in OpenAPI - Made Authorization header compulsory in OpenAPI - All 'enum' values in OpenAPI lowercase (in linr with CPS/NCMP conventions) - Added impl with some basic functionality - Added testware - Pom includes docker image creation - Docker compose updated to deploy stub Issue-ID: CPS-2301 Change-Id: I462ad5c70474b2813fc04005c0d20a1b15b574ec Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-07-01Setup automated tests with two replicas of cps-and-ncmphalil.cakal3-5/+69
- this change for two test suits: CSIT and K6 - increase cps-and-ncmp replicas to 2 - introduce nginx as load-balancer - set proxies inside nginx for the cps-and-ncmp URI paths - upgrade hazelcast-spring to 5.3.7 Issue-ID: CPS-2038 Change-Id: If1b6c1e1218af16b24cc061507ea9b25d277d143 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-06-26Disable the java debug port permanentlyhalil.cakal1-2/+2
Issue-ID: CPS-1864 Change-Id: I0366b126920cf0bc706545bc5b4e469d3faa2486 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-06-14Update ONAP DMI version in NCMPmpriyank1-1/+1
- update the docker compose file to point to latest version on onap-dmi-plugin. Same to be used to run the CSITs during build. Issue-ID: CPS-2271 Change-Id: Ifbb2ee9c9548d54979a04595e19062d9192bfd1d Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2024-03-19Set Postgres shared_buffers in docker-composedanielhanrahan2-0/+3
By default, Postgres uses value of 128MB for shared_buffers, regardless of available memory. It is recommended to use between 25% and 40% of memory for shared_buffers in Postgres. Current value can be checked in SQL using 'SHOW shared_buffers'. Issue-ID: CPS-2156 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ib20502f159f005ba941f39758555551f447d2b67
2024-02-23Use default values for timers in docker-composedanielhanrahan1-3/+0
docker-compose.yml is overriding NCMP timers, e.g. module sync timer is overridden from default of 5 seconds to 30 seconds, which negatively affects performance testing. Issue-ID: CPS-2119 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic664169ddeed9cce59416dde50ec919cfcea522a
2024-02-12Add a configurable delay to the DMI stubJvD_Ericsson1-0/+3
Added realistic default delay to application.yaml Added realistic default delay to docker-compose.yml Issue-ID: CPS-2053 Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech> Change-Id: I756774e9538926aa341f1338fcb6882951ce3e69
2024-02-01Set resource limits of docker-compose containersdanielhanrahan1-1/+17
Without explicit limits, docker will use all available system resources for containers. In order to better detect/reproduce performance issues, limits are set to reflect those used in our production deployment. Issue-ID: CPS-2048 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I0549b7f9e57dff10c8b09d839ffb4979d66c643e
2024-01-08Clean up cps temporal related notification codeJvD_Ericsson1-1/+0
- remove 'test' level folder - remove schemas but leave structure - remove 'data-updated' from test application.yaml - remove NOTIFICATION_DATASPACE_FILTER_PATTERNS from docker-compose Issue-ID: CPS-2005 Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech> Change-Id: I3962d795c760de6ba5ff353e85894895437ad5e7
2023-12-18Remove Notification code for updated eventsmpriyank2-4/+0
- removed existing code for sending notifications to topic cps.dataupdated events formerly used by cps-temporal - corresponding testware removed or updated - unnecessary to fetch anchor details for delete and replace operation removed which might gain minor performance boost - yaml configurations , documentation removal and update - Added missing test for AsyncConfig to comply with coverage check Issue-ID: CPS-2005 Change-Id: I1848f7f229cb713fe8c0302ea50328e7451652ee Signed-off-by: mpriyank <priyank.maheshwari@est.tech>