aboutsummaryrefslogtreecommitdiffstats
path: root/k6-tests
AgeCommit message (Collapse)AuthorFilesLines
4 daysSet a permanent directory for the container logshalil.cakal1-1/+1
- Currently the container logs (cps-and-ncmp) being stored in the cps repository (in k6-tests/logs directory). In every build, cps code base being fetched freshly. Thus, the zipped container logs being lost. - Now, by using the WORKSPACE variable set by Jenkins job, the logs will be stored under the root directory of WORKSPACE permanently. Issue-ID: CPS-2676 Change-Id: I0f7f9a46b94fc3e438de39bbab04f9a59daf446b Signed-off-by: halil.cakal <halil.cakal@est.tech>
4 daysMerge "Refactor CM Handle Properties to be more realistic into K6 tests"Toine Siebelink3-15/+26
4 daysMerge "Zip and store logs for containers"Toine Siebelink2-5/+46
5 daysZip and store logs for containershalil.cakal2-5/+46
- keep console logs printed for k6 results - zip and store container logs into jenkins' workspace so that it will be downloadable whenever needed Issue-ID: CPS-2676 Change-Id: If2864c0c372d0b7319bba06a3732b49b53c371a4 Signed-off-by: halil.cakal <halil.cakal@est.tech>
8 daysRefactor CM Handle Properties to be more realistic into K6 testssourabh_sourabh3-15/+26
Issue-ID: CPS-2637 Change-Id: I627cb8be0738e6b0f82dace361eb02889f193ace Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
8 daysOptimize registration parametershalil.cakal1-1/+1
- change internal batch size from 100 to 300 - change input batch size from 100 to 2000 in k6 tests Issue-ID: CPS-2686 Change-Id: If9645facaee6c0d745a1e9dd12962a27e110f787 Signed-off-by: halil.cakal <halil.cakal@est.tech>
9 daysRandomize cm handle ids in a legacy batch arrayleventecsanyi2-4/+17
- added new JS method to get a random list of CmHandle IDs Issue-ID: CPS-2687 Change-Id: I3b02d4b7c03af3a05860e30b32877475b7e70dba Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2025-02-28Add CM Data Notification background Load to K6sourabh_sourabh3-28/+27
- Implement Kafka message producer with controlled throughput - Configured message rate to 1000 (default: 500). - Implemented constant-arrival-rate scenario to maintain controlled throughput. - Adjusted maxVUs from 1000 to prevent resource overload. - Added graceful shutdown with a 10s stop period. - Added it into main K6 suite Issue-ID:CPS-2587 Change-Id: I75fd1fa83892fd4775b27132059f73e9c0ea4d75 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2025-02-17Merge "Add actuator port based on testProfile provided"Daniel Hanrahan1-1/+7
2025-02-17Enable the container logs and disable the container restart forhalil.cakal1-1/+7
cps-and-ncmp Issue-ID: CPS-2643 Change-Id: I57ba18dcaa71f4c2c5d985e9c870569e40d26880 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2025-02-17Add actuator port based on testProfile providedmpriyank1-1/+7
- kpi uses 8883 port for healthcheck and endurance profile uses 8884 port. Issue-ID: CPS-2642 Change-Id: I0ee63f14e30141d894eb0e8b9a646a3e4112dac8 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2025-02-13Add healthchecks for docker-compose for k6 and CSITdanielhanrahan1-18/+8
To improve reliability and consistency of healthchecks in k6 and CSIT tests, they are implemented in docker-compose. This commit adds fail-fast logic, where k6 and CSITs will abort immediately if the containers won't start. Implementation: - Add healthchecks for docker containers used in tests. - Change k6 & CSIT tests to use docker healthchecks. - Tests will abort if containers are not healthy. - Start-up timeout for CPS containers is 90 seconds - Start-up timeout for other containers is 60 seconds Other Improvements: - Add --quiet-pull option to suppress junk output in Jenkins logs. - Add kpi.env file containing environment variables for KPI pipeline, just like endurance.env. This allows same code to run either suite. - Changed from port range to port number for Postgres exporter, since only a single instance runs, to be consistent with other containers Issue-ID: CPS-2630 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I50929ca4061bb844fca87a0d6c3103aaa2c45e0b
2025-01-31Realistic Examples: K6 Test-waresourabh_sourabh1-3/+3
- resourceIdentifiers must be url encoded fdns from ManagedElement. e.g.: ManagedElement=NRNode1/GNBDUFunction=1/NRCellDU=cell1 - resourceIdentifer refers to the object where the search starts from. This request will search for NRCellDUs and return their cellLocalId. The resourceIdentifier should be: ManagedElement=NRNode1/GNBDUFunction=1 Issue-ID: CPS-2482 Change-Id: Ic6ac360c71fcfc80f27367b626ad85d40a58a292 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2025-01-24[k6] 50000 CM-handlesdanielhanrahan1-2/+2
- Increase CM-handles to 50000 in k6 tests - Increase container memory to 3GB (crashes with 2GB) - Decrease max heap percentage to 70% to ensure container has at least 200MB free memory, to prevent JVM from crashing at higher peak loads. Issue-ID: CPS-2584 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8daabff58723114bdc3838b379e83c3ffbbb0ba9
2025-01-17Merge "[k6] Adjust thresholds"Sourabh Sourabh1-3/+3
2025-01-17[k6] Adjust thresholdsdanielhanrahan1-3/+3
Some cases improved performance after introduction of one schemaset per module set tag. Issue-ID: CPS-2540 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ib544d425758016df2fb3eeb1b0d0d2529b80c698
2025-01-17Realistic Examples: K6 Test-waresourabh_sourabh2-9/+14
- Updated resourceIdentifier query parameter to align with GNBDUFunctionConfig structure. - Introduced hierarchical notation (parent=..., child=...) for precise resource identification. - Added examples for querying all GNBDUFunctions, specific gNB IDs, and sub-resources like RIM-RS reporting configurations. Issue-ID: CPS-2482 Change-Id: I9e10db9bed7a3503f617a9f8dbd9fed8787402dc Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2025-01-13Update the cps's threshold for de-registration apihalil.cakal1-1/+1
- Due to the performance improvment of de-registration flow updating the cps' limit to 120 cmhandles/second Issue-ID: CPS-2192 Change-Id: Ia0224a585deba9181de89c6bb4600e8d5529df14 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-12-10Extend the container-up time after the tests finishedhalil.cakal4-4/+9
- extend it 5 mins for kpi test suit - extend it 7 mins for endurance test suit Issue-ID: CPS-2535 Change-Id: Ifecf2cdea67c12ff671da5f90e6715de7089e755 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-12-05[k6] Use constant arrival ratedanielhanrahan2-70/+116
Currently, k6 KPI tests send requests using 'constant-vus' executor. This continuously sends requests on different threads - as soon as one request completes, another is sent. This leads to weird results where improving performance of one operation ends up degrading other test cases, due to more requests being sent per second. This change makes KPI tests send requests at a constant rate, making the mixed-load tests less prone to such anomalies. - Send requests at constant rates for test cases 3, 4, 5, 6. - Throughput tests 1, 2 & 7 still send at the fastest rate. - Adjust expected performance timings. Issue-ID: CPS-2519 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I641ead7d2d9d3b29116e1cf0d4bf8cd345b01a98
2024-12-04Add persistent storage to prometheushalil.cakal1-1/+1
- add healthcheck to ensure prometheus service is running correctly and restarts it if it becomes unhealthy - set the data retention time to 30d - update readme Issue-ID: CPS-2466 Change-Id: I0f6186f8577e8e0113831ed040298b823e33d624 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-12-02[k6] Adjust expectationsdanielhanrahan1-12/+12
Some performance improvements were made over last months, so this commit adjusts expected performance to match. Issue-ID: CPS-1975 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iff6a303e8424a9d8d8b1effe27724844eadbf7fd
2024-11-22Define new Suite (js): Endurancehalil.cakal6-139/+218
- have one ncmp-test-runner.js with different configs: kpi.json and endurance.json - move scenarios and thresholds settings into the json configs Issue-ID: CPS-2444 Change-Id: I72f65036dc3f7762284b32903e1d05474dbed6c2 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-11-18Add new k6 test profile for running edurance testshalil.cakal9-38/+86
- assign a profile name to the existing k6 tests (kpi) - add control which profile is run using k6 config file - both test profiles can run in parallel Issue-ID: CPS-2464 Change-Id: I9fea13f12e2da46bd55b4315c68209843c1abe06 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-11-15[k6] Lower load for legacy async batch operation testdanielhanrahan2-3/+2
Legacy async batch operation test is failing and reporting zero as result due to timeouts in the test. - Lower total batch requests to 100 instead of 1000 - Limit the number of kafka messages per batch to 200 instead of 1000 to avoid timeout issues. Issue-ID: CPS-2274 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I06c04173ca1ed0f3580df6a73b5d647ab7ed0c71
2024-11-13[k6] Output git build info in k6 logdanielhanrahan1-1/+6
- output build info in k6 log using actuator endpoint - use quiet output while pulling images for less log noise Issue-ID: CPS-2445 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I4464b70650828c842c94677dddcc4d112f9758e0
2024-11-04Cm Avc Event to have same keympriyank1-1/+7
- incoming Cm Avc Event from DMI Plugin is consumed and forwarded to target topic - the key from source topic to be used in the target topic while forwarding - with same key the ordering of the message will be preserved - NOTE: the RTD related changes will be a separate patchset Issue-ID: CPS-2436 Change-Id: Ie692663706b378022ec0d621d92ca5054bad8d1b Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2024-10-29[k6] Increase timeouts after DMI stub changesdanielhanrahan1-2/+2
This commit changes k6 setup and teardown timeouts to 20 minutes. This fixes current test failures due to time-outs. Issue-ID: CPS-2410 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ifdbf26ed1669cdb3a2fc9afb551308757511281b
2024-10-25[k6] Update tests for DMI stub changes for modulesdanielhanrahan3-4/+4
It is needed to update k6 tests due to changes made in the DMI stub for dynamic module generation: - Increase setup timeout to 30 minutes - Use 5 module set tags: tagA to tagE - Update module search to look for module100 Issue-ID: CPS-2410 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I17830e4d858883743476d297af90d1ceaba1eb44
2024-10-23Resize cps expectationhalil.cakal1-1/+1
- set expectation to 70 (cmhanldes/second) for discovery case Issue-ID: CPS-2350 Change-Id: I3d0f9f5e20de87f79ebbe569c7905f4eefed35e8 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-10-17Refactor and Optimize ModuleSyncWatchdog for Improved Lock Handling and ↵sourabh_sourabh1-1/+1
Queue Management - Enhanced logging for populateWorkQueueIfNeeded and resetPreviouslyLockedCmHandles methods, improving clarity and error handling. - Improved readability and maintainability of the locking mechanism with Hazelcast's FencedLock. - Optimized error handling in catch blocks, logging detailed exception messages and stack traces for better troubleshooting. - Refined lock acquisition and release flow, with clear log messages for both successful and failed lock operations, ensuring safe handling of Hazelcast distributed locks. Issue-ID: CPS-2403 Change-Id: Ie089f36a817d4965782235b51ee987ef054516b1 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-10-09Merge "Added K6 script test to check a throughput of CPS kafka consumer"Daniel Hanrahan2-0/+137
2024-10-03Added K6 script test to check a throughput of CPS kafka consumersourabh_sourabh2-0/+137
- Added new K6 script to produce cm avc enents. (It is not a part regular automated tests) - Added a json resource (avc event) Issue-ID: CPS-2329 Change-Id: I6446bc120382257c5039df17fd34b84b7c6f6550 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-10-03Re-adjust cps expectationshalil.cakal1-13/+13
- aligning cps expectation for new plots Issue-ID: CPS-2350 Change-Id: I5b300c4f5432bf2bdfe8a8b0d52b2ce60bd338d6 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-09-24[k6] CM handles searches using different filtersdanielhanrahan4-30/+181
Each 5 VUs for searches uses a different filter: no filter, module, property, cps path, and trust level. This is needed since each search may have different performance. This also helps reduce load versus doing a combined module and property search all the time. Issue-ID: CPS-2349 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I33a75260f97697d2061d10b80ba7eeb2f6c346f5
2024-09-18Change execution order of the legacy async batch read operation in k6 suitehalil.cakal1-2/+3
- run the test after the parallel cases - add 30 seconds safe-period of time waiting for completion of active threads (remained from previous test cases) Issue-ID: CPS-2414 Change-Id: I1260b37cd4f1974eebcccf867b36172b407a9646 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-09-13Add CPS limit to the plotshalil.cakal1-16/+16
- as of now the cps limits given manually according to the latest result - there will be another commit will change the plot job in cicd repo to reflect the values on the plots Issue-ID: CPS-2350 Change-Id: Ic371f46894f1825dca1d26ec65ae0f62aa06c327 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-09-12Calculate throughput for legacy batch read data operationshalil.cakal3-53/+66
- stress the cps-and-ncmp module for batch read data operation - for this aim use shared-ireations during async http request - as parallel to this, consume expected number of messages as fast as possible Issue-ID: CPS-2268 Change-Id: I1b6724479dac7391cbb6407fda52d15176aa8985 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-09-10[k6] Fix incorrect results when errors occur in k6 tests (case 2)egernug1-1/+5
De-registration is reported as 200,000 handles per second if all operations fail - Added a running total for successful de-registrations of CM Handles Issue-ID: CPS-2346 Change-Id: Ib4302d3b605ff1eaf4ecadac11b4e851a69210df Signed-off-by: egernug <gerard.nugent@est.tech>
2024-09-10Fix high-cardinality metrics in k6, causing high memory usesourabh_sourabh4-58/+101
- Added unique IDs as metric tags for all the endpoints. - Re-arranged order of public and prive js methods. Issue-ID: CPS-2331 Change-Id: Ib876a647fb35110c50670c7222986e8a8a6f5ca0 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-09-05[k6] Minor correction to plot titlesdanielhanrahan1-2/+2
Plot title say Module filter instead of Module and Property. Issue-ID: CPS-2349 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie23080a6085960b89132ad38d07d0c2b6f12471b
2024-09-04[k6] Align tests with updated requirementsdanielhanrahan5-76/+75
- Test load increased to 5 CM handle searches and 5 Id searches to reflect current requirements. - CM handles searches now use a combined search filter including 1 module and 1 public property, so we are testing with a heavier load than required, for early warning of trouble. - Test load reduced for passthrough operations to 4 VUs per operation to reflect addition of batch operation. - Added test of passthrough write operation with alternate ID. - Additional refactoring. Issue-ID: CPS-2349 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I88d2fe431f74821e8e0e976441efcf82aa320849
2024-09-03Merge "Fix huge logs produced when errors occur in k6"Daniel Hanrahan1-1/+3
2024-08-30[k6] Report failure rate for all tests in a single plotdanielhanrahan2-16/+2
Instead of separate plots of test failures for each test case, we have one single plot showing failures. Issue-ID: CPS-2347 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie79b75c34903a7df2037474b3536d4867c1a61e0
2024-08-30[k6] Fix k6 tests after DMI stub changesdanielhanrahan2-6/+6
A change to DMI stub has broken k6 module search due to module rename and increased registration time. - increase setupTimeout by 2 minutes to factor in DMI startup delay - module search uses module common to all module set tags - only record times for searches returning number of CM-handles Issue-ID: CPS-2353 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I9067dc73015cfb475cdce1e7a3cc2d5401449caf
2024-08-30Fix huge logs produced when errors occur in k6sourabh_sourabh1-1/+3
- Changed k6 config to ignore all the error logs. Issue-ID: CPS-2344 Change-Id: Iad7be43ae5bb909518f9f7d8e386a97c96c349e3 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-08-27[k6] Exclude timings when error occurdanielhanrahan2-28/+37
If an error occurs during k6 test, e.g. tcp connection refused, the timing of failed response is included in the average response time. This can lead to incorrect plots when errors occur, e.g. operations appear to be faster when really the operations are failing. - use custom Trend metrics for all measurements - only log results when operations return expected response codes Issue-ID: CPS-2345 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iab1db5e5ca9530f33da239187348cabcd0605648
2024-08-27Merge "Plot request failures for each k6 test"Daniel Hanrahan3-37/+52
2024-08-27Merge "[k6] Always pull latest cps and dmi stub images"Priyank Maheshwari1-1/+7
2024-08-27Install dependencies for k6 and csitdanielhanrahan2-0/+51
- install docker-compose in csit and k6 tests - install k6 with kafka extension in k6 tests Issue-ID: CPS-2376 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8a67813a96dc7f321d4abe0aa91e236fd1060f3e