summaryrefslogtreecommitdiffstats
path: root/cps-ncmp-service/src/test/groovy/org
AgeCommit message (Collapse)AuthorFilesLines
2022-09-21Robustness cleaning of in progress cachesourabh_sourabh5-22/+69
- Removed cm handle entry from in progress map while moving from locked to advised and de-registration - Fixed groovy tests Issue-ID: CPS-1239 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ic527c7b5d209e0c7788888942e95ce0c2c59f7c6
2022-09-13CmHandle registration/module sync watchdog performance improvmentsourabh_sourabh1-9/+8
- Tuned CPS DB parameters - Removed deprecated CPS DB parameter - Modified module sync watch logic - Added aditional logs - Removed logic to get FragmentEntities as stream - Removed detaching of fragment entity as it was causing NCMP to break with an exception "SQLSTATE(08006)" Issue-ID: CPS-1126 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I0bde11895f754602dece132efde701d82b377f12
2022-09-13Handle partial failure (improvements)ToineSiebelink2-5/+29
- catching of failures on retry of individual nodes - extract cm handle id from xpaths (can only report xpaths in cps core) - add test for same Issue-ID: CPS-1232 Issue-ID: CPS-1126 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ice2032c8b15fea97ae0aaa4d1ed642b3499228fa
2022-09-12Handle partial failurempriyank1-14/+10
- Removing the transaction boundaries as it was getting rollbacked on partial failures - Handled adding the elements in batch and if it fails try them individually - Refactored code a bit and when there is partial failure we try one more time in sequence and even if there are failures we collect the failures Issue-ID: CPS-1232 Issue-ID: CPS-1126 Change-Id: I7824c9f37f80cbaeedd5dc06d598ca0e3a69c59b Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-09-05Code to Interface : InventoryPersistencempriyank1-2/+2
- Extracted interface for InventoryPersistence using automatic extraction of IDE. - We call the InventoryPersistence to InventoryPersistenceImpl - Used the correct type in the test Issue-ID: CPS-1241 Change-Id: Ieceaec69c1063d7762abc7d8e389d36011a93860 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-09-05Performance Improvement: Fix Insert Yang Resource IDs (Schemset)ToineSiebelink1-7/+6
- Add robustness to handle insertion/get of empty collection - Fix business logic to pass down ALL module references during module Sync - Update Java doc etc to clarify ALL module references are needed (not just new) Issue-ID: CPS-1246 Issue-ID: CPS-1126 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ic8ff4bdfef646e98ef61a6732c6d5ecb4b762e29
2022-09-02Performance Improvement: Use save batches of cmhandlessourabh_sourabh3-34/+50
-Used cm handle batch to persist from state handler. Issue-ID: CPS-1230 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I68b7fde7dc85818b818f1af588344c26b549d87b
2022-09-02Code to Interface: CmHandleQueriesmpriyank3-5/+3
- Extracting the interface CmHandleQueries using IDE intelligence - Used intreface when mocking in the tests Issue-ID: CPS-1240 Change-Id: I5f537eaac992c68b65468e344830a04f1dda7e45 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-09-01Merge "Get all cm handles by DMI plugin Identifier"Sourabh Sourabh3-2/+34
2022-09-01Merge "Performance Improvement: Enhance state handler"Sourabh Sourabh3-3/+100
2022-09-01Performance Improvement: Enhance state handlermpriyank3-3/+100
- Introduced batch handling capability to state handler - Refactored methods in CompositeStateUtils - Renamed saveCmHandleStates to saveCmHandleStateBatch - Decoupled processing of events in bulk - Test scenarios for the new functionality Issue-ID: CPS-1231 Issue-ID: CPS-1126 Change-Id: Ifacdeb7bbed14712ecf4f5e2a4d9b324bae278d8 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-09-01Get all cm handles by DMI plugin Identifieremaclee3-2/+34
- api added to get cm handles by dmi plugin identifier - response object refactored from RestOutputCmHandle to collection of Strings (cm handle ids) - added public and private methods in CmHandleQueries to get cm handle ID by dmi plugin - added unit tests including test to show that there are no duplicates on response Issue-ID: CPS-1136 Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech> Change-Id: Ia3bdc16172a90ad3a3f9ae11cddcad1352188726
2022-09-01Performance Improvement: Watchdog Parallel execution with configurationsourabh_sourabh3-22/+100
- Introduced AsyncSyncExecutor to get task and execute it with configured number of parallel threads. - Number of parallel thread can be configured from application.yml. - AsyncTaskExecutorSpec is added - Fixed existing grovvy test now async task would be submitted. Issue-ID: CPS-1200 Change-Id: I58c0368b945c90e619c2acfc7458ba58de047484 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2022-08-29Performance Improvement:save cmhandles capabilitympriyank1-0/+16
- add saveCmHandleBatch in InventoryPersistence - add saveListElementsBatch in CpsDataService - have addListElementsBatch in CpsDataPersistenceService - Test scenarios for the same Issue-ID: CPS-1229 Issue-ID: CPS-1126 Change-Id: I0a1401818da5a4e523d7d0751cac6a526d1611b2 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-08-29Performance Improvement: Use hazelcast blocking queueToineSiebelink6-131/+216
- Introducing hazelcast for queue and progress map - process batch of 100 at the time - decreased module sync watchdog sleeptime to 5 seconds - separate module sync tasks in new class and some other async preparations and easier testing - tests for batching in module sync watchdog - remove qualifiers annotation (support) where no longer needed Issue-ID: CPS-1210 Issue-ID: CPS-1126 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I0a7d3755bf774e27c5688741bddb01f427d4a8a7
2022-08-25Performance Improvement: Batch Update DataNodeslukegleeson1-2/+17
Implemented methods to perform a batch operation on updating datanodes Refactored replace data node(s) tree methods to update data node(s) and descendants Issue-ID: CPS-1203 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I365d657422b19c9ce384110c9a23d041eaed06f4
2022-08-22Merge "Distributed datastore solution for Data Sync Watchdog"Toine Siebelink5-52/+48
2022-08-22Distributed datastore solution for Data Sync Watchdogkissand5-52/+48
- update lombok config to handle Qualifier annotation - update Semaphore config to use ConcurrentMap - update SyncUtils to return a list of cm handles - update DataSyncWatchdog and ModuleSyncWatchdog with Qualifier - update DataSyncWatchdog to handle a list of cm handles - Use get with xpath to check cm handle state Issue-ID: CPS-1015 Change-Id: Icb39bd29f89e0020d49a1f8960476ffe81b12362 Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2022-08-22Merge "Fix: CM Handles for only hasAllModules Query"Sourabh Sourabh1-1/+1
2022-08-22Fix: CM Handles for only hasAllModules Querylukegleeson1-1/+1
hasAllModules was returning no CMHandles when queried by itself, only when paired with another query condition e.g. hasAllProperties This fix means endpoint works as expected. Reverted null check Issue-ID: CPS-977 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: Idafad4c9877941d121f5cbb98617c56a35a1cc94
2022-08-18Ensure Additional Properties for CM Handles are includedToineSiebelink2-2/+13
- seems like bug is not present on master - modified test to ensure the behavior is tested now - (intelliJ) cleaned up so unused stuff Issue-ID: CPS-1212 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I6882df4277e50ef869c6a1e9b093ee83dbcdf78e
2022-08-10CmHandleState transition using state handlermpriyank4-32/+71
- Raise LCM Event when state transition happens. - Raised events for ADVISED to READY, ADVISED to LOCKED, LOCKED to ADVISED. - Refactor existing code to raise the events correctly. - Refactored existing test scenarios to comply with the code change. Issue-ID: CPS-1034 Change-Id: Ie548e644f6133304d7fa36c892ca2bec7393c074 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-08-04LcmEvent state handler refactoringmpriyank3-11/+124
- State handler will now handle new structure of LcmEvent. - We have 3 types of events i.e create, update and delete. - Introduced the LcmEventCreatorHelper to delegate some of the responsibility of event creation based on event type and the state. - New tests and existing refactoring - Code rebased - Refactored name as per group code review - Code rebase with other commits - Used copy constructor for deep copy operation of NcmpServiceCmhandle and CompositeState - UPCOMING : Related user stories to trigger the event publishing using state handler. Issue-ID: CPS-1128 Change-Id: I94b5a87d37d6a174c017ee0aa37cd0f0f74ba084 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-08-02Merge "Enable/Disable Data Sync for Cm Handle"Joseph Keenan6-34/+77
2022-08-02Merge "Incorrect response to {} for cmhandle id-searches"Joseph Keenan1-2/+0
2022-08-02Merge "Query CmHandles using CPS path"Joseph Keenan4-157/+266
2022-08-02Incorrect response to {} for cmhandle id-searcheslukegleeson1-2/+0
An empty body for the CmHandles id-searches api returns an anchor instead of all cm-handles Now returns all childnode ids for the dmi-registry Issue-ID: CPS-1175 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I358970170a4eb7905e9646605f0f373fbcd78945
2022-07-29Query CmHandles using CPS pathlukegleeson4-157/+266
Added withCpsPath condition parameter Validated to prevent misuse and blocking of querying using private properties Updated OpenAPI with examples and links to documentation Moved methods related to cmHandle querying using cps path from InventoryPersistence to CmHandleQueries Renamed private method deleteSchemaSetAndListElementByCmHandleId to deleteCmHandleByCmHandleId Issue-ID: CPS-977 Change-Id: I83827215b7e58de74f8f62cd0140516d217d93f1 Signed-off-by: lukegleeson <luke.gleeson@est.tech>
2022-07-28Enable/Disable Data Sync for Cm HandleDylanB95EST6-34/+77
-Create API Which will enable/disable data sync enabled flag -Default functionality of module sync watchdog is to set to false -Remove global config param -Will set initial sync state based on data sync enabled flag -Throws an Exception if the same data sync enabled flag tries to be set -Throws Exception if state is not in READY -Data Sync enabled must be true to complete data sync process - Delete all resource data within fragment table related to synced cm handle when data sync is set to false Issue-ID: CPS-1133 Change-Id: Ib47bbd8293f083c1d705d91bd0def74e6a105c72 Signed-off-by: DylanB95EST <dylan.byrne@est.tech>
2022-07-27Distributed datastore solution for Module Sync Watchdogkissand1-1/+6
- use semaphore map in ModuleSyncWatchdog - increase test timeout, because it needs more time for hazelcast initialization Issue-ID: CPS-1015 Change-Id: I71feed8fbbd047af9fabba29a5f762a1f17a1c78 Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2022-07-25CmHandle creation performance degradationsourabh_sourabh3-11/+70
- Created a dedicated threadpool for scheduler. - Tuned async threadpool of notification executor from setting RejectedExecutionHandler and application.yml. Issue-ID: CPS-1126 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I2afe3c76c1aec78751777df0d2f08ddb8dcee102
2022-07-25De-registration: send event(s) using central statesourabh_sourabh3-19/+64
- refactored method "parseAndRemoveCmHandlesInDmiRegistration" - added method to send events for DELETING and DELETED - introduced private method in state handler - updated unit tests for registration spec - added unit test to cover new private method in state handler Issue-ID: CPS-1003 Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech> Change-Id: Ifc1d1e94b7103d85b6352b196edf7075257c211a Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2022-07-20Refractor code to use the new LcmEvent schemabmiklos3-36/+29
-Old schema deleted and comments from tests are removed -EventDateTimeFormatter utility added -From LcmEventsCreator.populateLcmEvent the NcmpServiceCmHandle param removed and tests are fixed accordingly -Changes made in test of the LcmEventsCreator for payload testing to make it more clear Issue-ID: CPS-1147 Change-Id: Id87c81f2755a6617a0fc68ba92a3d0e3170fd20f Signed-off-by: bmiklos <miklos.baranyak@est.tech>
2022-07-14Code Refactoring Ncmp* to Lcm* as per new scopempriyank7-46/+58
- Refactored code to reflect the LcmEvents scope now. - Test cases updated and dependencies reviewed. - You can find few occurences of NcmpEvent for now but that will be cleaned up once we implement CPS-1104 which is for LcmEvent schema - UPCOMING: LcmEvent schema and then the code to publish the events on cmHandleStates - LATER : LcmEvents based on public properties update. Issue-ID: CPS-1137 Change-Id: I9f395ed733b4028e706205894c36a38412e3452e Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-07-08CmHandleState Handlermpriyank2-27/+133
- State handler taking care of ADVISED, READY and LOCKED state transition at the moment. - For now I have not removed the actual code but eventually state handler will take care of persisting the state and publishing of the events - Rebased code to add the global parameter related to dataSyncCache - UPCOMING : The classes will be prefixed/renamed with LCM once we have the LcmEvent created with new schema Issue-ID: CPS-1118 Change-Id: Ic45d95169eb0c06cfb35c907d34380dbcbf2da11 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-07-07Merge "Define Initial Data Sync Enabled Flag and state"Joseph Keenan1-3/+11
2022-07-07Define Initial Data Sync Enabled Flag and stateDylanB95EST1-3/+11
- Define the initial Data Sync Cache enabled through configuration parameter - Set the data sync enabled flag based on this - And in turn define the initial sync state of the Data Sync Issue-ID: CPS-1119 Change-Id: I43bf03c79481291bf47c9b672f7bf408d789df61 Signed-off-by: DylanB95EST <dylan.byrne@est.tech>
2022-07-07Merge "Improve test coverage on CompositeStateBuilder"Toine Siebelink1-0/+22
2022-07-07Improve test coverage on CompositeStateBuilderbmiklos1-0/+22
Issue-ID: CPS-475 Change-Id: Id24f2b0c6fe95ca9c7559a8dbd160c4566ddc32f Signed-off-by: bmiklos <miklos.baranyak@est.tech>
2022-07-06Merge "Move persistence methods from NCMPproperty handler"Joseph Keenan3-30/+51
2022-07-06Merge "Distributed map setup for Module and Data Sync"Toine Siebelink1-0/+49
2022-07-06Distributed map setup for Module and Data Syncmpriyank1-0/+49
- Introduce the hazelcast dependency and check how the builds react to it. - Data Sync and Module Sync Map config. - Added basic test scenarios. - Updating the artifact-id from hazelcast-all to hazelcast-spring and version is 4.2.5. (5.1.2 is latest but has known vulnerabilities) - Refactored the code as per the onsite review session - Upcoming : Actual sub-task which will use the distributed maps in respective use cases. Issue-ID: CPS-1050 Change-Id: I60314c4ff95c267b186b20862aef93a5c389a971 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2022-07-05Move persistence methods from NCMPproperty handleremaclee3-30/+51
Issue-ID: CPS-1117 Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech> Change-Id: If6a0c620970a2a34a601267c6610f85e4bc07f60
2022-07-04Move persistence related methodsemaclee4-61/+129
-created new method in inventory persistence service for getting module references Issue-ID: CPS-1117 Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech> Change-Id: Ic1f3c180a62a7211e19982d8f3570829db58370e
2022-07-04Simplified 'External' lock reason Mappinglukegleeson6-25/+25
Refactored LOCKED_MISBEHAVING -> LOCKED_MODULE_SYNC_FAILED CompositeStateMapper will change internal reason LOCKED_MODULE_SYNC_FAILED to external reason LOCKED_MISBEHAVING for client payloads Changed openapi description of lock-reason to reflect only enum currently available LOCKED_MISBEHAVING Issue-ID: CPS-1099 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I9cda45f6c30b94684ee1c8ad0c49e35a3a824d52
2022-07-01Merge "use separated get methods for every cmHandle instead of one "get all" ↵Toine Siebelink1-1/+3
query"
2022-07-01use separated get methods for every cmHandle instead of one "get all"kissand1-1/+3
query Issue-ID: CPS-1085 Change-Id: Ifac152f606b25b7a7d143315dba6d73698ef914e Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2022-07-01Merge "Allow Module Re-Sync"Toine Siebelink2-2/+46
2022-06-30Allow Module Re-Synclukegleeson2-2/+46
Allows the Resync of an already synced CmHandle Currently: Advised -(create schemaset)-> Ready -(manual write)-> Advised -> Locked as schemaset for cmhandle already exists With this: Advised -(create schemaset)-> Ready -(manual write)-> Advised -(delete schemaset, create schemaset)-> Ready Included some logging Renamed ModuleSyncSpec -> ModuleSyncWatchdogSpec to match class Issue-ID: CPS-1045 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I408fbea698b7926dbf5d0cddc74acf1b00235b1f
2022-06-30Add method to get YANG module sources for CM handleemaclee8-16/+35
- part of this commit includes renaming the enum SyncState to DataStoreSyncState Issue-ID: CPS-1064 Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech> Change-Id: I6bf419141a1b33f09871946445cdfff422c8c354