summaryrefslogtreecommitdiffstats
path: root/cps-service/src
AgeCommit message (Collapse)AuthorFilesLines
2024-04-26Faster module searches (CPS-2190 #3)danielhanrahan3-6/+4
This greatly improves performance of module searches by eliminating unneeded SQL queries via Hibernate lazy fetching. Issue-ID: CPS-2190 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie9e65017d0027366456f1741cc37b10679317b25
2024-03-19Configure Hazelcast to have 1 backup to reduce memorydanielhanrahan2-8/+5
Currently Hazelcast is configured to have 3 synchronous backups and 3 asynchronous backups. These are separate, meaning there will be 7 copies in the cluster (1 original + 3 sync + 3 async backups). Even if only 1 instance of CPS/NCMP is running, it will have 7 copies in memory. Given that CPS/NCMP is typically only deployed using 1 or 2 instances, the settings are changed to 1 synchronous backup (same as Hazelcast's default configuration). This change has been tested and shown to reduce heap usage by around 100MB during 20K CM-handle registration. Issue-ID: CPS-2146 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I4a5f6d83bc35e2c13cfb32002e38dc365da34c8e
2024-02-12Fix test failure by ordering leaf-listsdanielhanrahan1-4/+9
YANG specifies two ways that leaf-lists can be ordered: - ordered-by user: original order in JSON is preserved - ordered-by system (default): it is up to the system how to order For leaf-lists to preserve same order as the JSON, the Yang module must specify 'ordered-by user'. To ensure consistent behaviour even when system ordering is used, the leaf-list is sorted during parsing. - Add 'ordered-by user' to authors field in bookstore.yang - Sort leaf-list during parsing when using 'ordered-by system' - Add new tests to verify ordering Issue-ID: CPS-2057 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I6ab688ec2fa4a22182e853d1a8b26642f278c40a
2024-02-07Orphaned modules are removed from the DB before cmhandle removalsourabh_sourabh1-1/+1
- changed sequence of delete schema set by cascade to remove unused yang resource modules at last. Issue-ID: CPS-2031 Change-Id: Icefc075165231cc4b9b170e04a2d12af645529e9 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-02-01Add integration test for extending API: Get Module Definitionshalil.cakal1-1/+5
- Add new module: bookstore-types - Import the new module from stores module - Change integration testwares to comply with the new module - Add new integration test for retrieving module definition Issue-ID: CPS-1135 Change-Id: Ib51c998ceeab50d3fcea212ed55c8ab50ca09257 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-02-01Extend API: Get Module Definitionshalil.cakal4-7/+50
- add query parameters: module-name and revision to OpenAPI - extend the controller method to hande the new parameters - add the new method stack to the service layer - extend the SQL query to support the new parameters - add unit and integration testwares Issue-ID: CPS-1135 Change-Id: I089ad2ad71effb58ac0ba809e9f441d6cdb59c4f Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-01-25CloudEvents support for cps-corempriyank2-0/+330
- Moving cloud events dependencies to the cps-service module instead of cps-ncmp-service - Testware has moved and new tests also introduced - EventsPublisher also moved Issue-ID: CPS-2040 Change-Id: Ibafb15e7e9efbdbe1e00e2b4f0da820bbcead004 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2024-01-19CPS 1824: Delta Between 2 Anchors release notesArpit Singh2-18/+17
Updated release notes for Delta Feature Issue-ID: CPS-1824 Signed-off-by: Arpit Singh <as00745003@techmahindra.com> Change-Id: I1c2403391e85aec6a9e34f0ff616c17b100bf6a9
2024-01-18Merge "CPS Delta API: Update action for delta service"Toine Siebelink4-42/+185
2024-01-17Clear instance based Schema Context Cache upon validation errorsToineSiebelink14-505/+628
- retry yang parser exceptions one time by clearing cache - split yangUtils into YangParserHelper (instance based) and YangUtils for non-parsering static methods - removed public methods only used from test. Refactored to use variation with (empty) parent path - removed use of optional for parentPath, easier to handle with just an empty string! - make methods no longer used in production code in YangUtils are private - udpate testware to use proper public methods instead of private methods of yang utils / parser (helper) Issue-ID:CPS-2000 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I0c7590a5e1495d047006e7136f1bd873be37f7b0
2024-01-16CPS Delta API: Update action for delta serviceArpit Singh4-42/+185
- Added code for Update action in Delta service - added method to get updated Leaf data: getUpdatedLeavesBetweenSourceAndTargetDataNode - added method to compare Leaf data common in source and target data node - added method to process leaves unique to target data node - added method to compare leaves: compareLeaves - added method to store updated data to a DeltaReport: addUpdatedLeavesToDeltaReport - Added corresponding testware Issue-ID: CPS-1824 Signed-off-by: Arpit Singh <as00745003@techmahindra.com> Change-Id: I3de07ea3227988784a0892f6a92c238ecf00a7fa
2024-01-09Remove inefficient saveListElementsBatch APIdanielhanrahan4-70/+7
CpsDataService::saveListElementsBatch method is not needed as saveListElements supports saving multiple list elements in a single operation. This both simplifies implementation and greatly improves performance when saving list elements, as the Yang parser need only run once for the whole batch. - Change InventoryPersistence to save CM-handles in batches of 100 using existing CpsDataService::saveListElements method. - Remove not needed CpsDataService::saveListElementBatch. Issue-ID: CPS-2019 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8b74dda2917e094d064b42f2c0e4d57029b90395
2024-01-08Clean up cps temporal related notification codeJvD_Ericsson1-7/+0
- remove 'test' level folder - remove schemas but leave structure - remove 'data-updated' from test application.yaml - remove NOTIFICATION_DATASPACE_FILTER_PATTERNS from docker-compose Issue-ID: CPS-2005 Signed-off-by: JvD_Ericsson <jeff.van.dam@est.tech> Change-Id: I3962d795c760de6ba5ff353e85894895437ad5e7
2023-12-18Remove the dependency-cycle between beansToineSiebelink13-182/+273
- Splitting admin service into AnchorService and DataspaceService (this resolves the cyclic dependency) - Improved Delete dataspace integration error tests (were depending on execution order, now independent) Issue-ID: CPS-871 Change-Id: I47efedb6eb4bd2900f72d689616b7b7b62df2938 Signed-off-by: halil.cakal <halil.cakal@est.tech> Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2023-12-18Remove Notification code for updated eventsmpriyank17-1149/+51
- removed existing code for sending notifications to topic cps.dataupdated events formerly used by cps-temporal - corresponding testware removed or updated - unnecessary to fetch anchor details for delete and replace operation removed which might gain minor performance boost - yaml configurations , documentation removal and update - Added missing test for AsyncConfig to comply with coverage check Issue-ID: CPS-2005 Change-Id: I1848f7f229cb713fe8c0302ea50328e7451652ee Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-12-13Introduce Schema Set upgrade methodsourabh_sourabh4-16/+65
- Incl. integration test - Prod code is changed to use new method that update yand module. - updated module resource to return multiple modules. Issue-ID: CPS-1806 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I751d9393ce78a3be9daeaff6d0252738c02115e0 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-11-27Expose hazelcast cluster infompriyank2-0/+22
- exposing hazelcast cluster and health related rest endpoints to be used in the testware - added test to support it - also added RTD docs Issue-ID: CPS-1980 Change-Id: I926013bee05603a43050b861f677885a2511fffc Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-11-14CPS Delta API 1: Delta between 2 anchorsArpit Singh10-4/+461
- CPS Delta Feature Part 1: To find delta between two anchors - created new endpoint deltaByDataspaceAndAnchors - endpoint to take dataspaceName, source anchor, target anchor, xpath, descendants as input - added new service CpsDeltaService - added method to find delta between DataNodes: getDeltaReport - added method to find removed data nodes: getRemovedDeltaReports - added method to get Added DataNodes: getAddedDeltaReports - added method to get Map of xpath to DataNode: convertToXPathToDataNodesMap - added a POJO for delta report - Added new JSON data for delta feature testing - Added groovy test files CpsDeltaServiceImplSpec and DeltaReportBuilderSpec - code related to update operation, will be added in separate commit Issue-ID: CPS-1824 Signed-off-by: Arpit Singh <as00745003@techmahindra.com> Change-Id: I313f0f71d04b03878be7643f709d8af1aa6df6ba
2023-11-03[BUG] Set Hazelcast class loaderdanielhanrahan1-0/+1
ClassNotFoundException is being thrown during CM-handle registration when running the cps-application JAR. This fixes the issue by using the class loader for the CPS classes. Issue-ID: CPS-1933 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I02e2d19d9f668f0c4af36e9061e68f9b9ddb9434
2023-10-23Introduce and use new Hazelcast map pt. 2sourabh_sourabh1-1/+0
- modified ModuleSyncService - created ModuleSetTagCacheConfig and tests - ensure both Create and Upgrade cm Handle use cases work - Moved inventory pkg to its right patha as production code. Issue-ID: CPS-1859 Signed-off-by: leventecsanyi <levente.csanyi@est.tech> Change-Id: I173dcd81fe2e74d86d85365b2ead461302cad974 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-10-11Expose REST endpoint to update YANG schema set using moduleSetTagsourabh_sourabh3-4/+4
- Added new schema to upgrade model into component. - Modified Ncmp rest input mapper to add upgradedCmHandles attributes. - Modified cm handle state mapper to add new lock reason. - Added new method to parse and upgrade Cm handles in DmiRegistration. - YANG data converter is modified to add "module-set-tag" atribute. - Cm handle new query method is added for cps path without appending ancestor. - Modified setCompositeStateForRetry to add lock reason. - New lock reason category is added. - Existing module sync service is modified to upgrade the model "syncAndCreateOrUpgradeSchemaSetAndAnchor". - Sync util method "getModuleSyncFailedCmHandles" to modified to add another lock reason "LOCKED_MISBEHAVING". - Added new attribute "UpgradedCmHandles" into DmiPluginRegistration and DmiPluginRegistrationResponse. - New attribute "moduleSetTag" is added into NcmpServiceCmHandle. - New model "UpgradedCmHandles" is added. - New method "updateSchemaSetWithExistingModules" is added into cps module service to update cm handle with exsting model. - Code coverage is reducced to 96 percentage that would be addressed and pushed into new patch. Issue-ID: CPS-1798 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I540acb404e38fc434de87a0d959bfde710a18b03 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-09-28JSON list support when updating multiple datanodesdanielhanrahan2-27/+15
updateDataNodesAndDescendants if supplied with a JSON list such as {"branch": [{"name":"Name1"}, {"name":"Name2"}]} would only replace the first node /test-tree/branch[@name='Name1'], and ignore any remaining list items. This is caused by the use of a legacy buildDataNode, which returns only a single DataNode from JSON, even if the JSON contained a list. Issue-ID: CPS-1889 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I257491b6bc3f047a64eb241eaac70fd457b24347
2023-09-22Reinstate Spring Boot 3.0 after revertegernug4-4/+4
Issue-ID: CPS-1789 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: Iee5c514ccdba36a387f83948d8a3ba26e6b1de5b
2023-09-20Revert "Migrate CPS to Spring-boot 3.0"Toine Siebelink4-4/+4
This reverts commit 9693ec51cf6526082f0ad0c3ad208d144cbbb163. Reason for revert: Bug fix delivery Change-Id: I73bdc1528192c662983b0bbef73b10b6d612a719 Signed-off-by: egernug <gerard.nugent@est.tech> Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2023-09-20Merge "Use YangParserFactory to generate schema context"Toine Siebelink1-9/+13
2023-09-20Use YangParserFactory to generate schema contextdanielhanrahan1-9/+13
This is presented as a fix for ClassDefNotFoundErrors from opendaylight yang parser Issue-ID: CPS-1866 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: If2d1a8f0319cad0a9517ea3652446e2e43f51fd4
2023-09-14Migrate CPS to Spring-boot 3.0egernug4-4/+4
Issue-ID: CPS-1789 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: I932fca8a50cedf2ae0604401c5e2b27896e4d449
2023-08-18Merge "Device heartbeat listener"Luke Gleeson2-3/+22
2023-08-18Device heartbeat listenermpriyank2-3/+22
- Infrastructure code to have the kafka listener and distributed set in place - performance tested locally - testware added Issue-ID: CPS-1642 Change-Id: I775dbe6e6b520b8777faa08610db439877757572 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-08-18CPS-Core : Expose a java interface to update schema setsourabh_sourabh4-0/+32
- Exposed an interface to update anchor by schema set name. - New interface is implemented into RI model. - New native query is exposed to update id with given schema set name. - A new integration test is written to test new interface into cps core. Issue-ID: CPS-1800 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ibf44712e11b53cb6673b04b9e3fd864321c90839 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-08-03Merge "Support pagination in query across all anchors(ep4)"Luke Gleeson9-23/+166
2023-08-02Support pagination in query across all anchors(ep4)rajesh.kumar9-23/+166
Add pagination query parameters in query across all anchors API pagination parameters (pageIndex and pageSize) are optional default is to query all fragments each pageSize represents number of records(number of anchors) TotalRecords is returned in response header to find number of pages. - If pagination option is provided in request then query number of anchors equal to pageSize. pageIndex is used for setting offset. - return number of records(one anchor per record) as per pagesize and pageSize Issue-ID: CPS-1605 Change-ID: I73f97f986a817d423f93a8d922dcd9647b2504bc Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2023-08-01Merge "Fix: Make bookstore data consistent"Toine Siebelink3-26/+16
2023-07-31Add 'direct' keyword to descendants option to query direct children (ep1)Rudrangi Anupriya3-5/+5
-added 'direct' keyword also to Fetch Descendants Option along with 'all' and 'none' to Query direct child. -added unit tests to test direct keyword Issue-ID: CPS-1784 Change-Id: Icb7f59fbeebb03703626132d6d5d2cfde0e5ab4d Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-07-31Increase code coverage in cps-service moduleToineSiebelink21-281/+421
- After last rebase I had to remove 3 unused recent cloud eventd specific exceptions/constructors - Moved the only used new exception from SPI to the relevant util package (please NOTE not all exceptions belong in SPI and always question need for new exception when there is no specific handling, try to use standard or existign CPS exception instead!) - Increased cps-service module (line) coverage from 95 to 100% - Added tests for missing exceptions (handling i.e. thrown up) - Removed incorrect SPI defined OperationNotYetSupportedException (replaced with standard java exception instead) - Fixed some legacy issues with existign test classes I modified (unnecessary setup, conventions etc) - Increased coverage for DataNodeBuilder - Added or modified test to include more spi models - Added tests for Hazelcast Configs - Added more tests for json object mapper - Added test and fixed error handling in YangUtils/XmlFileUtils (it was incorrectly converting a config exception to a data validation exception) Issue-ID: CPS-475 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I5852ba01bc5b33ae361b8f29daae9868f05baa35
2023-07-28Add 'direct' keyword to descendants option to query direct children (ep1)Rudrangi Anupriya3-5/+9
-added 'direct' keyword also to Fetch Descendants Option along with 'all' and 'none' to Query direct child. -added unit tests to test direct keyword Issue-ID:CPS-1784 Change-Id: Iab7f59fbeebb03703626132c6d5c2afde0e5ab4d Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-07-27Merge "Subscription Creation: NCMP to Client CloudEvent transformation"Luke Gleeson2-0/+98
2023-07-27Fix: Make bookstore data consistentArpit Singh3-26/+16
- Removed list named invoice from all bookstore files - Added a new parent list as bookstore-address - Refactored tests Issue-ID: CPS-1813 Signed-off-by: Arpit Singh <as00745003@techmahindra.com> Change-Id: I5c7e83e1ee0ad9ac1d94d1906c2361c7a00d49af
2023-07-26Subscription Creation: NCMP to Client CloudEvent transformationhalil.cakal2-0/+98
- Delete legacy avc subscription event and event outcome schemas - Change subscription response and outcome sample json file contents - Change ncmp event response code to support avc subscriptions - Add mapper that maps cloud event to subscription event response - Add mapper that maps subscription event outcome to cloud event - Change subscription event response consumer to consume CloudEvents - Change time out task to support response event instead client id and name - Change subscription event response mapper to support Cloud Event - Change subscription outcome mapper to group subscription responses as per details and status - Change subscription status to have fromString functionality - Change all unit tests to support new functionalities - Add cps exceptions for cloud event and outcome type - Add details field in yang model - Change data node helper to supoort details field - Consolidate final subscription response codes - Fix code smells reported by SonarLint Issue-ID: CPS-1739 Change-Id: I5eadeb8ef40d3d7befb762b5a8d2139fe3c85d7e Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-07-20Merge "Apostrophe handling in CpsPathParser"Toine Siebelink1-1/+1
2023-07-20Apostrophe handling in CpsPathParserdanielhanrahan1-1/+1
Apostrophe is not currently handled correctly, and having apostrophe in the xpath will lead to various errors. For example, normalizing this xpath works: /path[@name="I'm quoted"] -> /path[@name='I\'m quoted'] However the resulting xpath will throw a PathParsingException if parsed! (Thus path normalization is not idempotent.) - Use '' for escaping apostrophe in single quoted leaf value, to comply with XPath standard (and use "" for escaping in "). - Use Liquibase to make existing data comply with new rules. - Leaf values in data leaves are now unescaped, e.g. "I'm quoted" - Quoting is now consistent for leaf/text/contains conditions. Issue-ID: CPS-1769 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iafc287f738254d7f99706c6bc548091c0ecd5aa0
2023-07-18Persisting a list element to a parent list (ep2)Rudrangi Anupriya4-2/+68
Post List Element does not allow for create List Element, only appends onto existing node as children -Add a check in saveListElements to see if the parent xpath is a root path ("/").If root node store list element as top node. Else add passed list element to parent xpath node. -Add test for scenario for above -Add test scenario Saving list element data fragment under Root node -Add Integration Tests Add and Delete top-level list (element) data nodes with root node -Update bookstore model with TopLevelList datanode Issue-ID: CPS-1586 Change-Id: Iaa7f59fbeebb03703626132c6d5c2afde0e7ab4b Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-07-17Improved code coverage for CpsDataServiceImplToineSiebelink3-46/+38
- Added (extended) test to cover missed scenarios - Removed unnecesarry null check from Production code - Improved error messages in production code - Removed duplicate test - Cleaned up existign test somewhat (labels and aligment) Issue-ID: CPS-475 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I9761ad6d4777a6bcfee4cbe1b876dd39fa218fc8
2023-07-13Fix handling of special characters in prefix namesdanielhanrahan4-18/+20
This fixes issues with special characters like square brackets - Make PrefixResolver use CpsPathParser instead of regex - Make DataMapUtils use CpsPathParser instead of String parsing Issue-ID: CPS-1758 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I6dd66eee398a46a69c0229059195f5096ab6fdec
2023-07-11Combine alreadyDefinedException classesdanielhanrahan3-64/+25
Issue-ID: CPS-1774 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I52ff9074a9f8188e8635a811b0d2713a97cb4b10
2023-06-29Merge "Normalize parent xpath when building datanodes in CpsDataService"Toine Siebelink1-2/+4
2023-06-29Improved code coverage (branches) around multipart file utilsToineSiebelink1-37/+37
- added test - moved wrongly placed test (from MultipartFileUtilSpec to DataMapUtilsSpec) - applied curent conventions to DataMapUtilsSpec Issue-ID: CPS-475 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ie8dc7f6802bb6ff0a256dc05df73da6af75409f3
2023-06-27Normalize parent xpath when building datanodes in CpsDataServicedanielhanrahan1-2/+4
Data nodes are being saved with non-normalized xpaths, resuling in data nodes that cannot be operated on. This affects all operations including get, query, update, and delete. Issue-ID: CPS-1765 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I5352182d79daec67805753ca5943b1a86c18159f
2023-06-27NCMP : Handle non-existing and non-ready cm handlessourabh_sourabh2-3/+21
- Modified data operation schema to contains cm handle as steing and response content field name as result. - Added a new common cloud event builder for NCMP to create an event. - Added data operation event creater that uses cloud event builder to create a cloud event. - Introduced a new method onto json object mapper to convert json object to bytes. - Modified EventDateTimeFormatter and added a new method to convert date timestamp into offsetdateTime. - Added a new code into ResourceRequestUtil to identify non-ready cm handle and non-existing cm handle and later publish it as cloud event to tha client given topic. - Introduced CpsApplicationContext to get spring mannaged bean into non spring managed java object. Issue-ID: CPS-1583, CPS-1614 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I24a39d2cb2c54dea25cd2f17e7748e21cd83a088 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-05-16Fixing minor compilationwaqas.ikram1-1/+1
- Complilation issues while importing project in eclipse - Fixing CSIT Change-Id: Idb85b08066d27a7f2a9b297d1580915df2b67541 Issue-ID: CPS-1672 Signed-off-by: waqas.ikram <waqas.ikram@est.tech>