aboutsummaryrefslogtreecommitdiffstats
path: root/cps-service/src/main
AgeCommit message (Collapse)AuthorFilesLines
2025-02-12Store yang resources with recommended RFC-6020 file-nameToineSiebelink3-13/+11
- Ignore input filename and create filename from module name and revision - added integration test to verify names and edge cases (before and after change) - Some code cleanup (vars etc) - Implemented NB comments from last merge(https://gerrit.onap.org/r/c/cps/+/140180) - fixed SQ warning Out of scope: - BLANK revision, test it but failed in ODL Yang Parser and many other places: not supported! Issue-ID: CPS-138 Change-Id: I6fe6d0f8f3683196b183c6e6582ad8eefdfbb7d7 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2025-02-11Merge "Query data nodes with limit"Toine Siebelink3-2/+49
2025-02-11Query data nodes with limitleventecsanyi3-2/+49
- added new methods to java interfaces - added integration test - removed unused methods Issue-ID: CPS-2394 Change-Id: Iac4094a5daedbf593d17f55928136a80391c6d23 Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2025-02-06Merge "Fix duplicated yang resource references V2"Daniel Hanrahan8-60/+62
2025-02-06Fix duplicated yang resource references V2ToineSiebelink8-60/+62
- Store Yang module references using SchemaSetEntity instead of SQL - Added integration test to check number of references - Update dispatcher for integration test to only return yang resources requested modules - refactor schema set create & upgrade methods for better re-uses and readability - consistent naming in related methods like(new)yangResourceContentPerName (the name can be filename, module name or schema set name) depending on the context - replaced 'var' with actual class names in affected classes Issue-ID: CPS-2605 Change-Id: I8870c70832ac533bd17ce8af409a071f659e4acf Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2025-02-04Simple implementation of attribute-axisdanielhanrahan2-3/+15
This minimally implements attribute-axis using existing queryDataNodes API. Acceptance tests are un-ignored now. Issue-ID: CPS-2416 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia06be3dd85dfce261d9d78529784d54d84b71bcd
2025-01-29Address sonar violations related to hazelcastmpriyank2-4/+4
- Supressing warning caused by deprecation of RestApiConfig in hazelcast 5.5 community version , which will be removed in 6.0 so created https://lf-onap.atlassian.net/browse/CPS-2599 to address the issues - Other sonar fixes Issue-ID: CPS-2594 Change-Id: I2e95cd851eae74d8bbb4535cfbaed1dc1ef89543 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2025-01-27Moved CpsValidator under the utils packageleventecsanyi7-7/+7
Issue-ID: CPS-2515 Change-Id: I57f6c10c8596281aed686552ad16e1dd6c15ff98 Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2025-01-23Merge "Add DB Clean-up to CPS Rest Interface"Daniel Hanrahan3-6/+12
2025-01-23Add DB Clean-up to CPS Rest InterfaceToineSiebelink3-6/+12
- add dataspace filter to module cleanup impl - updated integration test to only delete orphaned modules in relevant dataspaces (this also fixes CPS-2571) - improved labels on admin controller test - add dataspaces/../clean rest endpoint for cps core - added 'Regular Maintenance' section to RTD Admin Guide - consistent spelling of 'CM Handle' in RTD Docs Issue-ID: CPS-2554 Change-Id: Ica70c3495758f073eaac9eeeadcc0e1be2c8cc1c Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2025-01-20Moved builders under impl in CPS Serviceleventecsanyi6-10/+8
- moved builders and fixed the Arch test Issue-ID: CPS-2542 Change-Id: Iac9d47f20fff2e6fa415ada8bc17880b0b032591 Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2025-01-17Moved impl package out from api packageleventecsanyi9-9/+9
- refactored cps service packages and the architecture test Issue-ID: CPS-2543 Change-Id: I0fab54cebd157b19ff6105b7d4b6d8265a1af485 Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2025-01-16One SchemaSet per moduleSetTagToineSiebelink6-138/+48
- Registration: create and upgrade cases. - Handle moduleSetTag deletion (all orphans) for testware - Unit tests updated - additional logging of details for upgrade scenarios - Integration Tests updated - Remove cache for module sets being processed - Removed DbCleaner (startup) - Removed redundant methods in NCMP Inventory for deleting schema set(s) - Removed validation check for all schema set interactions - Updated some schema set tests to use special characters previously not allowed - Checked integration test scenarios for upgrades with and without tags: all scenarios covered! TODO - REST endpoint to remove orphaned schema set data, separate story: CPS-2554 - Investigate exception handling regarding DuplicateYangResourceException: CPS-2555 Issue-ID: CPS-2540 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Iaa59cbdb86b7a4a8044624829bc002506ff40cc7
2024-12-20Add schema to persist notification subscription informationrajesh.kumar5-0/+351
Add required schema to persist notification subscription information. It should contain - Schema yang file - New Dataspace, Anchors or any other database entity - Refactore duplicate code in NCMP Issue-ID:CPS-2427 Change-Id: I56c34400dc73c71b936a51260efd300924ababdc Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2024-12-05Moved common classes under the api package under apileventecsanyi58-129/+131
- moved common classes under the api package - moved Spec files Issue-ID: CPS-2512 Change-Id: I88046280ce359ce8cc488ad37d4e6f85406b0dbf Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2024-12-03Add tests of attribute-axis featuredanielhanrahan2-6/+28
Following TDD, functional acceptance tests are added for attribute-axis feature. This covers xpaths such as '//books/@title' - Add stub to CpsQueryService throwing UnsupportedOperationException - Add tests showing expected behaviour (tests currently ignored) - Implementation will be provided in following commmit for CPS-2416 Issue-ID: CPS-2416 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I71817e66b28dfc21e7b75243fd0135f3cceddb8e
2024-11-29Merge "XML content support on Replace list content"Priyank Maheshwari2-5/+6
2024-11-28De-Registration without Orphaned Module CheckToineSiebelink3-2/+58
- Testware updates to measure time spent for de-registration - Removed orphan check from module delete methods - Using @scheduled to run Db Cleaner method once (see https://lf-onap.atlassian.net/wiki/spaces/DW/pages/52494359/CPS-2478+Module+Sync+Inefficiencies#De-Registration:-Test-Measurements-With-and-Without-Orphanage-removal for reasoning) - Updated integration tests to call orphan check where need (after schema set deletion) Issue-ID: CPS-2478 Change-Id: I513af9d8bb88486a242284b58e0363a527346dd4 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2024-11-28XML content support on Replace list contentRudrangi Anupriya2-5/+6
Here to bring Support for XML Response Entity in Replace List content - Add ContentTypeInheadr in cpsData.yml to support application/xml - Add contentTypeInHeader parameter to accept xml in DataRestController.java - Modify the code return xml Data - written testcase for above changes made Issue-ID: CPS-2411 Change-Id: Ibb7ffb66ccdd03703266123c6d5c2eade0e7cb4a Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2024-11-19Merge "Test to highlight ModuleSetTag Inefficiencies"Sourabh Sourabh1-2/+2
2024-11-18Test to highlight ModuleSetTag InefficienciesToineSiebelink1-2/+2
- Add (micrometer) instrumentation to expose inefficiencies - Add test config for micrometer - Add setup methods in base to create many cm handles - Set module sync parallelism to 2 for testing - Add clean up methods for hazelcast related tests - added test to show inefficiencies - POC 1 use hazelcast set to prevent multiple threads working on same ModuleSetTag - POC 2 'cache' module set tags per thread to prevent DB looks ups - Main inefficiency left: create schemaset for EACH cm Handled even if same tag. No easy PoC... Change-Id: Idf46b44c475a24727dd7084bb613459f4c29be55 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2024-11-17XML content support for only cps Query v2Rudrangi Anupriya1-21/+25
Here to bring Support for XML Response Entity in query data nodes - Add ContentTypeInheadr in cpsQueryV2.yml to support application/xml - Add contentTypeInHeader parameter to accept xml in QueryRestController.java - Implement logic to convert data to xml - written testcase for above changes made Issue-ID: CPS-2359 Change-Id: Ieb7eeb66ccbb03703626132c6d5c2eade0e7cb4b Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2024-11-07Merge "Removing ONAP O-Parent as parent from CPS"Toine Siebelink1-2/+2
2024-11-05Removing ONAP O-Parent as parent from CPSsourabh_sourabh1-2/+2
- Fixed checkstyle and license validation. - Updated checkstyle configuration (`onap-java-style.xml` and `check-license.xml`) to ensure compatibility with Maven Checkstyle Plugin version 3.3.1 - Removed obsolete properties causing build failures. - Improved code style rules to meet ONAP Java guidelines. - Vulnerabilities scan are now configured on parent pom.xml - Fixed all code style. Issue-ID: CPS-2470 Change-Id: I1e72af604415aa7587cbe1dff471ed0323e29aa3 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-11-05XML content support on get a nodeRudrangi Anupriya2-1/+96
Here to bring Support for XML Response Entity In GET A NODE - Made changes in components.yml to support contentType as application/xml - Add ContentTypeInheadr in cpsDataV2.yml to support application/xml - Add contentTypeInHeader parameter to accept xml in DataRestController.java - Implemented Logic to convert DataMaps To XML Data - written testcase for above changes made Issue-ID: CPS-2280 Change-Id: Ibe7ffb66ccbb03703626132c6d5c2eade0e7ab4b Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2024-10-24Refactor: Improve code quality and compliance with SonarQube standardssourabh_sourabh1-13/+12
- Consolidated multiple `if` statements to enhance readability and maintainability. - Replaced traditional `instanceof` checks followed by casting with the new pattern matching syntax for improved type safety and clarity. - Removed unnecessary casts and streamlined code in `addYangLeafList` and `addDataNodeFromNormalizedNode` methods. Issue-ID: CPS-1957 Change-Id: I46696dc184b6b3c3d0fe3a6bd12267a4e0d76193 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-10-09Implementation of Data validation feature in Create a Node APIArpit Singh4-13/+82
Added support to validate JSON/XML data without the need of persisting it in the databse. - added "dryRunInQuery" flag as a new query parameter - added new method as part of CpsDataService layer to perform data validation - added new method in yang parser "validateData" to validate data without persisting it Issue-ID: CPS-2361 Change-Id: I43dd33cc6120576d0fac606d5c4b0168d107311d Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
2024-10-08Merge "Policy Executor: handle errors, part 2 (fighting between IntelliJ and ↵Sourabh Sourabh1-0/+2
Checkstyle best practices)"
2024-10-03[BUG] Fix memory leak related to using arrays in Hibernatedanielhanrahan1-12/+10
The use of arrays like String[] instead of Collection<String> in JpaRepository methods is causing a memory leak, most likely due to a bug in the hypersistence-utils dependency which provides the feature. Note code using arrays in JDBC (via jdbcTemplate) is not affected. This patch removes most uses of arrays in Java, except one needed for FragmentPrefetchRepository using JDBC setArray(), which is safe. Issue-ID: CPS-2430 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I94f8c3d4c8c32ebe0978c08d3226a196a95bf3b9
2024-10-03Policy Executor: handle errors, part 2ToineSiebelink1-0/+2
(fighting between IntelliJ and Checkstyle best practices) - non-2xx responses are processed using web client exceptions - handle unknown host exception - upgraded spotbugs (checkstyle and related mvn plugin) - fixed some small spotbugs due to upgrade - added commented instructions in docker compose to enable debugging - added some environment variables for policy executor configuration - extract out Sleeper in stub service to achieve 100% coverage - added cause to Policy Executor exceptions where applicable - ignored (new) spotbug rule about catch NPE because of issue in 3pp - ignored (new) spotbug rule about \n in string due to multiline string block Issue-ID: CPS-2412 Change-Id: I6835a73320c436cbeea12cc7a06f15899eec7bf1 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2024-10-03Update JIRA and Wiki links host name from wiki.onap.org to lf-onap.atlassian.netsourabh_sourabh1-2/+2
Issue-ID: CPS-2432 Change-Id: I53f380a4fe0968d1ec9f1393651c7c43f03d0d95 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-09-24Remove Hazelcast cache for prefix resolver (CPS-2417 #2)danielhanrahan4-256/+16
This patch removes the AnchorDataCache from CPS, which is used for prefix resolution in get/query operations. As such, Hazelcast is no longer a dependency of CPS, only NCMP. - Changed PrefixResolver to be more efficient. - Removed AnchorDataCache and associated classes. - Moved HazelcastCacheConfig to NCMP. - Removed Hazelcast dependency from cps-service/pom.xml This shows good performance improvements in some APIs such as v2 GET which is nearly 2x faster (also 5x faster including base patch). Issue-ID: CPS-2417 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I24768469f24e90b70f7a6187faa4f5b3d75777d2
2024-09-24Reduce anchor lookups related to PrefixResolver (CPS-2417 #1)danielhanrahan2-15/+1
Instead of looking up same Anchor many times inside a for-loop, do it once outside the loop. This greatly improves performance in some cases, such as v2 GET API: - /cps/api/v2/dataspaces/{dataspace}/anchors/{anchor}/node Testing shows 3x faster response time. Issue-ID: CPS-2417 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I80d97d8cc24372eed70626ed840cad985cbe0a4b
2024-09-17Refactored cps-ri package structureleventecsanyi8-8/+7
- fixed import order and moved package structure Issue-ID: CPS-2293 Change-Id: Ie2f9f057f261577054530feee7480850ba4b41e1 Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2024-09-10Renaming Delta Operations as per RFC 9144Arpit Singh3-8/+8
RFC 9144 recommends naming convention for operations that can be performed on any datastore. The recommended names are: - create - remove - replace Whereas in CPS Delta feature these operaitions were named: - add - remove - update So, "add" and "update" operations have been changed to "create" and "replace" respectively. Issue-ID: CPS-2393 Change-Id: Ie39f5f8ccc91e96cf484ca7fbc833a6be8758054 Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
2024-08-13CPS-NCMP: Slow cmHandle registration when we use moduleSetTag, alternateId ↵sourabh_sourabh3-0/+60
and dataProducerIdentifier - Created a new repo. service for fragment table that executes a native sql query to find first ready cm handle id based on moduleset tag and then returns list of module references. - Exposed a new interface into module service that is used by module sync service to get list of midule refs by module set tag. Issue-ID: CPS-2353 Change-Id: I438dbd1ed37c1ff4e15f792e93a095aa604120bc Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-08-01XML content support on add list elementRudrangi Anupriya2-6/+8
Issue-ID: CPS-2287 Change-Id: Ibb7ffb66ccbb03703266132c6d5c2eade0e7cb4a Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2024-07-24Update response message for data validation failureArpit Singh1-7/+3
- Make the response message more informative and cleaner in case of data validation failure. - Updated message body will not contain the entire jsonData as part of response sent back to the user Issue-ID: CPS-2306 Change-Id: I28aab1450779af98ccdd11701e78bdfcdbc18cf0 Signed-off-by: Arpit Singh <as00745003@techmahindra.com>
2024-07-16XML content support on replace a node with descendantsRudrangi Anupriya2-18/+21
Issue-ID: CPS-2282 Change-Id: Ibb7ffb65ccbb03703266712c6d5c2eade0e7ab4b Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2024-06-25Fix incorrect use of getAnchors(dataspace, schemasets)danielhanrahan4-45/+72
There is an issue in a method deleteDataNodes(dataspace, anchors) where it doesn't send data update events. This is because it fetches anchors using a method getAnchors(dataspace, schemaSetNames), when it needs to fetch anchors by name: getAnchors(dataspace, anchorNames) Changes: - Rename getAnchors method using schemasets to getAnchorsBySchemaSetNames - Add a method getAnchors(dataspace, anchorNames) - Update test of deleteDataNodes method using getAnchors so it checks that data update events are sent Issue-ID: CPS-2254 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I55fe853f0a9278a66a5724bf4cf2723b0e2fbc8b
2024-06-19Addressed an adaptability issuesourabh_sourabh1-5/+5
- Defined a constant instead of duplicating literal "No data nodes 4 times. Issue-ID: CPS-478 Change-Id: I2b980a228217e85424342d4c2b9e97c5cfa618d1 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-06-06CPS Delta API 2: Delta between anchor and payloadArpit Singh4-14/+201
- Second API to get Delta between an anchor and JSON payload - added new API getDeltaByDataspaceAnchorAndPayload - added controller and service layer methods getDeltaByDataspaceAnchorAndPayload - Core Delta algorithm remains same as the first API. getDeltaByDataspaceAnchorAndPayload will call getDeltaBetweenDataNodes Issue-ID: CPS-1836 Signed-off-by: Arpit Singh <as00745003@techmahindra.com> Change-Id: Id74cd930ce48e5cb414aa62c5381b79675788a37
2024-06-05XML content support on update node leavesRudrangi Anupriya2-7/+8
Issue-ID: CPS-2071 Change-Id: Ibe7f59fbfcbb03703626132c6d5c2afde0e7ab4b Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2024-05-08Conditional cps change eventsmpriyank1-2/+6
- introduced a parameter to control the cps core change event notifications. we should be able to disable these notifications without affecting other notification flows. - fixed the LayeredArchitectureTest as we are accessing the Anchor model in the events package now. Issue-ID: CPS-2213 Change-Id: Id875925bc14de1cc6e8fa3193c0df470e09fe43f Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2024-05-03[BUG] Correctly parse observedTimestampmpriyank2-1/+38
- Using the DateTimeUtility to parse the passed OffsetDateTime and using it for conversion to string - Moving to actual src/main/java from test side as it is to be used in the prod code as well and only keeping the methods in use. - Added test to fix the coverage issue Issue-ID: CPS-2211 Change-Id: I2b4325bb77a6da7fd470986b1da293f455650375 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2024-04-30Add Notification support in cps corerajesh.kumar3-1/+209
Add notification support using cloud events Issue-ID:CPS-2068 Change-Id: I56c34400dc73c71b936a51260efd240223babacd Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2024-04-26Faster module searches (CPS-2190 #3)danielhanrahan2-4/+2
This greatly improves performance of module searches by eliminating unneeded SQL queries via Hibernate lazy fetching. Issue-ID: CPS-2190 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie9e65017d0027366456f1741cc37b10679317b25
2024-03-19Configure Hazelcast to have 1 backup to reduce memorydanielhanrahan1-6/+3
Currently Hazelcast is configured to have 3 synchronous backups and 3 asynchronous backups. These are separate, meaning there will be 7 copies in the cluster (1 original + 3 sync + 3 async backups). Even if only 1 instance of CPS/NCMP is running, it will have 7 copies in memory. Given that CPS/NCMP is typically only deployed using 1 or 2 instances, the settings are changed to 1 synchronous backup (same as Hazelcast's default configuration). This change has been tested and shown to reduce heap usage by around 100MB during 20K CM-handle registration. Issue-ID: CPS-2146 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I4a5f6d83bc35e2c13cfb32002e38dc365da34c8e
2024-02-12Fix test failure by ordering leaf-listsdanielhanrahan1-4/+9
YANG specifies two ways that leaf-lists can be ordered: - ordered-by user: original order in JSON is preserved - ordered-by system (default): it is up to the system how to order For leaf-lists to preserve same order as the JSON, the Yang module must specify 'ordered-by user'. To ensure consistent behaviour even when system ordering is used, the leaf-list is sorted during parsing. - Add 'ordered-by user' to authors field in bookstore.yang - Sort leaf-list during parsing when using 'ordered-by system' - Add new tests to verify ordering Issue-ID: CPS-2057 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I6ab688ec2fa4a22182e853d1a8b26642f278c40a
2024-02-07Orphaned modules are removed from the DB before cmhandle removalsourabh_sourabh1-1/+1
- changed sequence of delete schema set by cascade to remove unused yang resource modules at last. Issue-ID: CPS-2031 Change-Id: Icefc075165231cc4b9b170e04a2d12af645529e9 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>