aboutsummaryrefslogtreecommitdiffstats
path: root/cps-ri/src
AgeCommit message (Collapse)AuthorFilesLines
2024-10-03[BUG] Fix memory leak related to using arrays in Hibernatedanielhanrahan7-83/+59
The use of arrays like String[] instead of Collection<String> in JpaRepository methods is causing a memory leak, most likely due to a bug in the hypersistence-utils dependency which provides the feature. Note code using arrays in JDBC (via jdbcTemplate) is not affected. This patch removes most uses of arrays in Java, except one needed for FragmentPrefetchRepository using JDBC setArray(), which is safe. Issue-ID: CPS-2430 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I94f8c3d4c8c32ebe0978c08d3226a196a95bf3b9
2024-09-17Refactored cps-ri package structureleventecsanyi38-259/+258
- fixed import order and moved package structure Issue-ID: CPS-2293 Change-Id: Ie2f9f057f261577054530feee7480850ba4b41e1 Signed-off-by: leventecsanyi <levente.csanyi@est.tech>
2024-09-06Refactor FragmentQueryBuilderdanielhanrahan1-79/+89
This is a refactoring of FragmentQueryBuilder, there are no functional changes. This refactoring is needed before main patch of CPS-2365. Issue-ID: CPS-2365 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I648616159719e084ba0e7cb4b09649cd17560cab
2024-08-31[Cps Path Parser] Refactoring leaf conditionsdanielhanrahan1-11/+9
Instead of DataLeaf and ComparativeOperators classes, we more simply have LeafConditions class with all info. Issue-ID: CPS-2365 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8fd18bb56b8ed9d26a3f9d36f487d00a9274c8bc
2024-08-27Cleanup unused yang models from CPS coredanielhanrahan5-356/+0
Yang models and generated CSVs are no longer used in CPS liquibase, rather Inventory model loader does this. Issue-ID: CPS-2382 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I0944c3778716450a88355c669ac806fd07b85193
2024-08-13CPS-NCMP: Slow cmHandle registration when we use moduleSetTag, alternateId ↵sourabh_sourabh4-11/+112
and dataProducerIdentifier - Created a new repo. service for fragment table that executes a native sql query to find first ready cm handle id based on moduleset tag and then returns list of module references. - Exposed a new interface into module service that is used by module sync service to get list of midule refs by module set tag. Issue-ID: CPS-2353 Change-Id: I438dbd1ed37c1ff4e15f792e93a095aa604120bc Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2024-06-25Fix incorrect use of getAnchors(dataspace, schemasets)danielhanrahan1-7/+16
There is an issue in a method deleteDataNodes(dataspace, anchors) where it doesn't send data update events. This is because it fetches anchors using a method getAnchors(dataspace, schemaSetNames), when it needs to fetch anchors by name: getAnchors(dataspace, anchorNames) Changes: - Rename getAnchors method using schemasets to getAnchorsBySchemaSetNames - Add a method getAnchors(dataspace, anchorNames) - Update test of deleteDataNodes method using getAnchors so it checks that data update events are sent Issue-ID: CPS-2254 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I55fe853f0a9278a66a5724bf4cf2723b0e2fbc8b
2024-04-26Faster module searches (CPS-2190 #3)danielhanrahan2-12/+11
This greatly improves performance of module searches by eliminating unneeded SQL queries via Hibernate lazy fetching. Issue-ID: CPS-2190 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie9e65017d0027366456f1741cc37b10679317b25
2024-04-25[BUG] Remove slow validation check during module search (CPS-2190 #2)danielhanrahan2-56/+1
The check for existing modules is extremely slow, being dependent on the file sizes of the Yang modules being queries. The validation check is also completely unnecessary, since it simply ignores any exceptions and returns an empty set, which the main logic does implicitly. (The Dataspace check is also not needed, as the NCMP dataspace is guaranteed to exist.) Issue-ID: CPS-2190 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I329c1f8aac3f50bda0333e6c9c686f47af2e009f
2024-03-29Uplift Spring boot 3.2.4emaclee1-2/+3
- uplifted spring boot from 3.1.2 to 3.2.4 - removed redundant dependencies - upgraded relevant dependencies - use of @Bean(some_name) is give a specific bean name - using @Bean(some_name) with @Qualifier so on the injection point to specify which bean to actually inject, this is used as we deal with multiple beans of the same type. Issue-ID: CPS-2101 Change-Id: I64d5bf567b9eb03fb39034a46528d0160b332715 Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech>
2024-03-26Condense Liquibase stepsdavid.mcweeney45-8454/+239
Change-Id: I1493f49bb93c359df3ff7d4f928f9a94fb8b2604 Signed-off-by: david.mcweeney <david.mcweeney@est.tech> Issue-ID: CPS-2061
2024-02-28Revert Spring Boot Upliftmpriyank1-3/+2
- reverting back to spring 3.1.2 - manual changes to revert as other apps not fully ready for the delivery Issue-ID: CPS-2133 Change-Id: I670b67916fa4a5d4e5bced6548b7fcd5ee12e855 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2024-02-22Uplift Spring Boot to 3.2.2waqas.ikram1-2/+3
- Upgraded hypersistence-utils-hibernate-60 to hypersistence-utils-hibernate-63 - Introduced @Qualifier annotations where needed Issue-ID: CPS-2101 Change-Id: I8303592ed91584deb6ca5cd4e49800210b6ea989 Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
2024-02-14Faster absolute xpath query using parent path lookupdanielhanrahan1-6/+27
Issue-ID: CPS-2087 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I62b452f34f08a32a6d80671c7af560fbceec1285
2024-02-01Extend API: Get Module Definitionshalil.cakal2-8/+33
- add query parameters: module-name and revision to OpenAPI - extend the controller method to hande the new parameters - add the new method stack to the service layer - extend the SQL query to support the new parameters - add unit and integration testwares Issue-ID: CPS-1135 Change-Id: I089ad2ad71effb58ac0ba809e9f441d6cdb59c4f Signed-off-by: halil.cakal <halil.cakal@est.tech>
2024-01-19Remove Lambda & Restructure Fileseanbeirne1-270/+272
-Removed Lambda from CompositeStateUtils -Restructured CpsDataPersistenceServiceImpl to more logical layout Issue-ID: CPS-1238 Change-Id: I09096dab474230d04b15c204cbea95a2f1285366 Signed-off-by: seanbeirne <sean.beirne@est.tech>
2024-01-11Merge "Enable hibernate write batching"Lee Anjella Macabuhay5-3/+75
2024-01-10Enable hibernate write batchingdanielhanrahan5-3/+75
- Implement fragment ID sequence generator with allocation size of 100 - Enable write batching for insert and update, with batch size of 100 - Write performance is over 2x faster Issue-ID: CPS-1795 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7ebd0f834abb57c593603e2cd3fafc7e3425ab1e
2024-01-09Remove inefficient saveListElementsBatch APIdanielhanrahan1-17/+0
CpsDataService::saveListElementsBatch method is not needed as saveListElements supports saving multiple list elements in a single operation. This both simplifies implementation and greatly improves performance when saving list elements, as the Yang parser need only run once for the whole batch. - Change InventoryPersistence to save CM-handles in batches of 100 using existing CpsDataService::saveListElements method. - Remove not needed CpsDataService::saveListElementBatch. Issue-ID: CPS-2019 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8b74dda2917e094d064b42f2c0e4d57029b90395
2024-01-04Normalize JSON attributes during updatedanielhanrahan1-9/+26
This change fixes a few issues related to JSON encoding of FragmentEntity attributes (data leaves). This significantly improves update performance in cases of partial updates. - Normalize JSON and order leaves by name when comparing data leaves during update operations. - Update performance test timings. Issue-ID: CPS-2018 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia764a353bf96c05758827845e1358745247ee237
2023-12-13Introduce Schema Set upgrade methodsourabh_sourabh3-11/+58
- Incl. integration test - Prod code is changed to use new method that update yand module. - updated module resource to return multiple modules. Issue-ID: CPS-1806 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I751d9393ce78a3be9daeaff6d0252738c02115e0 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-10-11Expose REST endpoint to update YANG schema set using moduleSetTagsourabh_sourabh2-4/+1
- Added new schema to upgrade model into component. - Modified Ncmp rest input mapper to add upgradedCmHandles attributes. - Modified cm handle state mapper to add new lock reason. - Added new method to parse and upgrade Cm handles in DmiRegistration. - YANG data converter is modified to add "module-set-tag" atribute. - Cm handle new query method is added for cps path without appending ancestor. - Modified setCompositeStateForRetry to add lock reason. - New lock reason category is added. - Existing module sync service is modified to upgrade the model "syncAndCreateOrUpgradeSchemaSetAndAnchor". - Sync util method "getModuleSyncFailedCmHandles" to modified to add another lock reason "LOCKED_MISBEHAVING". - Added new attribute "UpgradedCmHandles" into DmiPluginRegistration and DmiPluginRegistrationResponse. - New attribute "moduleSetTag" is added into NcmpServiceCmHandle. - New model "UpgradedCmHandles" is added. - New method "updateSchemaSetWithExistingModules" is added into cps module service to update cm handle with exsting model. - Code coverage is reducced to 96 percentage that would be addressed and pushed into new patch. Issue-ID: CPS-1798 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I540acb404e38fc434de87a0d959bfde710a18b03 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-09-29Fix Sonar Code Smellsmpriyank2-49/+70
- convert normal java strings to text block - feature from java 15 - handling callback for legacy events. Once the deprecated methods are removed then this would not be needed. Issue-ID: CPS-89 Change-Id: I9b537ebee2c284c8109e6cc6c8961a430ee9599d Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-09-27Uplift Spring Boot to 3.1.2egernug1-1/+1
To align with Ericsson Spring Boot is being uplifted to version 3.1.2 Issue-ID: CPS-1888 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: Icfe97df15ab67a7fb25e0a98e4bbbbe91f332321
2023-09-22Reinstate Spring Boot 3.0 after revertegernug16-77/+84
Issue-ID: CPS-1789 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: Iee5c514ccdba36a387f83948d8a3ba26e6b1de5b
2023-09-20Revert "Migrate CPS to Spring-boot 3.0"Toine Siebelink16-83/+77
This reverts commit 9693ec51cf6526082f0ad0c3ad208d144cbbb163. Reason for revert: Bug fix delivery Change-Id: I73bdc1528192c662983b0bbef73b10b6d612a719 Signed-off-by: egernug <gerard.nugent@est.tech> Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2023-09-14Migrate CPS to Spring-boot 3.0egernug16-77/+83
Issue-ID: CPS-1789 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: I932fca8a50cedf2ae0604401c5e2b27896e4d449
2023-08-18CPS-Core : Expose a java interface to update schema setsourabh_sourabh2-0/+16
- Exposed an interface to update anchor by schema set name. - New interface is implemented into RI model. - New native query is exposed to update id with given schema set name. - A new integration test is written to test new interface into cps core. Issue-ID: CPS-1800 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ibf44712e11b53cb6673b04b9e3fd864321c90839 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-08-17Update of top-level data node fails with SQL errordanielhanrahan1-1/+1
The error is caused by the fetch-descendants code using ResultSet.getLong which returns a 'long' primitive instead of a 'Long' object. Thus a parent ID of 'NULL' becomes '0', which causes an error during update. To preserve the NULL value, ResultSet.getObject must be used. Issue-ID: CPS-1841 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ida6837a04954cd3c23f0f2faabd7d0712d8ee19d
2023-08-16Fixing sonar issuesdanielhanrahan3-6/+3
Issue-ID: CPS-475 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7623a8301fd4567bf96c605eb26bf6c02b1d3c76
2023-08-03Merge "Support pagination in query across all anchors(ep4)"Luke Gleeson7-21/+159
2023-08-02Support pagination in query across all anchors(ep4)rajesh.kumar7-21/+159
Add pagination query parameters in query across all anchors API pagination parameters (pageIndex and pageSize) are optional default is to query all fragments each pageSize represents number of records(number of anchors) TotalRecords is returned in response header to find number of pages. - If pagination option is provided in request then query number of anchors equal to pageSize. pageIndex is used for setting offset. - return number of records(one anchor per record) as per pagesize and pageSize Issue-ID: CPS-1605 Change-ID: I73f97f986a817d423f93a8d922dcd9647b2504bc Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2023-07-27Allow getDataNodes to read whole lists(ep 1)danielhanrahan3-31/+53
- getDataNodes can now retrieve list data nodes. Issue-ID: CPS-1696 Signed-off-by: arpitsingh <as00745003@techmahindra.com> Change-Id: I320a368d6cb73599d3f7c13fe9b8ab7d0cc69470
2023-07-20Apostrophe handling in CpsPathParserdanielhanrahan8-9/+84
Apostrophe is not currently handled correctly, and having apostrophe in the xpath will lead to various errors. For example, normalizing this xpath works: /path[@name="I'm quoted"] -> /path[@name='I\'m quoted'] However the resulting xpath will throw a PathParsingException if parsed! (Thus path normalization is not idempotent.) - Use '' for escaping apostrophe in single quoted leaf value, to comply with XPath standard (and use "" for escaping in "). - Use Liquibase to make existing data comply with new rules. - Leaf values in data leaves are now unescaped, e.g. "I'm quoted" - Quoting is now consistent for leaf/text/contains conditions. Issue-ID: CPS-1769 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iafc287f738254d7f99706c6bc548091c0ecd5aa0
2023-07-17Allow duplicate leaf names in Cps Path leaf conditiondanielhanrahan1-12/+9
Presently, a query using the same leaf name twice will fail: //books[@price > 10 and @price < 20] It is caused by storing data leaves in a Map. This is fixed by storing data leaves in a List<DataLeaf>. Issue-ID: CPS-1779 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie6990ea5e622cf37e986b720a0a07fb69ce4f03b
2023-07-13Upgrade to Java 17egernug1-3/+4
Upgrade CPS component to Java 17 Issue-ID:CPS-1767 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: Ide1e75205851e025371b4822ee161f71f04a6af1
2023-07-11Combine alreadyDefinedException classesdanielhanrahan1-7/+7
Issue-ID: CPS-1774 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I52ff9074a9f8188e8635a811b0d2713a97cb4b10
2023-07-10Handle special characters in CpsPath queries (CPS-1760 #2)danielhanrahan2-29/+7
This fixes issues with special characters for CPS-500, CPS-1756, CPS-1758, and CPS-1760. It also improves query performance. - use SQL LIKE instead of regex in Cps Path queries Issue-ID: CPS-1763 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I5c179882bfba71d3b009c60059e9073f46227e7d
2023-07-06Escape SQL LIKE wildcards in queries (CPS-1760 #1)danielhanrahan4-5/+75
If '%' and '_' are used in the contains-condition of a CpsPath query, incorrect results will be returned. For example: /bookstore/categories[contains(@code, "%")] Special characters in the contains-condition value must be escaped. Issue-ID: CPS-1762 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2fdd5a26433d510cd7d6af5b734a6779b537d63d
2023-06-21Improve performance of updateDataLeavesdanielhanrahan1-12/+9
Avoid fetching descendants during batchUpdateDataLeaves, as descendants are not needed: - Remove prefetch descendants step from getFragmentEntities; - Explicitly prefetch descendants in operations requiring it; - Update performance tests (5x faster for batch update). Issue-ID: CPS-1675 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7442a6f799cc0803b3a78f09d1ee53377f24b0b7
2023-06-19Move integration test (clean-up, last phase)ToineSiebelink14-677/+82
- Moved session manager test - Improved schemaset concurrency test (retry testing) - Cleaned up all ri (container based) integration test (setup) - Applied some groovy best practice where needed - internal ri module cover now down to 29%, covred by integration instead with - Line coverage up by 41 lines to 99% (was 97%) - Branch coverage up by 3 branches to 96% (was 93%) Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ifb77a053e5a5db62a3f6a32ae60a3a8b10918efd
2023-06-14Lower memory usage in FragmentRepositorydanielhanrahan6-238/+174
Avoid using Spring Data "interface projection" in FragmentRepository. The use of FragmentExtract in FragmentRepository is causing an overhead of around 5 kilobytes per fragment, which is leading to abnormally high memory usage when queries return a large number of nodes. For example, around 250MB of additional memory is needlessly used when fetching 50,000 datanodes. - Remove FragmentExtract interface and FragmentEntityArranger class. - Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper to fetch FragmentEntity descendants in a single SQL query. - Many CpsDataService operations have memory reductions: - queryDataNodes - getDataNodesForMultipleXpaths - updateDataNodesAndDescendants - updateNodeLeaves - and any NCMP methods using the above. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c
2023-06-13Merge "Move integration test (DataService)"Toine Siebelink4-747/+17
2023-06-12Fix for recursive SQL returning extra level of descendantsdanielhanrahan1-2/+2
Recursive SQL queries are incorrectly returning maxDepth + 1 levels of fragments due to incorrect comparison in the WHERE clause. This code is used by getDataNodes and queryDataNodes to prefetch the descendants before converting to datanodes. Prefetching extra descendants causes high memory usage. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6
2023-06-12Move integration test (DataService)ToineSiebelink4-747/+17
- Moved all Data Intergation Test - Added/improved some module Moudle Servcie test too - Removed some unused (impossible) code-paths in prod code - Replaced null's with meaningful strings (in edge case scenarios) - Replaced @Shared with static (shorter) in some test Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ib4b1ec1706ace7e3f5ac03d85177a6f7d4f6b8d0
2023-05-30Add <,> operators support to cps-pathRudrangi Anupriya1-17/+32
Issue-ID: CPS-1273 Change-Id: I5d562463b9a49abfe0436047a637857d10596fff Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-05-17Entity ID types do not match types in database definitionToineSiebelink17-36/+86
As Liquibase is used, JPA does not manage database definition. Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long) IDs. Other entities have INTEGER IDs. - Change AnchorEntity/AnchorRepository ID type to Long - Change YangResourceEntity/YangResourceRepository and ModuleReferenceRepository ID to Integer - Modify datatypes of foreign key references in liquibase changelog to use correct types for dataspace_id, schema_set_id, yang_resource_id - Update test liquibase changelog Issue-ID: CPS-1676 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2023-05-17Move integration test for adminServiceToineSiebelink2-267/+2
- finalized moving of admin service interation test (just some scenarios were missing) - deleted old test class (code coverage in RI module dropped) - line/branch coverage now 100/100 (was 97/91) - small refactoring; renaming and removed unnecessary check Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ic683548ea5e1e4e252f257c0f1034c5cf76e498d
2023-05-16Fixing minor compilationwaqas.ikram1-1/+1
- Complilation issues while importing project in eclipse - Fixing CSIT Change-Id: Idb85b08066d27a7f2a9b297d1580915df2b67541 Issue-ID: CPS-1672 Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
2023-05-12Remove dataspace_id column from Fragment tabledanielhanrahan12-91/+243
- Remove dataspace_id column from fragment table and drop associated index, foreign key constraint and uniqueness constaint. - Add uniqueness constraint using only fragment xpath and anchor_id. - Add not-null constraint to anchor_id. - Update code and test data to remove references to dataspace_id. - Repopulate the dataspace_id in Liquibase rollback (verified with manual testing of update/rollback). Issue-ID: CPS-1677 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I467cccba25ac5d884ec790064ca58150281b7e1d