aboutsummaryrefslogtreecommitdiffstats
path: root/cps-ri/src/main
AgeCommit message (Collapse)AuthorFilesLines
2023-05-04Fix for quickfind with descendants incl. list entriesToineSiebelink2-24/+38
- add specialized regex - fixed legacy issues with absolute paths (added ^ in regex's) - split into 2 methods for normal queries and quickfind - fixed order of private methods Issue-ID: CPS-1671 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I785a230cd0e3e0359e3cbb0fa754a21b6e2be7ff
2023-05-03canUseRegexQuickfind does not test for contains-conditiondanielhanrahan1-1/+2
This bug causes queries using contains-condition to return wrong results, when INCLUDE_ALL_DESCENDANTS option is set. Issue-ID: CPS-1663 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: If323e0e3b54d8c9a213dffda46897b2efb01fe2a
2023-04-26[CPS] RI, Service: Code Refactoringraviteja.karumuri2-22/+0
# Remove unused method "updateDataNodeAndDescendants" # Remove unused production code Issue-ID: CPS-1594 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Id909e9da3e9181c181dfdfe62f051451d8e488a2
2023-04-24Remove 32K limit from queries with collection parametersdanielhanrahan7-160/+113
SQL queries taking collection parameters currently create a seperate query parameter for each collection element. There is a limit of around 2^15 (32,768) query parameters. Postgres DB natively supports array types, in which the whole array is transmitted in binary as a single parameter. Changing queries to use arrays removes the 32K limit on queries. - Add support for Postgres arrays to queries - Change repository methods to use arrays, and provide overloaded versions taking collection parameters - Update tests to reflect 32K limit being lifted Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I64f2aeaedbe54bfe12e3079cba0f2216759142c3
2023-04-21Merge "Add contains condition support to cps-path"Luke Gleeson1-0/+11
2023-04-21Add contains condition support to cps-pathRudrangi Anupriya1-0/+11
Issue-ID: CPS-1272 Change-Id: Ic81d1322cacc64a8752916324b801d02be47d34f Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-04-20Migrate hibernate-types to hypersistence-utilsdanielhanrahan1-1/+1
There is newer release of the dependency that provides JSONB support. The newer version provides additional features, such as Postgres arrays. It was renamed from hibernate-types to hypersistence-utils. Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iab556850041b592a8c4d515270dfe4bd9547957a
2023-04-17Remove CollectionUtils dependencylukegleeson1-2/+1
Remove dependency creating pipeline issue Issue-ID: CPS-1626 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I64a6019a8196aa98dd3874d08f373c4fb9e3b8c9
2023-04-14Use cascade delete in fragments tabledanielhanrahan4-30/+40
- Add Liquibase change to cascade delete fragments - Remove cascade delete logic from FragmentNativeRepository - Remove unused method deleteFragmentEntity Issue-ID: CPS-1617 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iecf4f3022b402e0328609ddc11a879c83e5d7f30
2023-04-14Remove @NotNull from repository methodsdanielhanrahan4-29/+23
Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I55312bdc5454ed9ca60441b3968d19a61028eb66
2023-04-14Add OR operator to cps-pathRudrangi Anupriya1-16/+22
Issue-ID: CPS-1215 Change-Id: I91fdf5bddcc4fc12a8cf9dbce75f77c832c55871 Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-04-11Populate Dataspace field of DataNodedanielhanrahan1-0/+1
Issue-ID: CPS-1603 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I85aca8bccc28d36aa4cefa0a32cb9bdf2156618a
2023-03-21Query data nodes across all anchors under one dataspacerajesh.kumar6-8/+88
Issue-ID: CPS-1396 Change-ID: I73f97f986a817d423f93a8d922dcd9647b1412ab Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2023-03-14Reduce dataspace/anchor lookups in CpsDataServicedanielhanrahan1-1/+1
- Use Anchor object instead of names in processDataUpdatedEventAsync - Use Anchor object instead of names in buildDataNodes - Avoid unnecessary ArrayList copy in updateDataNodesAndDescendants Issue-ID: CPS-1536 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I46936a655a3d151357e55b68e1c4161e07100e84
2023-03-13Fetch fragment entities using recursive SQL querydanielhanrahan5-131/+39
- Add SQL query that can fetch needed fragments to any given depth - Update getFragmentEntities to use new query - Remove now unused FragmentRepositoryMultiPathQuery - Remove unused TempTableCreator method - Result: getDataNodesForMultipleXpaths is up to 10 times faster Issue-ID: CPS-1525 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I07cbc9da5ab994ce7e0c2b02d7ca05089f05dab0
2023-03-13Merge "Reduce dataspace/anchor lookups in CpsDataPersistenceService"Toine Siebelink2-143/+83
2023-03-13Merge "Add new metrics to the get data node method"Toine Siebelink1-1/+8
2023-03-13Reduce dataspace/anchor lookups in CpsDataPersistenceServicedanielhanrahan2-144/+84
- Remove unneeded calls to DataspaceRepository::getByName - Remove unneeded calls to AnchorRepository::getByDataspaceAndName - Refactor FragmentRepository Issue-ID: CPS-1536 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2121f6247070ee4a7c000e06ec66a6278b758540
2023-03-13Make single deleteDataNode use plural deleteDataNodesdanielhanrahan1-71/+31
- Make deleteDataNode and deleteListDataNode call deleteDataNodes - Add onlySupportListDeletion option to deleteDataNodes to support original deleteListDataNode behaviour - Allow delete root xpath in deleteDataNodes - Fix incorrect use of PathParsingException in deleteDataNode - Update performance tests timings Issue-ID: CPS-1523 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I92c3c3ce606a5ab2cb8e6779d1ee0f9853529982
2023-03-09Add new metrics to the get data node methodkissand1-1/+8
Issue-ID: CPS-1509 Change-Id: I2eded9a23b56f5263591c710c7a707cf281f93d7 Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2023-03-08Add index to schema_set_yang_resources tabledanielhanrahan2-2/+19
Currently, the database is sequentially scanning when deleting from schema_set_yang_resources table as there are no indexes on the table. Adding an index doubles performance of NCMP CM-handle deregistration. Issue-ID: CPS-1535 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia6ac7ef95d1662f49a77fc72112f1e29a67079ff
2023-03-07Drop temp table when transaction finishesdanielhanrahan1-1/+2
By default, temp tables are persisted until the end of the session. This causes "Out of shared memory" errors in the database, which then auto-vacuums the orphaned temp tables to free memory. To fix this, we drop temp tables when the transaction finishes, using ON COMMIT DROP. Issue-ID: CPS-1533 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iac47d85e9621531a4a95b99c4becc8cc95242721
2023-03-01Add DataNodeNotFoundException to deleteDataNodesdanielhanrahan2-10/+24
Current implementation of NCMP handle de-registration relies on DataNodeNotFoundException being thrown to report errors. - Make deleteDataNodes throw DataNodeFoundExceptionBatch - Update performance test timings Issue-ID: CPS-1481 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ib833bdb4a23d24e1784bdaf4e5e5e8a9acb41c54
2023-02-28Skip deleting list xpaths that are list elementsdanielhanrahan1-4/+10
List elements cannot be lists. Filtering list elements before trying to delete lists by xpaths doubles performance when deleting list elements (which doubles performance of CM handle de-registration). Issue-ID: CPS-1511 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ieb6002212d006396d468f27f853708b5aa1e31f2
2023-02-24Improve performance for update data nodesdanielhanrahan1-39/+34
- Extract method getFragmentEntities from getDataNodes - Remove unused code from getFragmentEntity - Use bulk getFragmentEntities in updateDataNodesAndDescendants - Update performance test timings Issue-ID: CPS-1504 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Icd2c6a0e6009e152de43090cbc23a21349703da2
2023-02-21Improve performance of deleteDataNodes SQLdanielhanrahan1-35/+38
- Use SQL IN operator instead of temp table when deleting nodes - Use Postgresql LIKE ANY array operator when deleting lists - Update delete perf test timings - Refactor adding cascade delete constraint Issue-ID: CPS-1502 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic90b867e7c71ec1981f05a9122322ece84dd8bde
2023-02-20Merge "FragmentEntity stuck in memory on "Already exists" error"Toine Siebelink1-0/+2
2023-02-20Merge "CPS-1401 Implement V2 of GET Data Node API"Luke Gleeson1-10/+15
2023-02-17CPS-1401 Implement V2 of GET Data Node APIarpitsingh1-10/+15
- Modified the GET Data Node API so it returns all the data nodes when xpath set to root "/" - Fragment Repository now returns a collection of Fragment Entities - Instead of returning only the first Fragment Entity now all fragment entities are returned when xpath is set to root - The Fragemnt Entities are further processed to a Collection of Data Nodes. As opposed to singular Data Node in current implementation. - Finally the DataRestController also returns a Collection of Data Nodes when xpath is set to root and valid data is present - Response body changed from JSON object to JSON Array. - Exception handling for invalid xpath and non-existing xpath is now done separately at persistence layer. - Refactored code against CPS-1422 - Deprecated getDataNode method from Service and Persistence layer - Modified V1 of Get Data Node API to use the getDataNodes method and get the first data node from the collection returned. - Modified NCMP to use getDataNodes method - NCMP still does not support multiple data nodes. It retrieves the first data node from the collection returned by getDataNodes Signed-off-by: arpitsingh <as00745003@techmahindra.com> Change-Id: I494a5740a53f65376d135fcb9f1e2e8900a2803e
2023-02-16FragmentEntity stuck in memory on "Already exists" errorkissand1-0/+2
- exclude childFragments from toString method Issue-ID: CPS-1493 Change-Id: I3d0c7ef601f06ed36265d88cadb6fb0ab0cb6ea2 Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2023-02-16Improve batch delete schemasets performancedanielhanrahan3-2/+34
- Bulk delete anchors and datanodes associated with schemasets. Improves de-registration performance by approx 10% Issue-ID: CPS-1423 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie38e8b4c64356bf5935d8c7a5d3f5bfa73fb1714
2023-02-15Bulk delete schemasets in CM handle deregistrationdanielhanrahan3-4/+31
- Batch delete schema sets in single query - Call deleteUnusedYangResourceModules once per batch, not per CM handle - Results for deregistering 10k: 14 mins before; 6 mins after Issue-ID: CPS-1423 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia3a86a0dc88677323e2f386253a99022a7f02603
2023-02-01Create plural version of deleteDataNodedanielhanrahan6-45/+123
- Add method to CpsDataService to batch delete data nodes and lists - Use native queries to batch delete fragment entities by xpaths, for data nodes and lists - Add performance tests for batch delete - Refactor FragmentNativeRepository - Add single-column version of createTemporaryTable - Renamed metric cps.data.service.datanode.batch.delete to cps.data.service.datanode.all.delete Issue-ID: CPS-1438 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I1851f9c7ef0b1be4bd421b3352d9697a2dd23f79
2023-01-26Merge "Springboot Integration tests improvements"Sourabh Sourabh1-1/+3
2023-01-25Springboot Integration tests improvementslukegleeson1-1/+3
Creation of CpsIntegrationSpecBase Demonstration of test class implementing CpsIntegrationSpecBase in CpsPersistenceSpec Tests use reduced liquibase steps, basic bookstore yang model and bookstore json payload Issue-ID: CPS-1379 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I38202d0888808d08d85fce1aab45fc43e8b0cec3
2023-01-24Handle root xpaths in getDataNodesdanielhanrahan1-4/+13
Issue-ID: CPS-1458 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I64abf97317afe4335c8d04169689ee1396e75860
2023-01-23Use native query to delete data nodessourabh_sourabh4-2/+96
- Used hashcode to remove child fragment based on it's xpath - Used native query to delete fragment by altering fragment_parent_id_fkey CONSTRAINT. Issue-ID: CPS-1439 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: If19c449818e18f8fd666503b7346704eeb4a95d0 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-01-19Normalize xpaths for getDataNodesdanielhanrahan2-8/+27
Issue-ID: CPS-1457 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I93d19666c168aa69da73eadbfef0fc54181aec52
2023-01-16Merge "Fetch CM handles by collection of xpaths"Sourabh Sourabh5-14/+134
2023-01-11Fetch CM handles by collection of xpathsseanbeirne5-14/+134
- Added FragmentRepositoryMultiPathQuery - Removed Hibernate method for same - Added perf. test - Handle escaping of single qoutes in sql-data - Increased timing for path paser performance test Issue-ID: CPS-1422 Signed-off-by: seanbeirne <sean.beirne@est.tech> Change-Id: Ibea12a44bffd29ed43cc1560b507d1fa7e968b8b
2023-01-11CPS PATCH operation does not merge existing datalukegleeson1-2/+14
Altered code to add attributes and not overwrite attributes + tests Issue-ID: CPS-1442 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I23c5f6a65b98ea1b05af62a38a874c228cc82067
2022-12-22Temp Table Creation improvementsToineSiebelink4-43/+111
- extracted methods for more generic temp table creation - optimized - added perf test on use case that depends on temp table creation - had some doubts about stopwatch use in recent tests, but was all OK, just renamed some Issue-ID: CPS-1422 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I22cabb9b0ba1b0aa8576a2d756d77af46eebc1b0
2022-12-21CpsPath Query OptimizationToineSiebelink5-112/+231
- Optimized CpsPathqueries with descendants that only care about the xpath (no attribuets checks) - Use native query with regular expression for target xpath and descendants - Refactored so existing sql-geneartion code can be re-used in different repository implementations - Adjusted related performance test expectations Issue-ID: CPS-1421 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I3a807a14478c4b3272a5335d31c9aa3615eb2bee
2022-12-19Fix Delete uses case with '/' in pathToineSiebelink1-8/+2
-Extend and use cpsPath parser (util) classes instead of regex Issue-ID: CPS-1409 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I230c9eb71cc34264983830b39149511b95c4b4a6
2022-12-15CPS-341 Support for multiple data tree instances under 1 anchorarpitsingh1-4/+42
- Updated the parseJsonData method so it can parse JSON with multiple data trees, now it returns a ContainerNode - ContainerNode holds a collection of NormalizedNodes - Updated DataNodeBuilder and FragmentRepository as well to support collection of NormalizedNodes - Added new methods in CpsDataPersistenceService to store multiple Data Nodes - Added new test cases - Updated existing test cases and fixed code coverage - Addressed comments from previous patch Issue-ID: CPS-341 Change-Id: Ie893e91c0fbfb139a1a406e962721b0f52412ced Signed-off-by: arpitsingh <as00745003@techmahindra.com>
2022-12-13Merge "Fixing SonarQube violations"Toine Siebelink2-6/+9
2022-12-12Fixing SonarQube violationsleventecsanyi2-6/+9
- Code smell fixes Signed-off-by: leventecsanyi <levente.csanyi@est.tech> Issue-ID: CPS-475 Change-Id: I4b47ece82aed7e4b31aaa28e601e419fb6f06718
2022-12-09Merge "Added API to get all schema sets for a given dataspace."Toine Siebelink2-0/+37
2022-12-09Added API to get all schema sets for a given dataspace.rajesh.kumar2-0/+37
Issue-ID: CPS-1187 Change-ID: I73f97f986a817d423f93a8d922dcd9647b0914aa Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2022-12-08Merge "Fix SonarQube Violations"Sourabh Sourabh1-2/+1