summaryrefslogtreecommitdiffstats
path: root/cps-ri/src
AgeCommit message (Collapse)AuthorFilesLines
2023-05-05[CPS] RI: Code Refactoringraviteja.karumuri4-31/+17
# Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method. Issue-ID: CPS-1594 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Iaa5f8b0cc8d66473c993060a8fc5a54e37e373b1
2023-05-04Merge "Sensible equals and hashCode for FragmentEntity (CPS-1664 #1)"Toine Siebelink4-12/+19
2023-05-04Fix for quickfind with descendants incl. list entriesToineSiebelink2-24/+38
- add specialized regex - fixed legacy issues with absolute paths (added ^ in regex's) - split into 2 methods for normal queries and quickfind - fixed order of private methods Issue-ID: CPS-1671 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I785a230cd0e3e0359e3cbb0fa754a21b6e2be7ff
2023-05-04Sensible equals and hashCode for FragmentEntity (CPS-1664 #1)danielhanrahan4-12/+19
Include Anchor and Xpath in equals and hashCode methods for FragmentEntity. (This also requires adding equals and hashCode for AnchorEntity and DataspaceEntity.) The combination of dataspace, anchor, and xpath uniquely identify a fragment/datanode. This allows FragmentEntity objects returned from query across anchors to be stored in Set collections. Performance was observed to be unaffected by the change. Issue-ID: CPS-1664 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2c7e3957e392af36f5230d08c9bbd550f44e7444
2023-05-03canUseRegexQuickfind does not test for contains-conditiondanielhanrahan1-1/+2
This bug causes queries using contains-condition to return wrong results, when INCLUDE_ALL_DESCENDANTS option is set. Issue-ID: CPS-1663 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: If323e0e3b54d8c9a213dffda46897b2efb01fe2a
2023-04-26[CPS] RI, Service: Code Refactoringraviteja.karumuri7-100/+27
# Remove unused method "updateDataNodeAndDescendants" # Remove unused production code Issue-ID: CPS-1594 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Id909e9da3e9181c181dfdfe62f051451d8e488a2
2023-04-24Remove 32K limit from queries with collection parametersdanielhanrahan8-161/+114
SQL queries taking collection parameters currently create a seperate query parameter for each collection element. There is a limit of around 2^15 (32,768) query parameters. Postgres DB natively supports array types, in which the whole array is transmitted in binary as a single parameter. Changing queries to use arrays removes the 32K limit on queries. - Add support for Postgres arrays to queries - Change repository methods to use arrays, and provide overloaded versions taking collection parameters - Update tests to reflect 32K limit being lifted Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I64f2aeaedbe54bfe12e3079cba0f2216759142c3
2023-04-21Merge "Add contains condition support to cps-path"Luke Gleeson1-0/+11
2023-04-21Add contains condition support to cps-pathRudrangi Anupriya1-0/+11
Issue-ID: CPS-1272 Change-Id: Ic81d1322cacc64a8752916324b801d02be47d34f Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-04-20Migrate hibernate-types to hypersistence-utilsdanielhanrahan1-1/+1
There is newer release of the dependency that provides JSONB support. The newer version provides additional features, such as Postgres arrays. It was renamed from hibernate-types to hypersistence-utils. Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iab556850041b592a8c4d515270dfe4bd9547957a
2023-04-17Remove CollectionUtils dependencylukegleeson1-2/+1
Remove dependency creating pipeline issue Issue-ID: CPS-1626 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I64a6019a8196aa98dd3874d08f373c4fb9e3b8c9
2023-04-14Use cascade delete in fragments tabledanielhanrahan4-30/+40
- Add Liquibase change to cascade delete fragments - Remove cascade delete logic from FragmentNativeRepository - Remove unused method deleteFragmentEntity Issue-ID: CPS-1617 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iecf4f3022b402e0328609ddc11a879c83e5d7f30
2023-04-14Remove @NotNull from repository methodsdanielhanrahan4-29/+23
Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I55312bdc5454ed9ca60441b3968d19a61028eb66
2023-04-14Add OR operator to cps-pathRudrangi Anupriya1-16/+22
Issue-ID: CPS-1215 Change-Id: I91fdf5bddcc4fc12a8cf9dbce75f77c832c55871 Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-04-13Migrate query tests to integration-test module #6danielhanrahan2-178/+0
- Remove old tests and broken test data, where fragments in ANCHOR-005 have parent fragments in ANCHOR-004 - Migrate tests to new test framework using bookstore model - Add two bookstore anchors to test querying across both - Add bookstore data to another test dataspace, to verify querying across anchors should be limited to one dataspace - Lower minimum module coverage, since cps-ri tests are now in integration-test - @Ignore failing tests for now Issue-ID: CPS-1595 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I808de288961cb84a486052f9e1dc8fed5f2afe03
2023-04-12Migrate query tests to integration-test module #5danielhanrahan1-17/+0
- Migrate query tests for composite keys to integration-test - Add addresses with composite key to bookstore model Issue-ID: CPS-1597 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I6f8dfc2c44ae6ba3bac1cce804841671ef5a009d
2023-04-12Migrate query tests to integration-test module #3danielhanrahan1-18/+0
Issue-ID: CPS-1597 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I1d92355f272271424a7014057d251cf88eac0203
2023-04-12Merge "Migrate query tests to integration-test module #2"Sourabh Sourabh1-26/+0
2023-04-11Performance tests for getDataNodes and queryDataNodesdanielhanrahan1-58/+0
- Generate openroadm data from a single innerNode.json template - Double the number of openroadm device nodes (25 -> 50) in tests - Add new performance tests for getDataNodes and queryDataNodes to integration-test module, using openroadm and bookstore data - Remove old performance tests from cps-ri Issue-ID: CPS-1524 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Id9ec2a86d984d6c50c9ae6093e7a62729cb851da
2023-04-11Migrate query tests to integration-test module #2danielhanrahan1-26/+0
- Migrate some query tests to integration-test - Edit bookstore model to add integer leaf-list 'editions' - Lower cps-ri code coverage Issue-ID: CPS-1597 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: If50bf15ad7d7f147448f6b60d464efc1cdc91005
2023-04-11Merge "Migrate query tests to integration-test module #1"Toine Siebelink1-50/+1
2023-04-11Migrate query tests to integration-test module #1danielhanrahan1-50/+1
Issue-ID: CPS-1597 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I0403641e2e5293571c61a58aa2b67b144cf68ac4
2023-04-11Populate Dataspace field of DataNodedanielhanrahan1-0/+1
Issue-ID: CPS-1603 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I85aca8bccc28d36aa4cefa0a32cb9bdf2156618a
2023-04-05Migrate query tests to integration-test module #4danielhanrahan1-33/+0
Issue-ID: CPS-1597 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Id3b4a3829b6f9aec1a649ed0001d7c11db91ccfe
2023-03-30Update performance test timingsdanielhanrahan2-28/+28
Issue-ID: CPS-1524 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iaa8e9a0ca3170a2b487df312834245d609fdfe13
2023-03-21Query data nodes across all anchors under one dataspacerajesh.kumar9-10/+150
Issue-ID: CPS-1396 Change-ID: I73f97f986a817d423f93a8d922dcd9647b1412ab Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2023-03-14Reduce dataspace/anchor lookups in CpsDataServicedanielhanrahan1-1/+1
- Use Anchor object instead of names in processDataUpdatedEventAsync - Use Anchor object instead of names in buildDataNodes - Avoid unnecessary ArrayList copy in updateDataNodesAndDescendants Issue-ID: CPS-1536 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I46936a655a3d151357e55b68e1c4161e07100e84
2023-03-13Fetch fragment entities using recursive SQL querydanielhanrahan8-155/+81
- Add SQL query that can fetch needed fragments to any given depth - Update getFragmentEntities to use new query - Remove now unused FragmentRepositoryMultiPathQuery - Remove unused TempTableCreator method - Result: getDataNodesForMultipleXpaths is up to 10 times faster Issue-ID: CPS-1525 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I07cbc9da5ab994ce7e0c2b02d7ca05089f05dab0
2023-03-13Merge "Reduce dataspace/anchor lookups in CpsDataPersistenceService"Toine Siebelink4-194/+110
2023-03-13Merge "Add new metrics to the get data node method"Toine Siebelink1-1/+8
2023-03-13Reduce dataspace/anchor lookups in CpsDataPersistenceServicedanielhanrahan4-195/+111
- Remove unneeded calls to DataspaceRepository::getByName - Remove unneeded calls to AnchorRepository::getByDataspaceAndName - Refactor FragmentRepository Issue-ID: CPS-1536 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2121f6247070ee4a7c000e06ec66a6278b758540
2023-03-13Make single deleteDataNode use plural deleteDataNodesdanielhanrahan3-90/+49
- Make deleteDataNode and deleteListDataNode call deleteDataNodes - Add onlySupportListDeletion option to deleteDataNodes to support original deleteListDataNode behaviour - Allow delete root xpath in deleteDataNodes - Fix incorrect use of PathParsingException in deleteDataNode - Update performance tests timings Issue-ID: CPS-1523 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I92c3c3ce606a5ab2cb8e6779d1ee0f9853529982
2023-03-09Merge "Create Base and Sample Performance Integration Tests"Sourabh Sourabh2-33/+33
2023-03-09Create Base and Sample Performance Integration TestsToineSiebelink2-33/+33
- added data (folders) for different models and json data - added base and data for cps-ncmp (registry) specific test cases - integrated Ahila's (large) openroadm model and data (corrected version provided by Lee Anjella) - created profiles to exclude/include *PerfTest - fixed dependency test check - included integration test for overall coverage - increased margings in existing perf test to reduce nordix intermitten failures Issue-ID: CPS-1516 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ia82826f610636c14aa7e8939b385c278e5039817
2023-03-09Add new metrics to the get data node methodkissand1-1/+8
Issue-ID: CPS-1509 Change-Id: I2eded9a23b56f5263591c710c7a707cf281f93d7 Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2023-03-08Add index to schema_set_yang_resources tabledanielhanrahan2-2/+19
Currently, the database is sequentially scanning when deleting from schema_set_yang_resources table as there are no indexes on the table. Adding an index doubles performance of NCMP CM-handle deregistration. Issue-ID: CPS-1535 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia6ac7ef95d1662f49a77fc72112f1e29a67079ff
2023-03-07Drop temp table when transaction finishesdanielhanrahan1-1/+2
By default, temp tables are persisted until the end of the session. This causes "Out of shared memory" errors in the database, which then auto-vacuums the orphaned temp tables to free memory. To fix this, we drop temp tables when the transaction finishes, using ON COMMIT DROP. Issue-ID: CPS-1533 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iac47d85e9621531a4a95b99c4becc8cc95242721
2023-03-01Add DataNodeNotFoundException to deleteDataNodesdanielhanrahan4-25/+50
Current implementation of NCMP handle de-registration relies on DataNodeNotFoundException being thrown to report errors. - Make deleteDataNodes throw DataNodeFoundExceptionBatch - Update performance test timings Issue-ID: CPS-1481 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ib833bdb4a23d24e1784bdaf4e5e5e8a9acb41c54
2023-02-28Skip deleting list xpaths that are list elementsdanielhanrahan2-9/+12
List elements cannot be lists. Filtering list elements before trying to delete lists by xpaths doubles performance when deleting list elements (which doubles performance of CM handle de-registration). Issue-ID: CPS-1511 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ieb6002212d006396d468f27f853708b5aa1e31f2
2023-02-24Improve performance for update data nodesdanielhanrahan3-52/+72
- Extract method getFragmentEntities from getDataNodes - Remove unused code from getFragmentEntity - Use bulk getFragmentEntities in updateDataNodesAndDescendants - Update performance test timings Issue-ID: CPS-1504 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Icd2c6a0e6009e152de43090cbc23a21349703da2
2023-02-22Add performance tests for update data nodesdanielhanrahan2-8/+52
Issue-ID: CPS-1504 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I1356e4a67a40d03cd71a98dad1571583229f414d
2023-02-21Improve performance of deleteDataNodes SQLdanielhanrahan2-43/+46
- Use SQL IN operator instead of temp table when deleting nodes - Use Postgresql LIKE ANY array operator when deleting lists - Update delete perf test timings - Refactor adding cascade delete constraint Issue-ID: CPS-1502 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic90b867e7c71ec1981f05a9122322ece84dd8bde
2023-02-21Merge "Update performance test timings"Toine Siebelink2-23/+23
2023-02-20Merge "FragmentEntity stuck in memory on "Already exists" error"Toine Siebelink1-0/+2
2023-02-20Update performance test timingsdanielhanrahan2-23/+23
- Tighten delete perf test timings - Fix timings for getDataNodes Issue-ID: CPS-1502 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ib89a70ef90b1b30e2720e48ebfb66650a2fafa12
2023-02-20Merge "CPS-1401 Implement V2 of GET Data Node API"Luke Gleeson6-91/+114
2023-02-17CPS-1401 Implement V2 of GET Data Node APIarpitsingh6-91/+114
- Modified the GET Data Node API so it returns all the data nodes when xpath set to root "/" - Fragment Repository now returns a collection of Fragment Entities - Instead of returning only the first Fragment Entity now all fragment entities are returned when xpath is set to root - The Fragemnt Entities are further processed to a Collection of Data Nodes. As opposed to singular Data Node in current implementation. - Finally the DataRestController also returns a Collection of Data Nodes when xpath is set to root and valid data is present - Response body changed from JSON object to JSON Array. - Exception handling for invalid xpath and non-existing xpath is now done separately at persistence layer. - Refactored code against CPS-1422 - Deprecated getDataNode method from Service and Persistence layer - Modified V1 of Get Data Node API to use the getDataNodes method and get the first data node from the collection returned. - Modified NCMP to use getDataNodes method - NCMP still does not support multiple data nodes. It retrieves the first data node from the collection returned by getDataNodes Signed-off-by: arpitsingh <as00745003@techmahindra.com> Change-Id: I494a5740a53f65376d135fcb9f1e2e8900a2803e
2023-02-16FragmentEntity stuck in memory on "Already exists" errorkissand1-0/+2
- exclude childFragments from toString method Issue-ID: CPS-1493 Change-Id: I3d0c7ef601f06ed36265d88cadb6fb0ab0cb6ea2 Signed-off-by: kissand <andras.zoltan.kiss@est.tech>
2023-02-16Improve batch delete schemasets performancedanielhanrahan7-12/+90
- Bulk delete anchors and datanodes associated with schemasets. Improves de-registration performance by approx 10% Issue-ID: CPS-1423 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie38e8b4c64356bf5935d8c7a5d3f5bfa73fb1714
2023-02-15Bulk delete schemasets in CM handle deregistrationdanielhanrahan5-7/+61
- Batch delete schema sets in single query - Call deleteUnusedYangResourceModules once per batch, not per CM handle - Results for deregistering 10k: 14 mins before; 6 mins after Issue-ID: CPS-1423 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia3a86a0dc88677323e2f386253a99022a7f02603