summaryrefslogtreecommitdiffstats
path: root/cps-ri/src/main
AgeCommit message (Collapse)AuthorFilesLines
2023-08-18CPS-Core : Expose a java interface to update schema setsourabh_sourabh2-0/+16
- Exposed an interface to update anchor by schema set name. - New interface is implemented into RI model. - New native query is exposed to update id with given schema set name. - A new integration test is written to test new interface into cps core. Issue-ID: CPS-1800 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ibf44712e11b53cb6673b04b9e3fd864321c90839 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-08-17Update of top-level data node fails with SQL errordanielhanrahan1-1/+1
The error is caused by the fetch-descendants code using ResultSet.getLong which returns a 'long' primitive instead of a 'Long' object. Thus a parent ID of 'NULL' becomes '0', which causes an error during update. To preserve the NULL value, ResultSet.getObject must be used. Issue-ID: CPS-1841 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ida6837a04954cd3c23f0f2faabd7d0712d8ee19d
2023-08-16Fixing sonar issuesdanielhanrahan3-6/+3
Issue-ID: CPS-475 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7623a8301fd4567bf96c605eb26bf6c02b1d3c76
2023-08-03Merge "Support pagination in query across all anchors(ep4)"Luke Gleeson6-21/+149
2023-08-02Support pagination in query across all anchors(ep4)rajesh.kumar6-21/+149
Add pagination query parameters in query across all anchors API pagination parameters (pageIndex and pageSize) are optional default is to query all fragments each pageSize represents number of records(number of anchors) TotalRecords is returned in response header to find number of pages. - If pagination option is provided in request then query number of anchors equal to pageSize. pageIndex is used for setting offset. - return number of records(one anchor per record) as per pagesize and pageSize Issue-ID: CPS-1605 Change-ID: I73f97f986a817d423f93a8d922dcd9647b2504bc Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2023-07-27Allow getDataNodes to read whole lists(ep 1)danielhanrahan2-31/+52
- getDataNodes can now retrieve list data nodes. Issue-ID: CPS-1696 Signed-off-by: arpitsingh <as00745003@techmahindra.com> Change-Id: I320a368d6cb73599d3f7c13fe9b8ab7d0cc69470
2023-07-20Apostrophe handling in CpsPathParserdanielhanrahan7-8/+78
Apostrophe is not currently handled correctly, and having apostrophe in the xpath will lead to various errors. For example, normalizing this xpath works: /path[@name="I'm quoted"] -> /path[@name='I\'m quoted'] However the resulting xpath will throw a PathParsingException if parsed! (Thus path normalization is not idempotent.) - Use '' for escaping apostrophe in single quoted leaf value, to comply with XPath standard (and use "" for escaping in "). - Use Liquibase to make existing data comply with new rules. - Leaf values in data leaves are now unescaped, e.g. "I'm quoted" - Quoting is now consistent for leaf/text/contains conditions. Issue-ID: CPS-1769 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iafc287f738254d7f99706c6bc548091c0ecd5aa0
2023-07-17Allow duplicate leaf names in Cps Path leaf conditiondanielhanrahan1-12/+9
Presently, a query using the same leaf name twice will fail: //books[@price > 10 and @price < 20] It is caused by storing data leaves in a Map. This is fixed by storing data leaves in a List<DataLeaf>. Issue-ID: CPS-1779 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie6990ea5e622cf37e986b720a0a07fb69ce4f03b
2023-07-11Combine alreadyDefinedException classesdanielhanrahan1-7/+7
Issue-ID: CPS-1774 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I52ff9074a9f8188e8635a811b0d2713a97cb4b10
2023-07-10Handle special characters in CpsPath queries (CPS-1760 #2)danielhanrahan1-27/+5
This fixes issues with special characters for CPS-500, CPS-1756, CPS-1758, and CPS-1760. It also improves query performance. - use SQL LIKE instead of regex in Cps Path queries Issue-ID: CPS-1763 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I5c179882bfba71d3b009c60059e9073f46227e7d
2023-07-06Escape SQL LIKE wildcards in queries (CPS-1760 #1)danielhanrahan3-5/+39
If '%' and '_' are used in the contains-condition of a CpsPath query, incorrect results will be returned. For example: /bookstore/categories[contains(@code, "%")] Special characters in the contains-condition value must be escaped. Issue-ID: CPS-1762 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2fdd5a26433d510cd7d6af5b734a6779b537d63d
2023-06-21Improve performance of updateDataLeavesdanielhanrahan1-12/+9
Avoid fetching descendants during batchUpdateDataLeaves, as descendants are not needed: - Remove prefetch descendants step from getFragmentEntities; - Explicitly prefetch descendants in operations requiring it; - Update performance tests (5x faster for batch update). Issue-ID: CPS-1675 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7442a6f799cc0803b3a78f09d1ee53377f24b0b7
2023-06-14Lower memory usage in FragmentRepositorydanielhanrahan5-205/+151
Avoid using Spring Data "interface projection" in FragmentRepository. The use of FragmentExtract in FragmentRepository is causing an overhead of around 5 kilobytes per fragment, which is leading to abnormally high memory usage when queries return a large number of nodes. For example, around 250MB of additional memory is needlessly used when fetching 50,000 datanodes. - Remove FragmentExtract interface and FragmentEntityArranger class. - Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper to fetch FragmentEntity descendants in a single SQL query. - Many CpsDataService operations have memory reductions: - queryDataNodes - getDataNodesForMultipleXpaths - updateDataNodesAndDescendants - updateNodeLeaves - and any NCMP methods using the above. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c
2023-06-13Merge "Move integration test (DataService)"Toine Siebelink2-5/+4
2023-06-12Fix for recursive SQL returning extra level of descendantsdanielhanrahan1-2/+2
Recursive SQL queries are incorrectly returning maxDepth + 1 levels of fragments due to incorrect comparison in the WHERE clause. This code is used by getDataNodes and queryDataNodes to prefetch the descendants before converting to datanodes. Prefetching extra descendants causes high memory usage. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6
2023-06-12Move integration test (DataService)ToineSiebelink2-5/+4
- Moved all Data Intergation Test - Added/improved some module Moudle Servcie test too - Removed some unused (impossible) code-paths in prod code - Replaced null's with meaningful strings (in edge case scenarios) - Replaced @Shared with static (shorter) in some test Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ib4b1ec1706ace7e3f5ac03d85177a6f7d4f6b8d0
2023-05-30Add <,> operators support to cps-pathRudrangi Anupriya1-17/+32
Issue-ID: CPS-1273 Change-Id: I5d562463b9a49abfe0436047a637857d10596fff Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-05-17Entity ID types do not match types in database definitionToineSiebelink17-36/+86
As Liquibase is used, JPA does not manage database definition. Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long) IDs. Other entities have INTEGER IDs. - Change AnchorEntity/AnchorRepository ID type to Long - Change YangResourceEntity/YangResourceRepository and ModuleReferenceRepository ID to Integer - Modify datatypes of foreign key references in liquibase changelog to use correct types for dataspace_id, schema_set_id, yang_resource_id - Update test liquibase changelog Issue-ID: CPS-1676 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2023-05-17Move integration test for adminServiceToineSiebelink1-4/+2
- finalized moving of admin service interation test (just some scenarios were missing) - deleted old test class (code coverage in RI module dropped) - line/branch coverage now 100/100 (was 97/91) - small refactoring; renaming and removed unnecessary check Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ic683548ea5e1e4e252f257c0f1034c5cf76e498d
2023-05-12Remove dataspace_id column from Fragment tabledanielhanrahan8-26/+150
- Remove dataspace_id column from fragment table and drop associated index, foreign key constraint and uniqueness constaint. - Add uniqueness constraint using only fragment xpath and anchor_id. - Add not-null constraint to anchor_id. - Update code and test data to remove references to dataspace_id. - Repopulate the dataspace_id in Liquibase rollback (verified with manual testing of update/rollback). Issue-ID: CPS-1677 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I467cccba25ac5d884ec790064ca58150281b7e1d
2023-05-11Move integration test for moduleServiceToineSiebelink1-41/+23
- all modules service interation test moved - added some aditional scenarios to increase coverage - removed unused method(s) - remove javdoc on private methods - some minor refactoring like replace 'var' with correct type Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I03ff1f3562cfe38318e8b9af81be47a1fe667da2
2023-05-10Move integration test (DataService)ToineSiebelink1-7/+0
- dropped ri module coverage to alow for move - removed performance tests (already better tests in intergration module) - added 'save' iue cases in new integration-test - removed no longer used version for save single data node (todo: remove old tests for similar functionality, will wait until new test cover same) Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I4c2e788e0a313938a923cd1e7e52584d739ec6f4
2023-05-09Support for Patch across multiple data nodes--global2-13/+31
- Added new method updateMultipleDataLeaves to perform Update on multiple data nodes - Deprecated singular method of update data node(updateDataLeaves) - Refactored code where singular version was used - Updated release notes Issue-ID: CPS-1006 Signed-off-by: <as00745003@techmahindra.com> Change-Id: If67280e2dd3ad566de9a8217489f168415e624bc
2023-05-08Merge "Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)"Sourabh Sourabh3-99/+37
2023-05-08Merge "Fix and refactor query across anchors (CPS-1664 #3)"Sourabh Sourabh7-68/+154
2023-05-08Merge "[CPS] RI: Code Refactoring # Replace '[\s\S]' to '.' as it is same as ↵Toine Siebelink1-8/+3
'.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method."
2023-05-08Merge "Don't handle ancestor-axis using regex quickfind (CPS-1664 #2)"Sourabh Sourabh2-10/+6
2023-05-05[CPS] RI: Code Refactoringraviteja.karumuri1-8/+3
# Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method. Issue-ID: CPS-1594 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Iaa5f8b0cc8d66473c993060a8fc5a54e37e373b1
2023-05-04Fragment Entity does not represent the correct relationship with Anchordanielhanrahan1-2/+1
FragmentEntity incorrectly has OneToOne relationship to AnchorEntity, instead of ManyToOne. The reason this has not caused errors so far is that Liquibase is being used to generate database definition, not JPA. Issue-ID: CPS-863 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8d3c5dc19193a84fa59526de7f2cbab7ab20a8fc
2023-05-04Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)danielhanrahan3-99/+37
- Add recursive SQL method to fetch descendants from queries. This changes worst-case complexity from quadratic to linear, resulting in extremely large performance increase for large number of datanodes. - Remove RegexQuickFind algorithm as it is no longer faster. - Updated query performance test timings Issue-ID: CPS-1664 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: If5f0b54a88af1cb681006bbeca7043345dcdc8da
2023-05-04Fix and refactor query across anchors (CPS-1664 #3)danielhanrahan7-68/+154
- Fix CPS-1580: Query Across All Anchors Does NOT Filter on Dataspace - Fix CPS-1582: NullPointerException in queryDataNodesAcrossAnchors - Improve performance of queryDataNodesAcrossAnchors - Refactor queryDataNodes - Refactor FragmentQueryBuilder Issue-ID: CPS-1580 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8b83d4c580280d8719ab2ac33f3914f53e798774
2023-05-04Don't handle ancestor-axis using regex quickfind (CPS-1664 #2)danielhanrahan2-10/+6
This is a preparatory patch to make refactoring easier in CPS-1664 #3 Issue-ID: CPS-1664 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I1f94fef137ae41f7dedc585d8314f13493885d1b
2023-05-04Merge "Sensible equals and hashCode for FragmentEntity (CPS-1664 #1)"Toine Siebelink3-1/+10
2023-05-04Fix for quickfind with descendants incl. list entriesToineSiebelink2-24/+38
- add specialized regex - fixed legacy issues with absolute paths (added ^ in regex's) - split into 2 methods for normal queries and quickfind - fixed order of private methods Issue-ID: CPS-1671 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I785a230cd0e3e0359e3cbb0fa754a21b6e2be7ff
2023-05-04Sensible equals and hashCode for FragmentEntity (CPS-1664 #1)danielhanrahan3-1/+10
Include Anchor and Xpath in equals and hashCode methods for FragmentEntity. (This also requires adding equals and hashCode for AnchorEntity and DataspaceEntity.) The combination of dataspace, anchor, and xpath uniquely identify a fragment/datanode. This allows FragmentEntity objects returned from query across anchors to be stored in Set collections. Performance was observed to be unaffected by the change. Issue-ID: CPS-1664 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2c7e3957e392af36f5230d08c9bbd550f44e7444
2023-05-03canUseRegexQuickfind does not test for contains-conditiondanielhanrahan1-1/+2
This bug causes queries using contains-condition to return wrong results, when INCLUDE_ALL_DESCENDANTS option is set. Issue-ID: CPS-1663 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: If323e0e3b54d8c9a213dffda46897b2efb01fe2a
2023-04-26[CPS] RI, Service: Code Refactoringraviteja.karumuri2-22/+0
# Remove unused method "updateDataNodeAndDescendants" # Remove unused production code Issue-ID: CPS-1594 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Id909e9da3e9181c181dfdfe62f051451d8e488a2
2023-04-24Remove 32K limit from queries with collection parametersdanielhanrahan7-160/+113
SQL queries taking collection parameters currently create a seperate query parameter for each collection element. There is a limit of around 2^15 (32,768) query parameters. Postgres DB natively supports array types, in which the whole array is transmitted in binary as a single parameter. Changing queries to use arrays removes the 32K limit on queries. - Add support for Postgres arrays to queries - Change repository methods to use arrays, and provide overloaded versions taking collection parameters - Update tests to reflect 32K limit being lifted Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I64f2aeaedbe54bfe12e3079cba0f2216759142c3
2023-04-21Merge "Add contains condition support to cps-path"Luke Gleeson1-0/+11
2023-04-21Add contains condition support to cps-pathRudrangi Anupriya1-0/+11
Issue-ID: CPS-1272 Change-Id: Ic81d1322cacc64a8752916324b801d02be47d34f Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-04-20Migrate hibernate-types to hypersistence-utilsdanielhanrahan1-1/+1
There is newer release of the dependency that provides JSONB support. The newer version provides additional features, such as Postgres arrays. It was renamed from hibernate-types to hypersistence-utils. Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iab556850041b592a8c4d515270dfe4bd9547957a
2023-04-17Remove CollectionUtils dependencylukegleeson1-2/+1
Remove dependency creating pipeline issue Issue-ID: CPS-1626 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: I64a6019a8196aa98dd3874d08f373c4fb9e3b8c9
2023-04-14Use cascade delete in fragments tabledanielhanrahan4-30/+40
- Add Liquibase change to cascade delete fragments - Remove cascade delete logic from FragmentNativeRepository - Remove unused method deleteFragmentEntity Issue-ID: CPS-1617 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Iecf4f3022b402e0328609ddc11a879c83e5d7f30
2023-04-14Remove @NotNull from repository methodsdanielhanrahan4-29/+23
Issue-ID: CPS-1573 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I55312bdc5454ed9ca60441b3968d19a61028eb66
2023-04-14Add OR operator to cps-pathRudrangi Anupriya1-16/+22
Issue-ID: CPS-1215 Change-Id: I91fdf5bddcc4fc12a8cf9dbce75f77c832c55871 Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-04-11Populate Dataspace field of DataNodedanielhanrahan1-0/+1
Issue-ID: CPS-1603 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I85aca8bccc28d36aa4cefa0a32cb9bdf2156618a
2023-03-21Query data nodes across all anchors under one dataspacerajesh.kumar6-8/+88
Issue-ID: CPS-1396 Change-ID: I73f97f986a817d423f93a8d922dcd9647b1412ab Signed-off-by: rajesh.kumar <rk00747546@techmahindra.com>
2023-03-14Reduce dataspace/anchor lookups in CpsDataServicedanielhanrahan1-1/+1
- Use Anchor object instead of names in processDataUpdatedEventAsync - Use Anchor object instead of names in buildDataNodes - Avoid unnecessary ArrayList copy in updateDataNodesAndDescendants Issue-ID: CPS-1536 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I46936a655a3d151357e55b68e1c4161e07100e84
2023-03-13Fetch fragment entities using recursive SQL querydanielhanrahan5-131/+39
- Add SQL query that can fetch needed fragments to any given depth - Update getFragmentEntities to use new query - Remove now unused FragmentRepositoryMultiPathQuery - Remove unused TempTableCreator method - Result: getDataNodesForMultipleXpaths is up to 10 times faster Issue-ID: CPS-1525 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I07cbc9da5ab994ce7e0c2b02d7ca05089f05dab0
2023-03-13Merge "Reduce dataspace/anchor lookups in CpsDataPersistenceService"Toine Siebelink2-143/+83