aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2023-06-15Release cps3.3.2jenkins-releng18-18/+18
2023-06-15Update release docsdanielhanrahan2-55/+44
- Also synched up the API changes in openapi yml file - Step 1 and 2 of the release process Issue-ID: CPS-1703 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Id47822737df3b89dffddb90413a53cac2b7ced6a
2023-06-14Lower memory usage in FragmentRepositorydanielhanrahan7-239/+175
Avoid using Spring Data "interface projection" in FragmentRepository. The use of FragmentExtract in FragmentRepository is causing an overhead of around 5 kilobytes per fragment, which is leading to abnormally high memory usage when queries return a large number of nodes. For example, around 250MB of additional memory is needlessly used when fetching 50,000 datanodes. - Remove FragmentExtract interface and FragmentEntityArranger class. - Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper to fetch FragmentEntity descendants in a single SQL query. - Many CpsDataService operations have memory reductions: - queryDataNodes - getDataNodesForMultipleXpaths - updateDataNodesAndDescendants - updateNodeLeaves - and any NCMP methods using the above. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c
2023-06-13Merge "Move integration test (DataService)"Toine Siebelink8-820/+322
2023-06-12Fix for recursive SQL returning extra level of descendantsdanielhanrahan1-2/+2
Recursive SQL queries are incorrectly returning maxDepth + 1 levels of fragments due to incorrect comparison in the WHERE clause. This code is used by getDataNodes and queryDataNodes to prefetch the descendants before converting to datanodes. Prefetching extra descendants causes high memory usage. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6
2023-06-12Move integration test (DataService)ToineSiebelink8-820/+322
- Moved all Data Intergation Test - Added/improved some module Moudle Servcie test too - Removed some unused (impossible) code-paths in prod code - Replaced null's with meaningful strings (in edge case scenarios) - Replaced @Shared with static (shorter) in some test Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ib4b1ec1706ace7e3f5ac03d85177a6f7d4f6b8d0
2023-06-07Agreed outstanding comments including package refactoringhalil.cakal11-33/+55
- Change package of AVC Subscription relevent codes into their place - Change package of Subscription Event Cache config inot its place - Add more branches for subscription outcome mapper - Add more branches for subscription event response consumer - Change unit test method params in order not to use deprecated methods Issue-ID: CPS-1730 Change-Id: Ieda587d5be318db8360d52d49dc38d7ce3dd85cd Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-07Fix some code smellshalil.cakal2-7/+7
Issue-ID: CPS-1507 Change-Id: Ic11ccbb37538242cb65ab1dbdde4fc77d9f94bdf Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-07Merge "NCMP : forward bulk response messages to client topic"Sourabh Sourabh2-8/+8
2023-06-06NCMP : forward bulk response messages to client topicraviteja.karumuri2-8/+8
# Fixing the avc subscription event is not consuming even there is a record published on to the topic. Issue-ID: CPS-1557 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: If09fd1849f467785141cc56639839ddda9f2c0de
2023-06-06Merge "<,> operators document"Luke Gleeson2-2/+5
2023-06-01Subscription Create Event Outcome Kafka Parthalil.cakal25-123/+829
- Add subscription event outcome schema with java type for pojos - Add subscription event outcome json for testing - Add mapper to convert subscription response to event outcome - Add a bean to handle subscription response outcome tasks - Change response consumer to publish outcome for client app - Change response timeout task to publish outcome for client app - Change subscription persistance to read datanodes - Add helper to extract cm handle to status mapping from data nodes event - Fix code smells Issue-ID: CPS-1507 Change-Id: I70195073490f456f014e53c1f59d1b6761d18cd4 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-01NCMP : forward bulk response messages to client topicraviteja.karumuri1-4/+9
# Fixing the NullPointer Exception if the 'eventType' header not availabe. # eventType header is a mandatory header but still we are doing null check for supporting old events (AsyncResponseEvent is not moved to separate kafka headers) Issue-ID: CPS-1557 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Ie7923d0e2674402fa36cb28cde966575b899cedb
2023-06-01<,> operators documentRudrangi Anupriya2-2/+5
Issue-ID: CPS-1273 Change-Id: Ibbbbe90a28c3711ef211136cd8f8c0f265a51537 Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-05-31Merge "Add <,> operators support to cps-path"Luke Gleeson8-54/+194
2023-05-30Add <,> operators support to cps-pathRudrangi Anupriya8-54/+194
Issue-ID: CPS-1273 Change-Id: I5d562463b9a49abfe0436047a637857d10596fff Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-05-30Merge "Improve error scenarios SubscriptionEventForwarder"Priyank Maheshwari4-7/+51
2023-05-29Merge "NCMP: Update existing Batch endpoint (Moving url param into rest body)"Toine Siebelink36-329/+1030
2023-05-26Improve error scenarios SubscriptionEventForwarderlukegleeson4-7/+51
- Added placeholder for emtpy outcome response when no cmhandles match targets Issue-ID: CPS-1564 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: Ia04bb656c639946d259da779971ab29b6e91acea
2023-05-25NCMP: Update existing Batch endpoint (Moving url param into rest body)sourabh_sourabh36-329/+1030
- NCMP batch endpoint is updated to accept details into request payload. - Removed unused code of previous impl. Issue-ID: CPS-1635 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ic290b750557da06b861c5a4a9bb12debc495ec2e Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-05-24Avc Event compliant with RFC8641mpriyank2-6/+126
- The event section of the payload will be now in the format of RFC8641. - Once this event is updated , the corresponding code in the dmi plugin will be changed - Schema is present in cps-ncmp codebase so we will create a release and then it will be used in the dmi plugin Issue-ID: CPS-1668 Change-Id: Ibd811f318658d8b510b42575fad5d21357af9047 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-05-23NCMP : forward bulk response messages to client topicraviteja.karumuri7-11/+401
Issue-ID: CPS-1557 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: I0ea040e98987f992f46105afea0a171c4031d64f
2023-05-22Merge "LcmEvent to have header now"Luke Gleeson14-46/+252
2023-05-17Entity ID types do not match types in database definitionToineSiebelink18-39/+89
As Liquibase is used, JPA does not manage database definition. Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long) IDs. Other entities have INTEGER IDs. - Change AnchorEntity/AnchorRepository ID type to Long - Change YangResourceEntity/YangResourceRepository and ModuleReferenceRepository ID to Integer - Modify datatypes of foreign key references in liquibase changelog to use correct types for dataspace_id, schema_set_id, yang_resource_id - Update test liquibase changelog Issue-ID: CPS-1676 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2023-05-17Merge "Change dmi plugin version to 1.4.0-SNAPHOST"Luke Gleeson1-1/+1
2023-05-17Change dmi plugin version to 1.4.0-SNAPHOSThalil.cakal1-1/+1
Issue-ID: CPS-1507 Change-Id: Ic812bd1be455d9df663e018383dd1d5c15359362 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-05-17Move integration test for adminServiceToineSiebelink4-269/+48
- finalized moving of admin service interation test (just some scenarios were missing) - deleted old test class (code coverage in RI module dropped) - line/branch coverage now 100/100 (was 97/91) - small refactoring; renaming and removed unnecessary check Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ic683548ea5e1e4e252f257c0f1034c5cf76e498d
2023-05-16LcmEvent to have header nowmpriyank14-46/+252
- Introduce LcmEventHeader as per our agreement with stakeholders - The version remains v1 as we just add the header and dont change anything in the existing event payload. - Later we will remove the header fields from the event payload. - tests modification and little code refractor for dmi data avc as well Issue-ID: CPS-1695 Change-Id: Ibef1138a6d0cc7ffec50b4c201a4d3417b99e27e Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-05-16Fixing minor compilationwaqas.ikram5-8/+8
- Complilation issues while importing project in eclipse - Fixing CSIT Change-Id: Idb85b08066d27a7f2a9b297d1580915df2b67541 Issue-ID: CPS-1672 Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
2023-05-12Remove dataspace_id column from Fragment tabledanielhanrahan13-123/+247
- Remove dataspace_id column from fragment table and drop associated index, foreign key constraint and uniqueness constaint. - Add uniqueness constraint using only fragment xpath and anchor_id. - Add not-null constraint to anchor_id. - Update code and test data to remove references to dataspace_id. - Repopulate the dataspace_id in Liquibase rollback (verified with manual testing of update/rollback). Issue-ID: CPS-1677 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I467cccba25ac5d884ec790064ca58150281b7e1d
2023-05-11Move integration test for moduleServiceToineSiebelink6-366/+304
- all modules service interation test moved - added some aditional scenarios to increase coverage - removed unused method(s) - remove javdoc on private methods - some minor refactoring like replace 'var' with correct type Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I03ff1f3562cfe38318e8b9af81be47a1fe667da2
2023-05-11Subscription Create Event Outcome Database Parthalil.cakal9-19/+225
- Add mapper to convert subscription response to Yang model - Add update operation to update Yang model into database - Change Subscription persistence to support both save and update operations Issue-ID: CPS-1669 Change-Id: I40cab1052ada5846945c67cac8640c393358e988 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-05-10Merge "DMI Data AVC to use kafka headers"Toine Siebelink9-61/+121
2023-05-10DMI Data AVC to use kafka headersmpriyank9-61/+121
- POC done keeping AvcEvent schema in mind. - Approach to have header schema per event schema. - Moved the header information from AvcEvent to separate AvcEventHeader schema. - Added Jsr303 annotation support for required field check Issue-ID: CPS-1671 Change-Id: I2e4f969e8ca4f6282d1b9aa5fd52d16174a26084 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-05-10Move integration test (DataService)ToineSiebelink6-26/+73
- dropped ri module coverage to alow for move - removed performance tests (already better tests in intergration module) - added 'save' iue cases in new integration-test - removed no longer used version for save single data node (todo: remove old tests for similar functionality, will wait until new test cover same) Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I4c2e788e0a313938a923cd1e7e52584d739ec6f4
2023-05-10Merge "Change default use of Model Loader to TRUE"Priyank Maheshwari2-2/+2
2023-05-09Migrate cps-ri performance tests to integration-test moduledanielhanrahan11-526/+264
- Migrate update and delete data service tests, using openroadm model - Move module service tests to integration-test module - Update performance test timings - Remove all performance-related files from cps-ri - Remove performance profile from cps-ri pom.xml Issue-ID: CPS-1687 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Id9d864f8cab0377cb37c7967943d738748e1b6c5
2023-05-09Merge "Support for Patch across multiple data nodes"Toine Siebelink11-98/+155
2023-05-09Support for Patch across multiple data nodes--global11-98/+155
- Added new method updateMultipleDataLeaves to perform Update on multiple data nodes - Deprecated singular method of update data node(updateDataLeaves) - Refactored code where singular version was used - Updated release notes Issue-ID: CPS-1006 Signed-off-by: <as00745003@techmahindra.com> Change-Id: If67280e2dd3ad566de9a8217489f168415e624bc
2023-05-09NCMP: Delete DatastoreType enum from cps-ncmp-restsourabh_sourabh14-74/+34
- Delete duplicate enum of datastore - Use datastore from cps-ncmp-service Issue-ID: CPS-1635 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I36484d6a327c81d81741b18a932fc6a1febfc520
2023-05-09Change default use of Model Loader to TRUEemaclee2-2/+2
Issue-Id: CPS-1688 Signed-off-by: emaclee <lee.anjella.macabuhay@est.tech> Change-Id: I18bb5e3ad45ccdf5f3a1e88dd5a34dc830e06f4a
2023-05-08Merge "Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)"Sourabh Sourabh4-108/+46
2023-05-08Merge "Fix and refactor query across anchors (CPS-1664 #3)"Sourabh Sourabh10-75/+156
2023-05-08Merge "[CPS] RI: Code Refactoring # Replace '[\s\S]' to '.' as it is same as ↵Toine Siebelink5-43/+17
'.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method."
2023-05-08Merge "Don't handle ancestor-axis using regex quickfind (CPS-1664 #2)"Sourabh Sourabh2-10/+6
2023-05-05[CPS] RI: Code Refactoringraviteja.karumuri5-43/+17
# Replace '[\s\S]' to '.' as it is same as '.' while grouping in regex # Updated the groovy files to add 'assert' and to replace deprecated method # Removed deprecated 'storeDataNode' method. Issue-ID: CPS-1594 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Iaa5f8b0cc8d66473c993060a8fc5a54e37e373b1
2023-05-04Fragment Entity does not represent the correct relationship with Anchordanielhanrahan1-2/+1
FragmentEntity incorrectly has OneToOne relationship to AnchorEntity, instead of ManyToOne. The reason this has not caused errors so far is that Liquibase is being used to generate database definition, not JPA. Issue-ID: CPS-863 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8d3c5dc19193a84fa59526de7f2cbab7ab20a8fc
2023-05-04Use recursive SQL to fetch descendants in CpsPath queries (CPS-1664 #4)danielhanrahan4-108/+46
- Add recursive SQL method to fetch descendants from queries. This changes worst-case complexity from quadratic to linear, resulting in extremely large performance increase for large number of datanodes. - Remove RegexQuickFind algorithm as it is no longer faster. - Updated query performance test timings Issue-ID: CPS-1664 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: If5f0b54a88af1cb681006bbeca7043345dcdc8da
2023-05-04Fix and refactor query across anchors (CPS-1664 #3)danielhanrahan10-75/+156
- Fix CPS-1580: Query Across All Anchors Does NOT Filter on Dataspace - Fix CPS-1582: NullPointerException in queryDataNodesAcrossAnchors - Improve performance of queryDataNodesAcrossAnchors - Refactor queryDataNodes - Refactor FragmentQueryBuilder Issue-ID: CPS-1580 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I8b83d4c580280d8719ab2ac33f3914f53e798774
2023-05-04Don't handle ancestor-axis using regex quickfind (CPS-1664 #2)danielhanrahan2-10/+6
This is a preparatory patch to make refactoring easier in CPS-1664 #3 Issue-ID: CPS-1664 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I1f94fef137ae41f7dedc585d8314f13493885d1b