summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2023-06-21Merge "Remove irrelevant groovy class that tests nothing"Toine Siebelink1-57/+0
2023-06-21Improve handling of legacy eventsToineSiebelink2-19/+62
Legacy (non-cloud) events would cause a NPE in the event filter Although the event woudl still be ignored it would lead to uncessary error logging This fix addressed this issue using a trace level message instead (also some improvements in the associated test) Issue-ID: CPS-1724 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ibf71b13e4a47aaf705e32df5fe50cf41de5f558f
2023-06-21Remove irrelevant groovy class that tests nothinghalil.cakal1-57/+0
Issue-ID: CPS-1741 Change-Id: If27a74e2496b952eb4803bbabb8363400c57915a Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-21Merge "Patch # 3: Data operation response event (NCMP → Client App) to ↵Toine Siebelink10-74/+151
comply with CloudEvents"
2023-06-21Add performance tests for updateDataLeavesdanielhanrahan1-2/+31
Issue-ID: CPS-1675 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I4dcd2ff0d0f6c3acf7373357da0ef7217a691573
2023-06-21Patch # 3: Data operation response event (NCMP → Client App) to comply ↵sourabh_sourabh10-74/+151
with CloudEvents - Modified data operation record strategy to consume cloud event. - Modified NCMP data operation event consumer to read cloud event header. (prefixed with ce_) - Modified event publisher to support legacy and cloud event based on event type (if legacy event use legacy kafka template else cloud kafka template). - Introduced a new method onto json object mapper to convert json object to bytes. - Modified data operation consumer spec to produce a cloud event and validate it. - Added Kafka Integration Test (for filtering) Issue-ID: CPS-1724 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ide701b1ff952f57413cb4e4aa0d55c08753f0298 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-06-21publishLcmEventBatchAsynchronously is not asyncdanielhanrahan4-48/+85
Spring @Async has a limitation where calling the async method from within the same class won't work, as the method needs to be proxied, and calling from the same class bypasses the proxy and calls the underlying method directly, thus running synchronously. Issue-ID: CPS-1749 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7f198487dc18b359654dc38b5cf8fd600d33e189
2023-06-21Apply latest ReedTheDoc config rulesToineSiebelink1-6/+4
Issue-ID: CPS-1757 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Icdf5fe5b5b1d6628bed786a1d99816e7990302cc
2023-06-20Refactored code to enable its utilization in Junitwaqas.ikram11-329/+219
tests for the client Change-Id: I847ae0f48444af907039bfddb9879581003c0f35 Issue-ID: CPS-1751 Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
2023-06-20Merge "Fix test-deregistration script"Toine Siebelink3-23/+25
2023-06-20Merge "DMI Data AVC to cloud events"Luke Gleeson7-146/+55
2023-06-20Merge "Patch # 1: Data operation response event (NCMP → Client App) to ↵Toine Siebelink6-129/+58
comply with CloudEvents"
2023-06-20Merge "Replace deprecated WebSecurityConfigurerAdapter"Toine Siebelink2-13/+41
2023-06-19Fix test-deregistration scriptdanielhanrahan3-23/+25
- Fix issue where script breaks with newer version of docker-compose caused by profile names being omitted during `docker-compose down` - Use main .gitignore file to ignore metrics-reports directory - Fix Intelli-j/shellcheck warnings (mostly variables needing quotes) - Script prints summary report at exit Issue-ID: CPS-1552 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia51e51c1c199de8cc2f4878bfb04cb09fbb0b0e7
2023-06-19Update performance test timingsdanielhanrahan4-25/+25
The fixes to lower memory usage in FragmentRepository have positively affected read performance, so tests must be updated. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2c8b882f56fb8c1eaefe5b802da1d308d16b1f02
2023-06-19Patch # 1: Data operation response event (NCMP → Client App) to comply ↵sourabh_sourabh6-129/+58
with CloudEvents - Removed header definitions (since we now use CloudEvents) - Used 'dataOperation' instead of batch where appropriate. - Modified test json Issue-ID: CPS-1724 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ic0f65297b944adf9cf5f3c2cbec679a031a675ec Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-06-19Merge "Code to persist or update the yang model subscription into db"Priyank Maheshwari7-54/+220
2023-06-19Code to persist or update the yang model subscription into dbhalil.cakal7-54/+220
- Change subscription persistence for subscription registry to handle adding or updating cm handles into db Issue-ID: CPS-1735 Change-Id: Icae609e810b2a097a4a174de1691892526f6b5c0 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-19DMI Data AVC to cloud eventsmpriyank7-146/+55
- DMI Data AVC to be consumed as CloudEvents now - Removed the schema header as it is taken care by cloudevent headers - Implemented naming and packaging comments on the schema - Test cases refactoring Issue-ID: CPS-1719 Change-Id: I9864f90446720fe903fb3c1502d86035d886751d Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-06-19Move integration test (clean-up, last phase)ToineSiebelink17-640/+115
- Moved session manager test - Improved schemaset concurrency test (retry testing) - Cleaned up all ri (container based) integration test (setup) - Applied some groovy best practice where needed - internal ri module cover now down to 29%, covred by integration instead with - Line coverage up by 41 lines to 99% (was 97%) - Branch coverage up by 3 branches to 96% (was 93%) Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ifb77a053e5a5db62a3f6a32ae60a3a8b10918efd
2023-06-15Merge "Patch #2 : Introduce kafka template for cloud events"Toine Siebelink12-38/+269
2023-06-15Patch #2 : Introduce kafka template for cloud eventssourabh_sourabh12-38/+269
- Introduced a new cloud kafka template for cloud events that reads it's configuration from application.yml - Kept legacy kafka template for backward compatibility utill all cps events moved to cloud event comply. - Modified application.yml producer and consumer value deserializer properties to support cloud events. - Added new cloudevents-bom used into cps-ncmp-service pom. - For the time being we will have 2 kafkatemplates (legacyEventKafkaTemplate, cloudEventKafkaTemplate) into EventsPublisher until we fully move to cloudevents for all events. Once all cps events will be cloud event compy, we have TODO task where Deprecated: legacyKafkaEventTemplate will be removed with its java configuration file KafkaTemplateConfig. Issue-ID: CPS-1724 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I78c15bd480db063b89c6630c46c2d3a328b4fae4 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-06-15Bump CPS version to 3.3.3-SNAPSHOTdanielhanrahan20-20/+47
- Update version properties as well as we prepare for the next release. Issue-ID: CPS-1750 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I0626eee7834499d2fa96812fb416ce7daebf0529
2023-06-15Add container yaml for 3.3.2danielhanrahan1-0/+8
- Step 6 of the release process Issue-ID: CPS-1703 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I58e54ecd17878d3b9b6bc69ac9d254e454bd68b2
2023-06-15Add maven stage yaml for 3.3.2danielhanrahan1-0/+4
- Step 5 of the release process Issue-ID: CPS-1703 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I105dedd6b1a634ca8ac4409ed682f6a10d1b018a
2023-06-15Replace deprecated WebSecurityConfigurerAdapteregernug2-13/+41
WebSecurityConfigurerAdapter has been deprecated in Spring 2.7. Change needed to use SecurityFilterChain and InMemoryUserDetailsManager. Note: WebMvcTest does not autoimport configurations that use beans. Issue-Id: CPS-1742 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: I100884f10beaac6ce87e603302280aeb78559017
2023-06-15Update release docsdanielhanrahan2-55/+44
- Also synched up the API changes in openapi yml file - Step 1 and 2 of the release process Issue-ID: CPS-1703 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Id47822737df3b89dffddb90413a53cac2b7ced6a
2023-06-14Lower memory usage in FragmentRepositorydanielhanrahan7-239/+175
Avoid using Spring Data "interface projection" in FragmentRepository. The use of FragmentExtract in FragmentRepository is causing an overhead of around 5 kilobytes per fragment, which is leading to abnormally high memory usage when queries return a large number of nodes. For example, around 250MB of additional memory is needlessly used when fetching 50,000 datanodes. - Remove FragmentExtract interface and FragmentEntityArranger class. - Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper to fetch FragmentEntity descendants in a single SQL query. - Many CpsDataService operations have memory reductions: - queryDataNodes - getDataNodesForMultipleXpaths - updateDataNodesAndDescendants - updateNodeLeaves - and any NCMP methods using the above. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c
2023-06-13Merge "Move integration test (DataService)"Toine Siebelink8-820/+322
2023-06-12Fix for recursive SQL returning extra level of descendantsdanielhanrahan1-2/+2
Recursive SQL queries are incorrectly returning maxDepth + 1 levels of fragments due to incorrect comparison in the WHERE clause. This code is used by getDataNodes and queryDataNodes to prefetch the descendants before converting to datanodes. Prefetching extra descendants causes high memory usage. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6
2023-06-12Move integration test (DataService)ToineSiebelink8-820/+322
- Moved all Data Intergation Test - Added/improved some module Moudle Servcie test too - Removed some unused (impossible) code-paths in prod code - Replaced null's with meaningful strings (in edge case scenarios) - Replaced @Shared with static (shorter) in some test Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ib4b1ec1706ace7e3f5ac03d85177a6f7d4f6b8d0
2023-06-07Agreed outstanding comments including package refactoringhalil.cakal11-33/+55
- Change package of AVC Subscription relevent codes into their place - Change package of Subscription Event Cache config inot its place - Add more branches for subscription outcome mapper - Add more branches for subscription event response consumer - Change unit test method params in order not to use deprecated methods Issue-ID: CPS-1730 Change-Id: Ieda587d5be318db8360d52d49dc38d7ce3dd85cd Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-07Fix some code smellshalil.cakal2-7/+7
Issue-ID: CPS-1507 Change-Id: Ic11ccbb37538242cb65ab1dbdde4fc77d9f94bdf Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-07Merge "NCMP : forward bulk response messages to client topic"Sourabh Sourabh2-8/+8
2023-06-06NCMP : forward bulk response messages to client topicraviteja.karumuri2-8/+8
# Fixing the avc subscription event is not consuming even there is a record published on to the topic. Issue-ID: CPS-1557 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: If09fd1849f467785141cc56639839ddda9f2c0de
2023-06-06Merge "<,> operators document"Luke Gleeson2-2/+5
2023-06-01Subscription Create Event Outcome Kafka Parthalil.cakal25-123/+829
- Add subscription event outcome schema with java type for pojos - Add subscription event outcome json for testing - Add mapper to convert subscription response to event outcome - Add a bean to handle subscription response outcome tasks - Change response consumer to publish outcome for client app - Change response timeout task to publish outcome for client app - Change subscription persistance to read datanodes - Add helper to extract cm handle to status mapping from data nodes event - Fix code smells Issue-ID: CPS-1507 Change-Id: I70195073490f456f014e53c1f59d1b6761d18cd4 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-01NCMP : forward bulk response messages to client topicraviteja.karumuri1-4/+9
# Fixing the NullPointer Exception if the 'eventType' header not availabe. # eventType header is a mandatory header but still we are doing null check for supporting old events (AsyncResponseEvent is not moved to separate kafka headers) Issue-ID: CPS-1557 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Ie7923d0e2674402fa36cb28cde966575b899cedb
2023-06-01<,> operators documentRudrangi Anupriya2-2/+5
Issue-ID: CPS-1273 Change-Id: Ibbbbe90a28c3711ef211136cd8f8c0f265a51537 Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-05-31Merge "Add <,> operators support to cps-path"Luke Gleeson8-54/+194
2023-05-30Add <,> operators support to cps-pathRudrangi Anupriya8-54/+194
Issue-ID: CPS-1273 Change-Id: I5d562463b9a49abfe0436047a637857d10596fff Signed-off-by: Rudrangi Anupriya <ra00745022@techmahindra.com>
2023-05-30Merge "Improve error scenarios SubscriptionEventForwarder"Priyank Maheshwari4-7/+51
2023-05-29Merge "NCMP: Update existing Batch endpoint (Moving url param into rest body)"Toine Siebelink36-329/+1030
2023-05-26Improve error scenarios SubscriptionEventForwarderlukegleeson4-7/+51
- Added placeholder for emtpy outcome response when no cmhandles match targets Issue-ID: CPS-1564 Signed-off-by: lukegleeson <luke.gleeson@est.tech> Change-Id: Ia04bb656c639946d259da779971ab29b6e91acea
2023-05-25NCMP: Update existing Batch endpoint (Moving url param into rest body)sourabh_sourabh36-329/+1030
- NCMP batch endpoint is updated to accept details into request payload. - Removed unused code of previous impl. Issue-ID: CPS-1635 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ic290b750557da06b861c5a4a9bb12debc495ec2e Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-05-24Avc Event compliant with RFC8641mpriyank2-6/+126
- The event section of the payload will be now in the format of RFC8641. - Once this event is updated , the corresponding code in the dmi plugin will be changed - Schema is present in cps-ncmp codebase so we will create a release and then it will be used in the dmi plugin Issue-ID: CPS-1668 Change-Id: Ibd811f318658d8b510b42575fad5d21357af9047 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-05-23NCMP : forward bulk response messages to client topicraviteja.karumuri7-11/+401
Issue-ID: CPS-1557 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: I0ea040e98987f992f46105afea0a171c4031d64f
2023-05-22Merge "LcmEvent to have header now"Luke Gleeson14-46/+252
2023-05-17Entity ID types do not match types in database definitionToineSiebelink18-39/+89
As Liquibase is used, JPA does not manage database definition. Within Liquibase changelog, only Anchor and Fragment have BIGINT (Long) IDs. Other entities have INTEGER IDs. - Change AnchorEntity/AnchorRepository ID type to Long - Change YangResourceEntity/YangResourceRepository and ModuleReferenceRepository ID to Integer - Modify datatypes of foreign key references in liquibase changelog to use correct types for dataspace_id, schema_set_id, yang_resource_id - Update test liquibase changelog Issue-ID: CPS-1676 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I26de6712b2e74d166fc5a48b1f8149de6a1e5199 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech>
2023-05-17Merge "Change dmi plugin version to 1.4.0-SNAPHOST"Luke Gleeson1-1/+1