summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2023-06-27NCMP : Handle non-existing and non-ready cm handlessourabh_sourabh21-187/+664
- Modified data operation schema to contains cm handle as steing and response content field name as result. - Added a new common cloud event builder for NCMP to create an event. - Added data operation event creater that uses cloud event builder to create a cloud event. - Introduced a new method onto json object mapper to convert json object to bytes. - Modified EventDateTimeFormatter and added a new method to convert date timestamp into offsetdateTime. - Added a new code into ResourceRequestUtil to identify non-ready cm handle and non-existing cm handle and later publish it as cloud event to tha client given topic. - Introduced CpsApplicationContext to get spring mannaged bean into non spring managed java object. Issue-ID: CPS-1583, CPS-1614 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I24a39d2cb2c54dea25cd2f17e7748e21cd83a088 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-06-27Generate docs using OpenAPI 3.0egernug3-2/+37
Doc generation in OpenAPI 3.0 requires executions for each file being generated. This change adds those executions and updates the dependency to copy these files. Issue-ID:CPS-1745 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: I144b49b0066f798f15408d955dcd914dd2c0bc91
2023-06-26Streamline outcome response for subscription creationhalil.cakal11-53/+182
- Pivot to outcome response generated from db for the corresponding subscription - Fine tuning to wait till the cutoff time for all cm handles that subject to subscription - Update cm handles as rejected if dmi plugin or cm handles does not exist in db - Add more test for missed branches in mappers and response outcome Issue-ID: CPS-1741 Change-Id: I82ab6958e673059977abc7708974d5a2cb13e339 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-26Merge "KafkaListener config for CloudEvents"Sourabh Sourabh4-18/+48
2023-06-26KafkaListener config for CloudEventsmpriyank4-18/+48
- Introduced kafkalistener factory configs to correctly use the configs and convert the incoming events to CloudEvents - Also legacy events to use a separate listener factory - Increased the wait time from 100ms to 300ms to listen to the message in the test Issue-ID: CPS-1764 Change-Id: I8b9fe0ba82ef87d52b7731941ccd3af8ae980109 Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-06-26Change exclude path for mappers into more generalhalil.cakal1-3/+1
Issue-ID: CPS-1741 Change-Id: I5ad92f19d65bbd7577014b6ea2ffbfffd41e0b00 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-26Merge "Adding logic to read response files from"Toine Siebelink6-22/+220
2023-06-26Merge "#2 NCMP : Replacing the word 'Batch' as 'DataOperation'."Toine Siebelink8-74/+76
2023-06-26Merge "Chsnge to OpenApi 3.0"Toine Siebelink11-51/+60
2023-06-26Adding logic to read response files fromwaqas.ikram6-22/+220
filesystem for client integration tests Issue-ID: CPS-1751 Change-Id: I62e24d09474184716c56149a06deae4500b7b286 Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
2023-06-26Chsnge to OpenApi 3.0egernug11-51/+60
In preparation for the Java 17 and Springboot 3 migrations Swagger Codegen has to upgrade to OpenApi 3.0 This effects cps-rest and cps-ncmp-rest Issue-Id: CPS-1745 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: I18b8e9ca26d0123009702c2474efbfbeec6d4c44
2023-06-23#2 NCMP : Replacing the word 'Batch' as 'DataOperation'.raviteja.karumuri8-74/+76
Issue-ID: CPS-1754 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Ib04e9d8a909e8868bf094e2625412996409b3a67
2023-06-23Merge "Improve code coverage"Luke Gleeson1-12/+25
2023-06-23Merge "NCMP : Replacing the word 'Batch' as 'DataOperation'."Sourabh Sourabh22-193/+195
2023-06-22Improve code coverageToineSiebelink1-12/+25
cleaned up the modified test a bit as well Issue-ID: CPS-475 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: I0a6ac50bb5720935c5c1c7036ada8423b3174fda
2023-06-22Subscription Response Event Schemaseanbeirne1-0/+69
Issue-ID: CPS-1738 Signed-off-by: seanbeirne <sean.beirne@est.tech> Change-Id: I409baf66ca702166308821eeeff4887e9385148e
2023-06-22NCMP : Replacing the word 'Batch' as 'DataOperation'.raviteja.karumuri22-193/+195
Issue-ID: CPS-1754 Signed-off-by: raviteja.karumuri <raviteja.karumuri@est.tech> Change-Id: Ie9f56effb160582ba7177331fb6260b2bde2342b
2023-06-22Merge "Improve performance of updateDataLeaves"Toine Siebelink2-16/+13
2023-06-22Add RTD link to CPS in Release notesToineSiebelink1-2/+4
Issue-ID: REQ-1580 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Id7f561a4686f62697bb98f7049199b44ebd387d4
2023-06-21Merge "Remove irrelevant groovy class that tests nothing"Toine Siebelink1-57/+0
2023-06-21Improve handling of legacy eventsToineSiebelink2-19/+62
Legacy (non-cloud) events would cause a NPE in the event filter Although the event woudl still be ignored it would lead to uncessary error logging This fix addressed this issue using a trace level message instead (also some improvements in the associated test) Issue-ID: CPS-1724 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ibf71b13e4a47aaf705e32df5fe50cf41de5f558f
2023-06-21Remove irrelevant groovy class that tests nothinghalil.cakal1-57/+0
Issue-ID: CPS-1741 Change-Id: If27a74e2496b952eb4803bbabb8363400c57915a Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-21Merge "Patch # 3: Data operation response event (NCMP → Client App) to ↵Toine Siebelink10-74/+151
comply with CloudEvents"
2023-06-21Improve performance of updateDataLeavesdanielhanrahan2-16/+13
Avoid fetching descendants during batchUpdateDataLeaves, as descendants are not needed: - Remove prefetch descendants step from getFragmentEntities; - Explicitly prefetch descendants in operations requiring it; - Update performance tests (5x faster for batch update). Issue-ID: CPS-1675 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7442a6f799cc0803b3a78f09d1ee53377f24b0b7
2023-06-21Add performance tests for updateDataLeavesdanielhanrahan1-2/+31
Issue-ID: CPS-1675 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I4dcd2ff0d0f6c3acf7373357da0ef7217a691573
2023-06-21Patch # 3: Data operation response event (NCMP → Client App) to comply ↵sourabh_sourabh10-74/+151
with CloudEvents - Modified data operation record strategy to consume cloud event. - Modified NCMP data operation event consumer to read cloud event header. (prefixed with ce_) - Modified event publisher to support legacy and cloud event based on event type (if legacy event use legacy kafka template else cloud kafka template). - Introduced a new method onto json object mapper to convert json object to bytes. - Modified data operation consumer spec to produce a cloud event and validate it. - Added Kafka Integration Test (for filtering) Issue-ID: CPS-1724 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ide701b1ff952f57413cb4e4aa0d55c08753f0298 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-06-21publishLcmEventBatchAsynchronously is not asyncdanielhanrahan4-48/+85
Spring @Async has a limitation where calling the async method from within the same class won't work, as the method needs to be proxied, and calling from the same class bypasses the proxy and calls the underlying method directly, thus running synchronously. Issue-ID: CPS-1749 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I7f198487dc18b359654dc38b5cf8fd600d33e189
2023-06-21Apply latest ReedTheDoc config rulesToineSiebelink1-6/+4
Issue-ID: CPS-1757 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Icdf5fe5b5b1d6628bed786a1d99816e7990302cc
2023-06-20Refactored code to enable its utilization in Junitwaqas.ikram11-329/+219
tests for the client Change-Id: I847ae0f48444af907039bfddb9879581003c0f35 Issue-ID: CPS-1751 Signed-off-by: waqas.ikram <waqas.ikram@est.tech>
2023-06-20Merge "Fix test-deregistration script"Toine Siebelink3-23/+25
2023-06-20Merge "DMI Data AVC to cloud events"Luke Gleeson7-146/+55
2023-06-20Merge "Patch # 1: Data operation response event (NCMP → Client App) to ↵Toine Siebelink6-129/+58
comply with CloudEvents"
2023-06-20Merge "Replace deprecated WebSecurityConfigurerAdapter"Toine Siebelink2-13/+41
2023-06-19Fix test-deregistration scriptdanielhanrahan3-23/+25
- Fix issue where script breaks with newer version of docker-compose caused by profile names being omitted during `docker-compose down` - Use main .gitignore file to ignore metrics-reports directory - Fix Intelli-j/shellcheck warnings (mostly variables needing quotes) - Script prints summary report at exit Issue-ID: CPS-1552 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ia51e51c1c199de8cc2f4878bfb04cb09fbb0b0e7
2023-06-19Update performance test timingsdanielhanrahan4-25/+25
The fixes to lower memory usage in FragmentRepository have positively affected read performance, so tests must be updated. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I2c8b882f56fb8c1eaefe5b802da1d308d16b1f02
2023-06-19Patch # 1: Data operation response event (NCMP → Client App) to comply ↵sourabh_sourabh6-129/+58
with CloudEvents - Removed header definitions (since we now use CloudEvents) - Used 'dataOperation' instead of batch where appropriate. - Modified test json Issue-ID: CPS-1724 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: Ic0f65297b944adf9cf5f3c2cbec679a031a675ec Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-06-19Merge "Code to persist or update the yang model subscription into db"Priyank Maheshwari7-54/+220
2023-06-19Code to persist or update the yang model subscription into dbhalil.cakal7-54/+220
- Change subscription persistence for subscription registry to handle adding or updating cm handles into db Issue-ID: CPS-1735 Change-Id: Icae609e810b2a097a4a174de1691892526f6b5c0 Signed-off-by: halil.cakal <halil.cakal@est.tech>
2023-06-19DMI Data AVC to cloud eventsmpriyank7-146/+55
- DMI Data AVC to be consumed as CloudEvents now - Removed the schema header as it is taken care by cloudevent headers - Implemented naming and packaging comments on the schema - Test cases refactoring Issue-ID: CPS-1719 Change-Id: I9864f90446720fe903fb3c1502d86035d886751d Signed-off-by: mpriyank <priyank.maheshwari@est.tech>
2023-06-19Move integration test (clean-up, last phase)ToineSiebelink17-640/+115
- Moved session manager test - Improved schemaset concurrency test (retry testing) - Cleaned up all ri (container based) integration test (setup) - Applied some groovy best practice where needed - internal ri module cover now down to 29%, covred by integration instead with - Line coverage up by 41 lines to 99% (was 97%) - Branch coverage up by 3 branches to 96% (was 93%) Issue-ID: CPS-1687 Signed-off-by: ToineSiebelink <toine.siebelink@est.tech> Change-Id: Ifb77a053e5a5db62a3f6a32ae60a3a8b10918efd
2023-06-15Merge "Patch #2 : Introduce kafka template for cloud events"Toine Siebelink12-38/+269
2023-06-15Patch #2 : Introduce kafka template for cloud eventssourabh_sourabh12-38/+269
- Introduced a new cloud kafka template for cloud events that reads it's configuration from application.yml - Kept legacy kafka template for backward compatibility utill all cps events moved to cloud event comply. - Modified application.yml producer and consumer value deserializer properties to support cloud events. - Added new cloudevents-bom used into cps-ncmp-service pom. - For the time being we will have 2 kafkatemplates (legacyEventKafkaTemplate, cloudEventKafkaTemplate) into EventsPublisher until we fully move to cloudevents for all events. Once all cps events will be cloud event compy, we have TODO task where Deprecated: legacyKafkaEventTemplate will be removed with its java configuration file KafkaTemplateConfig. Issue-ID: CPS-1724 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech> Change-Id: I78c15bd480db063b89c6630c46c2d3a328b4fae4 Signed-off-by: sourabh_sourabh <sourabh.sourabh@est.tech>
2023-06-15Bump CPS version to 3.3.3-SNAPSHOTdanielhanrahan20-20/+47
- Update version properties as well as we prepare for the next release. Issue-ID: CPS-1750 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I0626eee7834499d2fa96812fb416ce7daebf0529
2023-06-15Add container yaml for 3.3.2danielhanrahan1-0/+8
- Step 6 of the release process Issue-ID: CPS-1703 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I58e54ecd17878d3b9b6bc69ac9d254e454bd68b2
2023-06-15Add maven stage yaml for 3.3.2danielhanrahan1-0/+4
- Step 5 of the release process Issue-ID: CPS-1703 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: I105dedd6b1a634ca8ac4409ed682f6a10d1b018a
2023-06-15Replace deprecated WebSecurityConfigurerAdapteregernug2-13/+41
WebSecurityConfigurerAdapter has been deprecated in Spring 2.7. Change needed to use SecurityFilterChain and InMemoryUserDetailsManager. Note: WebMvcTest does not autoimport configurations that use beans. Issue-Id: CPS-1742 Signed-off-by: egernug <gerard.nugent@est.tech> Change-Id: I100884f10beaac6ce87e603302280aeb78559017
2023-06-15Update release docsdanielhanrahan2-55/+44
- Also synched up the API changes in openapi yml file - Step 1 and 2 of the release process Issue-ID: CPS-1703 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Id47822737df3b89dffddb90413a53cac2b7ced6a
2023-06-14Lower memory usage in FragmentRepositorydanielhanrahan7-239/+175
Avoid using Spring Data "interface projection" in FragmentRepository. The use of FragmentExtract in FragmentRepository is causing an overhead of around 5 kilobytes per fragment, which is leading to abnormally high memory usage when queries return a large number of nodes. For example, around 250MB of additional memory is needlessly used when fetching 50,000 datanodes. - Remove FragmentExtract interface and FragmentEntityArranger class. - Add FragmentPrefetchRepository, using JdbcTemplate and RowMapper to fetch FragmentEntity descendants in a single SQL query. - Many CpsDataService operations have memory reductions: - queryDataNodes - getDataNodesForMultipleXpaths - updateDataNodesAndDescendants - updateNodeLeaves - and any NCMP methods using the above. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ic47a2c9eb34150ed76bd5ce452fe1c9aaf9b4c5c
2023-06-13Merge "Move integration test (DataService)"Toine Siebelink8-820/+322
2023-06-12Fix for recursive SQL returning extra level of descendantsdanielhanrahan1-2/+2
Recursive SQL queries are incorrectly returning maxDepth + 1 levels of fragments due to incorrect comparison in the WHERE clause. This code is used by getDataNodes and queryDataNodes to prefetch the descendants before converting to datanodes. Prefetching extra descendants causes high memory usage. Issue-ID: CPS-1716 Signed-off-by: danielhanrahan <daniel.hanrahan@est.tech> Change-Id: Ie22215c0c7018cd293f73cf75adca2cd55f218a6