summaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/commons
AgeCommit message (Collapse)AuthorFilesLines
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine4-8/+54
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-24Merge "Secure Kafka Authentication"Dan Timoney11-168/+794
2020-04-21Secure Kafka AuthenticationJulien Fontaine11-168/+794
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-16Fix default behavior for DatabaseResourceAssignmentProcessorJozsef Csongvai2-70/+37
If a data-dictionary did not declare an endpoint-selector, it would default to primary cds-db instead of using data-dictionary source to look up database configuration. Processor-db configurations were not used because of this. Also improved error messages and some refactoring. Issue-ID: CCSDK-2315 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I7122e1334951db4eec5b9ae8b01c1f66596636f6
2020-04-10Merge "Publish execution input/output into Kafka topics"Dan Timoney1-1/+5
2020-04-09Remove unneeded backup filesDan Timoney9-568/+0
Remove unneeded backup files created by updParentVersion script Issue-ID: CCSDK-2308 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I8d76b8ee3ccb2dbe563f22aadf1125f8226a104f
2020-04-08Publish execution input/output into Kafka topicsJulien Fontaine1-1/+5
- Creation and implementation of PublishAudit services (NoAudit,Kafka) - Conceal of sensitive data (request input parameters) - Update ExecutionServiceHandler tests - Update application.properties file Issue-ID: CCSDK-2183 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ifc28a91ce588f5ed0010acb4ed22b64429c6c1a0
2020-04-07Roll to next Guilin snapshotDan Timoney18-9/+586
Roll to next Guilin snapshot release Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I1a5ea68da8821ffe28ac84e60177af05e26b3585
2020-03-30Roll to next Frankfurt snapshot and fix sonar buildDan Timoney9-9/+9
Roll to next Frankfurt snapshot. Also, fixed config to skip sonar for error-catalog, which has no scannable components. Sonar builds were failing because error-catalog does not have standard maven source and test directories (src/main, src/test) Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Iacb76b240e71206de8306af7e2447eaa07b0d33b
2020-03-18Use released parent pomsDan Timoney1-1/+0
Use released version of parent poms Issue-ID: CCSDK-2166 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Ib45fc3134e2ba0e97908d9efa12e3cbec265b381
2020-02-26Merge SW Upgrade Blueprint into PNF_AAI and create one UAT BP for PNFgummar2-3/+8
UAT: Add support to multiple responses for a single request Set property IN_UAT=1 during UAT execution so blueprints can tune their settings to values more suitable for testing (like timeouts) Add 'times' field to specify expected number of invocations Add UAT blueprint script for PNF SW Upgrade UC Add current thread check for Hazelcast distributed lock Resolve URI before returning Issue-ID: CCSDK-2091 Change-Id: Id256bad043488f88f1b60015ebf9ade4be607fa2 Signed-off-by: gummar <raj.gumma@est.tech>
2020-02-25Changed the default multicast address.Sebastien Premont-Tendland6-69/+30
There's an issue with macOS when docker is installed and a multicast address different than 224.0.0.1 is used. https://stackoverflow.com/questions/46341715/hazelcast-multicast-does-not-work-because-of-vboxnet-which-is-used-by-docker-mac Re-organized the test hazelcast yaml since we now fill the cluster-name and instance-name information from the ClusterInfo object during creation. No need to pass it in the config file anymore. This is causing every file to be the exact same thus the consolidation. Also fix an issue with test case using the same hazelcast service in two different threads causing intermittent issue during the build process. Issue-ID: CCSDK-2119 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I7f8328bfd93cdcabe8ab2fa844cbf9dd57f1493a
2020-02-18Add ability to resolve environment variables when readingSebastien Premont-Tendland1-5/+3
PrimaryDatabaseConfiguration This is needed to remove hardcoded password in application.properties file by being able to resolve environment variables. OOM would pass the username and password through environment variables. Issue-ID: CCSDK-2095 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: Ia19ef0ddee29509b0378b048bf58c700870ecce4
2020-02-17Removed manual shutdown hook for hazelcastSebastien Premont-Tendland1-3/+5
Shutdown hook for hazelcast are already working with spring boot. Just changed the policy to GRACEFUL (default is TERMINATE). This allow hazelcast to shutdown gracefully and recover from a rolling upgrade Also updated the Dockerfile to use multi stage to reduce by ~200Mb the docker image size. Issue-ID: CCSDK-2011 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I0034145c7acefa88ed8d16f23a29249c7225ace3
2020-02-14Cluster CP member promotion as default.Brinda Santh7-5/+151
Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4df4b442c020fbf9da294a9b35739ca3acb8ee02
2020-02-12Cluster co-ordination with Hazelcast.Brinda Santh18-551/+631
Remove Atomix implementation, due to Kubernetes clustering issues. Cluster environment property changes. Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I23f40c92c0adc6b3ab8690871385f78525c76433
2020-02-01Expose rest API with non blocking call.Brinda Santh1-0/+28
Convert Mono and Flux to coroutines Convert reactor mdc to coroutine mdc Issue-ID: CCSDK-2052 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic58c0b74866d28fd2d803b96626b08f8e8b2db56
2020-01-31Spring boot, Kotlin version upgradesBrinda Santh1-1/+1
Dependent Patch : https://gerrit.onap.org/r/c/ccsdk/parent/+/100990 Fixed Jackson set method according to latest version. Fixed Security properties issues for failed JUnit test cases. Reused maven properties from parent for Spring boot, Nats, Kafka, etc Issue-ID: CCSDK-1737 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I04bb0e535161e05897f587a0f08a2689e10c5f41
2020-01-17Fix scheduler shutdown config function.Brinda Santh1-1/+1
Optimize application name accessing in Docker Compose. Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I6b055de7ef5a8ac4a0c103499997ed272bd74ac6
2020-01-16Prioritization Optional NATS consumer supportBrinda Santh7-8/+114
Add prioritization NATS consumer service and configuration data beans. Optimizing message prioritization service interface. Added Integration testing for NATS simulation. Updated sample docker compose for NATS support Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Icd21e5e2ab7b64d6e6e4b0610599ca947555ee15
2020-01-13Add uri-encoding to webclientJozsef Csongvai1-6/+9
Path params are not uri-encoded, which can cause failed rest calls if resolved params include characters such as whitespace Issue-ID: CCSDK-2024 Change-Id: Iac57219260ac94897e09e3cb9d1d5884d86b83ae Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-01-08Rest endpoint for message PrioritizationBrinda Santh1-0/+8
Refactored to support both Kafka and Rest endpoint prioritization. Simplified number for Kafka processors. Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Iba77ed94be3398940840ff01a298f0bec785401f
2020-01-07Include correlationId in group lock.Brinda Santh2-14/+37
Message prioritization optimization by checking and including correlation Id, so that non related correlation message won't get locked. Optimized Atomix Junit Test cases. Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I090ed4c193c7f9af4014cfeee4c6208c12b542c1
2020-01-02Message Prioritization message group lock.Brinda Santh2-0/+8
Implementation to avoid concurrent procession of message group while prioritization. Sample message prioritization Kafka listener properties. Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ifbf39985b03c662b6ccf7740be711cfeb7bfbebb
2019-12-31Flexible DSL Types and Templates definition.Brinda Santh12-62/+183
Easy search definitions Types and Templates inside ServiceTemplate DSL builder. Unit test modifications to support this change. Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ie944ff5f75f80c852555306e1a4e0fa7f5b803d7
2019-12-30Runtime services for Relationship Type and TemplatesBrinda Santh5-2/+497
Added and improved blueprint runtime service property resolution api. Relationship template CBA sample Performance improvement on Artifact resolutions. Added connection config DSL for DB, GRPC connections. Added starter relationship connection config models. Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I73e1deed708dc924352867ddd1d1b2271c8ea106
2019-12-30Relationship Type and Templates modelsBrinda Santh12-62/+852
Enrichment Support for Relationship Types and Templates. Relationship DSL support for ConnectTo connections ( RestClient, SshClient, MessageProducer, MessageConsume, Nats) Moved datatype map from collection to complex type Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I0f18db2cb52e1e93dfab04498b8298587cba2540
2019-12-30Cluster partition master APIBrinda Santh9-26/+81
Added cluster lock close method. Added NATS connection name and default services. Junit test cases improvements. Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I44c7c1aaeae2ddbf768903152fb00bc160172fc1
2019-12-26add logging to value not found from propertiesOleg Mitsura1-1/+10
Issue-ID: CCSDK-1995 rev1: commit rev2: formatting rev3: cleanup rev4: review comment from Brinda Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I5a575dad5191b72934667d2c4e965882ccc2a905
2019-12-23Cluster distributed lock service.Brinda Santh5-34/+183
Included and fixed clustered env properties and utils. Fixed docker compose instance sequence numbers. Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ie28935ae7cb3de8c77cd7110993304eb49799b6c
2019-12-19Cluster communication channelsBrinda Santh14-5/+855
Add NATS property and library services both . NATS Messaging Services with Token Auth and TLS Auth implementation Docker Compose for NATS Streaming instance. Documentation : https://docs.nats.io/ Issue-ID: CCSDK-2007 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ieebaa8f2b18ae89d02a4f38a8027eda495a9db43
2019-12-16Cluster distributed data storeBrinda Santh9-0/+480
Add experimental cluster co-ordination service using Atomic framework. Included distributed data store creation utilities. Sample docker compose data cluster between cds controller and resource-resolution instances. Issue-ID: CCSDK-2000 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4de00e773a996e08fd1d260fc27ed18832433883
2019-12-11Metadata for name, version, tags and typeBrinda Santh3-13/+18
Mandate Tosca.meta template name, version, type and tags. Auto copy metadata from Tosca.meta to ServiceTemplate definitions. Optimize Blueprint context and runtime creation from file path. Removed attached CBA zip file in test repository dirs Issue-ID: CCSDK-1992 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I5d9d7a4599234a38d431328dbd9b74bd831e0115
2019-12-09Cli executor doesn't keep/support execution contextSteve Siani4-53/+284
Issue-ID: CCSDK-1927 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ib417bfd62662676fe7520a5500df82ade716f66c
2019-12-06Refractor controller blueprint modulesBrinda Santh7-16/+16
Move controller blueprints modules to blueprints processor and change the maven group name. Fix test cba and model type paths Fix dependencies issues. Issue-ID: CCSDK-1663 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I3654e6f04811470327acba90e8a452b66e3e189b
2019-12-06Remote Script Executor ComponentBrinda Santh7-36/+164
Define and Implement component-remote-script-executor component and DSL. Get the timeout from model definitions fix the dat pattern issues Define API data extension functions. Issue-ID: CCSDK-1975 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I41a429eb21a050e5ab512a7a73a394b975543f31
2019-11-29Upgrade py executor ms GRPC version.Brinda Santh2-75/+85
Regenerate TLS certs with config file. Issue-ID: CCSDK-1737 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I2b0982c4e230524ad1e58f150da2a05e4474a0fa
2019-11-29Optimize spring data JPA UT.Brinda Santh7-50/+80
Test case based database configuration, so that we can define what repositories and entities can be used for testing. Issue-ID: CCSDK-1735 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4f8a7eb4ed47fec9ab17eb9552648ebd0de01236
2019-11-27Upgrade CDS GRPC implementations.Brinda Santh8-37/+38
Dependent on : https://gerrit.onap.org/r/c/ccsdk/parent/+/98823 Issue-ID: CCSDK-1737 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I0b5087f7e98490d7066f389af29424aed40a6520
2019-11-26Formatting Code base with ktlintSingal, Kapil (ks220y)87-838/+1216
No Business logic change, just the code format. Competible with IntelliJ: https://github.com/pinterest/ktlint#option-3 To format run: mvn process-sources -P format Issue-ID: CCSDK-1947 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic9e9209fb7023d77f434693ad5a01229f8d09331
2019-11-19Merge "add service for searching in meta data pageable"Brinda Santh Muthuramalingam1-0/+24
2019-11-18Property based GRPC server service.Brinda Santh3-1/+33
Issue-ID: CCSDK-1747 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I948d4fac974a0d6d0c855ad4e24766a9ce6b7a6a
2019-11-18add service for searching in meta data pageableShaaban Ebrahim1-0/+24
Issue-ID: CCSDK-1770 Signed-off-by: Shaaban Ebrahim <shaaban.eltanany.ext@orange.com> Change-Id: Ibf718314f2f818108af3133641ab7be48c334dce
2019-11-18Fix Message Prioritization merge conflictsBrinda Santh2-4/+4
Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I92cdca04036c23c212b7b333ca14455eef20555c
2019-11-18Merge "Add message prioritization module"Dan Timoney3-0/+209
2019-11-18Merge "Add Kafka Streams consumer service"Dan Timoney9-2/+356
2019-11-18Merge "Convert component functions IT to UT."Dan Timoney19-135/+101
2019-11-14Removed deprecated class. The correct implementation isSebastien Premont-Tendland1-127/+0
in the class BlueprintProcessorCatalogServiceImpl. Issue-ID: CCSDK-1924 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I0d4ccfb69979ec3be35a46f154cd20586175d671
2019-11-13Add message prioritization moduleBrinda Santh3-0/+209
Kafka streams based solution for message prioritization using database store. Implement initial Abstract Processors, Puntuations and sample Topology for easy plug and play based on situations Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I9c135604574cc3c642186545e076d6a7c60048d4
2019-11-09Convert component functions IT to UT.Brinda Santh19-135/+101
Issue-ID: CCSDK-1735 Convert Netconf and Restconf Executor IT to UT. Remove duplicate BluePrintProperties. Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I2eafdbabb17a4df72541ab93c46e7fc9eb0fe8d7