summaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/inbounds/selfservice-api
AgeCommit message (Collapse)AuthorFilesLines
2021-03-24Refactoring to enable on_failure for imperative workflowJozsef Csongvai2-7/+3
BlueprintError needs to associate errors with the steps in which they occurred in order for imperative workflow to handle on_failure properly. Made stepName more accessible and corrected places where stepName was assigned to nodeTemplateName. Issue-ID: CCSDK-3219 Change-Id: I7e5805745c63558cff6be533e1b99c32ad06c3db Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> (cherry picked from commit b96b44d6d7ca11dbbc3ad4bd2194df31fba5efb6)
2021-03-04Release version 1.1.1 of CDS dockersDan Timoney1-1/+1
Release new CDS dockers to correct issues found with release 1.1.0 dockers. Issue-ID: CCSDK-3192 Issue-ID: CCSDK-3197 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I3bc1acbb93f0bb95ef14a6565e66d8da7a5df21d
2021-02-23Release version 1.1.0 docker containers and roll to next versionDan Timoney1-1/+1
Release version 1.1.0 docker containers and roll to next version Issue-ID: CCSDK-3179 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I2518ba1dd5a4bd926499867f5f9714bc3f716732
2021-02-11Added Kafka metrics for CDS workersJulien Fontaine5-32/+67
Added counters to gather metrics on CDS Kafka workers. This will enable us to get metrics on how many messages we consumer and produce to/from kafka. For consumers we count how many messages we consume and how many failed ie. consumed but not able to be processed (parsing error). For producers we count how many messages we produce and how many failed ie. failed to be pushed to the cluster (unavailable brokers, network error, ...). Relocated metrics tag constants to BlueprintConstants so that they can be use by any CDS module. If they make sense for other metrics then they should be shared. Issue-ID: CCSDK-3155 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iad6aba588766f655f3a74cd626e0f74e29188f96
2021-02-11Improve logging for CDS Kafka workersJulien Fontaine2-16/+18
Modified CDS Kafka consumersand producers logs to provide more details about the topic of the consumer record being consumed or published. Refactored the publish callback to make it more readable. Refactored audit service log error messages. Issue-ID: CCSDK-3154 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I7b42930e926bc15ce175974a78d3bfe2f219b0a8
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL13-90/+90
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-12-16Fixing typo in message-prioritizationSingal, Kapil (ks220y)1-1/+1
Refactoring few POMs name tag Issue-ID: CCSDK-3053 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I14447ea7f93efcc970213bbe7d42663cb87e33d7
2020-10-28Blueprints Processor MetricsJulien Fontaine9-4/+140
Add counter and timer for Blueprints Processor process to get success/failure and execution time for each blueprint execution using blueprint name, version and action. Issue-ID: CCSDK-2950 Change-Id: I38e8997de26effe69ec2ee9e2b6ed0da14de4a43 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-09-22Roll to initial Honolulu versionDan Timoney1-1/+1
Roll to initial Honolulu version Issue-ID: CCSDK-2742 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I563033c6c93a4121219eaa1d7bb8a026e9e490d4
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)19-82/+117
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-09-11Refactoring poms groupId artifactIdSingal, Kapil (ks220y)1-8/+8
Issue-ID: CCSDK-2787 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ica64f0c793beabb47950c7bf1eede63a0dbf3cea
2020-09-02Adding parent pom referenceSingal, Kapil (ks220y)1-2/+1
Adding parent pom to cba-assembly-descriptor as it needs to inherit repository from parent so maven-deploy-plugin works Issue-ID: CCSDK-2720 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I44ce3721f280555b81475580ca47539ae83d07ca
2020-08-10Merge "Make use of Kafka Key for Audit service and Kafka listener"KAPIL SINGAL2-5/+9
2020-08-07Kafka Audit Service: Error during sensitive data hidingJulien Fontaine1-54/+118
Changed behaviour during sensitive data hiding. Instead of looking for every node template defined in the CBA we only check for the ones declared for the current executed workflow. This prevent the issue when you have two independant workflows defined in the same CBA and one of those has a node template using a get_input expression for template-prefix in a component-resource-resolution. If you execute the other workflow audit service was still trying to resolve this expression which was making it fail. Issue-ID: CCSDK-2636 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ib232c26ebf930917e19e7970e1a56c15e676888f
2020-08-04Make use of Kafka Key for Audit service and Kafka listenerJulien Fontaine2-5/+11
* When message is sent by audit service, key will be the CBA name * When sent by kafka listener (self-service api), key is the same as the request message key consumed. If not specified, a random UUID * MessageProducer interface refactoring : * add 'key' parameter to specify a key * add default value null to paramater 'headers' to remove some unnecessary method Issue-ID: CCSDK-2628 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I68580151184c87104c07037f379276dd8c8c71c7
2020-07-24Change EventListener functions to non-blocking scopeJozsef Csongvai1-1/+2
BluePrintProcessingKafkaConsumer and BluePrintProcessorCluster were both defining EventListener functions which were executed with runBlocking. When both were enabled, kafkaconsumer would block and cluster would never run. Issue-ID: CCSDK-2609 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I91813208f0474869f174da6c99280b5159859232
2020-06-22Kafka Audit Service : Improve error handling and miscellaneous refactoringJulien Fontaine5-87/+96
When Kafka Audit Service fails it no longer stops Blueprint Processor execution * Add error handling when trying to hide sensitive data * Add error handling when trying to send kafka message * Set timeout for blocking loop when sending messages with kafka producer -> When broker is not available producer tries to reconnect in a blocking loop * Refactor Audit Service interface to give more explict name for publish methods * Modify publishExecutionOutput() to a non-blocking function Issue-ID: CCSDK-2459 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I809a5f34f81889aa9eed499608348f149984bc38
2020-06-19Error when template-prefix-names list is set as an inputJulien Fontaine1-1/+12
Case when template-prefix-names list for resource resolution is passed as an input was not handled. When the audit service was trying to get input-prefix-names values he was only getting {"get_input": "..."} and was not resolving this expression. Issue-ID: CCSDK-2456 Change-Id: If9ebbf0aa6b6eae3e44a7bfe0b930f622dfd3c1e Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-05-13Kafka Audit Service : Error with CBA using dg-generic node templateJulien Fontaine1-27/+35
* Fixed issue with dg-generic node template * Changed behaviour for hiding sensitive data -> Instead of just checking the node-template targeted by the workflow, we check every node-template declared in the CBA. If we find one or more component-resource-resolution node-template, we verify if we need to hide sensitive data in the kafka message. Issue-ID: CCSDK-2372 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I079ee29ae532b3947cdc8efe3438678309ca50b7
2020-05-11Kafka Audit Service : CorrelationUUID from request is not matching the ↵Julien Fontaine5-9/+7
correct response in Kafka Moved out CorrelationUUID linking between the request and the response from the Kafka Audit Service to the ExecutionServiceHandler. This prevents the race condition happening when several ExecutionServiceOutputs try to set the CorrelationUUID related to their ExecutionServiceInput. Issue-ID: CCSDK-2370 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I0c5934d4486961fbfcb34fd2d2492cd843350025
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine2-10/+25
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-21Secure Kafka AuthenticationJulien Fontaine3-1/+12
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-10Merge "Publish execution input/output into Kafka topics"Dan Timoney7-46/+349
2020-04-09Remove unneeded backup filesDan Timoney1-81/+0
Remove unneeded backup files created by updParentVersion script Issue-ID: CCSDK-2308 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I8d76b8ee3ccb2dbe563f22aadf1125f8226a104f
2020-04-08Publish execution input/output into Kafka topicsJulien Fontaine7-46/+349
- Creation and implementation of PublishAudit services (NoAudit,Kafka) - Conceal of sensitive data (request input parameters) - Update ExecutionServiceHandler tests - Update application.properties file Issue-ID: CCSDK-2183 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ifc28a91ce588f5ed0010acb4ed22b64429c6c1a0
2020-04-08Merge "Refactoring BP Code with ErrorCatalog"Dan Timoney12-21/+122
2020-04-07Roll to next Guilin snapshotDan Timoney2-1/+83
Roll to next Guilin snapshot release Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I1a5ea68da8821ffe28ac84e60177af05e26b3585
2020-04-06Refactoring BP Code with ErrorCatalogBrinda Santh12-21/+122
Issue-ID: CCSDK-2124 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ief468a56f9e7b3ef86c357965aa7b15f0a4cfa22
2020-03-30Roll to next Frankfurt snapshot and fix sonar buildDan Timoney1-1/+1
Roll to next Frankfurt snapshot. Also, fixed config to skip sonar for error-catalog, which has no scannable components. Sonar builds were failing because error-catalog does not have standard maven source and test directories (src/main, src/test) Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Iacb76b240e71206de8306af7e2447eaa07b0d33b
2020-03-18Fix GroupId and package name in Error CatalogDan Timoney1-1/+1
Issue-ID: CCSDK-2180 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ibe00fc1f2a905821b7100ae4f221ea1e0b934894
2020-03-11Error Catalog Management Core Library.Brinda Santh2-1/+49
Issue-ID: CCSDK-2076 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I9e8e67bdee77fb359a791f4ef4c2963aa78ab93a Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com>
2020-02-01Expose rest API with non blocking call.Brinda Santh1-8/+5
Convert Mono and Flux to coroutines Convert reactor mdc to coroutine mdc Issue-ID: CCSDK-2052 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic58c0b74866d28fd2d803b96626b08f8e8b2db56
2020-01-31Spring boot, Kotlin version upgradesBrinda Santh1-2/+1
Dependent Patch : https://gerrit.onap.org/r/c/ccsdk/parent/+/100990 Fixed Jackson set method according to latest version. Fixed Security properties issues for failed JUnit test cases. Reused maven properties from parent for Spring boot, Nats, Kafka, etc Issue-ID: CCSDK-1737 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I04bb0e535161e05897f587a0f08a2689e10c5f41
2019-12-11Metadata for name, version, tags and typeBrinda Santh3-5/+13
Mandate Tosca.meta template name, version, type and tags. Auto copy metadata from Tosca.meta to ServiceTemplate definitions. Optimize Blueprint context and runtime creation from file path. Removed attached CBA zip file in test repository dirs Issue-ID: CCSDK-1992 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I5d9d7a4599234a38d431328dbd9b74bd831e0115
2019-12-06Refractor controller blueprint modulesBrinda Santh1-4/+4
Move controller blueprints modules to blueprints processor and change the maven group name. Fix test cba and model type paths Fix dependencies issues. Issue-ID: CCSDK-1663 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I3654e6f04811470327acba90e8a452b66e3e189b
2019-11-29Optimize spring data JPA UT.Brinda Santh4-8/+95
Test case based database configuration, so that we can define what repositories and entities can be used for testing. Issue-ID: CCSDK-1735 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4f8a7eb4ed47fec9ab17eb9552648ebd0de01236
2019-11-26Formatting Code base with ktlintSingal, Kapil (ks220y)14-139/+201
No Business logic change, just the code format. Competible with IntelliJ: https://github.com/pinterest/ktlint#option-3 To format run: mvn process-sources -P format Issue-ID: CCSDK-1947 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic9e9209fb7023d77f434693ad5a01229f8d09331
2019-11-09Convert component functions IT to UT.Brinda Santh1-3/+3
Issue-ID: CCSDK-1735 Convert Netconf and Restconf Executor IT to UT. Remove duplicate BluePrintProperties. Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I2eafdbabb17a4df72541ab93c46e7fc9eb0fe8d7
2019-11-07Fix missing status event type workflow response.Brinda Santh5-265/+2
Issue-ID: CCSDK-1896 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Iffb4c5e9d4e5788895bda6c7ee4197ee9e0bd60e
2019-11-05Rolling upgrade support for in-flight requestsSebastien Premont-Tendland3-1/+39
Three entry points are being handled : 1 - REST endpoint 2 - gRPC endpoint 3 - Kafka consumer We make use of Phaser object to make sure the PreDestroy callback wait for all requests to be executed before stopping the process. The docker image was also modified to make sure the java process becomes PID 1 in the container in order to catch the SIGTERM signal which triggers the PreDestroy callback of Spring. This was done by using the "exec" command in bash. Issue-ID: CCSDK-1885 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I3e2a72e26a4c8b7768ebc374ea40aa8d55fb6761
2019-11-04Fixed CDS liveness probe failure during high load.prathamesh1-3/+5
-Added Dispatchers.IO so we can utilize CPU resources efficiently. Issue-ID: CCSDK-1874 Signed-off-by: prathamesh <prathamesh_morde@yahoo.ca> Change-Id: I8bb3c078b22663ca6c2420189b0477b8a66a3bd6
2019-10-29Add remote service function execution.Brinda Santh2-10/+115
Issue-ID: CCSDK-1875 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I18279b1d2ff1a0b2962afa74fc616bfd99e93c91
2019-10-22Add GRPC log tracing service.Brinda Santh1-1/+1
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4ba6ed11d8fb63c21b9c49774ed733cca05c5646
2019-10-02Add reactive log tracing service.Brinda Santh1-11/+15
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic20013045dd5d2681243c03f9e4cdfe557b630be
2019-09-25Add bi-directional GRPC python executor.Brinda Santh1-0/+96
Python based bidirectional communication script executor microservice. Issue-ID: CCSDK-1747 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: If2e8f862ca4cb97d10714df2ab1cccccef2d576a
2019-09-18Optimizing Imports and Formatting codeSingal, Kapil (ks220y)2-6/+1
No Business logic change, no code change.. just optimizing imports and reformatting code Issue-ID: CCSDK-1744 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I1fb2ed10525a6f392f3c0d963c2ed03cfe8dec7e
2019-09-17Plugin to format/validate POMSINGAL, KAPIL (ks220y)1-0/+2
Adding tidy-maven-plugin and related profile to format/vaidate pom files To format POM Files just run mvn process-sources -P format Issue-ID: CCSDK-1739 Signed-off-by: SINGAL, KAPIL (ks220y) <ks220y@att.com> Change-Id: I58dc261ac7b3179a91a0ab5a06237990707543f5
2019-09-12Fix proto cyclic dependency.Brinda Santh1-3/+2
Change-Id: I84e0ba25b78e8212eff6b4c91fca2d4457dbf7eb Issue-ID: CCSDK-1682 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-11Merge "Move GRPC management api to designer api."Dan Timoney9-565/+56
2019-09-11Roll versions for Frankfurt developmentTimoney, Dan (dt5972)1-1/+1
Roll master version for Frankfurt development Change-Id: Ife7ecc5bfb2d8d515805c40c381f6acb34af538b Issue-ID: CCSDK-1701 Signed-off-by: Timoney, Dan (dt5972) <dtimoney@att.com>