aboutsummaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/inbounds/selfservice-api
AgeCommit message (Collapse)AuthorFilesLines
2021-02-11Improve logging for CDS Kafka workersJulien Fontaine2-16/+18
Modified CDS Kafka consumersand producers logs to provide more details about the topic of the consumer record being consumed or published. Refactored the publish callback to make it more readable. Refactored audit service log error messages. Issue-ID: CCSDK-3154 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I7b42930e926bc15ce175974a78d3bfe2f219b0a8
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL13-90/+90
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-12-16Fixing typo in message-prioritizationSingal, Kapil (ks220y)1-1/+1
Refactoring few POMs name tag Issue-ID: CCSDK-3053 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I14447ea7f93efcc970213bbe7d42663cb87e33d7
2020-10-28Blueprints Processor MetricsJulien Fontaine9-4/+140
Add counter and timer for Blueprints Processor process to get success/failure and execution time for each blueprint execution using blueprint name, version and action. Issue-ID: CCSDK-2950 Change-Id: I38e8997de26effe69ec2ee9e2b6ed0da14de4a43 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-09-22Roll to initial Honolulu versionDan Timoney1-1/+1
Roll to initial Honolulu version Issue-ID: CCSDK-2742 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I563033c6c93a4121219eaa1d7bb8a026e9e490d4
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)19-82/+117
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-09-11Refactoring poms groupId artifactIdSingal, Kapil (ks220y)1-8/+8
Issue-ID: CCSDK-2787 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ica64f0c793beabb47950c7bf1eede63a0dbf3cea
2020-09-02Adding parent pom referenceSingal, Kapil (ks220y)1-2/+1
Adding parent pom to cba-assembly-descriptor as it needs to inherit repository from parent so maven-deploy-plugin works Issue-ID: CCSDK-2720 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I44ce3721f280555b81475580ca47539ae83d07ca
2020-08-10Merge "Make use of Kafka Key for Audit service and Kafka listener"KAPIL SINGAL2-5/+9
2020-08-07Kafka Audit Service: Error during sensitive data hidingJulien Fontaine1-54/+118
Changed behaviour during sensitive data hiding. Instead of looking for every node template defined in the CBA we only check for the ones declared for the current executed workflow. This prevent the issue when you have two independant workflows defined in the same CBA and one of those has a node template using a get_input expression for template-prefix in a component-resource-resolution. If you execute the other workflow audit service was still trying to resolve this expression which was making it fail. Issue-ID: CCSDK-2636 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ib232c26ebf930917e19e7970e1a56c15e676888f
2020-08-04Make use of Kafka Key for Audit service and Kafka listenerJulien Fontaine2-5/+11
* When message is sent by audit service, key will be the CBA name * When sent by kafka listener (self-service api), key is the same as the request message key consumed. If not specified, a random UUID * MessageProducer interface refactoring : * add 'key' parameter to specify a key * add default value null to paramater 'headers' to remove some unnecessary method Issue-ID: CCSDK-2628 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I68580151184c87104c07037f379276dd8c8c71c7
2020-07-24Change EventListener functions to non-blocking scopeJozsef Csongvai1-1/+2
BluePrintProcessingKafkaConsumer and BluePrintProcessorCluster were both defining EventListener functions which were executed with runBlocking. When both were enabled, kafkaconsumer would block and cluster would never run. Issue-ID: CCSDK-2609 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I91813208f0474869f174da6c99280b5159859232
2020-06-22Kafka Audit Service : Improve error handling and miscellaneous refactoringJulien Fontaine5-87/+96
When Kafka Audit Service fails it no longer stops Blueprint Processor execution * Add error handling when trying to hide sensitive data * Add error handling when trying to send kafka message * Set timeout for blocking loop when sending messages with kafka producer -> When broker is not available producer tries to reconnect in a blocking loop * Refactor Audit Service interface to give more explict name for publish methods * Modify publishExecutionOutput() to a non-blocking function Issue-ID: CCSDK-2459 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I809a5f34f81889aa9eed499608348f149984bc38
2020-06-19Error when template-prefix-names list is set as an inputJulien Fontaine1-1/+12
Case when template-prefix-names list for resource resolution is passed as an input was not handled. When the audit service was trying to get input-prefix-names values he was only getting {"get_input": "..."} and was not resolving this expression. Issue-ID: CCSDK-2456 Change-Id: If9ebbf0aa6b6eae3e44a7bfe0b930f622dfd3c1e Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-05-13Kafka Audit Service : Error with CBA using dg-generic node templateJulien Fontaine1-27/+35
* Fixed issue with dg-generic node template * Changed behaviour for hiding sensitive data -> Instead of just checking the node-template targeted by the workflow, we check every node-template declared in the CBA. If we find one or more component-resource-resolution node-template, we verify if we need to hide sensitive data in the kafka message. Issue-ID: CCSDK-2372 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I079ee29ae532b3947cdc8efe3438678309ca50b7
2020-05-11Kafka Audit Service : CorrelationUUID from request is not matching the ↵Julien Fontaine5-9/+7
correct response in Kafka Moved out CorrelationUUID linking between the request and the response from the Kafka Audit Service to the ExecutionServiceHandler. This prevents the race condition happening when several ExecutionServiceOutputs try to set the CorrelationUUID related to their ExecutionServiceInput. Issue-ID: CCSDK-2370 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I0c5934d4486961fbfcb34fd2d2492cd843350025
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine2-10/+25
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-21Secure Kafka AuthenticationJulien Fontaine3-1/+12
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-10Merge "Publish execution input/output into Kafka topics"Dan Timoney7-46/+349
2020-04-09Remove unneeded backup filesDan Timoney1-81/+0
Remove unneeded backup files created by updParentVersion script Issue-ID: CCSDK-2308 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I8d76b8ee3ccb2dbe563f22aadf1125f8226a104f
2020-04-08Publish execution input/output into Kafka topicsJulien Fontaine7-46/+349
- Creation and implementation of PublishAudit services (NoAudit,Kafka) - Conceal of sensitive data (request input parameters) - Update ExecutionServiceHandler tests - Update application.properties file Issue-ID: CCSDK-2183 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ifc28a91ce588f5ed0010acb4ed22b64429c6c1a0
2020-04-08Merge "Refactoring BP Code with ErrorCatalog"Dan Timoney12-21/+122
2020-04-07Roll to next Guilin snapshotDan Timoney2-1/+83
Roll to next Guilin snapshot release Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I1a5ea68da8821ffe28ac84e60177af05e26b3585
2020-04-06Refactoring BP Code with ErrorCatalogBrinda Santh12-21/+122
Issue-ID: CCSDK-2124 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ief468a56f9e7b3ef86c357965aa7b15f0a4cfa22
2020-03-30Roll to next Frankfurt snapshot and fix sonar buildDan Timoney1-1/+1
Roll to next Frankfurt snapshot. Also, fixed config to skip sonar for error-catalog, which has no scannable components. Sonar builds were failing because error-catalog does not have standard maven source and test directories (src/main, src/test) Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Iacb76b240e71206de8306af7e2447eaa07b0d33b
2020-03-18Fix GroupId and package name in Error CatalogDan Timoney1-1/+1
Issue-ID: CCSDK-2180 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ibe00fc1f2a905821b7100ae4f221ea1e0b934894
2020-03-11Error Catalog Management Core Library.Brinda Santh2-1/+49
Issue-ID: CCSDK-2076 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I9e8e67bdee77fb359a791f4ef4c2963aa78ab93a Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com>
2020-02-01Expose rest API with non blocking call.Brinda Santh1-8/+5
Convert Mono and Flux to coroutines Convert reactor mdc to coroutine mdc Issue-ID: CCSDK-2052 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic58c0b74866d28fd2d803b96626b08f8e8b2db56
2020-01-31Spring boot, Kotlin version upgradesBrinda Santh1-2/+1
Dependent Patch : https://gerrit.onap.org/r/c/ccsdk/parent/+/100990 Fixed Jackson set method according to latest version. Fixed Security properties issues for failed JUnit test cases. Reused maven properties from parent for Spring boot, Nats, Kafka, etc Issue-ID: CCSDK-1737 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I04bb0e535161e05897f587a0f08a2689e10c5f41
2019-12-11Metadata for name, version, tags and typeBrinda Santh3-5/+13
Mandate Tosca.meta template name, version, type and tags. Auto copy metadata from Tosca.meta to ServiceTemplate definitions. Optimize Blueprint context and runtime creation from file path. Removed attached CBA zip file in test repository dirs Issue-ID: CCSDK-1992 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I5d9d7a4599234a38d431328dbd9b74bd831e0115
2019-12-06Refractor controller blueprint modulesBrinda Santh1-4/+4
Move controller blueprints modules to blueprints processor and change the maven group name. Fix test cba and model type paths Fix dependencies issues. Issue-ID: CCSDK-1663 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I3654e6f04811470327acba90e8a452b66e3e189b
2019-11-29Optimize spring data JPA UT.Brinda Santh4-8/+95
Test case based database configuration, so that we can define what repositories and entities can be used for testing. Issue-ID: CCSDK-1735 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4f8a7eb4ed47fec9ab17eb9552648ebd0de01236
2019-11-26Formatting Code base with ktlintSingal, Kapil (ks220y)14-139/+201
No Business logic change, just the code format. Competible with IntelliJ: https://github.com/pinterest/ktlint#option-3 To format run: mvn process-sources -P format Issue-ID: CCSDK-1947 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic9e9209fb7023d77f434693ad5a01229f8d09331
2019-11-09Convert component functions IT to UT.Brinda Santh1-3/+3
Issue-ID: CCSDK-1735 Convert Netconf and Restconf Executor IT to UT. Remove duplicate BluePrintProperties. Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I2eafdbabb17a4df72541ab93c46e7fc9eb0fe8d7
2019-11-07Fix missing status event type workflow response.Brinda Santh5-265/+2
Issue-ID: CCSDK-1896 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Iffb4c5e9d4e5788895bda6c7ee4197ee9e0bd60e
2019-11-05Rolling upgrade support for in-flight requestsSebastien Premont-Tendland3-1/+39
Three entry points are being handled : 1 - REST endpoint 2 - gRPC endpoint 3 - Kafka consumer We make use of Phaser object to make sure the PreDestroy callback wait for all requests to be executed before stopping the process. The docker image was also modified to make sure the java process becomes PID 1 in the container in order to catch the SIGTERM signal which triggers the PreDestroy callback of Spring. This was done by using the "exec" command in bash. Issue-ID: CCSDK-1885 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I3e2a72e26a4c8b7768ebc374ea40aa8d55fb6761
2019-11-04Fixed CDS liveness probe failure during high load.prathamesh1-3/+5
-Added Dispatchers.IO so we can utilize CPU resources efficiently. Issue-ID: CCSDK-1874 Signed-off-by: prathamesh <prathamesh_morde@yahoo.ca> Change-Id: I8bb3c078b22663ca6c2420189b0477b8a66a3bd6
2019-10-29Add remote service function execution.Brinda Santh2-10/+115
Issue-ID: CCSDK-1875 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I18279b1d2ff1a0b2962afa74fc616bfd99e93c91
2019-10-22Add GRPC log tracing service.Brinda Santh1-1/+1
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4ba6ed11d8fb63c21b9c49774ed733cca05c5646
2019-10-02Add reactive log tracing service.Brinda Santh1-11/+15
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic20013045dd5d2681243c03f9e4cdfe557b630be
2019-09-25Add bi-directional GRPC python executor.Brinda Santh1-0/+96
Python based bidirectional communication script executor microservice. Issue-ID: CCSDK-1747 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: If2e8f862ca4cb97d10714df2ab1cccccef2d576a
2019-09-18Optimizing Imports and Formatting codeSingal, Kapil (ks220y)2-6/+1
No Business logic change, no code change.. just optimizing imports and reformatting code Issue-ID: CCSDK-1744 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I1fb2ed10525a6f392f3c0d963c2ed03cfe8dec7e
2019-09-17Plugin to format/validate POMSINGAL, KAPIL (ks220y)1-0/+2
Adding tidy-maven-plugin and related profile to format/vaidate pom files To format POM Files just run mvn process-sources -P format Issue-ID: CCSDK-1739 Signed-off-by: SINGAL, KAPIL (ks220y) <ks220y@att.com> Change-Id: I58dc261ac7b3179a91a0ab5a06237990707543f5
2019-09-12Fix proto cyclic dependency.Brinda Santh1-3/+2
Change-Id: I84e0ba25b78e8212eff6b4c91fca2d4457dbf7eb Issue-ID: CCSDK-1682 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-11Merge "Move GRPC management api to designer api."Dan Timoney9-565/+56
2019-09-11Roll versions for Frankfurt developmentTimoney, Dan (dt5972)1-1/+1
Roll master version for Frankfurt development Change-Id: Ife7ecc5bfb2d8d515805c40c381f6acb34af538b Issue-ID: CCSDK-1701 Signed-off-by: Timoney, Dan (dt5972) <dtimoney@att.com>
2019-09-05Move GRPC management api to designer api.Brinda Santh9-565/+56
Change-Id: I58ee303d361cf4f1996c966c094ec66886587b52 Issue-ID: CCSDK-1682 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-05Add Config based blueprint process consumerBrinda Santh12-448/+329
Change-Id: I9e37ecb5032047f835f3b2ea20b2689c76353497 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-04Add Kafka message lib consumer servicesBrinda Santh1-10/+9
Change-Id: Iaf1df07a0d8f4fb54d5d06047520010d3bfe5465 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-04Add draft and publish grpc upload options.Brinda Santh3-19/+53
Change-Id: Id1476f1cb449c4070a7ef53a918b62d26da26391 Issue-ID: CCSDK-1682 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>