aboutsummaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/inbounds/selfservice-api/src
AgeCommit message (Collapse)AuthorFilesLines
2021-08-27CCSDK-3434 CBA workflow status storeKavitha P5-6/+82
Change-Id: Iaeac6fa534c569bbc152e6c8a78c2dd23b6c4b1a Signed-off-by: Kavitha P <pkavitha@aarnanetworks.com> Issue-ID: CCSDK-3434
2021-07-28Revert "Renaming Files having BluePrint to have Blueprint"Jozsef Csongvai15-112/+112
The renaming in CCSDK-3098 caused breaking changes to the grpc api and compile issues for kotlin scripts. Issue-ID: CCSDK-3385 Change-Id: I0d745cb858371678eabcb2284671c1fd76a1ab6d Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-04-28Add check for cluster status to readiness endpointJozsef Csongvai1-3/+14
When Cluster is enabled, BlueprintsProcessor should not process any requests until cluster is fully joined. Issue-ID: CCSDK-3275 Change-Id: I779159346976f7af0c0add69883f27d7f359f413 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-03-17Refactoring to enable on_failure for imperative workflowJozsef Csongvai2-7/+3
BlueprintError needs to associate errors with the steps in which they occurred in order for imperative workflow to handle on_failure properly. Made stepName more accessible and corrected places where stepName was assigned to nodeTemplateName. Issue-ID: CCSDK-3219 Change-Id: I7e5805745c63558cff6be533e1b99c32ad06c3db Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-03-16Assign a unique worker ID for CDS Kafka workerJulien Fontaine1-1/+1
* Modified CDS Kafka workers to add the 5 lasts characters of the env var HOSTNAME to their worker ID. * Small refactoring to move some utilitary functions to BlueprintMessageUtils Issue-ID: CCSDK-3204 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iaacd35e9cbe4705d17548518040c679185eaf30a
2021-02-11Added Kafka metrics for CDS workersJulien Fontaine5-32/+67
Added counters to gather metrics on CDS Kafka workers. This will enable us to get metrics on how many messages we consumer and produce to/from kafka. For consumers we count how many messages we consume and how many failed ie. consumed but not able to be processed (parsing error). For producers we count how many messages we produce and how many failed ie. failed to be pushed to the cluster (unavailable brokers, network error, ...). Relocated metrics tag constants to BlueprintConstants so that they can be use by any CDS module. If they make sense for other metrics then they should be shared. Issue-ID: CCSDK-3155 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iad6aba588766f655f3a74cd626e0f74e29188f96
2021-02-11Improve logging for CDS Kafka workersJulien Fontaine2-16/+18
Modified CDS Kafka consumersand producers logs to provide more details about the topic of the consumer record being consumed or published. Refactored the publish callback to make it more readable. Refactored audit service log error messages. Issue-ID: CCSDK-3154 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I7b42930e926bc15ce175974a78d3bfe2f219b0a8
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL13-90/+90
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-10-28Blueprints Processor MetricsJulien Fontaine8-4/+135
Add counter and timer for Blueprints Processor process to get success/failure and execution time for each blueprint execution using blueprint name, version and action. Issue-ID: CCSDK-2950 Change-Id: I38e8997de26effe69ec2ee9e2b6ed0da14de4a43 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)19-82/+117
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-08-10Merge "Make use of Kafka Key for Audit service and Kafka listener"KAPIL SINGAL2-5/+9
2020-08-07Kafka Audit Service: Error during sensitive data hidingJulien Fontaine1-54/+118
Changed behaviour during sensitive data hiding. Instead of looking for every node template defined in the CBA we only check for the ones declared for the current executed workflow. This prevent the issue when you have two independant workflows defined in the same CBA and one of those has a node template using a get_input expression for template-prefix in a component-resource-resolution. If you execute the other workflow audit service was still trying to resolve this expression which was making it fail. Issue-ID: CCSDK-2636 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ib232c26ebf930917e19e7970e1a56c15e676888f
2020-08-04Make use of Kafka Key for Audit service and Kafka listenerJulien Fontaine2-5/+11
* When message is sent by audit service, key will be the CBA name * When sent by kafka listener (self-service api), key is the same as the request message key consumed. If not specified, a random UUID * MessageProducer interface refactoring : * add 'key' parameter to specify a key * add default value null to paramater 'headers' to remove some unnecessary method Issue-ID: CCSDK-2628 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I68580151184c87104c07037f379276dd8c8c71c7
2020-07-24Change EventListener functions to non-blocking scopeJozsef Csongvai1-1/+2
BluePrintProcessingKafkaConsumer and BluePrintProcessorCluster were both defining EventListener functions which were executed with runBlocking. When both were enabled, kafkaconsumer would block and cluster would never run. Issue-ID: CCSDK-2609 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I91813208f0474869f174da6c99280b5159859232
2020-06-22Kafka Audit Service : Improve error handling and miscellaneous refactoringJulien Fontaine5-87/+96
When Kafka Audit Service fails it no longer stops Blueprint Processor execution * Add error handling when trying to hide sensitive data * Add error handling when trying to send kafka message * Set timeout for blocking loop when sending messages with kafka producer -> When broker is not available producer tries to reconnect in a blocking loop * Refactor Audit Service interface to give more explict name for publish methods * Modify publishExecutionOutput() to a non-blocking function Issue-ID: CCSDK-2459 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I809a5f34f81889aa9eed499608348f149984bc38
2020-06-19Error when template-prefix-names list is set as an inputJulien Fontaine1-1/+12
Case when template-prefix-names list for resource resolution is passed as an input was not handled. When the audit service was trying to get input-prefix-names values he was only getting {"get_input": "..."} and was not resolving this expression. Issue-ID: CCSDK-2456 Change-Id: If9ebbf0aa6b6eae3e44a7bfe0b930f622dfd3c1e Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-05-13Kafka Audit Service : Error with CBA using dg-generic node templateJulien Fontaine1-27/+35
* Fixed issue with dg-generic node template * Changed behaviour for hiding sensitive data -> Instead of just checking the node-template targeted by the workflow, we check every node-template declared in the CBA. If we find one or more component-resource-resolution node-template, we verify if we need to hide sensitive data in the kafka message. Issue-ID: CCSDK-2372 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I079ee29ae532b3947cdc8efe3438678309ca50b7
2020-05-11Kafka Audit Service : CorrelationUUID from request is not matching the ↵Julien Fontaine5-9/+7
correct response in Kafka Moved out CorrelationUUID linking between the request and the response from the Kafka Audit Service to the ExecutionServiceHandler. This prevents the race condition happening when several ExecutionServiceOutputs try to set the CorrelationUUID related to their ExecutionServiceInput. Issue-ID: CCSDK-2370 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I0c5934d4486961fbfcb34fd2d2492cd843350025
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine2-10/+25
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-21Secure Kafka AuthenticationJulien Fontaine3-1/+12
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-08Publish execution input/output into Kafka topicsJulien Fontaine7-46/+349
- Creation and implementation of PublishAudit services (NoAudit,Kafka) - Conceal of sensitive data (request input parameters) - Update ExecutionServiceHandler tests - Update application.properties file Issue-ID: CCSDK-2183 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ifc28a91ce588f5ed0010acb4ed22b64429c6c1a0
2020-04-06Refactoring BP Code with ErrorCatalogBrinda Santh11-21/+118
Issue-ID: CCSDK-2124 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ief468a56f9e7b3ef86c357965aa7b15f0a4cfa22
2020-03-18Fix GroupId and package name in Error CatalogDan Timoney1-1/+1
Issue-ID: CCSDK-2180 Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com> Change-Id: Ibe00fc1f2a905821b7100ae4f221ea1e0b934894
2020-03-11Error Catalog Management Core Library.Brinda Santh2-1/+49
Issue-ID: CCSDK-2076 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I9e8e67bdee77fb359a791f4ef4c2963aa78ab93a Signed-off-by: Steve Siani <alphonse.steve.siani.djissitchi@ibm.com>
2020-02-01Expose rest API with non blocking call.Brinda Santh1-8/+5
Convert Mono and Flux to coroutines Convert reactor mdc to coroutine mdc Issue-ID: CCSDK-2052 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic58c0b74866d28fd2d803b96626b08f8e8b2db56
2020-01-31Spring boot, Kotlin version upgradesBrinda Santh1-2/+1
Dependent Patch : https://gerrit.onap.org/r/c/ccsdk/parent/+/100990 Fixed Jackson set method according to latest version. Fixed Security properties issues for failed JUnit test cases. Reused maven properties from parent for Spring boot, Nats, Kafka, etc Issue-ID: CCSDK-1737 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I04bb0e535161e05897f587a0f08a2689e10c5f41
2019-12-11Metadata for name, version, tags and typeBrinda Santh3-5/+13
Mandate Tosca.meta template name, version, type and tags. Auto copy metadata from Tosca.meta to ServiceTemplate definitions. Optimize Blueprint context and runtime creation from file path. Removed attached CBA zip file in test repository dirs Issue-ID: CCSDK-1992 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I5d9d7a4599234a38d431328dbd9b74bd831e0115
2019-11-29Optimize spring data JPA UT.Brinda Santh4-8/+95
Test case based database configuration, so that we can define what repositories and entities can be used for testing. Issue-ID: CCSDK-1735 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4f8a7eb4ed47fec9ab17eb9552648ebd0de01236
2019-11-26Formatting Code base with ktlintSingal, Kapil (ks220y)14-139/+201
No Business logic change, just the code format. Competible with IntelliJ: https://github.com/pinterest/ktlint#option-3 To format run: mvn process-sources -P format Issue-ID: CCSDK-1947 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic9e9209fb7023d77f434693ad5a01229f8d09331
2019-11-09Convert component functions IT to UT.Brinda Santh1-3/+3
Issue-ID: CCSDK-1735 Convert Netconf and Restconf Executor IT to UT. Remove duplicate BluePrintProperties. Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I2eafdbabb17a4df72541ab93c46e7fc9eb0fe8d7
2019-11-07Fix missing status event type workflow response.Brinda Santh4-260/+2
Issue-ID: CCSDK-1896 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Iffb4c5e9d4e5788895bda6c7ee4197ee9e0bd60e
2019-11-05Rolling upgrade support for in-flight requestsSebastien Premont-Tendland3-1/+39
Three entry points are being handled : 1 - REST endpoint 2 - gRPC endpoint 3 - Kafka consumer We make use of Phaser object to make sure the PreDestroy callback wait for all requests to be executed before stopping the process. The docker image was also modified to make sure the java process becomes PID 1 in the container in order to catch the SIGTERM signal which triggers the PreDestroy callback of Spring. This was done by using the "exec" command in bash. Issue-ID: CCSDK-1885 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I3e2a72e26a4c8b7768ebc374ea40aa8d55fb6761
2019-11-04Fixed CDS liveness probe failure during high load.prathamesh1-3/+5
-Added Dispatchers.IO so we can utilize CPU resources efficiently. Issue-ID: CCSDK-1874 Signed-off-by: prathamesh <prathamesh_morde@yahoo.ca> Change-Id: I8bb3c078b22663ca6c2420189b0477b8a66a3bd6
2019-10-29Add remote service function execution.Brinda Santh2-10/+115
Issue-ID: CCSDK-1875 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I18279b1d2ff1a0b2962afa74fc616bfd99e93c91
2019-10-22Add GRPC log tracing service.Brinda Santh1-1/+1
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4ba6ed11d8fb63c21b9c49774ed733cca05c5646
2019-10-02Add reactive log tracing service.Brinda Santh1-11/+15
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic20013045dd5d2681243c03f9e4cdfe557b630be
2019-09-25Add bi-directional GRPC python executor.Brinda Santh1-0/+96
Python based bidirectional communication script executor microservice. Issue-ID: CCSDK-1747 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: If2e8f862ca4cb97d10714df2ab1cccccef2d576a
2019-09-18Optimizing Imports and Formatting codeSingal, Kapil (ks220y)2-6/+1
No Business logic change, no code change.. just optimizing imports and reformatting code Issue-ID: CCSDK-1744 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I1fb2ed10525a6f392f3c0d963c2ed03cfe8dec7e
2019-09-05Move GRPC management api to designer api.Brinda Santh8-555/+56
Change-Id: I58ee303d361cf4f1996c966c094ec66886587b52 Issue-ID: CCSDK-1682 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-05Add Config based blueprint process consumerBrinda Santh12-448/+329
Change-Id: I9e37ecb5032047f835f3b2ea20b2689c76353497 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-04Add Kafka message lib consumer servicesBrinda Santh1-10/+9
Change-Id: Iaf1df07a0d8f4fb54d5d06047520010d3bfe5465 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-04Add draft and publish grpc upload options.Brinda Santh3-19/+53
Change-Id: Id1476f1cb449c4070a7ef53a918b62d26da26391 Issue-ID: CCSDK-1682 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-08-30Combine Blueprint entities and repositories.Brinda Santh1-3/+3
Change-Id: I9e8672d6d0252dcf125e27060322ddd2b455aa39 Issue-ID: CCSDK-1663 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-08-29Resolve duplicate application property valuesBrinda Santh2-10/+10
Change-Id: I0813ff96388bce77eba9fbc9eff0eccf59a6be6a Issue-ID: CCSDK-1663 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-08-27Make source-db consistent across samplesOleg Mitsura2-10/+10
Issue-Id: CCSDK-1623 Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Iec7c6909c39c34894be11bfdac45f6394010d9cf
2019-08-09Add property definition type DSL.Brinda Santh1-1/+3
Change-Id: I83a3d1391c60cd426236b55d0d399e273e7c15dc Issue-ID: CCSDK-1577 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-07-31Fix compiler cache to hold for next transaction.Brinda Santh1-0/+6
Change-Id: Ibede4a378980717d49708f7665ab295534a092a0 Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-07-31Fix conflicting catalog service instance names.Brinda Santh3-9/+8
Change-Id: Iec330d87f9609d42c05f8d4c7c8515fa37310a6e Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-07-24Fix missing capability cli models.Brinda Santh1-19/+21
Change-Id: Id5ad2f75cf27b7c09dc95a028997847098cd3d2e Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-07-23Merge "ExecutionServiceHandler error handler error fix."Brinda Santh Muthuramalingam1-2/+2