summaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/commons/message-lib/src
AgeCommit message (Collapse)AuthorFilesLines
2021-02-11Added Kafka metrics for CDS workersJulien Fontaine9-12/+156
Added counters to gather metrics on CDS Kafka workers. This will enable us to get metrics on how many messages we consumer and produce to/from kafka. For consumers we count how many messages we consume and how many failed ie. consumed but not able to be processed (parsing error). For producers we count how many messages we produce and how many failed ie. failed to be pushed to the cluster (unavailable brokers, network error, ...). Relocated metrics tag constants to BlueprintConstants so that they can be use by any CDS module. If they make sense for other metrics then they should be shared. Issue-ID: CCSDK-3155 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iad6aba588766f655f3a74cd626e0f74e29188f96
2021-02-11Improve logging for CDS Kafka workersJulien Fontaine2-12/+29
Modified CDS Kafka consumersand producers logs to provide more details about the topic of the consumer record being consumed or published. Refactored the publish callback to make it more readable. Refactored audit service log error messages. Issue-ID: CCSDK-3154 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I7b42930e926bc15ce175974a78d3bfe2f219b0a8
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL16-117/+117
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)13-231/+276
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-08-27KafkaMessageConsumerService: 'launch' was missingOleg Mitsura1-9/+12
Issue-ID: CCSDK-2704 This was accidentally removed few commits back. Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I8f08c72e8d5695c1262aad2d10d1081bbabbdfcb
2020-08-04Make use of Kafka Key for Audit service and Kafka listenerJulien Fontaine8-51/+36
* When message is sent by audit service, key will be the CBA name * When sent by kafka listener (self-service api), key is the same as the request message key consumed. If not specified, a random UUID * MessageProducer interface refactoring : * add 'key' parameter to specify a key * add default value null to paramater 'headers' to remove some unnecessary method Issue-ID: CCSDK-2628 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I68580151184c87104c07037f379276dd8c8c71c7
2020-07-29Improve Kafka Producer callback messagesJulien Fontaine1-2/+13
Check if message sent is a BP request/response to print CBA name and version in the callback message. Issue-ID: CCSDK-2623 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I526a01299a3bfca3d1c4a10ae19dead46393a9ae
2020-07-03Add error handling for CDS Kafka Producer callbackJulien Fontaine1-1/+2
Issue-ID: CCSDK-2522 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ia7aabaa6d978a13fbf657ea6b406e0b37bc02222
2020-06-22Kafka Audit Service : Improve error handling and miscellaneous refactoringJulien Fontaine4-8/+16
When Kafka Audit Service fails it no longer stops Blueprint Processor execution * Add error handling when trying to hide sensitive data * Add error handling when trying to send kafka message * Set timeout for blocking loop when sending messages with kafka producer -> When broker is not available producer tries to reconnect in a blocking loop * Refactor Audit Service interface to give more explict name for publish methods * Modify publishExecutionOutput() to a non-blocking function Issue-ID: CCSDK-2459 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I809a5f34f81889aa9eed499608348f149984bc38
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine4-8/+54
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-21Secure Kafka AuthenticationJulien Fontaine11-168/+794
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2019-12-31Flexible DSL Types and Templates definition.Brinda Santh2-15/+45
Easy search definitions Types and Templates inside ServiceTemplate DSL builder. Unit test modifications to support this change. Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ie944ff5f75f80c852555306e1a4e0fa7f5b803d7
2019-12-30Relationship Type and Templates modelsBrinda Santh3-2/+362
Enrichment Support for Relationship Types and Templates. Relationship DSL support for ConnectTo connections ( RestClient, SshClient, MessageProducer, MessageConsume, Nats) Moved datatype map from collection to complex type Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I0f18db2cb52e1e93dfab04498b8298587cba2540
2019-12-06Remote Script Executor ComponentBrinda Santh1-2/+1
Define and Implement component-remote-script-executor component and DSL. Get the timeout from model definitions fix the dat pattern issues Define API data extension functions. Issue-ID: CCSDK-1975 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I41a429eb21a050e5ab512a7a73a394b975543f31
2019-11-26Formatting Code base with ktlintSingal, Kapil (ks220y)16-133/+185
No Business logic change, just the code format. Competible with IntelliJ: https://github.com/pinterest/ktlint#option-3 To format run: mvn process-sources -P format Issue-ID: CCSDK-1947 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic9e9209fb7023d77f434693ad5a01229f8d09331
2019-11-18Fix Message Prioritization merge conflictsBrinda Santh2-4/+4
Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I92cdca04036c23c212b7b333ca14455eef20555c
2019-11-18Merge "Add message prioritization module"Dan Timoney3-0/+209
2019-11-18Merge "Add Kafka Streams consumer service"Dan Timoney8-2/+343
2019-11-13Add message prioritization moduleBrinda Santh3-0/+209
Kafka streams based solution for message prioritization using database store. Implement initial Abstract Processors, Puntuations and sample Topology for easy plug and play based on situations Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I9c135604574cc3c642186545e076d6a7c60048d4
2019-11-09Convert component functions IT to UT.Brinda Santh3-12/+12
Issue-ID: CCSDK-1735 Convert Netconf and Restconf Executor IT to UT. Remove duplicate BluePrintProperties. Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I2eafdbabb17a4df72541ab93c46e7fc9eb0fe8d7
2019-11-08Add Kafka Streams consumer serviceBrinda Santh8-2/+343
Issue-ID: CCSDK-1914 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I8d2b51c66e1304decadbb55656fe8a0b4c018feb
2019-10-29Add message consumer dynamic functions.Brinda Santh5-15/+134
Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I51be88598557a05ef5db7cd595689a8e4a653fdc
2019-10-28Add Message tracing logger service.Brinda Santh6-3/+199
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I509df82ec558cd96934043a5b41ea53aa040cc81
2019-10-28Add Message Header support for tracing.Brinda Santh6-59/+86
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: If6a416fe157a1a8ed4d93878142b3c959d49a035
2019-10-04Kafka Back pressure configurationBrinda Santh3-2/+9
Issue-ID: CCSDK-1666 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I173672b14e07a89e0bd2d1ea0e47543b37f389f8
2019-09-05Add Config based blueprint process consumerBrinda Santh6-22/+64
Change-Id: I9e37ecb5032047f835f3b2ea20b2689c76353497 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-04Add Kafka message lib consumer servicesBrinda Santh8-29/+375
Change-Id: Iaf1df07a0d8f4fb54d5d06047520010d3bfe5465 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-06-25Kafka Messaging Controller API.prathameshmo1-3/+2
Things done- Addressed review comments. Logic to consume events and process it. Added integration testing. Change-Id: If574a363f9fb8581018cc5a7ba106251a9d8caf1 Issue-ID:CCSDK-1356 Signed-off-by: prathamesh morde <prathamesh.morde@bell.ca> Signed-off-by: prathameshmo <prathamesh.morde@bell.ca>
2019-05-23Add Kafka client servicesBrinda Santh7-0/+388
Change-Id: I76cf52f6df10e114539c4d65620f431e0f747644 Issue-ID: CCSDK-1349 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>