aboutsummaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/commons/message-lib/src/main
AgeCommit message (Collapse)AuthorFilesLines
2023-03-17Enable JUnit tests and porting to java 17Frank Kimmlingen1-25/+30
JUnit tests are no more executed: spring-boot-starter-test does not execute any junit4 tests by default Fix enable tests and adopts the tests moslty to java 17 runtime Issue-ID: CCSDK-3859 Signed-off-by: Frank Kimmlingen <frank.kimmlingen@telekom.de> Signed-off-by: Lukasz Rajewski <lukasz.rajewski@t-mobile.pl> Change-Id: I02a8d25350ca62262bfc6e07c2865cd8d7b4e6b2
2022-08-04Kafka producer parameters updated to be consistent with frankfurtOleg Mitsura1-2/+2
Issue-ID: CCSDK-3737 Signed-off-by: kuldipr <kuldip.rai@amdocs.com> Change-Id: I00cba2f644abf85778abf17ee3fb9e102ac5f42f
2022-06-07Improve log format consistencyJozsef Csongvai2-0/+23
Logs related to request processing should include: requestId, subrequestId and originatorId. Each API (rest,grpc,kafka) would produce different outputs, due to MDC context not being passed properly between coroutine contexts or values not being populated. Issue-ID: CCSDK-3686 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: Ibafdffd3409b9724ad91633ca5840070f7e287f5
2022-05-16Re-enable formatting and fix formatting issuesJozsef Csongvai1-4/+4
Issue-ID: CCSDK-3657 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: Idd1871e82a319325b042c339741a66c06499692c
2022-03-02DMAAP-1706 - New Kafka Auth optiondavid.mcweeney3-0/+67
Change-Id: I8533721d23d6adb41f65cb96fb2b8f852bda47b8 Signed-off-by: david.mcweeney <david.mcweeney@est.tech> Issue-ID: DMAAP-1706
2021-09-10Upgrading velocity version to v2.3 due to some vulnerabilities in previous ↵meriem zidane1-1/+1
version Issue-ID: CCSDK-3430 Signed-off-by: Meriem Zidane <meriem.zidane@bell.ca> Change-Id: I82dad1c2dd7f8e4cc5712e9fddde5b1bb5429b7f
2021-07-28Revert "Renaming Files having BluePrint to have Blueprint"Jozsef Csongvai11-86/+86
The renaming in CCSDK-3098 caused breaking changes to the grpc api and compile issues for kotlin scripts. Issue-ID: CCSDK-3385 Change-Id: I0d745cb858371678eabcb2284671c1fd76a1ab6d Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-06-23Add Request ID and Sub Request ID to CDS kafka logsJulien Fontaine1-8/+5
Improved log message when a message is published or consumed by CDS Kafka to specify the RequestID and SubRequestID. Issue-ID: CCSDK-3348 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I50e068d9aa36df8fe4ee5f0f6a8e9bd7a1209467
2021-03-16Assign a unique worker ID for CDS Kafka workerJulien Fontaine3-33/+47
* Modified CDS Kafka workers to add the 5 lasts characters of the env var HOSTNAME to their worker ID. * Small refactoring to move some utilitary functions to BlueprintMessageUtils Issue-ID: CCSDK-3204 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iaacd35e9cbe4705d17548518040c679185eaf30a
2021-02-11Added Kafka metrics for CDS workersJulien Fontaine5-11/+103
Added counters to gather metrics on CDS Kafka workers. This will enable us to get metrics on how many messages we consumer and produce to/from kafka. For consumers we count how many messages we consume and how many failed ie. consumed but not able to be processed (parsing error). For producers we count how many messages we produce and how many failed ie. failed to be pushed to the cluster (unavailable brokers, network error, ...). Relocated metrics tag constants to BlueprintConstants so that they can be use by any CDS module. If they make sense for other metrics then they should be shared. Issue-ID: CCSDK-3155 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iad6aba588766f655f3a74cd626e0f74e29188f96
2021-02-11Improve logging for CDS Kafka workersJulien Fontaine2-12/+29
Modified CDS Kafka consumersand producers logs to provide more details about the topic of the consumer record being consumed or published. Refactored the publish callback to make it more readable. Refactored audit service log error messages. Issue-ID: CCSDK-3154 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I7b42930e926bc15ce175974a78d3bfe2f219b0a8
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL10-83/+83
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)8-103/+128
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-08-27KafkaMessageConsumerService: 'launch' was missingOleg Mitsura1-9/+12
Issue-ID: CCSDK-2704 This was accidentally removed few commits back. Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I8f08c72e8d5695c1262aad2d10d1081bbabbdfcb
2020-08-04Make use of Kafka Key for Audit service and Kafka listenerJulien Fontaine5-49/+26
* When message is sent by audit service, key will be the CBA name * When sent by kafka listener (self-service api), key is the same as the request message key consumed. If not specified, a random UUID * MessageProducer interface refactoring : * add 'key' parameter to specify a key * add default value null to paramater 'headers' to remove some unnecessary method Issue-ID: CCSDK-2628 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I68580151184c87104c07037f379276dd8c8c71c7
2020-07-29Improve Kafka Producer callback messagesJulien Fontaine1-2/+13
Check if message sent is a BP request/response to print CBA name and version in the callback message. Issue-ID: CCSDK-2623 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I526a01299a3bfca3d1c4a10ae19dead46393a9ae
2020-07-03Add error handling for CDS Kafka Producer callbackJulien Fontaine1-1/+2
Issue-ID: CCSDK-2522 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ia7aabaa6d978a13fbf657ea6b406e0b37bc02222
2020-06-22Kafka Audit Service : Improve error handling and miscellaneous refactoringJulien Fontaine2-7/+12
When Kafka Audit Service fails it no longer stops Blueprint Processor execution * Add error handling when trying to hide sensitive data * Add error handling when trying to send kafka message * Set timeout for blocking loop when sending messages with kafka producer -> When broker is not available producer tries to reconnect in a blocking loop * Refactor Audit Service interface to give more explict name for publish methods * Modify publishExecutionOutput() to a non-blocking function Issue-ID: CCSDK-2459 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I809a5f34f81889aa9eed499608348f149984bc38
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine2-6/+52
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-21Secure Kafka AuthenticationJulien Fontaine7-137/+590
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2019-12-31Flexible DSL Types and Templates definition.Brinda Santh1-0/+13
Easy search definitions Types and Templates inside ServiceTemplate DSL builder. Unit test modifications to support this change. Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ie944ff5f75f80c852555306e1a4e0fa7f5b803d7
2019-12-30Relationship Type and Templates modelsBrinda Santh2-2/+261
Enrichment Support for Relationship Types and Templates. Relationship DSL support for ConnectTo connections ( RestClient, SshClient, MessageProducer, MessageConsume, Nats) Moved datatype map from collection to complex type Issue-ID: CCSDK-1054 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I0f18db2cb52e1e93dfab04498b8298587cba2540
2019-12-06Remote Script Executor ComponentBrinda Santh1-2/+1
Define and Implement component-remote-script-executor component and DSL. Get the timeout from model definitions fix the dat pattern issues Define API data extension functions. Issue-ID: CCSDK-1975 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I41a429eb21a050e5ab512a7a73a394b975543f31
2019-11-26Formatting Code base with ktlintSingal, Kapil (ks220y)11-52/+81
No Business logic change, just the code format. Competible with IntelliJ: https://github.com/pinterest/ktlint#option-3 To format run: mvn process-sources -P format Issue-ID: CCSDK-1947 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ic9e9209fb7023d77f434693ad5a01229f8d09331
2019-11-18Fix Message Prioritization merge conflictsBrinda Santh1-1/+1
Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I92cdca04036c23c212b7b333ca14455eef20555c
2019-11-18Merge "Add message prioritization module"Dan Timoney3-0/+209
2019-11-18Merge "Add Kafka Streams consumer service"Dan Timoney6-2/+114
2019-11-13Add message prioritization moduleBrinda Santh3-0/+209
Kafka streams based solution for message prioritization using database store. Implement initial Abstract Processors, Puntuations and sample Topology for easy plug and play based on situations Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I9c135604574cc3c642186545e076d6a7c60048d4
2019-11-09Convert component functions IT to UT.Brinda Santh1-6/+6
Issue-ID: CCSDK-1735 Convert Netconf and Restconf Executor IT to UT. Remove duplicate BluePrintProperties. Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I2eafdbabb17a4df72541ab93c46e7fc9eb0fe8d7
2019-11-08Add Kafka Streams consumer serviceBrinda Santh6-2/+114
Issue-ID: CCSDK-1914 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I8d2b51c66e1304decadbb55656fe8a0b4c018feb
2019-10-29Add message consumer dynamic functions.Brinda Santh3-11/+82
Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I51be88598557a05ef5db7cd595689a8e4a653fdc
2019-10-28Add Message tracing logger service.Brinda Santh4-2/+129
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I509df82ec558cd96934043a5b41ea53aa040cc81
2019-10-28Add Message Header support for tracing.Brinda Santh4-48/+72
Issue-ID: CCSDK-1046 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: If6a416fe157a1a8ed4d93878142b3c959d49a035
2019-10-04Kafka Back pressure configurationBrinda Santh2-0/+6
Issue-ID: CCSDK-1666 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I173672b14e07a89e0bd2d1ea0e47543b37f389f8
2019-09-05Add Config based blueprint process consumerBrinda Santh4-15/+17
Change-Id: I9e37ecb5032047f835f3b2ea20b2689c76353497 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-09-04Add Kafka message lib consumer servicesBrinda Santh5-18/+264
Change-Id: Iaf1df07a0d8f4fb54d5d06047520010d3bfe5465 Issue-ID: CCSDK-1668 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>
2019-06-25Kafka Messaging Controller API.prathameshmo1-3/+2
Things done- Addressed review comments. Logic to consume events and process it. Added integration testing. Change-Id: If574a363f9fb8581018cc5a7ba106251a9d8caf1 Issue-ID:CCSDK-1356 Signed-off-by: prathamesh morde <prathamesh.morde@bell.ca> Signed-off-by: prathameshmo <prathamesh.morde@bell.ca>
2019-05-23Add Kafka client servicesBrinda Santh5-0/+274
Change-Id: I76cf52f6df10e114539c4d65620f431e0f747644 Issue-ID: CCSDK-1349 Signed-off-by: Brinda Santh <brindasanth@in.ibm.com>