summaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/commons
AgeCommit message (Collapse)AuthorFilesLines
2021-07-28Revert "Renaming Files having BluePrint to have Blueprint"Jozsef Csongvai96-630/+630
The renaming in CCSDK-3098 caused breaking changes to the grpc api and compile issues for kotlin scripts. Issue-ID: CCSDK-3385 Change-Id: I0d745cb858371678eabcb2284671c1fd76a1ab6d Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2021-06-23Add Request ID and Sub Request ID to CDS kafka logsJulien Fontaine3-22/+23
Improved log message when a message is published or consumed by CDS Kafka to specify the RequestID and SubRequestID. Issue-ID: CCSDK-3348 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I50e068d9aa36df8fe4ee5f0f6a8e9bd7a1209467
2021-05-13Add MS SQL driver in CDS blueprints andPrudence Au5-1/+77
DB resolution fails on an empty dataset with required template mapping. Issue-ID: CCSDK-3293 Issue-ID: CCSDK-3294 Signed-off-by: Prudence Au <prudence.au@amdocs.com> Change-Id: I9f302d12e7359b62180fcdfddbd8ae2c993a8324
2021-04-28Fixed error when uploading CBA when Hazelcast cluster is not initializedJulien Fontaine1-4/+2
Hazelcast is used to enable CDS pods to notify the rest of the CDS cluster when a CBA got updated locally so they clean their compile cache (if they have an entry for that CBA). Though, the code doesn't make sure that hazelcast cluster is fully initialized before trying to notify using a hazelcast topic. Fixed that error by making sure that the cluster is fully initialized before pushing notification to hazelcast topic. Issue-ID: CCSDK-3277 Change-Id: I44211c63c5a991269005bcc238f18b19979cb718 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2021-04-07Remove trust all for BasicAuthRestClientServiceClaudio D. Gasparini1-18/+0
following global requirement expected in Istanbul, which forbids disabling host name verification Issue-ID: CCSDK-3245 Signed-off-by: Claudio D. Gasparini <claudio.gasparini@intl.att.com> Change-Id: I89c37a94cb3999a8f1813d3fc33958421b183de1
2021-03-25Address some pendings items for SSLBasicAuthRestClientClaudio D. Gasparini5-27/+50
- FIXME under SSLBasicAuthRestClientProperties - TODO under SSLBasicAuthRestClientPropertiesBuilder Issue-ID: CCSDK-3235 Signed-off-by: Claudio D. Gasparini <claudio.gasparini@intl.att.com> Change-Id: If00c7491573c69eb5d3f9192f54c7b384943abf7
2021-03-23Fix Certificate Exception when using BasicAuthRestClientServiceClaudio D. Gasparini1-0/+18
by using TrustAllStrategy Issue-ID: CCSDK-3234 Signed-off-by: Claudio D. Gasparini <claudio.gasparini@intl.att.com> Change-Id: I7ee63739b4f1d7a4ddbc618ba660704aa058752a
2021-03-16Assign a unique worker ID for CDS Kafka workerJulien Fontaine6-36/+112
* Modified CDS Kafka workers to add the 5 lasts characters of the env var HOSTNAME to their worker ID. * Small refactoring to move some utilitary functions to BlueprintMessageUtils Issue-ID: CCSDK-3204 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iaacd35e9cbe4705d17548518040c679185eaf30a
2021-03-09Roll master version for IstanbulDan Timoney9-9/+9
Roll versions of ccskd parent poms to 2.2.0-SNAPSHOT and of ccsdk/cds to 1.2.0-SNAPSHOT for Istanbul release Issue-ID: CCSDK-3190 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Iab0bf047c8d2e4ddaaadf2e6278917ed2de338e5
2021-03-04Release version 1.1.1 of CDS dockersDan Timoney9-9/+9
Release new CDS dockers to correct issues found with release 1.1.0 dockers. Issue-ID: CCSDK-3192 Issue-ID: CCSDK-3197 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I3bc1acbb93f0bb95ef14a6565e66d8da7a5df21d
2021-02-23Release version 1.1.0 docker containers and roll to next versionDan Timoney9-9/+9
Release version 1.1.0 docker containers and roll to next version Issue-ID: CCSDK-3179 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I2518ba1dd5a4bd926499867f5f9714bc3f716732
2021-02-11Added Kafka metrics for CDS workersJulien Fontaine9-12/+156
Added counters to gather metrics on CDS Kafka workers. This will enable us to get metrics on how many messages we consumer and produce to/from kafka. For consumers we count how many messages we consume and how many failed ie. consumed but not able to be processed (parsing error). For producers we count how many messages we produce and how many failed ie. failed to be pushed to the cluster (unavailable brokers, network error, ...). Relocated metrics tag constants to BlueprintConstants so that they can be use by any CDS module. If they make sense for other metrics then they should be shared. Issue-ID: CCSDK-3155 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iad6aba588766f655f3a74cd626e0f74e29188f96
2021-02-11Improve logging for CDS Kafka workersJulien Fontaine2-12/+29
Modified CDS Kafka consumersand producers logs to provide more details about the topic of the consumer record being consumed or published. Refactored the publish callback to make it more readable. Refactored audit service log error messages. Issue-ID: CCSDK-3154 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I7b42930e926bc15ce175974a78d3bfe2f219b0a8
2021-01-26PV/PVC eliminationOleg Mitsura3-2/+38
Issue-ID: CCSDK-3086 1. initial commit 2. fix accidental paste / rebased cleaned up unneeded validation call in cmd-exec upload Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ife5460c5be59aa8d8592d82099b27c507b08c6c6
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL93-616/+616
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-12-16Fixed NoClassDefFoundError when USE_SCRIPT_COMPILE_CACHE is set to falseJulien Fontaine6-4/+178
USE_SCRIPT_COMPILE_CACHE set to false cleans the Class Loader cache after each kotlin script execution. When several kotlin script are executed in parallel (ie no dependency between them) and USE_SCRIPT_COMPILE_CACHE=false then the class loader from the cache may be deleted before one of those executed kotlin script get the time to finish which to the NoClassDefFoundError. Removed cleanupInstance method for kotlin script executors that where causing the class loader to be removed prematurely. Now the behaviour is to remove the class loader from the cache only when we publish a new CBA which was already the case when CDS run with a single instance. In cluster mode, a topic has been added to hazelcast to allow the instance publishing the updated CBA to communicate to the other instances by sending a message to clean the class loader for this CBA from their cache. Added mutex on kotlin script compilation to fix race condition. For concurrent kotlin script execution each process wanted to compile an executable but it was causing a race condition if a process tries to execute while another still compile. Mutex on the execution path prevent this behaviour Issue-ID: CCSDK-3052 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I6ab002352b3272898ad0b183341ee664652c8ae3
2020-12-14Updating parent pom version to 2.1.1-SNAPSHOTSingal, Kapil (ks220y)1-1/+1
Comenting out testSampleAaiReq from RestClientServiceTest.kt as this is causing verify job timeout Issue-ID: CCSDK-3046 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I38d88814335b83167597c1a24162ad3c743355f3
2020-12-08CDS add Swagger annotations for BP Model APIJakobKrieg1-0/+20
Issue-ID: CCSDK-3014 Change-Id: Ibca7cbaa2c70f04acf818e2b554664e13b116c5e Signed-off-by: JakobKrieg <jakob.krieg@bcmsolutions.de>
2020-11-12Remove URI encoding in BlueprintWebClientServiceJozsef Csongvai2-22/+6
Malformed uri's should be handled by resource resolution in the cba Issue-ID: CCSDK-2980 Change-Id: I5c1cf97519a882ff5cf99e9850c65df9ba6d9c36 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-10-21Update AAI endpoints to /aai/v22 for HonoluluLasse Kaihlavirta1-11/+11
- Update all /aai/v21 references to /aai/v22 for Honolulu - note that the previous https://gerrit.onap.org/r/c/ccsdk/cds/+/114057 was for Guilin only Issue-ID: CCSDK-2914 Signed-off-by: Lasse Kaihlavirta <l.kaihlavirt@partner.samsung.com> Change-Id: I8103b12610a9de7263789a0852140f2f0bb09393
2020-10-20Update AAI endpoints to /aai/v21Lasse Kaihlavirta1-11/+11
- Update all /aai/v14 references to /aai/v21 to resolve certificate problems in CDS when trying to resolve attribute values with aai-data source Issue-ID: CCSDK-2914 Signed-off-by: Lasse Kaihlavirta <l.kaihlavirt@partner.samsung.com> Change-Id: I6b150348b66e93b3d08ceaa49fa997d531ddb0e4
2020-09-22Roll to initial Honolulu versionDan Timoney9-9/+9
Roll to initial Honolulu version Issue-ID: CCSDK-2742 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I563033c6c93a4121219eaa1d7bb8a026e9e490d4
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)66-688/+872
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-09-11Refactoring poms groupId artifactIdSingal, Kapil (ks220y)9-56/+53
Issue-ID: CCSDK-2787 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ica64f0c793beabb47950c7bf1eede63a0dbf3cea
2020-08-27KafkaMessageConsumerService: 'launch' was missingOleg Mitsura1-9/+12
Issue-ID: CCSDK-2704 This was accidentally removed few commits back. Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I8f08c72e8d5695c1262aad2d10d1081bbabbdfcb
2020-08-25CommandResult info appears in CDS logs by defaultJulien Fontaine3-2/+13
Add config property to enable/disable logging of the command result. Now command result logs aren't printed by default. Issue-ID: CCSDK-2693 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I5107ed6acc150875264a65bf1c64962ff26cfd78
2020-08-24Add request IDs in command-executor logJulien Fontaine2-0/+14
* Update gRPC interface to handle SubRequestID and Originator ID * Update command-executor log format to handle request IDs in the log statement * Add SubRequest ID and Originator ID in BP log statement Issue-ID: CCSDK-2687 Change-Id: If261bb0912df5800ed2fc65371c5ee784d6a45fb Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-08-04Make use of Kafka Key for Audit service and Kafka listenerJulien Fontaine8-51/+36
* When message is sent by audit service, key will be the CBA name * When sent by kafka listener (self-service api), key is the same as the request message key consumed. If not specified, a random UUID * MessageProducer interface refactoring : * add 'key' parameter to specify a key * add default value null to paramater 'headers' to remove some unnecessary method Issue-ID: CCSDK-2628 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I68580151184c87104c07037f379276dd8c8c71c7
2020-07-29Improve Kafka Producer callback messagesJulien Fontaine1-2/+13
Check if message sent is a BP request/response to print CBA name and version in the callback message. Issue-ID: CCSDK-2623 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I526a01299a3bfca3d1c4a10ae19dead46393a9ae
2020-07-03Add error handling for CDS Kafka Producer callbackJulien Fontaine1-1/+2
Issue-ID: CCSDK-2522 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ia7aabaa6d978a13fbf657ea6b406e0b37bc02222
2020-06-23Fix hazelcast issuesJozsef Csongvai6-51/+63
- confined lock tests to individual threads to ensure correct unlocking - removed silent failure in clusterlock.unlock function when unlock is called by a thread that doesnt own the lock. - added isLockedByCurrentThread method to ClusterLock interface - disabled multicast discovery, tcp-ip should be more stable for tests - fix Hazlecast typo Issue-ID: CCSDK-2429 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: Idfe723fff04fcd9c48510cf429eb15b33662c49d
2020-06-23Merge "Kafka Audit Service : Improve error handling and miscellaneous ↵KAPIL SINGAL4-8/+16
refactoring"
2020-06-22Kafka Audit Service : Improve error handling and miscellaneous refactoringJulien Fontaine4-8/+16
When Kafka Audit Service fails it no longer stops Blueprint Processor execution * Add error handling when trying to hide sensitive data * Add error handling when trying to send kafka message * Set timeout for blocking loop when sending messages with kafka producer -> When broker is not available producer tries to reconnect in a blocking loop * Refactor Audit Service interface to give more explict name for publish methods * Modify publishExecutionOutput() to a non-blocking function Issue-ID: CCSDK-2459 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I809a5f34f81889aa9eed499608348f149984bc38
2020-06-22solving continuous savingShaabanEltanany1-1/+1
Issue-ID: CCSDK-2336 Signed-off-by: ShaabanEltanany <shaaban.eltanany.ext@orange.com> Change-Id: I7e488950a77a19b7b1b4ac2e4c9044698b9ff757
2020-06-19Implement nodetemplate locking featureJozsef Csongvai4-6/+109
Enables locking execution of a nodetemplate using a lock key and lock acquire timeout. Issue-ID: CCSDK-2460 Change-Id: I308d4d89dab44b7f7a766d5b62258e67b051eab1 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-06-16providing search packages functionalitiesShaabanEltanany2-2/+2
- provide search package by (UpdatedBy or Tags or ArtifactName or ArtifactVersion or ArtifactType) Containing IgnoreCase Issue-ID: CCSDK-2434 Signed-off-by: ShaabanEltanany <shaaban.eltanany.ext@orange.com> Change-Id: I92167e916643dbbf17601a0d9fb7654d8940a537
2020-06-15add package description to be at Entry definitionShaabanEltanany1-1/+2
Issue-ID: CCSDK-2438 Signed-off-by: ShaabanEltanany <shaaban.eltanany.ext@orange.com> Change-Id: I8657560cd55a21421dc6849d7aac64c02e83db42
2020-05-19Fix incorrect encoding for query paramsJozsef Csongvai2-1/+37
Issue-ID: CCSDK-2380 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I3a49f2c5a3a29383e43e2a4a3d6a9c13a9f933c2
2020-05-19Merge "Add Naming Service & IP Assign custom resource resolution capability"Dan Timoney1-3/+3
2020-05-18Add Naming Service & IP Assign custom resource resolution capabilityPaira, Saurav(sp694w)1-3/+3
Issue-ID: CCSDK-2185 Signed-off-by: Paira, Saurav(sp694w) <sp694w@att.com> Change-Id: I70550fe9582b2db0a70c7b66591438b674173411
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine4-8/+54
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-24Merge "Secure Kafka Authentication"Dan Timoney11-168/+794
2020-04-21Secure Kafka AuthenticationJulien Fontaine11-168/+794
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-16Fix default behavior for DatabaseResourceAssignmentProcessorJozsef Csongvai2-70/+37
If a data-dictionary did not declare an endpoint-selector, it would default to primary cds-db instead of using data-dictionary source to look up database configuration. Processor-db configurations were not used because of this. Also improved error messages and some refactoring. Issue-ID: CCSDK-2315 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I7122e1334951db4eec5b9ae8b01c1f66596636f6
2020-04-10Merge "Publish execution input/output into Kafka topics"Dan Timoney1-1/+5
2020-04-09Remove unneeded backup filesDan Timoney9-568/+0
Remove unneeded backup files created by updParentVersion script Issue-ID: CCSDK-2308 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I8d76b8ee3ccb2dbe563f22aadf1125f8226a104f
2020-04-08Publish execution input/output into Kafka topicsJulien Fontaine1-1/+5
- Creation and implementation of PublishAudit services (NoAudit,Kafka) - Conceal of sensitive data (request input parameters) - Update ExecutionServiceHandler tests - Update application.properties file Issue-ID: CCSDK-2183 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ifc28a91ce588f5ed0010acb4ed22b64429c6c1a0
2020-04-07Roll to next Guilin snapshotDan Timoney18-9/+586
Roll to next Guilin snapshot release Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I1a5ea68da8821ffe28ac84e60177af05e26b3585
2020-03-30Roll to next Frankfurt snapshot and fix sonar buildDan Timoney9-9/+9
Roll to next Frankfurt snapshot. Also, fixed config to skip sonar for error-catalog, which has no scannable components. Sonar builds were failing because error-catalog does not have standard maven source and test directories (src/main, src/test) Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Iacb76b240e71206de8306af7e2447eaa07b0d33b
2020-03-18Use released parent pomsDan Timoney1-1/+0
Use released version of parent poms Issue-ID: CCSDK-2166 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Ib45fc3134e2ba0e97908d9efa12e3cbec265b381