aboutsummaryrefslogtreecommitdiffstats
path: root/ms/blueprintsprocessor/modules/commons
AgeCommit message (Collapse)AuthorFilesLines
2021-02-23Release version 1.1.0 docker containers and roll to next versionDan Timoney9-9/+9
Release version 1.1.0 docker containers and roll to next version Issue-ID: CCSDK-3179 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I2518ba1dd5a4bd926499867f5f9714bc3f716732
2021-02-11Added Kafka metrics for CDS workersJulien Fontaine9-12/+156
Added counters to gather metrics on CDS Kafka workers. This will enable us to get metrics on how many messages we consumer and produce to/from kafka. For consumers we count how many messages we consume and how many failed ie. consumed but not able to be processed (parsing error). For producers we count how many messages we produce and how many failed ie. failed to be pushed to the cluster (unavailable brokers, network error, ...). Relocated metrics tag constants to BlueprintConstants so that they can be use by any CDS module. If they make sense for other metrics then they should be shared. Issue-ID: CCSDK-3155 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Iad6aba588766f655f3a74cd626e0f74e29188f96
2021-02-11Improve logging for CDS Kafka workersJulien Fontaine2-12/+29
Modified CDS Kafka consumersand producers logs to provide more details about the topic of the consumer record being consumed or published. Refactored the publish callback to make it more readable. Refactored audit service log error messages. Issue-ID: CCSDK-3154 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I7b42930e926bc15ce175974a78d3bfe2f219b0a8
2021-01-26PV/PVC eliminationOleg Mitsura3-2/+38
Issue-ID: CCSDK-3086 1. initial commit 2. fix accidental paste / rebased cleaned up unneeded validation call in cmd-exec upload Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: Ife5460c5be59aa8d8592d82099b27c507b08c6c6
2021-01-22Renaming Files having BluePrint to have BlueprintKAPIL SINGAL93-616/+616
Replacing BluePrint with Blueprint throughout Issue-ID: CCSDK-3098 Signed-off-by: KAPIL SINGAL <ks220y@att.com> Change-Id: Ibee8bad07ae7d9287073db2d4f2f2cd730fa8b96
2020-12-16Fixed NoClassDefFoundError when USE_SCRIPT_COMPILE_CACHE is set to falseJulien Fontaine6-4/+178
USE_SCRIPT_COMPILE_CACHE set to false cleans the Class Loader cache after each kotlin script execution. When several kotlin script are executed in parallel (ie no dependency between them) and USE_SCRIPT_COMPILE_CACHE=false then the class loader from the cache may be deleted before one of those executed kotlin script get the time to finish which to the NoClassDefFoundError. Removed cleanupInstance method for kotlin script executors that where causing the class loader to be removed prematurely. Now the behaviour is to remove the class loader from the cache only when we publish a new CBA which was already the case when CDS run with a single instance. In cluster mode, a topic has been added to hazelcast to allow the instance publishing the updated CBA to communicate to the other instances by sending a message to clean the class loader for this CBA from their cache. Added mutex on kotlin script compilation to fix race condition. For concurrent kotlin script execution each process wanted to compile an executable but it was causing a race condition if a process tries to execute while another still compile. Mutex on the execution path prevent this behaviour Issue-ID: CCSDK-3052 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I6ab002352b3272898ad0b183341ee664652c8ae3
2020-12-14Updating parent pom version to 2.1.1-SNAPSHOTSingal, Kapil (ks220y)1-1/+1
Comenting out testSampleAaiReq from RestClientServiceTest.kt as this is causing verify job timeout Issue-ID: CCSDK-3046 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I38d88814335b83167597c1a24162ad3c743355f3
2020-12-08CDS add Swagger annotations for BP Model APIJakobKrieg1-0/+20
Issue-ID: CCSDK-3014 Change-Id: Ibca7cbaa2c70f04acf818e2b554664e13b116c5e Signed-off-by: JakobKrieg <jakob.krieg@bcmsolutions.de>
2020-11-12Remove URI encoding in BlueprintWebClientServiceJozsef Csongvai2-22/+6
Malformed uri's should be handled by resource resolution in the cba Issue-ID: CCSDK-2980 Change-Id: I5c1cf97519a882ff5cf99e9850c65df9ba6d9c36 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-10-21Update AAI endpoints to /aai/v22 for HonoluluLasse Kaihlavirta1-11/+11
- Update all /aai/v21 references to /aai/v22 for Honolulu - note that the previous https://gerrit.onap.org/r/c/ccsdk/cds/+/114057 was for Guilin only Issue-ID: CCSDK-2914 Signed-off-by: Lasse Kaihlavirta <l.kaihlavirt@partner.samsung.com> Change-Id: I8103b12610a9de7263789a0852140f2f0bb09393
2020-10-20Update AAI endpoints to /aai/v21Lasse Kaihlavirta1-11/+11
- Update all /aai/v14 references to /aai/v21 to resolve certificate problems in CDS when trying to resolve attribute values with aai-data source Issue-ID: CCSDK-2914 Signed-off-by: Lasse Kaihlavirta <l.kaihlavirt@partner.samsung.com> Change-Id: I6b150348b66e93b3d08ceaa49fa997d531ddb0e4
2020-09-22Roll to initial Honolulu versionDan Timoney9-9/+9
Roll to initial Honolulu version Issue-ID: CCSDK-2742 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I563033c6c93a4121219eaa1d7bb8a026e9e490d4
2020-09-22Enabling Code FormatterSingal, Kapil (ks220y)66-688/+872
Code Formatter was turned off due to java 11 migation Issue-ID: CCSDK-2852 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: I3d02ed3cc7a93d7551fe25356512cfe8db1517d8
2020-09-11Refactoring poms groupId artifactIdSingal, Kapil (ks220y)9-56/+53
Issue-ID: CCSDK-2787 Signed-off-by: Singal, Kapil (ks220y) <ks220y@att.com> Change-Id: Ica64f0c793beabb47950c7bf1eede63a0dbf3cea
2020-08-27KafkaMessageConsumerService: 'launch' was missingOleg Mitsura1-9/+12
Issue-ID: CCSDK-2704 This was accidentally removed few commits back. Signed-off-by: Oleg Mitsura <oleg.mitsura@amdocs.com> Change-Id: I8f08c72e8d5695c1262aad2d10d1081bbabbdfcb
2020-08-25CommandResult info appears in CDS logs by defaultJulien Fontaine3-2/+13
Add config property to enable/disable logging of the command result. Now command result logs aren't printed by default. Issue-ID: CCSDK-2693 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I5107ed6acc150875264a65bf1c64962ff26cfd78
2020-08-24Add request IDs in command-executor logJulien Fontaine2-0/+14
* Update gRPC interface to handle SubRequestID and Originator ID * Update command-executor log format to handle request IDs in the log statement * Add SubRequest ID and Originator ID in BP log statement Issue-ID: CCSDK-2687 Change-Id: If261bb0912df5800ed2fc65371c5ee784d6a45fb Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-08-04Make use of Kafka Key for Audit service and Kafka listenerJulien Fontaine8-51/+36
* When message is sent by audit service, key will be the CBA name * When sent by kafka listener (self-service api), key is the same as the request message key consumed. If not specified, a random UUID * MessageProducer interface refactoring : * add 'key' parameter to specify a key * add default value null to paramater 'headers' to remove some unnecessary method Issue-ID: CCSDK-2628 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I68580151184c87104c07037f379276dd8c8c71c7
2020-07-29Improve Kafka Producer callback messagesJulien Fontaine1-2/+13
Check if message sent is a BP request/response to print CBA name and version in the callback message. Issue-ID: CCSDK-2623 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I526a01299a3bfca3d1c4a10ae19dead46393a9ae
2020-07-03Add error handling for CDS Kafka Producer callbackJulien Fontaine1-1/+2
Issue-ID: CCSDK-2522 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ia7aabaa6d978a13fbf657ea6b406e0b37bc02222
2020-06-23Fix hazelcast issuesJozsef Csongvai6-51/+63
- confined lock tests to individual threads to ensure correct unlocking - removed silent failure in clusterlock.unlock function when unlock is called by a thread that doesnt own the lock. - added isLockedByCurrentThread method to ClusterLock interface - disabled multicast discovery, tcp-ip should be more stable for tests - fix Hazlecast typo Issue-ID: CCSDK-2429 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: Idfe723fff04fcd9c48510cf429eb15b33662c49d
2020-06-23Merge "Kafka Audit Service : Improve error handling and miscellaneous ↵KAPIL SINGAL4-8/+16
refactoring"
2020-06-22Kafka Audit Service : Improve error handling and miscellaneous refactoringJulien Fontaine4-8/+16
When Kafka Audit Service fails it no longer stops Blueprint Processor execution * Add error handling when trying to hide sensitive data * Add error handling when trying to send kafka message * Set timeout for blocking loop when sending messages with kafka producer -> When broker is not available producer tries to reconnect in a blocking loop * Refactor Audit Service interface to give more explict name for publish methods * Modify publishExecutionOutput() to a non-blocking function Issue-ID: CCSDK-2459 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: I809a5f34f81889aa9eed499608348f149984bc38
2020-06-22solving continuous savingShaabanEltanany1-1/+1
Issue-ID: CCSDK-2336 Signed-off-by: ShaabanEltanany <shaaban.eltanany.ext@orange.com> Change-Id: I7e488950a77a19b7b1b4ac2e4c9044698b9ff757
2020-06-19Implement nodetemplate locking featureJozsef Csongvai4-6/+109
Enables locking execution of a nodetemplate using a lock key and lock acquire timeout. Issue-ID: CCSDK-2460 Change-Id: I308d4d89dab44b7f7a766d5b62258e67b051eab1 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca>
2020-06-16providing search packages functionalitiesShaabanEltanany2-2/+2
- provide search package by (UpdatedBy or Tags or ArtifactName or ArtifactVersion or ArtifactType) Containing IgnoreCase Issue-ID: CCSDK-2434 Signed-off-by: ShaabanEltanany <shaaban.eltanany.ext@orange.com> Change-Id: I92167e916643dbbf17601a0d9fb7654d8940a537
2020-06-15add package description to be at Entry definitionShaabanEltanany1-1/+2
Issue-ID: CCSDK-2438 Signed-off-by: ShaabanEltanany <shaaban.eltanany.ext@orange.com> Change-Id: I8657560cd55a21421dc6849d7aac64c02e83db42
2020-05-19Fix incorrect encoding for query paramsJozsef Csongvai2-1/+37
Issue-ID: CCSDK-2380 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I3a49f2c5a3a29383e43e2a4a3d6a9c13a9f933c2
2020-05-19Merge "Add Naming Service & IP Assign custom resource resolution capability"Dan Timoney1-3/+3
2020-05-18Add Naming Service & IP Assign custom resource resolution capabilityPaira, Saurav(sp694w)1-3/+3
Issue-ID: CCSDK-2185 Signed-off-by: Paira, Saurav(sp694w) <sp694w@att.com> Change-Id: I70550fe9582b2db0a70c7b66591438b674173411
2020-05-05Truncate message published on Kafka / Spike: Define solution for logs separationJulien Fontaine4-8/+54
Refactoring of cmd-exec component - Improve display of error messages within the response - Fix gRPC buffer limitation (4Mb) by truncating error messages and cmd-exec logs if too heavy (>3Mb) Truncation of BP responses (<4Kb) before sending them in kafka audit topics. - Truncation if needed of error messages for every response - Truncation of cmd-exec logs in cmd-exec responses (Spike) Add a flag in the application.properties to enable/disable the display of cmd-exec responses on the BP side (Fix) Correction of BP processing with kafka regression (Fix) Changed default SSL Endpoint Algo Issue-ID: CCSDK-2326 Change-Id: If4d0e661117d1dd156cf19c95774824e754d870a Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-24Merge "Secure Kafka Authentication"Dan Timoney11-168/+794
2020-04-21Secure Kafka AuthenticationJulien Fontaine11-168/+794
Implementation of kafka secure authentication : - SSL - SASL(SCRAM) & SSL Issue-ID: CCSDK-2313 Change-Id: I4b2fc7abab7478e360ebf461608a620d75708f54 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca>
2020-04-16Fix default behavior for DatabaseResourceAssignmentProcessorJozsef Csongvai2-70/+37
If a data-dictionary did not declare an endpoint-selector, it would default to primary cds-db instead of using data-dictionary source to look up database configuration. Processor-db configurations were not used because of this. Also improved error messages and some refactoring. Issue-ID: CCSDK-2315 Signed-off-by: Jozsef Csongvai <jozsef.csongvai@bell.ca> Change-Id: I7122e1334951db4eec5b9ae8b01c1f66596636f6
2020-04-10Merge "Publish execution input/output into Kafka topics"Dan Timoney1-1/+5
2020-04-09Remove unneeded backup filesDan Timoney9-568/+0
Remove unneeded backup files created by updParentVersion script Issue-ID: CCSDK-2308 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I8d76b8ee3ccb2dbe563f22aadf1125f8226a104f
2020-04-08Publish execution input/output into Kafka topicsJulien Fontaine1-1/+5
- Creation and implementation of PublishAudit services (NoAudit,Kafka) - Conceal of sensitive data (request input parameters) - Update ExecutionServiceHandler tests - Update application.properties file Issue-ID: CCSDK-2183 Signed-off-by: Julien Fontaine <julien.fontaine@bell.ca> Change-Id: Ifc28a91ce588f5ed0010acb4ed22b64429c6c1a0
2020-04-07Roll to next Guilin snapshotDan Timoney18-9/+586
Roll to next Guilin snapshot release Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: I1a5ea68da8821ffe28ac84e60177af05e26b3585
2020-03-30Roll to next Frankfurt snapshot and fix sonar buildDan Timoney9-9/+9
Roll to next Frankfurt snapshot. Also, fixed config to skip sonar for error-catalog, which has no scannable components. Sonar builds were failing because error-catalog does not have standard maven source and test directories (src/main, src/test) Issue-ID: CCSDK-2152 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Iacb76b240e71206de8306af7e2447eaa07b0d33b
2020-03-18Use released parent pomsDan Timoney1-1/+0
Use released version of parent poms Issue-ID: CCSDK-2166 Signed-off-by: Dan Timoney <dtimoney@att.com> Change-Id: Ib45fc3134e2ba0e97908d9efa12e3cbec265b381
2020-02-26Merge SW Upgrade Blueprint into PNF_AAI and create one UAT BP for PNFgummar2-3/+8
UAT: Add support to multiple responses for a single request Set property IN_UAT=1 during UAT execution so blueprints can tune their settings to values more suitable for testing (like timeouts) Add 'times' field to specify expected number of invocations Add UAT blueprint script for PNF SW Upgrade UC Add current thread check for Hazelcast distributed lock Resolve URI before returning Issue-ID: CCSDK-2091 Change-Id: Id256bad043488f88f1b60015ebf9ade4be607fa2 Signed-off-by: gummar <raj.gumma@est.tech>
2020-02-25Changed the default multicast address.Sebastien Premont-Tendland6-69/+30
There's an issue with macOS when docker is installed and a multicast address different than 224.0.0.1 is used. https://stackoverflow.com/questions/46341715/hazelcast-multicast-does-not-work-because-of-vboxnet-which-is-used-by-docker-mac Re-organized the test hazelcast yaml since we now fill the cluster-name and instance-name information from the ClusterInfo object during creation. No need to pass it in the config file anymore. This is causing every file to be the exact same thus the consolidation. Also fix an issue with test case using the same hazelcast service in two different threads causing intermittent issue during the build process. Issue-ID: CCSDK-2119 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I7f8328bfd93cdcabe8ab2fa844cbf9dd57f1493a
2020-02-18Add ability to resolve environment variables when readingSebastien Premont-Tendland1-5/+3
PrimaryDatabaseConfiguration This is needed to remove hardcoded password in application.properties file by being able to resolve environment variables. OOM would pass the username and password through environment variables. Issue-ID: CCSDK-2095 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: Ia19ef0ddee29509b0378b048bf58c700870ecce4
2020-02-17Removed manual shutdown hook for hazelcastSebastien Premont-Tendland1-3/+5
Shutdown hook for hazelcast are already working with spring boot. Just changed the policy to GRACEFUL (default is TERMINATE). This allow hazelcast to shutdown gracefully and recover from a rolling upgrade Also updated the Dockerfile to use multi stage to reduce by ~200Mb the docker image size. Issue-ID: CCSDK-2011 Signed-off-by: Sebastien Premont-Tendland <sebastien.premont@bell.ca> Change-Id: I0034145c7acefa88ed8d16f23a29249c7225ace3
2020-02-14Cluster CP member promotion as default.Brinda Santh7-5/+151
Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I4df4b442c020fbf9da294a9b35739ca3acb8ee02
2020-02-12Cluster co-ordination with Hazelcast.Brinda Santh18-551/+631
Remove Atomix implementation, due to Kubernetes clustering issues. Cluster environment property changes. Issue-ID: CCSDK-2011 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I23f40c92c0adc6b3ab8690871385f78525c76433
2020-02-01Expose rest API with non blocking call.Brinda Santh1-0/+28
Convert Mono and Flux to coroutines Convert reactor mdc to coroutine mdc Issue-ID: CCSDK-2052 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Ic58c0b74866d28fd2d803b96626b08f8e8b2db56
2020-01-31Spring boot, Kotlin version upgradesBrinda Santh1-1/+1
Dependent Patch : https://gerrit.onap.org/r/c/ccsdk/parent/+/100990 Fixed Jackson set method according to latest version. Fixed Security properties issues for failed JUnit test cases. Reused maven properties from parent for Spring boot, Nats, Kafka, etc Issue-ID: CCSDK-1737 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I04bb0e535161e05897f587a0f08a2689e10c5f41
2020-01-17Fix scheduler shutdown config function.Brinda Santh1-1/+1
Optimize application name accessing in Docker Compose. Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: I6b055de7ef5a8ac4a0c103499997ed272bd74ac6
2020-01-16Prioritization Optional NATS consumer supportBrinda Santh7-8/+114
Add prioritization NATS consumer service and configuration data beans. Optimizing message prioritization service interface. Added Integration testing for NATS simulation. Updated sample docker compose for NATS support Issue-ID: CCSDK-1917 Signed-off-by: Brinda Santh <bs2796@att.com> Change-Id: Icd21e5e2ab7b64d6e6e4b0610599ca947555ee15