summaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2019-10-08Add graphs for performance monitoringkjaniak6-104/+377
Performance tests need graphs representing: total, message queue on HV-VES, offset per kafka topic partition, incoming data rate. Added deletion of producers at test end. Gauges for offsets turned to Range vector (distinguishable based on tag). Since Producer is to fast it was reconfigured to send messages with 1 ms interval. Change-Id: I09fc1edbcd824294d0bb89acc2ea2306a6df00ca Issue-ID: DCAEGEN2-1807 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-10-03Setup Grafana on Cloud (hv-ves)pawel7-0/+2089
Issue-ID: DCAEGEN2-1784 Signed-off-by: pawel <pawel.kasperkiewicz@nokia.com> Change-Id: I87159f360854dccb56e9393bfbe87f66f71dcb6e
2019-09-26Add partition offset metric to each topic partitionkjaniak8-30/+50
Before this commit offset consumer was able to fetch offset just from one partition. This commit solve this. Change-Id: I2c2c300219e43ab422b237094ad775ca8795169e Issue-ID: DCAEGEN2-1783 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-09-10release 1.3.0 hv-ves5.0.2-ONAP5.0.1-ONAPelaltoVijay Venkatesh Kumar1-0/+4
Change-Id: Iacc2a85513a6de9e1ea32a98acf31c42568f1654 Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com> Issue-ID: DCAEGEN2-1669
2019-09-06Add Prometheus deployment for HV-VES performance tests - cloudkjaniak7-7/+226
Script for cloud is updated with proper prometheus config and deployment. Change-Id: Icf2527e76bc2028807653313c8ec978d779c6ec9 Issue-ID: DCAEGEN2-1692 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-08-06Merge "Setup grafana for local performance tests"Piotr Wielebski8-6/+1859
2019-08-01fix hv-ves build issueVijay Venkatesh Kumar1-5/+0
Change-Id: I549bcb1ab068a4983dfcbf8e49a0d4efed46f9cb Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com> Issue-ID: DCAEGEN2-1578
2019-07-21remove docker binding to mvnVijay Venkatesh Kumar1-0/+2
Change-Id: Idd3472fd3c9fb40c6f0312b2ab440126e33f3517 Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com> Issue-ID: DCAEGEN2-1578
2019-07-25Fix build failure in coverage moduleIzabela Zawadzka1-1/+1
xpath command had to be replaced as the versions available for CentOs 7 dont support used options Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com> Issue-ID: DCAEGEN2-1687 Change-Id: I0b9ee73982fa9a348a7822d040ca90435fbc264c
2019-07-19Fix of hv-ves dns configurationkjaniak1-5/+1
Change-Id: I2f859c5a683fb9de0efd2ba93dcd853d8f70df1d Issue-ID: DCAEGEN2-1664 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-07-17Adding script for Performance tests of HV-VES on cloudkjaniak5-22/+163
Change-Id: If71b0e8b04ce23a5daafef632788667ccd50c1a1 Issue-ID: DCAEGEN2-1664 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-07-17Setup grafana for local performance testsIzabela Zawadzka8-6/+1859
Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com> Issue-ID: DCAEGEN2-1666 Change-Id: I49bfccd30e4089a30a235957d3f1a404bfba5e76
2019-07-12Use consumers in mainIzabela Zawadzka6-13/+70
It includes --disable-processing flag. Also fixed some issues with script for local performance test. Also added KafkaConsumer::poll in OffsetKafka Consumer - without it KafkaConsumer::assignment returns empty set Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com> Issue-ID: DCAEGEN2-1657 Change-Id: I95fadb45f321398346094dfa0c4a6e9da954c186
2019-07-08Merge "Measure message travel time"Piotr Jaszczyk8-156/+293
2019-07-05Increase coverage in new consumerFilip Krzywka1-0/+140
Change-Id: I58e88182e84a7d807d481e8cbd7196d8b82309ac Issue-ID: DCAEGEN2-1619 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-07-05Merge "Add performance test producer deployment yaml"Piotr Jaszczyk9-27/+102
2019-07-05Add performance test producer deployment yamlJakub Dudycz9-27/+102
- create cloud performance test directory structure - move consumer deployment file to "tools/performence/cloud" directory Change-Id: Ia5c6d8ed15a2188ff9306cf788052f432dae1e33 Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com> Issue-ID: DCAEGEN2-1652
2019-07-04Measure message travel timeIzabela Zawadzka8-156/+293
Issue-ID: DCAEGEN2-1654 Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com> Change-Id: Ifd6597209c5be51d5b4ff5faf7b3b1b1a2871403
2019-07-04Merge "Change handling of offset metric for consumer app"Piotr Jaszczyk5-15/+26
2019-07-03Merge "Add K8S deployment for kafka-consumer"Piotr Jaszczyk1-0/+111
2019-07-03Merge "Create script to run performance test locally"Piotr Jaszczyk28-1/+354
2019-07-03Change handling of offset metric for consumer appkjaniak5-15/+26
Change-Id: Ia83469783d64d0174624489df671006e72bc8948 Issue-ID: DCAEGEN2-1635 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-07-03Create script to run performance test locallyJakub Dudycz28-1/+354
Change-Id: I6bbf0e9b029aef1558008286f0fbec83b76be44c Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com> Issue-ID: DCAEGEN2-1641
2019-07-02Reorganize kafka modulekjaniak19-197/+494
Change-Id: I2eb7a8a6e92c9d89586b877f4cae438497b62ae2 Issue-ID: DCAEGEN2-1635 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-06-28Add K8S deployment for kafka-consumerFilip Krzywka1-0/+111
Change-Id: Idba5c738dc2062b30c32bb928d701614743f0828 Issue-ID: DCAEGEN2-1619 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-06-27JJB alignment across DCAE components - DCAE HV-VESrajendrajaiswal22-24/+24
Change-Id: I815c32a91a3e710d9076495dbafcba420bde4da5 Issue-ID: DCAEGEN2-1578 Signed-off-by: rajendrajaiswal <rajendra.jaiswal@ericsson.com>
2019-06-26Implement kafka consumer metricsFilip Krzywka9-105/+200
- bump Micrometer version 1.0.8 -> 1.1.5 Change-Id: I7a00fbf252df0f31f12f8743e8719701bd282ce2 Issue-ID: DCAEGEN2-1626 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-06-26Add environment configuration to kafka consumerFilip Krzywka6-125/+174
- HV-VES-specific environment prefix moved inside HvVes modules to allow simpler no-prefix API for other modules - created OptionDSL for brevity Change-Id: I2fabbda1280cc0f913f8a0a04b4a055f39ed1fae Issue-ID: DCAEGEN2-1626 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-06-25Implement message counting in consumerkjaniak20-54/+473
Issue-ID: DCAEGEN2-1635 Change-Id: I2666de7bad27052d9cefa0f687ad0772d4c9a95d Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-06-25Merge "Expose Prometheus metrics in Kafka Consumer"Piotr Jaszczyk8-4/+210
2019-06-19Expose Prometheus metrics in Kafka ConsumerJakub Dudycz8-4/+210
Change-Id: I9d0568a5fc296604d7ac7a45b8bbd0289b845938 Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com> Issue-ID: DCAEGEN2-1626
2019-06-18Fix request intervalFilip Krzywka2-11/+8
In previous implementation DistinctUntilChangedSubscriber always requested from upstream 256 events, which resulted in immediate 256 requests to CBS. Request amount is not configurable in other way than hard-limiting using `limitRequest`, which limits request amount for single subscriber. (At least in our pipeline) To avoid multiple manual subscribes, this commit changed CbsClientAdapter to use Mono instead of Flux for CbsRequests and repeat this Mono conditionally. Flux inside of repeatWhen is emitting event after each onComplete received from upstream Mono and resubscribes to it if condition is met. This seemed like good place to put our interval mechanism, which is always-pass condition, but condition resolving blocks for variable duration. Change-Id: I04d1e657ec4d82185f6f07422c25c2d2ff23e60d Issue-ID: DCAEGEN2-1557 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-06-14Merge "Update docker-maven-plugin to latest version"Piotr Jaszczyk1-1/+1
2019-06-14Create Kafka Consumer project stubJakub Dudycz6-0/+171
Change-Id: Ib45ca6a9ab282dab509e34f9a3d3682373d794d3 Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com> Issue-ID: DCAEGEN2-1619
2019-06-13Update docker-maven-plugin to latest versionJakub Dudycz1-1/+1
Update fabric8io/docker-maven-plugin to version 0.30.0 Change-Id: I528405896443b77e521a783511898298bfd91d2e Issue-ID: DCAEGEN2-1617 Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com>
2019-06-12Retry infinitelyFilip Krzywka9-104/+429
- changed specification to retry infinitely - moved MDC to cbsAdapter constructor as in whole module it contains only local context (instanceID etc.). Also permuted constructor params to match ConfigurationProviders order - refactored module tests as ground for future enhancements Change-Id: Ic074b9c421b60662e5512c55c7b1dfb90ab0d2ea Issue-ID: DCAEGEN2-1557 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-06-07Run container commands as non-rootFilip Krzywka2-1/+8
- 1410 uig/gid is arbitrary, note however that any volume mounted into container should limit permissions to this number - also reduced firstRequestDelay in configuration provided through local docker-compose Change-Id: I77eeebeec5931db5c04f8f6f24d7c6fe7a121015 Issue-ID: DCAEGEN2-1557 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-06-06Merge "Support CBS request interval reconfiguration"Piotr Jaszczyk8-91/+297
2019-06-05Support CBS request interval reconfigurationkjaniak8-91/+297
Change-Id: Ie8892e33b2f6a58d6076f66e6cc6a2df830dfa48 Issue-ID: DCAEGEN2-1525 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-05-30Bump Kotlin and Reactor versionsPiotr Jaszczyk2-2/+12
Issue-ID: DCAEGEN2-1542 Change-Id: I1b14b29a780deedc9c64bdfbded4b3b05e990279 Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
2019-05-10Enable Kafka consumer offset committingFilip Krzywka1-5/+6
It appears that reactor-kafka is setting auto.commit property to false, which makes our CSITs fail nondeterministically due to automatic reset of consumer offset. By acknowledging manually, we will mark every message for committing, which will be performed according to ConsumerConfiguration. This way, Kafka broker should persist consumer offset. Change-Id: I0c5156ff8df9bb3341e733e50a3c6866fdd94976 Issue-ID: DCAEGEN2-1495 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-05-06Update versions for El Alto releasePiotr Jaszczyk21-23/+23
Change-Id: If86dcbf8695cc50a540fe892a4f70b3ad24ac5d1 Issue-ID: DCAEGEN2-1474 Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
2019-04-26Change of KAFKA_ADVERTISED_LISTENERS4.0.0-ONAP1.1.0kjaniak1-1/+1
Fix for DMaaP app bug Change-Id: I05312a6b5d1acc1f7894f17d89922c508bbbd41d Issue-ID: DCAEGEN2-1448 Signed-off-by: kjaniak <kornel.janiak@nokia.com>
2019-04-24Use AAF credentials from stream definitionPiotr Jaszczyk8-33/+229
Change-Id: I4fc20c116c60f6e7d46215a32c33884cd957e93b Issue-ID: DCAEGEN2-1448 Signed-off-by: Piotr Jaszczyk <piotr.jaszczyk@nokia.com>
2019-04-24Use SASL auth in kafka connectionsFilip Krzywka3-15/+58
Change-Id: I55a9289901a6a44f3d07a3cf4e5a028399a5d0dc Issue-ID: DCAEGEN2-1448 Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>
2019-04-16Add internal modifiers to comply with new HvVesCustomRuleIzabela Zawadzka23-51/+58
Change-Id: I33ad24179d05ded58f6784544be9bbdd56210720 Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com> Issue-ID: DCAEGEN2-1438
2019-04-15Merge "Create custom rule to report public modifiers in impl"Piotr Jaszczyk4-2/+199
2019-04-15Create custom rule to report public modifiers in implIzabela Zawadzka4-2/+199
Change-Id: I383ca27a835943bd2dc2508425264ad7f64c7725 Signed-off-by: Izabela Zawadzka <izabela.zawadzka@nokia.com> Issue-ID: DCAEGEN2-1430
2019-04-12Move cmd ssl logic to xnf-simulatorJakub Dudycz14-35/+14
Change-Id: I562acbb0b7522e6cefea2a2dc1a9e292c1227b41 Signed-off-by: Jakub Dudycz <jakub.dudycz@nokia.com> Issue-ID: DCAEGEN2-565
2019-04-12Creation of server modulekjaniak36-195/+340
Issue-ID: DCAEGEN2-1390 Change-Id: I07410b16ed6566b933d5f1efa35bddb965225794 Signed-off-by: kjaniak <kornel.janiak@nokia.com> Signed-off-by: Filip Krzywka <filip.krzywka@nokia.com>