aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2019-06-18Do not publish Dublin DFC images with 'latest' tagdublinmaximesson1-12/+0
Change-Id: Ia975d464c27a0e4df560f013e52732cb9a2bf33a Issue-ID: DCAEGEN2-1603 Signed-off-by: maximesson <maxime.bonneau@est.tech>
2019-05-20Fix, skip FTP retry in certain cases4.0.0-ONAP1.1.3PatrikBuhr11-17/+81
In certain conditions there is no reason to retry fetching files. For instance when the file is removed in the PNF or when the password/certificate is wrong. When the DFC is started there are sometimes queued VES events that referes to removed files which in turn results in that the DFC will retry fetching these files in vain. The DFC house keeps its number of concurrents tasks to not exeed quotas for memory,open file descriptors etc. As more threads are occupied with retrying, the fewer threads can do their intended work, which decreases the throughput. Testing has showed that already when the number of PNFs are 10, the throughput is radically decreased (and the problem is then escalating). Change-Id: I9fd57b6b5209bae1ffb4191f5274b591c346b79a Issue-ID: DCAEGEN2-1508 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-05-03Ownership fix in config directoryTamasBakai5-5/+5
Change-Id: Iec3ca75c42642da27c52bccd6d60811c434f778a Issue-ID: DCAEGEN2-1466 Signed-off-by: TamasBakai <tamas.bakai@est.tech>
2019-04-25Extend Swagger for actuator end-points also1.1.21.1.1TamasBakai1-10/+12
Change-Id: Ia036f7d79d25ceb2f1e465d79f17db1acbabbfbf Issue-ID: DCAEGEN2-1443 Signed-off-by: TamasBakai <tamas.bakai@est.tech>
2019-04-16Bugfix, serializing in DataRouterPublisherPatrikBuhr2-1/+8
This fixes a bug introduced in c7b0c7014688c447d411f791a094440403e65228. The new property must be removed before publishing to data router. The consequense of the bug is that the DR rejects the publishing. Change-Id: Ib42cbed4decdba4c13678cea47dff35ca77679a5 Issue-ID: DCAEGEN2-1305 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-04-12Improved loggingPatrikBuhr12-102/+169
File name is used in Requext_ID to make it more easy to trouble shoot. Change-Id: Ied98766f1a177a9bda35ec5892a60d06619ff3a7 Issue-ID: DCAEGEN2-1305 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-04-12Fixed build problemsPatrikBuhr3-31/+15
Fixed a build problem cause by a merge error from previous commits. Fixes warnings from maven compile Change-Id: I1ce88b2686b3be5ecd17113cbfde3e65297c27c6 Issue-ID: DCAEGEN2-1118 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-04-11Merge "Robustness issue"Vijay Venkatesh Kumar2-19/+40
2019-04-11Merge "Enabling Spring boot actuator"Vijay Venkatesh Kumar2-4/+65
2019-04-11Robustness issuePatrikBuhr3-20/+41
Limiting the number of paralell threads when retying of ftp or publish. Previously there was no upper limit on how many paralel threads that could be started by retry attempts. For instance, a worker thread with 100 new files would start 100 new threads when the ftt server was unavailable. Change-Id: Ia8792f03ea55c0c467ef248ddc9d59187c06a946 Issue-ID: DCAEGEN2-1118 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-04-11Enabling Spring boot actuatorPatrikBuhr2-4/+65
This makes it possible by a REST API: - change logging levels - get the application log - get build info for the DFC - get various metrixes for the execution such as heap consumption, number of threads etc. Change-Id: I570bc0db6a9b9977ecfd83389b463fe652e4ba16 Issue-ID: DCAEGEN2-1428 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-04-11Extend heartbeat to also include som health statuselinuxhenrik6-11/+146
Add info about the internal status of DFC to the healtCheck message. Change-Id: Ic2ad3e5b97c42d6affe1ac4e1d7cf2274027f881 Issue-ID: DCAEGEN2-1403 Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
2019-04-11Housekeepingelinuxhenrik74-1213/+1003
No functional changes made in this commit. Removed CheckStyle and Sonar warnings. Formatted code. Renamed methods and classes for better understanding. Removed unnecessary classes. Moved all code to single project. Change-Id: Ie3feb6c6a985e94a382812aa083dcf57bc46c7b3 Issue-ID: DCAEGEN2-1367 Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
2019-04-10Updating certificatesTamasBakai12-22/+72
Issue-ID: DCAEGEN2-1300 Change-Id: I9ce457b69e5866e77a465fb040d196b8905ca274 Signed-off-by: TamasBakai <tamas.bakai@est.tech>
2019-04-04Improved loggingPatrikBuhr22-298/+299
Fixed problem with startup which lead to that the REST API was not working running locally. Fixed problem with DmaapProducerHttpClient which would use no timeout, which can lead to infinitly haninging threads. A long timeout is used instead. Change-Id: I28469b1b3aaad0dab4cf247bb8af968e71a60133 Issue-ID: DCAEGEN2-1305 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-03-29Change file name sent to DataRouterelinuxhenrik6-30/+16
In the previous commit for this issue, the file name sent to DataRouter was changed. This turned out to be wrong, so in this commit it is changed back again. Change-Id: I9b0d726f406758846fd256626e73edf59d710c8b Issue-ID: DCAEGEN2-1256 Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
2019-03-28Merge "Fix DataFile - Align Dmaap Topic configuration"Vijay Venkatesh Kumar6-471/+8
2019-03-26update INFO.yamlVijay Venkatesh Kumar1-25/+9
Change-Id: I147b698c40bb80a09c885c16d08137dcb4a0cf69 Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com> Issue-ID: DCAEGEN2-1270 Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com>
2019-03-22Local filename updated, stability issuesPatrikBuhr23-310/+261
The local filename is changed so it contains PNF name instead of the PNF IP address. The paralellism is restricted to 100 worker threads in order to solve problems with too many open file descriptors and out of memory. Logging is improved. Change-Id: I24ce2e23020cc253a3c7bebac1ab5cf703b5b144 Issue-ID: DCAEGEN2-1118 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-03-21Add check to DataRouter if file has been publishedmaximesson30-917/+1512
For each file in the FileReady message that DFC does not know if it has been published yet, it should ask DataRouter if it has been published already to avoid downloading and publishing a file more than once. Change-Id: I18117a6e968ec929aa255052a4c44f890a8ed39d Issue-ID: DCAEGEN2-1256 Signed-off-by: maximesson <maxime.bonneau@est.tech>
2019-03-18Fix DataFile - Align Dmaap Topic configurationYongchaoWu6-471/+8
Remove additional configuration in Blueprints Remove additional configruation in component specs Change-Id: I0d546227af0be485f2ffed27a93c701a7654ff8e Issue-ID: DCAEGEN2-1145 Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
2019-03-18Merge "Thread safety issues"Tony Hansen21-782/+436
2019-03-14DataFile - Align Dmaap Topic configurationYongchaoWu6-57/+594
Blueprints alignment Configbinding service support Modify the policy model complaint to R4 model expected by Policy Change-Id: Ic53f4ab2a5867e5cdf0fc735be180715077154ee Issue-ID: DCAEGEN2-1145 Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
2019-03-14Thread safety issuesPatrikBuhr21-782/+436
The TrustManager is now loaded and initialized once in a thread safe way (instead of each time it is used). Removed some unneeded wrappers. Using AutoCloseable for FTP clients to make sure they are closed in case of exceptions. Made AppConfig thread safe. Change-Id: Ia6a2c8a76bf960013180fdd7c53ae0ff17b26505 Issue-ID: DCAEGEN2-1118 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-03-12Merge "DFC logging according to ONAP specification"Vijay Venkatesh Kumar17-135/+289
2019-03-11Merge "Update Readme.md"Vijay Venkatesh Kumar1-2/+2
2019-03-08DFC logging according to ONAP specificationRehanRaza17-135/+289
Change-Id: I6fe18ce3bdbc6d0b1cf5c5e65534cab694cfb898 Issue-ID: DCAEGEN2-1305 Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
2019-03-08Merge "Improvement of the parallelism"Tony Hansen3-67/+379
2019-03-07Change to docker-maven-pluginRehanRaza2-20/+26
Change-Id: I5dc3c5b69b2ef764992e2db76eb9cc154ff5adf7 Issue-ID: DCAEGEN2-1269 Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
2019-03-06Improvement of the parallelismPatrikBuhr3-67/+379
The reactive framework Scedulers uses to few threads. (the same number as the number of processors). That is too few for an io-intense application like this where CPU is not the limiting factor. Change-Id: Ia5f41e75716d309f47dce5f5273b739f7e6d136a Issue-ID: DCAEGEN2-1118 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-03-06Fix dockerfile-maven-plugin in pom.xmlRehanRaza1-3/+9
Change-Id: Id0389e1011fdce9dee4fc0d330973ef34bc2ed6a Issue-ID: DCAEGEN2-1269 Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
2019-03-06Merge "DFC DCAE design artifact support"Tony Hansen9-0/+3905
2019-03-05Fix Docker imageRehanRaza2-13/+10
Change-Id: Icf8a37ea63e78d70256571d1bce424e629a04b55 Issue-ID: DCAEGEN2-1269 Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
2019-03-04DFC DCAE design artifact supportYongchaoWu9-0/+3905
Add blueprints, dataformat, component specs, policy and tosca models. Issue-ID: DCAEGEN2-1156 Change-Id: Id1b6573c3cd72a90ab2feca2180ce1d04ecf98c4 Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
2019-02-27Purging of cached informationPatrikBuhr5-25/+157
The datafile collector has a cache will all previously published files. The cache is on regular intevals purged so that non used entries are removed so that it does not grow infinitely. Added a unit test. Change-Id: I8897fee4522c97031f735b1d6774803dcb73926b Issue-ID: DCAEGEN2-1118 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-02-27Merge "Fixed some sonar issues"Tony Hansen17-435/+109
2019-02-22Fixed some sonar issuesPatrikBuhr17-435/+109
Changed some Flux to Mono. Removed some obfuscating wrappers. Issue-ID: DCAEGEN2-1118 Change-Id: I76dcaea7c69608cf404389fad93f7539f735aad2 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-02-22Run DFC docker container as a non-root userRehanRaza3-27/+54
Change-Id: I82504ebebc8ddf48f6ee6b1c1dfbe433c60ab1aa Issue-ID: DCAEGEN2-1269 Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
2019-02-19Refactor: fix over engineering in ConfigurationYongchaoWu10-433/+153
Remove the duplicated function, move the over engineering code. Make the code simple and understandable. Issue-ID: DCAEGEN2-1222 Change-Id: Iab1737c084421d038b724d2fb41c9d17ebe4dbeb Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
2019-02-18Update Readme.mdyanhuanwang1-2/+2
Update DFC description Change-Id: I778014f3e0ec215c5ce761d087e28bcf06b7296c Signed-off-by: yanhuanwang <martin.yan.seu@gmail.com> Issue-ID: DCAEGEN2-1039
2019-02-15Merge "Fix security vulnerabilities"Vijay Venkatesh Kumar1-1/+1
2019-02-15Merge "Add Readme.md"Tony Hansen3-16/+84
2019-02-15Running of file collection in paralellPatrikBuhr61-1619/+1419
Each FileReady message is run in a separate thread to increase the thoughput. Fetching of files from PNFs is retryed by using the reactive framework. Robustness to temporary failures is increased by retrying to publish fetched files. Fixed so that well known ports (FTPS/SFTP) are used if omitted in the FileReady message URL. Change-Id: I5dfc75a08da0e870fafa3ee1bc83574aca16aabd Issue-ID: DCAEGEN2-1118 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-02-13Fix security vulnerabilitieselinuxhenrik1-3/+3
Change-Id: I1ffb840a18b0c75cffe21229db743f6570073148 Issue-ID: DCAEGEN2-1210 Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
2019-02-13Add Readme.mdyanhuanwang3-16/+84
Format code Change-Id: Iba56718868043d3b19c71892f363fbb838f4988a Signed-off-by: Chengkai Yan <martin.c.yan@est.tech> Issue-ID: DCAEGEN2-1039
2019-02-12Merge "Change ATT Meta Header for generic alternative"Vijay Venkatesh Kumar2-4/+4
2019-01-31Change ATT Meta Header for generic alternativeegernug2-4/+4
Change-Id: Ie4ba3d8afebf7cda3dc0a841c5c5fa5d4cd95046 Issue-ID: DMAAP-999 Signed-off-by: egernug <gerard.nugent@est.tech>
2019-01-30Fix for integration with SDKpwielebs7-9/+9
Change-Id: I9947531e2cad813ac8936769147bee710db5a335 Issue-ID: DCAEGEN2-1096 Signed-off-by: pwielebs <piotr.wielebski@nokia.com>
2019-01-25Integration with SDKpwielebs33-630/+307
* New fields added to DmaapPublisherConfiguraion * New fields added to DmaapConsumerConfiguration * Dmaap Consumer web client replaced by SDK's consumer web client * UTs aligned * Disable enforcer plugin in DFC * Update oparent to 1.2.1 Change-Id: I3569180c15227bc2c8df74fd070571b0aa56fa04 Issue-ID: DCAEGEN2-1096 Signed-off-by: pwielebs <piotr.wielebski@nokia.com>
2019-01-08use sdk-cbsmicdzied10-412/+21
Change-Id: I615d57927c163d022683ad3cae2703adda48cf88 Issue-ID: DCAEGEN2-1064 Signed-off-by: micdzied <michal.1.dziedzic@nokia.com>