aboutsummaryrefslogtreecommitdiffstats
path: root/datafile-app-server
AgeCommit message (Collapse)AuthorFilesLines
2021-02-23Add JWT support in HTTP/HTTPS based locations1.5.4Krzysztof Gajewski11-63/+519
Issue-ID: DCAEGEN2-2536 Signed-off-by: Krzysztof Gajewski <krzysztof.gajewski@nokia.com> Change-Id: I47a928159853333014b0fd413a085b7c50eeb7a0
2021-02-11Add HTTPS to collect files from xNFs1.5.3Krzysztof Gajewski28-98/+1109
- plus small refactoring related to above - update to version 1.5.3 Issue-ID: DCAEGEN2-2528 Signed-off-by: Krzysztof Gajewski <krzysztof.gajewski@nokia.com> Change-Id: I2531c85967964f1359bafd5b694afbf662edf54e
2021-02-10Merge "[DCAE-dfc] Vulnerability Removal for Datafile Collector"Tony Hansen6-343/+272
2021-02-08[DCAE-dfc] Vulnerability Removal for Datafile Collectorajay_dp0016-343/+272
- swagger version 2.9.2 to 3.0.0 - commons-io version 1.3.2 to 2.8.0 - tomcat-embed-core version 9.0.36 to 9.0.39 - pom.xml refactoring Issue-ID: DCAEGEN2-2591 Signed-off-by: ajay_dp001 <ajay.deep.singh@est.tech> Change-Id: I40e65a6936fc118299cd575df34c51730a25f424
2021-02-08Add config-volume to DFC for storing know host keysTomasz Pietruszkiewicz1-1/+7
Change-Id: If2de5453cbe262f251b77b9f6e25b45e36207ea8 Issue-ID: DCAEGEN2-2549 Signed-off-by: Tomasz Pietruszkiewicz <tomasz.pietruszkiewicz@nokia.com>
2021-01-05Update Datafile version to 1.5.11.5.1Krzysztof Gajewski1-1/+1
Issue-ID: DCAEGEN2-2527 Signed-off-by: Krzysztof Gajewski <krzysztof.gajewski@nokia.com> Change-Id: I271b521521cfd38c0da05ae65d52550908021540
2020-12-30Add HTTP as new protocol to collect files from xNFsKrzysztof Gajewski21-25/+529
- HTTP basic auth included - small code refactoring related to the task Issue-ID: DCAEGEN2-2527 Signed-off-by: Krzysztof Gajewski <krzysztof.gajewski@nokia.com> Change-Id: I13ec80e996861e14d2c561087c4af3b34d861030
2020-12-02Upgrade Reactor and SpringBoot1.5.0tkogut14-93/+90
Issue-ID: DCAEGEN2-2537 Signed-off-by: tkogut <tomasz.kogut@nokia.com> Change-Id: Ide560554f11d01737b351d6138430efe6005791d
2020-11-25Remove blueprintJerzySzachniewicz1-136/+0
All future blueprint updates will be done under blueprint repo Issue-ID: DCAEGEN2-2456 Signed-off-by: JerzySzachniewicz <jerzy.szachniewicz@nokia.com> Change-Id: I388a17c68caf33b96533d8c3463d519846cbe5ca Signed-off-by: JerzySzachniewicz <jerzy.szachniewicz@nokia.com>
2020-10-19Align component-spec propertiesTomasz Wrobel1-2/+2
Change FTPes certs to PKCS12 (extternal communication). Issue-ID: DCAEGEN2-1795 Signed-off-by: Tomasz Wrobel <tomasz.wrobel@nokia.com> Change-Id: I06a729adb32290e2ebd942a37a144ae034a08a42
2020-10-14Add use_external_tls property to component-specTomasz Wrobel1-1/+2
Issue-ID: DCAEGEN2-1795 Signed-off-by: Tomasz Wrobel <tomasz.wrobel@nokia.com> Change-Id: I71a9538b79deac7e58d1cd8131e623ae3758fdce
2020-10-07Align DFC component specTomasz Wrobel1-5/+40
Issue-ID: DCAEGEN2-2461 Signed-off-by: Tomasz Wrobel <tomasz.wrobel@nokia.com> Change-Id: Ib09be9a2f21643b8e6de95ecc295d16f7ba7e8cb
2020-10-01Update datafile component specRemigiusz Janeczek1-13/+8
Issue-ID: DCAEGEN2-2461 Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com> Change-Id: I03a75203ab6870d8799fb498c67abbc226a54fb0
2020-08-25Update dcae SDK from 1.1.6 to 1.4.2Remigiusz Janeczek28-878/+368
Bump project version from 1.4.2 to 1.4.3 Update deprecated calls to JsonParser Make logs always go to file and console Issue-ID: DCAEGEN2-2267 Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com> Change-Id: Ib8d7f82b3daf03ca327581c9a5dc4f6f27a20141
2020-08-14Remove DFC Vulnerabilities1.4.2ERIMROB1-0/+1
Issue-ID: DCAEGEN2-2267 Change-Id: Id9110c8b4b747bba3fcce1c1411a40062aa40740 Signed-off-by: ERIMROB <robertas.rimkus@est.tech>
2020-08-03openJDK image update to 11.0.8deen19852-2/+2
Change-Id: I341390e5bde3e894455ac7717990669f4e0f0f94 Signed-off-by: deen1985 <damian.nowak@nokia.com> Issue-ID: DCAEGEN2-2378 Signed-off-by: deen1985 <damian.nowak@nokia.com>
2020-07-30Removed support for FTPS protocol (from now only FTPeS and sFTP will be ↵Jan Malkiewicz11-67/+74
supported). Issue-ID: DCAEGEN2-2367 Signed-off-by: Jan Malkiewicz <jan.malkiewicz@nokia.com> Change-Id: Ifada2899431c56bdbfcf3aad73fdc9b376ae9c4f
2020-07-29Merge "Add log when file is skipped because it was already published"Henrik Andersson1-9/+18
2020-07-28Add log when file is skipped because it was already publishedRemigiusz Janeczek1-9/+18
Issue-ID: DCAEGEN2-2366 Signed-off-by: Remigiusz Janeczek <remigiusz.janeczek@nokia.com> Change-Id: I913b887833e412dd9c0093936a9231d37f63e7d1
2020-07-28Add sftp strict host key checking to DFC.Jan Malkiewicz21-85/+350
Issue-ID: DCAEGEN2-2219 Signed-off-by: Jan Malkiewicz <jan.malkiewicz@nokia.com> Change-Id: Iadf6c6bd743c42ebb3bf9ad8ac443fc0f3f58063
2020-07-23Update to Java 11JoeOLeary2-5/+5
minor version bump to 1.4.0 Issue-ID: DCAEGEN2-1691 Signed-off-by: JoeOLeary <joseph.o.leary@est.tech> Change-Id: Iaf9e8fdcf469fe79c63e8e3868b18168a60a7f2c
2019-10-16DataFileCollector use wrong KeyManagerFactoryburdziak3-17/+34
Issue-ID: DCAEGEN2-1854 Signed-off-by: burdziak <olaf.burdziakowski@nokia.com> Change-Id: I71c7526097014e10d0ef091e38a929b81ba1f627
2019-10-02Update blueprintJoeOLeary1-1/+11
* Make topic name, consumer group and consumer id configurable * Add use existing to DR feed Issue-ID: DCAEGEN2-1793 Signed-off-by: JoeOLeary <joseph.o.leary@est.tech> Change-Id: I454acf3390c112be16c1604d8d62098b7f3c45e6
2019-09-18Personalized client certificate authentication via FTPes in 5G Bulk PM ↵burdziak3-1/+34
usecase doesn't work Issue-ID: DCAEGEN2-1754 Signed-off-by: burdziak <olaf.burdziakowski@nokia.com> Change-Id: Idec3efce8412dd175ecf75f55790a24cec651104
2019-09-12Update keyPasswordPathecaiyanlinux1-4/+4
Change-Id: I126c018444d235a1972b4b9cecc93dfd2f5cb4c1 Signed-off-by: Martin Yan <martin.c.yan@est.tech> Issue-ID: DCAEGEN2-1702
2019-09-12Update Cloudify types file versionJoeOLeary1-2/+2
Issue-ID: DCAEGEN2-1686 Change-Id: Id7f5bd56bb8e0f80e8006d84b03ce7323e509a87 Signed-off-by: JoeOLeary <joseph.o.leary@est.tech>
2019-09-10update Dockerfile & copyrigt statementecaiyanlinux2-2/+3
Issue-ID: DCAEGEN2-1702 Change-Id: If990e180b96f943dd7381a758c11d10e4733706c Signed-off-by: Martin Yan <martin.c.yan@est.tech>
2019-09-06TLS init container usageecaiyanlinux21-121/+82
Change-Id: Iec51c722f6a3a519fd7eb0f37f8b801eebbc039b Issue-ID: DCAEGEN2-1702 Signed-off-by: ecaiyanlinux <martin.c.yan@est.tech>
2019-08-26Align spec and blueprint with bp-generator toolRehanRaza2-139/+109
Change-Id: Ifbafe8be8bcb34417873af3e6bb1c7323f891fe0 Issue-ID: DCAEGEN2-1580 Signed-off-by: RehanRaza <muhammad.rehan.raza@est.tech>
2019-08-14Merge "change mr default port to https:3905"Tony Hansen1-2/+2
2019-08-12DFC HousekeepingPatrikBuhr21-196/+100
Removed warnings. Improved code formatting Removed unused files Removed datafile_endpoints.json from the container Change-Id: I7334775be793cd9cb709ca1e031c620c2c1c2b3f Issue-ID: DCAEGEN2-1645 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-08-03bump dfc version to 1.2.2Vijay Venkatesh Kumar1-1/+1
Change-Id: I080c26db2639eff2174d4aac334990af67019f46 Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com> Issue-ID: DCAEGEN2-1669
2019-07-31change mr default port to https:3905yanhuanwang1-2/+2
Change-Id: I3fada4907601e999490d73bbc85e792c538547b2 Issue-ID: DCAEGEN2-1595 Signed-off-by: yanhuanwang <martin.c.yan@est.tech>
2019-07-20remove docker bind from mvnVijay Venkatesh Kumar1-2/+7
Change-Id: I5787075e559bcfb56373de5cfa7e09365f16f8b8 Signed-off-by: Vijay Venkatesh Kumar <vv770d@att.com> Issue-ID: DCAEGEN2-1690 Issue-ID: DCAEGEN2-1578
2019-07-12Cbs Client integrationYongchaoWu27-368/+445
Cbs Client is integrated to read configurations from consul Issue-ID: DCAEGEN2-1595 Change-Id: Idb0ebd34eba077f9c1cb584abab4d8722b56f6c5 Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
2019-07-11JJB alignment across DCAE components - DCAE Datafile Collectorelinuxhenrik1-1/+1
Change-Id: I27dd76a8f1b84a8ab26080381321c509c1117a9b Issue-ID: DCAEGEN2-1578 Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
2019-07-02DFC - housekeepingelinuxhenrik22-150/+655
Removed Sonar warnings. Improved code coverage. Improved code formatting Change-Id: I0e8287cecb18a8976635427697dfaeb823da204b Issue-ID: DCAEGEN2-1645 Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
2019-06-14Code formatting with mavenPatrikBuhr50-1081/+1104
This commit only contains code formastting changes done by command: mvn formatter:format spotless:apply process-sources Change-Id: I0fb9d166ad5d9171cdeee9f26b6d353bca74069c Issue-ID: DCAEGEN2-1538 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-06-14Bugfix, improved behaviour for large filesPatrikBuhr14-97/+300
Previously files was read into a buffer for publishing. This does not work when files are bigger than the available memory. After the fix , files are streamed instead. Implemented a new REST primitive for exposing status and statistics. To be used for test and trouble shooting. Change-Id: Iab5a1ee9ffcbf6836fcf709d115bf25ab0391732 Issue-ID: DCAEGEN2-1532 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-06-05Datafile - Improve JUnit testselinuxhenrik23-257/+501
Improve JUnit tests for the Datafile Collector. Add tests of logging. Remove dependency to old JUnit version. Improve some log messages. Improve code coverage. Change-Id: Ie8204619ab7fe4294c4ac0dd8f030973653ef4a3 Issue-ID: DCAEGEN2-1490 Signed-off-by: elinuxhenrik <henrik.b.andersson@est.tech>
2019-06-04Blueprint alignmentYongchaoWu7-146/+94
Blueprint is aligned with the dmaap topic configuration Change-Id: I4303724eed21047b87a8b7438970625b08811d22 Issue-ID: DCAEGEN2-1145 Signed-off-by: YongchaoWu <yongchao.wu@est.tech>
2019-06-04Merge "Fix a small bug for swagger-ui"Henrik Andersson1-0/+8
2019-06-04Merge "Bugfixes, Generalizing Data File Collection to handle any type of file"Henrik Andersson1-9/+7
2019-06-03Bugfixes, Generalizing Data File Collection to handle any type of filePatrikBuhr1-9/+7
- When a change ID was not configured, the task counter was not decreased. The result was that the DFC stopped polling, - When the check if a file is a already published fails (most likely to a problem is the DR), the DFC will try to publish it (instead of just ingnoring it). Change-Id: If9f5b962210f809d5d2ae0aa60d3a7f99099c058 Issue-ID: DCAEGEN2-1532 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-06-03DFC, Purging the cache of published files occasionally failsPatrikBuhr1-5/+5
Solved a threading issue. Change-Id: I1272cadd5d8bc8bb8ffc4c3d33c9b5c706470864 Issue-ID: DCAEGEN2-1588 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-05-29Generalizing Data File Collection to handle any type of filePatrikBuhr43-955/+1097
Extension of the DFC to be able to handle any file types types, which are published on different DR feeds. This association between file type and DR feed is defined by configuration. The file type is defined by the changeIdentifier in the fileReady VES message reported from the PNF. The creation of DR feeds and configuration will be done by the DMAAP plugin, but that is not tested yet. Change-Id: I13b36acd926a6941ee733e6b37922049fb54a5d9 Issue-ID: DCAEGEN2-1532 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-05-28Fix a small bug for swagger-uiyanhuanwang1-0/+8
swagger-ui and actuator-ui are missing. This commit is to fix this bug. Change-Id: I237ad1739dd23b7316a3efecb40a9bd78247ff33 Issue-ID: DCAEGEN2-1535 Signed-off-by: yanhuanwang <martin.yan.seu@gmail.com>
2019-05-22Merge "Fix for robustness"Henrik Andersson3-9/+16
2019-05-17Fix, skip FTP retry in certain casesPatrikBuhr7-13/+77
In certain conditions there is no reason to retry fetching files. For instance when the file is removed in the PNF or when the password/certificate is wrong. When the DFC is started there are always queued VES events that referes to removed files which in turn results in that the DFC will retry fetching these files in vain. The DFC house keeps its number of concurrents tasks to not exeed quotas for memory,open file descriptors etc. As more threads are occupied with retrying, the fewer threads can do their intended work, which decreases the throughput. Testing has showed that already when the number of PNFs are 10, the throughput is radically decreased (and the problem is then escalating). Change-Id: I4833724a3ef3509025f4a0a438c6c8025932b0f4 Issue-ID: DCAEGEN2-1508 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>
2019-05-17Fix for robustnessPatrikBuhr3-9/+16
In the long time stability test it has showed that when there are an exessive amount of FileReady events, the DFC will fetch these from the MR and build up an internal queue. This has the effect that the DFC will consume too much memory. Itmight also affect load balancing (in case of several DFC instances) so that one might consume all events and the others get nothing. The DFC should not fetch new FileReady events when it fully loaded with work. Change-Id: I58665edd678d2f1c8a32d0e56455228b522aab40 Issue-ID: DCAEGEN2-1509 Signed-off-by: PatrikBuhr <patrik.buhr@est.tech>