summaryrefslogtreecommitdiffstats
path: root/components/datalake-handler/feeder
AgeCommit message (Collapse)AuthorFilesLines
2019-07-07Unit testGuobiao Mo3-3/+55
Issue-ID: DCAEGEN2-1468 Change-Id: Ib86b58ce77d4e03e695140729d8abb280594842c Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-07Unit testGuobiao Mo30-648/+898
Issue-ID: DCAEGEN2-1468 Change-Id: Ib0a1bbfa02390a9093e0a8ac4da1ae3fe2c1cd11 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-04Merge "kafka"Yan Yang7-0/+354
2019-07-04kafkaZhangZihao7-0/+354
Change-Id: I3084858f2ddc06c42e062f65d65b4d3dec620fd7 Issue-ID: DCAEGEN2-1631 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-03unit test coverage DbTypeTest and DbServiceTest and PortalDesignServiceTestzm3303-4/+137
Change-Id: I4981a90d0e3fa569c2259e54501df62c95cd8cf5 Signed-off-by: zm330 <zhangminyj@chinamobile.com> Issue-ID: DCAEGEN2-1468
2019-07-02converage topicServiceTestzm3301-17/+95
Change-Id: I61851fde9a91c1bd87c69ead31b9904f0695b1be Signed-off-by: zm330 <zhangminyj@chinamobile.com> Issue-ID: DCAEGEN2-1468
2019-06-28supports multiple Kafka clusters and DBsGuobiao Mo14-110/+101
Read data from Kafka and store into DBs Issue-ID: DCAEGEN2-1631 Change-Id: Ib8fccfd84cfdcd2e284ba4f2503b0fbfe41eb5ae Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-27supports multiple Kafka clusters and DBsGuobiao Mo52-602/+925
Read data from Kafka and store into DBs Issue-ID: DCAEGEN2-1631 Change-Id: Ic2736b6e0497ac2084b1a7ce0da3a6e0e1379f43 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-25supports multiple Kafka clusters and DBsGuobiao Mo40-192/+876
Domain classes Issue-ID: DCAEGEN2-1631 Change-Id: I54a715b2d3d8e13f347e46b0faf9d120d9a60548 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-25topic portal portaldesignZhangZihao5-21/+7
Change-Id: I47fd9bdbdedb70ae719790cf24f75a6029faa6c4 Issue-ID: DCAEGEN2-1623 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24Merge "portaldesign and httpclientutil"Yan Yang4-57/+144
2019-06-24portaldesign and httpclientutilZhangZihao6-59/+150
Change-Id: Ib31248a8eee2fe1511472c127abb52a0875c5b21 Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24Merge "db modify 2"Yan Yang2-92/+0
2019-06-24Merge "db modify"Yan Yang4-54/+139
2019-06-24db modify 2ZhangZihao2-92/+0
Change-Id: I32cc643bf9c86d78edc0bc6bf8a7e94fb1721735 Issue-ID: DCAEGEN2-1622 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24portalDesign improveZhangZihao2-2/+6
Change-Id: I1fefccc3bc4fcc72a85a84b46b5cbcb205cf122c Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24designTypeZhangZihao5-19/+74
Change-Id: I8bcbac4a6e1a1f2779a3243615c742719d29f8e5 Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24db modifyZhangZihao4-54/+139
Change-Id: I848b9ac6d45dc60e93be27306cfbed97c73da51d Issue-ID: DCAEGEN2-1622 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-21Check Db enabled flagGuobiao Mo5-9/+31
If DB is disabled, don't send data Issue-ID: DCAEGEN2-1622 Change-Id: I24482259125853d28a9abed2b6e987c169411411 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-21No data is lost when program is killedGuobiao Mo14-36/+117
When the program is killed by unix 'kill' or crtl+c, program should exits gracefully after all data is saved to DBs. Issue-ID: DCAEGEN2-1633 Change-Id: Ic134440be507faa44d04434eeaea1035ce7d63f0 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-19Connect to secured KafkaGuobiao Mo4-10/+21
Issue-ID: DCAEGEN2-1632 Change-Id: I85767916fbf3b8f874ac367b7161b50a272d8595 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-18Add a column to table design_typeGuobiao Mo2-5/+9
Issue-ID: DCAEGEN2-1625 Change-Id: I9f5421f083957219336fec0478571014d26c819f Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-14Unit test 3ZhangZihao3-0/+376
Change-Id: I63afd2b6c70fb2ce0a182fcb988a579f42c9b23d Issue-ID: DCAEGEN2-1468 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-11Merge "Unit test for JsonUtil"Yan Yang1-0/+62
2019-06-11Unit test for JsonUtilGuobiao Mo1-0/+62
Issue-ID: DCAEGEN2-1598 Change-Id: I61ccf5a2859fe80351ea6bf31138b775fe68bead Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-11Unit test 1ZhangZihao5-0/+254
Change-Id: I44ad9a41a35a056a9b8bc9d6c760acb5e5413b80 Issue-ID: DCAEGEN2-1468 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-10Druid Kafka Indexing Service Supervisor Spec for topicGuobiao Mo6-204/+1813
HW and EPC Issue-ID: DCAEGEN2-1598 Change-Id: Iee4a3a3efc2aa04afb1ab234070cd393ca110055 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-06Merge "Dashboard and esTemplate"Yan Yang18-0/+1169
2019-06-06Dashboard and esTemplateZhangZihao18-0/+1169
Change-Id: Id87cb7846ac50ab5e0deee9582479c41aa4e5484 Issue-ID: DCAEGEN2-1329 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-05Bug fixKate Hsuan1-1/+1
Elasticsearch Type should be "doc" Issue-ID: DCAEGEN2-1598 Change-Id: If62b18d60f065e37af15c19597bcc798949d9ade Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-06-04Flatten and Aggregate features in JSON array processingGuobiao Mo14-18/+296
Issue-ID: DCAEGEN2-1598 Change-Id: I9f563bcfa18285daf7b48878e8427bfdb1aff21f Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-30Unit tests to improve sonar coverageGuobiao Mo11-106/+499
Issue-ID: DCAEGEN2-1468 Change-Id: Ie322142a2298ca55d32d64861e71ebc64cd8c09e Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-28DB schema updateGuobiao Mo1-11/+24
Since the terms 'dashboard' and 'template' are used in some dashboard_template entries. Rename: dashboard -> portal dashboard_type -> design_type dashboard_template -> portal_design Issue-ID: DCAEGEN2-1329 Change-Id: Icaa0874ce652cf95f3989712a23412db362b8db5 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-26Dynamically detect topic updates and new topicsGuobiao Mo17-188/+405
Issue-ID: DCAEGEN2-1195 Change-Id: I35d36a9aafe3a7681a9d4745bc509aded111b29d Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-22Bug fixKate Hsuan5-18/+36
1. Fix feeder start/stop exceptions- "The Kafka consumer is NOT thread-safe" exception. 2. Modify REST format for feeder start/stop/status. Issue-ID: DCAEGEN2-1437 Change-Id: I7f79243a0098e6fa17b06866ef6dfc3f71b20dc8 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-17Dependency fixKate Hsuan1-1/+1
1. Change spring boot version 2.1.4->2.1.0. 2. Modify URL of list dmaap. Issue-ID: DCAEGEN2-1437 Change-Id: Ieeb28a3cf4c9441d00b6eb278dcb067e391175df Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-16Bug fixKate Hsuan1-0/+1
1. Close zookeeper connection. Issue-ID: DCAEGEN2-1437 Change-Id: I7a2bfdcf14203cbe1708e6148b12ea00c49a07a1 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-14Set HDFS userGuobiao Mo2-3/+5
Use DB.login as HDFS write user. Issue-ID: DCAEGEN2-1498 Change-Id: I59ee83472595b1fc79e9562b68f4178e7fbd920b Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-13Support HDFS as a data storeGuobiao Mo19-334/+579
Issue-ID: DCAEGEN2-1498 Change-Id: Id203275bce01bd4a4d6ec131fb9696d78eda82f5 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-10Bug fixKate Hsuan2-2/+3
1. Fix DB config JSON and config format. Issue-ID: DCAEGEN2-1437 Change-Id: I3bf0778c61811cc636099d2dbd118fec49d72625 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-09Bug fixKate Hsuan4-2/+34
1. Add a topic delete API. 2. Fix JSON response body. 3. Fix DB config update. Issue-ID: DCAEGEN2-1437 Change-Id: I4cd5710baf796c8a6bcc9af5283a36bd37659ec8 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-07Merge "Add dashboard template configuration schema"Yan Yang2-0/+41
2019-05-07Add dashboard template configuration schemaKate Hsuan2-0/+41
Issue-ID: DCAEGEN2-1329 Change-Id: I29ef46a44a61ef7adceeabe6b65d406d07902be2 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-03Merge "Handle duplicate id"Yan Yang6-103/+116
2019-05-03Merge "Unit test code for datalake seed code"Yan Yang1-0/+2
2019-05-02Handle duplicate idGuobiao Mo6-103/+116
Issue-ID: DCAEGEN2-1411 Change-Id: Ib39b39ff276d7d5c3dbfaa281df6104926fa354d Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-04-29Use TopicConfig in backend4.0.0-ONAP1.0.0Guobiao Mo19-213/+269
Issue-ID: DCAEGEN2-1200 Change-Id: Ia18993524c272c42ba48485a636f04f94af0cd0a Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-04-29Modify TopicConfigGuobiao Mo8-72/+80
Issue-ID: DCAEGEN2-1309 Change-Id: Iccfd579f71e0ca971851ba66443973bdd67dcced Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-04-27Merge "unit test of service"Yan Yang4-0/+341
2019-04-27Merge "Unit test code for datalake seed code"Yan Yang2-25/+141