summaryrefslogtreecommitdiffstats
path: root/components/datalake-handler/feeder
AgeCommit message (Collapse)AuthorFilesLines
2019-07-24kakfa test improveZhangZihao1-0/+14
Change-Id: I76f8094c32b82812069ff00b9ec1c9618962771a Issue-ID: DCAEGEN2-1631 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-22kafka testZhangZihao3-2/+147
Change-Id: I1ff274c218a19fd7cedfeda7c07bfa0245687070 Issue-ID: DCAEGEN2-1631 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-19Merge "db of design"Yan Yang1-0/+14
2019-07-18Add maven-deploy-pluginGuobiao Mo1-29/+0
For JJB alignment across DCAE components - DCAE services Issue-ID: DCAEGEN2-1578 Change-Id: I3e3b4b70054a1799c88d65ae470309c32cb467e6 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-18db of designZhangZihao1-0/+14
Change-Id: I0119780b7903bd84874c5ff9eca1aadb981e01b3 Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-17Merge "design modify 2"Yan Yang18-653/+43
2019-07-17design modify 2ZhangZihao18-653/+43
Change-Id: I839bee562f03e12de9a3a4bfcd25f4db5fee6f7e Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-16JJB alignment across DCAE components - DCAE servicesGerard Hickey2-5/+6
Issue-ID: DCAEGEN2-1578 Signed-off-by: Gerard Hickey <gerard.hickey@ericsson.com> Change-Id: If6a25d2ea01e5cac84dba95504ffa09cf4796f7a
2019-07-11design modify 1ZhangZihao17-642/+610
Change-Id: I32ef0ff484b786b9c5b73df8fd1b8cc97c5dc227 Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-11Merge "supports multiple Kafka clusters and DBs"Yan Yang21-74/+76
2019-07-10supports multiple Kafka clusters and DBsGuobiao Mo21-74/+76
Change kafka table id to int Issue-ID: DCAEGEN2-1631 Change-Id: Ib5109b75a387f76709dc38161bf2d7ef084950ef Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-10coverage unit testzm3304-0/+245
Change-Id: Ib1b5a43fe1974b1276a7367b32bd63ee506c2232 Signed-off-by: zm330 <zhangminyj@chinamobile.com> Issue-ID: DCAEGEN2-1468
2019-07-07Unit testGuobiao Mo3-3/+55
Issue-ID: DCAEGEN2-1468 Change-Id: Ib86b58ce77d4e03e695140729d8abb280594842c Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-07Unit testGuobiao Mo30-648/+898
Issue-ID: DCAEGEN2-1468 Change-Id: Ib0a1bbfa02390a9093e0a8ac4da1ae3fe2c1cd11 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-04Merge "kafka"Yan Yang7-0/+354
2019-07-04kafkaZhangZihao7-0/+354
Change-Id: I3084858f2ddc06c42e062f65d65b4d3dec620fd7 Issue-ID: DCAEGEN2-1631 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-03unit test coverage DbTypeTest and DbServiceTest and PortalDesignServiceTestzm3303-4/+137
Change-Id: I4981a90d0e3fa569c2259e54501df62c95cd8cf5 Signed-off-by: zm330 <zhangminyj@chinamobile.com> Issue-ID: DCAEGEN2-1468
2019-07-02converage topicServiceTestzm3301-17/+95
Change-Id: I61851fde9a91c1bd87c69ead31b9904f0695b1be Signed-off-by: zm330 <zhangminyj@chinamobile.com> Issue-ID: DCAEGEN2-1468
2019-06-28supports multiple Kafka clusters and DBsGuobiao Mo14-110/+101
Read data from Kafka and store into DBs Issue-ID: DCAEGEN2-1631 Change-Id: Ib8fccfd84cfdcd2e284ba4f2503b0fbfe41eb5ae Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-27supports multiple Kafka clusters and DBsGuobiao Mo52-602/+925
Read data from Kafka and store into DBs Issue-ID: DCAEGEN2-1631 Change-Id: Ic2736b6e0497ac2084b1a7ce0da3a6e0e1379f43 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-25supports multiple Kafka clusters and DBsGuobiao Mo40-192/+876
Domain classes Issue-ID: DCAEGEN2-1631 Change-Id: I54a715b2d3d8e13f347e46b0faf9d120d9a60548 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-25topic portal portaldesignZhangZihao5-21/+7
Change-Id: I47fd9bdbdedb70ae719790cf24f75a6029faa6c4 Issue-ID: DCAEGEN2-1623 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24Merge "portaldesign and httpclientutil"Yan Yang4-57/+144
2019-06-24portaldesign and httpclientutilZhangZihao6-59/+150
Change-Id: Ib31248a8eee2fe1511472c127abb52a0875c5b21 Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24Merge "db modify 2"Yan Yang2-92/+0
2019-06-24Merge "db modify"Yan Yang4-54/+139
2019-06-24db modify 2ZhangZihao2-92/+0
Change-Id: I32cc643bf9c86d78edc0bc6bf8a7e94fb1721735 Issue-ID: DCAEGEN2-1622 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24portalDesign improveZhangZihao2-2/+6
Change-Id: I1fefccc3bc4fcc72a85a84b46b5cbcb205cf122c Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24designTypeZhangZihao5-19/+74
Change-Id: I8bcbac4a6e1a1f2779a3243615c742719d29f8e5 Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24db modifyZhangZihao4-54/+139
Change-Id: I848b9ac6d45dc60e93be27306cfbed97c73da51d Issue-ID: DCAEGEN2-1622 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-21Check Db enabled flagGuobiao Mo5-9/+31
If DB is disabled, don't send data Issue-ID: DCAEGEN2-1622 Change-Id: I24482259125853d28a9abed2b6e987c169411411 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-21No data is lost when program is killedGuobiao Mo14-36/+117
When the program is killed by unix 'kill' or crtl+c, program should exits gracefully after all data is saved to DBs. Issue-ID: DCAEGEN2-1633 Change-Id: Ic134440be507faa44d04434eeaea1035ce7d63f0 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-19Connect to secured KafkaGuobiao Mo4-10/+21
Issue-ID: DCAEGEN2-1632 Change-Id: I85767916fbf3b8f874ac367b7161b50a272d8595 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-18Add a column to table design_typeGuobiao Mo2-5/+9
Issue-ID: DCAEGEN2-1625 Change-Id: I9f5421f083957219336fec0478571014d26c819f Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-14Unit test 3ZhangZihao3-0/+376
Change-Id: I63afd2b6c70fb2ce0a182fcb988a579f42c9b23d Issue-ID: DCAEGEN2-1468 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-11Merge "Unit test for JsonUtil"Yan Yang1-0/+62
2019-06-11Unit test for JsonUtilGuobiao Mo1-0/+62
Issue-ID: DCAEGEN2-1598 Change-Id: I61ccf5a2859fe80351ea6bf31138b775fe68bead Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-11Unit test 1ZhangZihao5-0/+254
Change-Id: I44ad9a41a35a056a9b8bc9d6c760acb5e5413b80 Issue-ID: DCAEGEN2-1468 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-10Druid Kafka Indexing Service Supervisor Spec for topicGuobiao Mo6-204/+1813
HW and EPC Issue-ID: DCAEGEN2-1598 Change-Id: Iee4a3a3efc2aa04afb1ab234070cd393ca110055 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-06Merge "Dashboard and esTemplate"Yan Yang18-0/+1169
2019-06-06Dashboard and esTemplateZhangZihao18-0/+1169
Change-Id: Id87cb7846ac50ab5e0deee9582479c41aa4e5484 Issue-ID: DCAEGEN2-1329 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-05Bug fixKate Hsuan1-1/+1
Elasticsearch Type should be "doc" Issue-ID: DCAEGEN2-1598 Change-Id: If62b18d60f065e37af15c19597bcc798949d9ade Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-06-04Flatten and Aggregate features in JSON array processingGuobiao Mo14-18/+296
Issue-ID: DCAEGEN2-1598 Change-Id: I9f563bcfa18285daf7b48878e8427bfdb1aff21f Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-30Unit tests to improve sonar coverageGuobiao Mo11-106/+499
Issue-ID: DCAEGEN2-1468 Change-Id: Ie322142a2298ca55d32d64861e71ebc64cd8c09e Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-28DB schema updateGuobiao Mo1-11/+24
Since the terms 'dashboard' and 'template' are used in some dashboard_template entries. Rename: dashboard -> portal dashboard_type -> design_type dashboard_template -> portal_design Issue-ID: DCAEGEN2-1329 Change-Id: Icaa0874ce652cf95f3989712a23412db362b8db5 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-26Dynamically detect topic updates and new topicsGuobiao Mo17-188/+405
Issue-ID: DCAEGEN2-1195 Change-Id: I35d36a9aafe3a7681a9d4745bc509aded111b29d Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-22Bug fixKate Hsuan5-18/+36
1. Fix feeder start/stop exceptions- "The Kafka consumer is NOT thread-safe" exception. 2. Modify REST format for feeder start/stop/status. Issue-ID: DCAEGEN2-1437 Change-Id: I7f79243a0098e6fa17b06866ef6dfc3f71b20dc8 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-17Dependency fixKate Hsuan1-1/+1
1. Change spring boot version 2.1.4->2.1.0. 2. Modify URL of list dmaap. Issue-ID: DCAEGEN2-1437 Change-Id: Ieeb28a3cf4c9441d00b6eb278dcb067e391175df Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-16Bug fixKate Hsuan1-0/+1
1. Close zookeeper connection. Issue-ID: DCAEGEN2-1437 Change-Id: I7a2bfdcf14203cbe1708e6148b12ea00c49a07a1 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-14Set HDFS userGuobiao Mo2-3/+5
Use DB.login as HDFS write user. Issue-ID: DCAEGEN2-1498 Change-Id: I59ee83472595b1fc79e9562b68f4178e7fbd920b Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>