summaryrefslogtreecommitdiffstats
path: root/components/datalake-handler/feeder/src/main/java/org
AgeCommit message (Collapse)AuthorFilesLines
2020-01-07Merge "Auto populate table 'topic_name'"Tony Hansen3-1/+73
2019-12-30Auto populate table 'topic_name'Guobiao Mo3-1/+73
We continually monitor Kafka; when a new topic is created, its name is auto inserted into table 'topic_name', which is a look up table and UI's topic selection list is based on it. Issue-ID: DCAEGEN2-1715 Change-Id: I35329eb76ed8fb9d41a73f1827f443f7a10b450e Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-12-19Fixed kafka's ui and add getKafkaDetail()ZhangZihao1-3/+21
Change-Id: I4f5b3c06feb19c39a7d67b678704f726c305e7c8 Issue-ID: DCAEGEN2-1877 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-12-12Modified topic and db api functionZhangZihao4-23/+19
table DB's PK is now integer, instead of string. Change-Id: Id1ceb1bcd7a0455bc55e30c875e4398d12e8cf82 Issue-ID: DCAEGEN2-1715 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-11-27Merge "Fixed the related api function with topic"Tony Hansen5-8/+35
2019-11-26Fixed the related api function with topicEkko Chang5-8/+35
Issue-ID: DCAEGEN2-1715 Signed-off-by: Ekko Chang <ekko.chang@qct.io> Change-Id: Iab20a1044cb953fb490f7981b6fa36e5eeea3c52
2019-11-22Fix UI bugsGuobiao Mo3-8/+8
Issue-ID: DCAEGEN2-1716 Change-Id: I01a5a3c69ff65afde37650e559619723f100e69a Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-11-19db by toolZhangZihao2-13/+16
Change-Id: Ice7559bf89b79a1cf284eb94e23140406d97b0df Issue-ID: DCAEGEN2-1877 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-10-27dbs modify codeZhangZihao4-20/+79
Change-Id: I87f0a339382354fa9ca6cbded9d1403a234f39f4 Issue-ID: DCAEGEN2-1622 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-09-10Defined a constant for this literal "Db not found"Rama-Huawei1-4/+5
Removed the declaration of thrown exception, as it can't be thrown from method's body Issue-ID: DCAEGEN2-1468 Signed-off-by: Rama-Huawei <rama.subba.reddy.s@huawei.com> Change-Id: I341b5ea67dd643503fee9077a7073aeb8adbc15a
2019-09-09Static field is updated from a non-static method.Rama-Huawei1-272/+266
Converted while loops into for loop Removed the static fields and make them as a local variables Issue-ID: DCAEGEN2-1468 Signed-off-by: Rama-Huawei <rama.subba.reddy.s@huawei.com> Change-Id: Ib6c60ff5975cc7b645174497a8fcfac313cd825b
2019-08-14Merge "Return an empty collection instead of null"Yan Yang4-19/+4
2019-08-14Merge "System.out is replace by a logger"Yan Yang2-21/+12
2019-08-08Get all databases id and name by designTypeIdZhangZihao1-10/+16
Change-Id: I9127ad0f5aa3b77e7e3847f9fe8240ab4c1482f7 Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-08-07Return an empty collection instead of nullRama-Huawei4-19/+4
Issue-ID: DCAEGEN2-1468 Signed-off-by: Rama-Huawei <rama.subba.reddy.s@huawei.com> Change-Id: I294014cd009ae02983e575c94e830b2ff30e7b07
2019-08-07System.out is replace by a loggerRama-Huawei2-21/+12
Issue-ID: DCAEGEN2-1468 Signed-off-by: Rama-Huawei <rama.subba.reddy.s@huawei.com> Change-Id: Iefdbfd3480f831ce734e748ac0680ed53c60f0fb
2019-08-05Used isEmpty to check collection is empty or notRama-Huawei3-6/+6
Issue-ID: DCAEGEN2-1468 Signed-off-by: Rama-Huawei <rama.subba.reddy.s@huawei.com> Change-Id: Ice0fe7a0119442f7cf7f6171ea1e2a2c8a0c8cab
2019-08-01design modify 2ZhangZihao3-40/+96
Change-Id: I7c75460b30184bf2803f8a936a4252867300e898 Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-30deploy improve 1ZhangZihao3-70/+88
Change-Id: I9ba21e0e0f8e6ab75d63e65457bc382aa02cff91 Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-18db of designZhangZihao1-0/+14
Change-Id: I0119780b7903bd84874c5ff9eca1aadb981e01b3 Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-17design modify 2ZhangZihao10-387/+40
Change-Id: I839bee562f03e12de9a3a4bfcd25f4db5fee6f7e Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-11design modify 1ZhangZihao10-416/+389
Change-Id: I32ef0ff484b786b9c5b73df8fd1b8cc97c5dc227 Issue-ID: DCAEGEN2-1658 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-07-10supports multiple Kafka clusters and DBsGuobiao Mo16-65/+65
Change kafka table id to int Issue-ID: DCAEGEN2-1631 Change-Id: Ib5109b75a387f76709dc38161bf2d7ef084950ef Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-07Unit testGuobiao Mo8-85/+50
Issue-ID: DCAEGEN2-1468 Change-Id: Ib0a1bbfa02390a9093e0a8ac4da1ae3fe2c1cd11 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-07-04kafkaZhangZihao7-0/+354
Change-Id: I3084858f2ddc06c42e062f65d65b4d3dec620fd7 Issue-ID: DCAEGEN2-1631 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-28supports multiple Kafka clusters and DBsGuobiao Mo14-110/+101
Read data from Kafka and store into DBs Issue-ID: DCAEGEN2-1631 Change-Id: Ib8fccfd84cfdcd2e284ba4f2503b0fbfe41eb5ae Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-27supports multiple Kafka clusters and DBsGuobiao Mo27-443/+748
Read data from Kafka and store into DBs Issue-ID: DCAEGEN2-1631 Change-Id: Ic2736b6e0497ac2084b1a7ce0da3a6e0e1379f43 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-25supports multiple Kafka clusters and DBsGuobiao Mo20-51/+552
Domain classes Issue-ID: DCAEGEN2-1631 Change-Id: I54a715b2d3d8e13f347e46b0faf9d120d9a60548 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-25topic portal portaldesignZhangZihao5-21/+7
Change-Id: I47fd9bdbdedb70ae719790cf24f75a6029faa6c4 Issue-ID: DCAEGEN2-1623 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24Merge "portaldesign and httpclientutil"Yan Yang3-56/+141
2019-06-24portaldesign and httpclientutilZhangZihao4-57/+143
Change-Id: Ib31248a8eee2fe1511472c127abb52a0875c5b21 Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24Merge "db modify 2"Yan Yang1-45/+0
2019-06-24Merge "db modify"Yan Yang2-50/+85
2019-06-24db modify 2ZhangZihao1-45/+0
Change-Id: I32cc643bf9c86d78edc0bc6bf8a7e94fb1721735 Issue-ID: DCAEGEN2-1622 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24portalDesign improveZhangZihao1-1/+2
Change-Id: I1fefccc3bc4fcc72a85a84b46b5cbcb205cf122c Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24designTypeZhangZihao4-18/+73
Change-Id: I8bcbac4a6e1a1f2779a3243615c742719d29f8e5 Issue-ID: DCAEGEN2-1625 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-24db modifyZhangZihao2-50/+85
Change-Id: I848b9ac6d45dc60e93be27306cfbed97c73da51d Issue-ID: DCAEGEN2-1622 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-21Check Db enabled flagGuobiao Mo2-8/+15
If DB is disabled, don't send data Issue-ID: DCAEGEN2-1622 Change-Id: I24482259125853d28a9abed2b6e987c169411411 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-21No data is lost when program is killedGuobiao Mo10-33/+104
When the program is killed by unix 'kill' or crtl+c, program should exits gracefully after all data is saved to DBs. Issue-ID: DCAEGEN2-1633 Change-Id: Ic134440be507faa44d04434eeaea1035ce7d63f0 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-19Connect to secured KafkaGuobiao Mo2-6/+12
Issue-ID: DCAEGEN2-1632 Change-Id: I85767916fbf3b8f874ac367b7161b50a272d8595 Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-18Add a column to table design_typeGuobiao Mo1-0/+3
Issue-ID: DCAEGEN2-1625 Change-Id: I9f5421f083957219336fec0478571014d26c819f Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-06-06Merge "Dashboard and esTemplate"Yan Yang17-0/+1166
2019-06-06Dashboard and esTemplateZhangZihao17-0/+1166
Change-Id: Id87cb7846ac50ab5e0deee9582479c41aa4e5484 Issue-ID: DCAEGEN2-1329 Signed-off-by: ZhangZihao <zhangzihao@chinamobile.com>
2019-06-04Flatten and Aggregate features in JSON array processingGuobiao Mo5-6/+214
Issue-ID: DCAEGEN2-1598 Change-Id: I9f563bcfa18285daf7b48878e8427bfdb1aff21f Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-30Unit tests to improve sonar coverageGuobiao Mo4-9/+11
Issue-ID: DCAEGEN2-1468 Change-Id: Ie322142a2298ca55d32d64861e71ebc64cd8c09e Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-26Dynamically detect topic updates and new topicsGuobiao Mo11-141/+354
Issue-ID: DCAEGEN2-1195 Change-Id: I35d36a9aafe3a7681a9d4745bc509aded111b29d Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>
2019-05-22Bug fixKate Hsuan3-15/+27
1. Fix feeder start/stop exceptions- "The Kafka consumer is NOT thread-safe" exception. 2. Modify REST format for feeder start/stop/status. Issue-ID: DCAEGEN2-1437 Change-Id: I7f79243a0098e6fa17b06866ef6dfc3f71b20dc8 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-17Dependency fixKate Hsuan1-1/+1
1. Change spring boot version 2.1.4->2.1.0. 2. Modify URL of list dmaap. Issue-ID: DCAEGEN2-1437 Change-Id: Ieeb28a3cf4c9441d00b6eb278dcb067e391175df Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-16Bug fixKate Hsuan1-0/+1
1. Close zookeeper connection. Issue-ID: DCAEGEN2-1437 Change-Id: I7a2bfdcf14203cbe1708e6148b12ea00c49a07a1 Signed-off-by: Kate Hsuan <kate.hsuan@qct.io>
2019-05-14Set HDFS userGuobiao Mo1-2/+4
Use DB.login as HDFS write user. Issue-ID: DCAEGEN2-1498 Change-Id: I59ee83472595b1fc79e9562b68f4178e7fbd920b Signed-off-by: Guobiao Mo <guobiaomo@chinamobile.com>