blob: a6c2b86467f5531baf9f5d6d87dfaccf628e13d4 (
plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
|
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0
.. _design:
Design
======
Compatibility aspects (VES-JSON)
--------------------------------
HV-VES Collector is a high-volume variant of the existing VES (JSON) collector, and not a completely new collector.
HV-VES follows the VES-JSON schema as much as possible.
- HV-VES uses a Google Protocol Buffers (GPB, proto files) representation of the VES Common Header.
- The proto files use most encoding effective types defined by GPB to cover Common Header fields.
- HV-VES makes routing decisions based mostly on the content of the **Domain** parameter.
- HV-VES allows to embed Payload of different types (by default perf3gpp domain is included).
Analytics applications impacts
- An analytics application operating on high-volume data needs to be prepared to read directly from Kafka.
- An analytics application needs to operate on GPB encoded data in order to benefit from GPB encoding efficiencies.
- It is assumed, that due to the nature of high volume data, there would have to be dedicated applications provided, able to operate on such volumes of data.
Extendability
-------------
HV-VES is designed to allow extending by adding new domain-specific proto files.
The proto file (with the VES CommonHeader) comes with a binary-type **Payload** parameter, where domain-specific data should be placed.
Domain-specific data are encoded as well with GPB. A domain-specific proto file is required to decode the data.
This domain-specific proto has to be shared with analytics applications - HV-VES does not analyze domain-specific data.
In order to support the RT-PM use-case, HV-VES includes a **perf3gpp** domain proto file. Within this domain, high volume data are expected to be reported to HV-VES collector.
Additional domains can be defined based on existing VES domains (like Fault, Heartbeat) or completely new domains. New domains can be added when needed.
GPB proto files are backwards compatible, and a new domain can be added without affecting existing systems.
Analytics applications have to be equipped with the new domain-specific proto file as well.
Currently, these additional, domain specific proto files can be added to respective repos of HV-VES collector.
Implementation details
----------------------
- Project Reactor is used as a backbone of the internal architecture.
- Netty is used by means of reactor-netty library.
- Kotlin is used to write concise code with great interoperability with existing Java libraries.
- Types defined in Λrrow library are also used when it improves readability or general cleanness of the code.
|
