1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
|
DCAE VESCollector
======================================
This is the repository for VES Collector for Open DCAE.
Virtual Event Streaming (VES) Collector is RESTful collector for processing JSON messages into DCAE. The collector verifies the source (when authentication is enabled) and validates the events against VES schema before distributing to DMAAP MR topics for downstream system to subscribe. The VESCollector also provides configurable event transformation function and event distribution to DMAAP MR topics.
The collector supports individual events or eventbatch posted to collector end-point(s) and post them to interface/bus for other application to subscribe.
### Build Instructions
This project is organized as a mvn project and has "org.onap.dcaegen2" as parent project. The build generate a jar and package into docker container.
```
git clone ssh://vv770d@gerrit.onap.org:29418/dcaegen2/collectors/ves
mvn clean install
```
### Docker Image
```
git clone ssh://vv770d@gerrit.onap.org:29418/dcaegen2/collectors/ves
mvn clean deploy
```
For R1 - image/version pushed to nexus3
```
nexus3.onap.org:10003/snapshots/onap/org.onap.dcaegen2.collectors.ves.vescollector 1.1
```
### Deployment
VESCollector in DCAE will be deployed as mS via DCAEGEN2 controller. A blueprint will be generated (CLAMP/SDC) which will fetch the docker image and install on the dockerhost identified. VESCollector on startup will query the configbindingService for updated configuration and starts the service. When configuration change is detected by DCAEGEN2 controller (via policy flow) - then contoller will notify Collector to fetch new configuration again.
For testing purpose, the docker image includes preset configuration which can be ran without DCAEGEN2 platform.
### Testing
For R1 as only measurement and faults are expected in ONAP, configuration are preset currently sto support these two topics only.
```
STEPS FOR SETUP/TEST
1) Get the VESCollector image from Nexus
docker pull nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
2) Start the container (change the DMAAPHOST environment value to running DMAAP instance host)
docker run -d -p 8080:8080/tcp -p 8443:8443/tcp -P -e DMAAPHOST='10.0.0.174' nexus.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.1
3) Login into container and tail /opt/app/VESCollector/logs/collector.log
4) Simulate event into VEScollector (can be done from different vm or same)
curl -i -X POST -d @measurement.txt --header "Content-Type: application/json" https://localhost:8443/eventListener/v5 -k
or curl -i -X POST -d @measurement.txt --header "Content-Type: application/json" http://localhost:8080/eventListener/v5 -k
Note: If DMAAPHOST provided is invalid, you will see exception around publish on the collector.logs (collector queues and attempts to resend the event hence exceptions reported will be periodic). If you don’t want to see the error, publish to dmaap can be disabled by changing either “collector.dmaap.streamid” on etc/collector.properties OR by modifying the “name” defined on etc/DmaapConfig.json.
Any changes to property within container requires collector restart
/opt/app/VESCollector/bin/VESrestfulCollector.sh stop
/opt/app/VESCollector/bin/VESrestfulCollector.sh start
5) If DMAAP instance (and DMAAPHOST passed during VESCollector startup) and VES input is valid, then events will be pushed to below topics depending on the domain
Fault :http://<dmaaphost>:3904/events/unauthenticated.SEC_FAULT_OUTPUT
Measurement : http://<dmaaphost>:3904/events/unauthenticated.SEC_MEASUREMENT_OUTPUT
6) When test is done – do ensure to remove the container (docker rm -f <containerid>) to avoid port conflict
```
Authentication is disabled on the container for R1; below are the steps for enabling HTTPS/authentication for VESCollector.
```
1) Login to the container
2) Open /opt/app/VESCollector/etc/collector.properties and edit below properties
a) Comment below property (with authentication enabled, standard http should be disabled)
collector.service.port=8080
b) Enable basic-authentication
header.authflag=1
Note: The actual credentials is stored part of header.authlist parameter. This is list of userid,password (base64encoded) values. Default configuration has below set
sample1,c2FtcGxlMQ==|vdnsagg,dmRuc2FnZw==, where password maps to same value as username.
3) Restart the collector
cd /opt/app/VESCollector/bin
./VESrestfulCollector.sh stop
./VESrestfulCollector.sh start
4) Exit from container and ensure tcp port on VM is not hanging on finwait – you can execute “netstat -an | grep 8443” . If under FIN_WAIT2, wait for server to release.
5) Simulate via curl (Note - username/pwd will be required)
Example of successfull POST:
vv770d@osdcae-dev-16:~$ curl -i -u 'sample1:sample1' -X POST -d @volte.txt --header "Content-Type: application/json" https://localhost:8443/eventListener/v5 -k
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
X-Rathravane: ~ software is craft ~
Content-Type: application/json;charset=ISO-8859-1
Content-Length: 17
Date: Thu, 21 Sep 2017 22:23:49 GMT
Message Accepted
Example of authentication failure:
vv770d@osdcae-dev-16:~$ curl -i -X POST -d @volte.txt --header "Content-Type: application/json" https://localhost:8443/eventListener/v5 -k
HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
X-Rathravane: ~ software is craft ~
Content-Type: application/json;charset=ISO-8859-1
Content-Length: 96
Date: Thu, 21 Sep 2017 22:20:43 GMT
Connection: close
{"requestError":{"GeneralException":{"MessagID":"\"POL2000\"","text":"\"Unauthorized user\""}}}
Note: In general support for HTTPS also require certificate/keystore be installed on target VM with FS mapped into the container for VESCollector to load. For demo and testing purpose - a self signed certificate is included within docker build. When deployed via DCAEGEN2 platform - these configuration will be overridden dynamically to map to required path/certificate name. This will be exercised post R1 though.
```
|
