1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
|
========================
DFC (DataFile Collector)
========================
:Date: 2018-09-21
.. contents::
:depth: 3
..
Overview
========
Component description can be found under `DFC`_.
.. _DFC: ../../sections/services/dfc/index.html
Paths
=====
GET /events/unauthenticated.VES_NOTIFICATION_OUTPUT
---------------------------------------------------
Description
~~~~~~~~~~~
Reads fileReady events from DMaaP (Data Movement as a Platform)
Responses
~~~~~~~~~
+-----------+---------------------+
| HTTP Code | Description |
+===========+=====================+
| **200** | successful response |
+-----------+---------------------+
POST /publish
-------------
Description
~~~~~~~~~~~
Publish the collected file/s as a stream to DataRouter
- file as stream
- compression
- fileFormatType
- fileFormatVersion
Responses
~~~~~~~~~
+-----------+---------------------+
| HTTP Code | Description |
+===========+=====================+
| **200** | successful response |
+-----------+---------------------+
Compiling DFC
=============
Whole project (top level of DFC directory) and each module (sub module directory) can be compiled using
`mvn clean install` command.
Configuration file: Config/datafile_endpoints.json
Maven GroupId:
==============
org.onap.dcaegen2.collectors
Maven Parent ArtifactId:
========================
dcae-collectors
Maven Children Artifacts:
=========================
1. datafile-app-server: DFC server
2. datafile-dmaap-client: Contains implementation of DmaaP client
3. datafile-commons: Common code for whole DFC modules
4. docker-compose: Contains the docker-compose
Configuration of Certificates in test environment(For FTP over TLS):
====================================================================
DFC supports two protocols: FTPES and SFTP.
For FTPES, it is mutual authentication with certificates.
In our test environment, we use vsftpd to simulate xNF, and we generate self-signed
keys & certificates on both vsftpd server and DFC.
1. Generate key/certificate with openssl for DFC:
-------------------------------------------------
.. code:: bash
openssl genrsa -out dfc.key 2048
openssl req -new -out dfc.csr -key dfc.key
openssl x509 -req -days 365 -in dfc.csr -signkey dfc.key -out dfc.crt
2. Generate key & certificate with openssl for vsftpd:
------------------------------------------------------
.. code:: bash
openssl genrsa -out ftp.key 2048
openssl req -new -out ftp.csr -key ftp.key
openssl x509 -req -days 365 -in ftp.csr -signkey ftp.key -out ftp.crt
3. Configure java keystore in DFC:
----------------------------------
We have two keystore files, one for TrustManager, one for KeyManager.
**For TrustManager:**
1. First, create a jks keystore for TrustManager:
.. code:: bash
keytool -keystore ftp.jks -genkey -alias ftp
2. Second, convert your certificate in a DER format :
.. code:: bash
openssl x509 -outform der -in ftp.crt -out ftp.der
3. And after, import it in the keystore :
.. code:: bash
keytool -import -alias ftp -keystore ftp.jks -file ftp.der
**For KeyManager:**
1. First, create a jks keystore:
.. code:: bash
keytool -keystore dfc.jks -genkey -alias dfc
2. Second, import dfc.crt and dfc.key to dfc.jks. This is a bit troublesome.
1). Step one: Convert x509 Cert and Key to a pkcs12 file
.. code:: bash
openssl pkcs12 -export -in dfc.crt -inkey dfc.key -out dfc.p12 -name [some-alias]
Note: Make sure you put a password on the p12 file - otherwise you'll get a null reference exception when yy to import it. (In case anyone else had this headache).
Note 2: You might want to add the -chainoption to preserve the full certificate chain.
2). Step two: Convert the pkcs12 file to a java keystore:
.. code:: bash
keytool -importkeystore -deststorepass [changeit] -destkeypass [changeit] -destkeystore dfc.jks -srckeystore dfc.p12 -srcstoretype PKCS12 -srcstorepass [some-password] -alias [some-alias]
3. Finished
4. Configure vsftpd:
--------------------
update /etc/vsftpd/vsftpd.conf:
.. code-block:: bash
rsa_cert_file=/etc/ssl/private/ftp.crt
rsa_private_key_file=/etc/ssl/private/ftp.key
ssl_enable=YES
allow_anon_ssl=NO
force_local_data_ssl=YES
force_local_logins_ssl=YES
ssl_tlsv1=YES
ssl_sslv2=YES
ssl_sslv3=YES
require_ssl_reuse=NO
ssl_ciphers=HIGH
require_cert=YES
ssl_request_cert=YES
ca_certs_file=/home/vsftpd/myuser/dfc.crt
5. Configure config/datafile_endpoints.json:
--------------------------------------------
Update the file accordingly:
.. code-block:: javascript
"ftpesConfiguration": {
"keyCert": "/config/dfc.jks",
"keyPassword": "[yourpassword]",
"trustedCA": "/config/ftp.jks",
"trustedCAPassword": "[yourpassword]"
}
6. This has been tested with vsftpd and dfc, with self-signed certificates.
---------------------------------------------------------------------------
In real deployment, we should use ONAP-CA signed certificate for DFC, and vendor-CA signed certificate for xNF
|
