summaryrefslogtreecommitdiffstats
path: root/docs/sections/apis/DFC.rst
blob: d7892a1454717beb6e46834862c2fbbc10ee4541 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
========================
DFC (DataFile Collector)
========================

:Date: 2018-09-21

.. contents::
    :depth: 3
..

Overview
========

Component description can be found under `DFC`_.

.. _DFC: ../../services/dfc/index.html


Paths
=====

GET /events/unauthenticated.VES_NOTIFICATION_OUTPUT
---------------------------------------------------

Description
~~~~~~~~~~~

Reads fileReady events from DMaaP (Data Movement as a Platform)


Responses
~~~~~~~~~

+-----------+---------------------+
| HTTP Code | Description         |
+===========+=====================+
| **200**   | successful response |
+-----------+---------------------+



POST /publish
-------------

Description
~~~~~~~~~~~

Publish the collected file/s as a stream to DataRouter
    - file as stream
    - compression
    - fileFormatType
    - fileFormatVersion


Responses
~~~~~~~~~

+-----------+---------------------+
| HTTP Code | Description         |
+===========+=====================+
| **200**   | successful response |
+-----------+---------------------+

Compiling DFC
=============

Whole project (top level of DFC directory) and each module (sub module directory) can be compiled using
`mvn clean install` command.

Configuration file: Config/datafile_endpoints.json

Maven GroupId:
==============

org.onap.dcaegen2.collectors

Maven Parent ArtifactId:
========================

dcae-collectors

Maven Children Artifacts:
=========================

1. datafile-app-server: DFC server
2. datafile-dmaap-client: Contains implementation of DmaaP client
3. datafile-commons: Common code for whole DFC modules
4. docker-compose: Contains the docker-compose

Configuration of Certificates in test environment(For FTP over TLS):
====================================================================

DFC supports two protocols: FTPES and SFTP.
For FTPES, it is mutual authentication with certificates.
In our test environment, we use vsftpd to simulate xNF, and we generate self-signed
keys & certificates on both vsftpd server and DFC.

1. Generate key/certificate with openssl for DFC:
-------------------------------------------------
.. code:: bash

    openssl genrsa -out dfc.key 2048
    openssl req -new -out dfc.csr -key dfc.key
    openssl x509 -req -days 365 -in dfc.csr -signkey dfc.key -out dfc.crt

2. Generate key & certificate with openssl for vsftpd:
------------------------------------------------------
.. code:: bash

   openssl genrsa -out ftp.key 2048
   openssl req -new -out ftp.csr -key ftp.key
   openssl x509 -req -days 365 -in ftp.csr -signkey ftp.key -out ftp.crt

3. Configure java keystore in DFC:
----------------------------------
We have two keystore files, one for TrustManager, one for KeyManager.

**For TrustManager:**

1. First, create a jks keystore for TrustManager:

 .. code:: bash

   keytool -keystore ftp.jks -genkey -alias ftp

2. Second, convert your certificate in a DER format :

 .. code:: bash

   openssl x509 -outform der -in ftp.crt -out ftp.der

3. And after, import it in the keystore :

 .. code:: bash

   keytool -import -alias ftp -keystore ftp.jks -file ftp.der

**For KeyManager:**

1. First, create a jks keystore:

 .. code:: bash

    keytool -keystore dfc.jks -genkey -alias dfc

2. Second, import dfc.crt and dfc.key to dfc.jks. This is a bit troublesome.

 1). Step one: Convert x509 Cert and Key to a pkcs12 file

 .. code:: bash

    openssl pkcs12 -export -in dfc.crt -inkey dfc.key -out dfc.p12 -name [some-alias]

 Note: Make sure you put a password on the p12 file - otherwise you'll get a null reference exception when yy to import it. (In case anyone else had this headache).

 Note 2: You might want to add the -chainoption to preserve the full certificate chain.

 2). Step two: Convert the pkcs12 file to a java keystore:

 .. code:: bash

    keytool -importkeystore -deststorepass [changeit] -destkeypass [changeit] -destkeystore dfc.jks -srckeystore dfc.p12 -srcstoretype PKCS12 -srcstorepass [some-password] -alias [some-alias]

3. Finished

4. Configure vsftpd:
--------------------
    update /etc/vsftpd/vsftpd.conf:

  .. code-block:: javascript

      rsa_cert_file=/etc/ssl/private/ftp.crt
      rsa_private_key_file=/etc/ssl/private/ftp.key
      ssl_enable=YES
      allow_anon_ssl=NO
      force_local_data_ssl=YES
      force_local_logins_ssl=YES

      ssl_tlsv1=YES
      ssl_sslv2=YES
      ssl_sslv3=YES

      require_ssl_reuse=NO
      ssl_ciphers=HIGH

      require_cert=YES
      ssl_request_cert=YES
      ca_certs_file=/home/vsftpd/myuser/dfc.crt

5. Configure config/datafile_endpoints.json:
--------------------------------------------
   Update the file accordingly:

  .. code-block:: javascript

            "ftpesConfiguration": {
                "keyCert": "/config/dfc.jks",
                "keyPassword": "[yourpassword]",
                "trustedCA": "/config/ftp.jks",
                "trustedCAPassword": "[yourpassword]"
            }

6. This has been tested with vsftpd and dfc, with self-signed certificates.
---------------------------------------------------------------------------
   In real deployment, we should use ONAP-CA signed certificate for DFC, and vendor-CA signed certificate for xNF