summaryrefslogtreecommitdiffstats
path: root/docs/sections/components/dcae-cli/walkthrough.rst
blob: 56c9e28ed9026006b2a4e0099bb8b4d939083c8a (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
.. This work is licensed under a Creative Commons Attribution 4.0 International License.
.. http://creativecommons.org/licenses/by/4.0

.. _walkthrough:

Walk-through
============

This section demonstrates the flow and usage of the dcae_cli tool to
onboard a typical component to the DCAE platform. The commands are
explained in more detail in :doc:`dcae_cli Commands <commands>`.

-  `Add (and validate) a data format <#add-a-data-format>`__
-  `Add (and validate) the component <#add-the-component>`__
-  `View the platform generated
   configuration <#view-the-platform-generated-configuration>`__
-  `If needed, Create the dmaap file for Dmaap Testing <#create-the-input-file-for-dmaap-testing>`__
-  `If needed, Create the input file for *Sourced at Deployment* Testing <#create-the-input-file-for-sourced-at-deployment-testing>`__
-  `Run the component <#run-the-component>`__
-  :any:`Undeploy the component <dcae_cli_undeploy_the_component>`
-  :any:`Publish the component and data_format <dcae_cli_publish_the_component_and_data_format>` to let others
   know its ready for reuse
-  `List the Catalog Contents <#list-the-catalog-contents>`__ to see
   your published resources


--------------

Add a Data Format
-----------------

::

    $ dcae_cli data_format add $HOME/yourapp/data-formats/health.json

Verify that the data_format was added

::

    $ dcae_cli data_format list | grep yourapp                                                                                              
    | sandbox.platform.yourapp.health | 0.1.0   | Data format used for the /health endpoint                   | staged    | 2017-11-07 21:48:47.736518 |

(Note: Each of the data formats for your component need to be added,
unless already existing in the onboarding catalog )

--------------

Add the Component
-----------------

::

    $ dcae_cli component add $HOME/yourapp/component-spec.json

Verify that the component was added

::

    $ dcae_cli component list 
    Active profile: solutioning

    +-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
    | Name                          | Version | Type   | Description                                                   | Status | Modified                   | #Deployed |
    +-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
    | sandbox.platform.yourapp      | 0.7.0   | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-11-08 20:27:34.168854 | 0         |
    +-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+

--------------

.. _dcae-cli-view-the-platform:

View the platform generated configuration
-----------------------------------------

The ``component dev`` command is useful during onboarding. Running this
command is part of a multi-step process that sets up a temporary test
environment, generates your application configuration, makes it
available in that environment, and allows you to view that configuration
to help with debugging.

Here is a step-by-step example based on a component specification called
``component-spec.json``.

Step 1 - Run the component dev command
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

(This creates a file called env_$ENV (in the current directory)- where
$ENV is the name of the active profile. Note: SERVICE_NAME and HOSTNAME
always resolve to the same value).

::

    $ dcae_cli component dev component-spec.json
    Ready for component development

    Setup these environment variables. Run "source env_solutioning":

    export DOCKER_HOST=yourdockerhost.com:2376
    export SERVICE_CHECK_INTERVAL=15s
    export CONFIG_BINDING_SERVICE=config_binding_service
    export HOSTNAME=user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp
    export CONSUL_HOST=yourconsulhost.com
    export CDAP_BROKER=cdap_broker
    export SERVICE_NAME=user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp
    export SERVICE_CHECK_TIMEOUT=1s
    export SERVICE_CHECK_HTTP=/health

    Press any key to stop and to clean up

Step 2 - Setup the environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In another window, setup the temporary testing environment, by executing
the environment file created above.

::

    $ source env_solutioning

(The application configuration is now available under the SERVICE_NAME
shown above -
``user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp``).

Step 3 - Query CONSUL
~~~~~~~~~~~~~~~~~~~~~

Query CONSUL to get the IP/PORT of CONFIG BINDING SERVICE

::

    $ curl http://$CONSUL_HOST:8500/v1/catalog/service/$CONFIG_BINDING_SERVICE
    [
      {
        "ID": "bfbc220d-4603-7f90-ec2e-611d3c330f20",
        "Node":"docker00",
        "Address": "10.226.1.15",
        "Datacenter":"solutioning-central",
        "TaggedAddresses": {
          "lan":"10.226.1.15",
          "wan":"10.226.1.15"
        },
        "NodeMeta": {},
        "ServiceID": "472b116f9035:config_binding_service:10000",
        "ServiceName": "config_binding_service",
        "ServiceTags": [],
        "ServiceAddress":"135.205.226.126",
        "ServicePort":10000,
        "ServiceEnableTagOverride": false,
        "CreateIndex":1078990,
        "ModifyIndex":1078990
      }
    ]

Fetch the generated configuration from CONFIG BINDING SERVICE using the
‘serviceaddress’ and ‘serviceport’ from above along with $SERVICE_NAME
from earlier.

::

    $ curl http://135.205.226.126:10000/service_component/user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp

    {"streams_subscribes": {}, "services_calls": {}, "multiplier": 3, "streams_publishes": {}}

--------------

.. _dcae-cli-walkthrough-dmaap-testing:

Create the input file for Dmaap Testing
---------------------------------------

Currently, the dcae-cli tool does not have the capability to provision
topics or feeds. Therefore, in order to test with ``message router`` or
``data router`` feeds, the developer must manually provision the topic
or feed and then provide the connection details in the form of a DMaap
JSON file for testing. This file is then passed in on the
``component run`` or ``component dev`` commands by using the argument
``--dmaap-file``.

The structure of the DMaaP JSON is an object of config keys with the
topic or feed connection details. The config keys are the ``config_key``
values specified in the component specification streams section where
the streams must be type ``message router`` or ``data router``. This
file corresponds to the ``Dmaap Connection Object`` which is generated
by the platform and provided to the component at runtime. The exception
is that ``delivery_url`` cannot be provided in the dmaap-file because it
is not created until the component is deployed. Refer to :any:`Dmaap Connection Object <dmaap-connection-objects>`, for details on creating the dmaap-file for testing.

--------------

Create the input file for *Sourced at Deployment* Testing
---------------------------------------------------------

Components may have configuration parameters whose values are to be
sourced at deployment time. This is established in the
:any:`component specification <common-specification-parameters>`
by setting the property ``sourced_at_deployment`` to ``true`` for each
applicable parameter.

Then, use the ``--inputs-file`` command-line argument when running the
component ``dev`` or ``run`` command for your component. This is to
simulate providing the dynamic, deployment time values for those
parameters marked as ``sourced_at_deployment``.

For example, if your component specification has the following
configuration parameters:

::

    "parameters": [{
        "name": "vnf-ip",
        "value": "",
        "sourced_at_deployment": true
    },
    {
        "name": "static-param",
        "value": 5
    }]

Pass in an input file that looks like:

::

    {
        "vnf-ip": "10.100.1.100"
    }

The application configuration would look like:

::

    {
        "vnf-ip": "10.100.1.100",
        "static-param": 5
    }

--------------

Run the component
-----------------

The ``run`` operation is to be used for running your application in its
container remotely on the activated environment. Docker containers have
the additional option to run locally on your development machine. If the
component uses Dmaap, you can specify the Dmaap Connection Object as
well. Refer to :any:`Dmaap Connection Object <dmaap-connection-objects>`.

In order to run the component, the data formats and component must have
been added to the onboarding catalog.

To verify what’s in the catalog:

::

    $ dcae_cli catalog list --expanded                                                                                      
    Active profile: solutioning
    +---------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
    | Name                      | Version | Type   | Description                                                   | Status | Modified                   | #Deployed |
    +---------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
    | sandbox.platform.yourapp  | 0.7.0   | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-11-08 20:27:34.168854 | 0         |
    +---------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+

For Docker

**NOTE** Make sure the Docker image has been uploaded to the shared
registry.

A docker component can be run in either ``attached`` or ``unattached``
mode. (Default is unattached).

+------------------+-----------------------------------------------------------+
| Mode             | Description                                               |
+==================+===========================================================+
| attached         | component is run in the foreground, container             |
|                  | logs are streamed to stdout. Ctrl-C is used to            |
|                  | terminate the dcae_cli session.                           |
+------------------+-----------------------------------------------------------+
| unattached       | component is run in the background container              |
|                  | logs are viewed via ``docker logs`` command,              |
|                  | container runs until undeployed with dcae_cli             |
|                  | ``undeploy`` command.                                     |
+------------------+-----------------------------------------------------------+

Run a component in attached mode:
---------------------------------

::

    $ dcae_cli -v component run --attached sandbox.platform.yourapp:0.7.0
    DCAE.Docker | INFO | Running image 'nexus01.server.com:18443/repository/solutioning01-mte2-docker/dcae-platform/yourapp:0.7.0' as 'user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp'
    DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | Consul host: yourconsulhost.com

    DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | service name: user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp

    DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | get_config returned the following configuration: {"streams_subscribes": {}, "multiplier": 3, "services_calls": {}, "streams_publishes": {}}

    DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO |  * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)

    DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | 135.205.226.156 - - [08/Nov/2017 23:27:30] "GET /health HTTP/1.1" 200 -


    Hit Ctrl-C to terminate session.

    ^C
    DCAE.Docker | INFO | Stopping container 'user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp' and cleaning up...

Run a component in unattached mode:
-----------------------------------

::

    $ dcae_cli -v component run sandbox.platform.yourapp:0.7.0
    DCAE.Docker | INFO | Running image 'nexus01.server.com:18443/repository/solutioning01-mte2-docker/dcae-platform/yourapp:0.7.0' as 'user12.22629ebd-417e-4e61-a9a0-f0cb16d4cef2.0-7-0.sandbox-platform-yourapp'
    DCAE.Run | INFO | Deployed user12.22629ebd-417e-4e61-a9a0-f0cb16d4cef2.0-7-0.sandbox-platform-yourapp. Verifying..
    DCAE.Run | INFO | Container is up and healthy

**NOTE** You must undeploy this component when finished testing. This is
important to conserve resources in the environment.

Run a component that subscribes to Dmaap Message Router or Data Router
----------------------------------------------------------------------

::

    $ dcae_cli -v component run $component-that-uses-dmamp --dmaap-file $dmaap-connection-object

Run a component that expects input that is ``sourced at deployment``
--------------------------------------------------------------------

::

    $ dcae_cli -v component run $component-that-expects-dti --inputs-file $input-file-to-simulate-dti

--------------


.. _dcae_cli_undeploy_the_component:

Undeploy the component
----------------------

The ``undeploy`` command is used to undeploy any instance of a specified component/version that you have deployed. This includes cleaning up the configuration. 
Undeploy ``sandbox.platform.yourapp:0.7.0`` that was deployed above: 
``$ dcae_cli -v component undeploy sandbox.platform.yourapp:0.7.0 DCAE.Undeploy | WARNING | Undeploying components: 1 DCAE.Undeploy | WARNING | Undeployed components: 1`` 

.. _dcae_cli_publish_the_component_and_data_format:

Publish the component and data_format
-------------------------------------

Once a component has been tested, it (and the data_format(s)) should be
published in the onboarding catalog using the ``publish`` sub-command
for both the ``data_format`` and ``component`` command.

**Note** Before a component can be published, all data_formats that it
references must be published.

Publishing will change the status of a component or data_format,
indicating that it has been tested, make accessible for other developers
to use.

::

    $ dcae_cli data_format publish sandbox.platform.yourapp:0.7.0
    Data format has been published

    $dcae_cli component publish sandbox.platform.yourapp:0.7.0
    Component has been published

--------------

List the catalog contents
-------------------------

::

    $dcae_cli catalog list

    $ dcae_cli data_format list | grep sandbox
    | sandbox.platform.yourapp         | 0.7.0   | docker | Web service used as a stand-alone test DCAE service compone..      | user12    | published | 2017-11-13 |
    | sandbox.platform.yourapp.health            | 0.1.0  | Data format used for the /health endpoint                          | published | 2017-11-13 17:48:10.121588 |
    | sandbox.platform.any                       | 0.1.0  | Data format used when no data format is required.                  | published | 2017-11-13 17:47:51.622607 |
    | sandbox.platform.yourapp.identity.response | 0.1.0  | Data format used for the /identity endpoint response which should  | published | 2017-11-13 17:47:43.234715 |
    | sandbox.platform.yourapp.identity.request  | 0.1.0  | Data format used for the /identity endpoint request. This is       | published | 2017-11-13 17:47:36.693643 |
    | sandbox.platform.yourapp.rollcall.response | 0.1.0  | Data format used for the /rollcall endpoint respon..               | published | 2017-11-13 17:46:30.026846 |