diff options
| author |   Ralph Knag <rhknag@research.att.com> | 2018-04-02 16:27:46 -0400 |
|---|---|---|
| committer | Ralph Knag <rhknag@research.att.com> | 2018-04-02 16:54:16 -0400 |
| commit | d2cd31b73c0282f7aafd5b4adada00c0f4533d61 (patch) | |
| tree | 0ca259920246992703fe4c6c996c0e0639cdeef2 /docs/sections/components/dcae-cli | |
| parent | ef55ef2163dee32905bd10b0a0b3ea0f6a763322 (diff) | |
Onboarding documentation update for CLI
Change-Id: I1d4d0111063ea62c3759aa9b7232998b70229644
Issue-ID: DCAEGEN2-350
Signed-off-by: Ralph Knag <rhknag@research.att.com>
Diffstat (limited to 'docs/sections/components/dcae-cli')
| -rw-r--r-- | docs/sections/components/dcae-cli/commands.rst | 590 | ||||
| -rw-r--r-- | docs/sections/components/dcae-cli/dcae-cli-tool.rst | 2 | ||||
| -rwxr-xr-x | docs/sections/components/dcae-cli/dmaap-connection-objects.rst | 415 | ||||
| -rwxr-xr-x | docs/sections/components/dcae-cli/quickstart.rst | 140 | ||||
| -rwxr-xr-x | docs/sections/components/dcae-cli/walkthrough.rst | 687 |
5 files changed, 936 insertions, 898 deletions
diff --git a/docs/sections/components/dcae-cli/commands.rst b/docs/sections/components/dcae-cli/commands.rst new file mode 100644 index 00000000..d34ba43d --- /dev/null +++ b/docs/sections/components/dcae-cli/commands.rst @@ -0,0 +1,590 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+
+.. _dcae_cli_commands:
+
+dcae_cli Commands
+=================
+
+The dcae_cli tool has four command groups. Each has several
+sub-commands.
+
+``catalog``
+-----------
+
+The ``catalog`` command lists and shows resources (components and
+data_formats) in the ‘onboarding’ catalog (regardless of the owner). A
+resource can have a status of ``staged`` or ``published``. By default,
+only ``published`` resources are displayed. To see ``staged`` resources,
+add the –expanded argument.
+
++-----------------------------------------------+-----------------------+
+| Catalog Status | Meaning |
++===============================================+=======================+
+| staged | resource has be added |
+| | (and validated), but |
+| | is under development |
++-----------------------------------------------+-----------------------+
+| staged data_formats can only be referenced in |
+| their owners component specs |
++-----------------------------------------------+-----------------------+
+| staged components can only be deployed by |
+| their owners |
++-----------------------------------------------+-----------------------+
+| published | resource has been |
+| | tested and can be |
+| | shared |
++-----------------------------------------------+-----------------------+
+| published data_formats can be used in |
+| anyone’s component spec |
++-----------------------------------------------+-----------------------+
+| published components and be deployed by |
+| anyone |
++-----------------------------------------------+-----------------------+
+
+::
+
+ $ dcae_cli catalog --help
+ Usage: dcae_cli catalog [OPTIONS] COMMAND [ARGS]...
+
+ Options:
+ --help Show this message and exit.
+
+ Commands:
+ list Lists resources in the onboarding catalog
+ show Provides more information about resource
+
+List onboarding catalog contents
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ $ dcae_cli catalog list
+ Components:
+ +--------------------------------+---------+--------+---------------------------------------------------------------------+--------+-----------+------------+
+ | Name | Version | Type | Description | Owner | Status | Published |
+ +--------------------------------+---------+--------+---------------------------------------------------------------------+--------+-----------+------------+
+ | DcaeSyslogCollector | 2.0.0 | docker | DCAE Control Plane Syslog Collector | sh1986 | published | 2017-08-04 |
+ | cdap.dmaap.spec.example | 0.2.0 | cdap | dmaap spec example. Not a functioning application, only for showing | tc677g | published | 2017-07-24 |
+ | | | | how to pub/sub dmaap. Pretend this is like MAP with VES in and ou.. | | | |
+ | cdap.event.proc.enrich.app | 1.0.3 | cdap | CDAP Event Processing Enrich application | an4828 | published | 2017-09-20 |
+ | cdap.event.proc.map.app | 1.0.3 | cdap | CDAP Event Processing Map application | an4828 | published | 2017-09-20 |
+
+ ...
+
+ Data formats:
+ +--------------------------------------------+---------+-----------------------------------------------------------------------+--------+-----------+------------+
+ | Name | Version | Description | Owner | Status | Published |
+ +--------------------------------------------+---------+-----------------------------------------------------------------------+--------+-----------+------------+
+ | FOI_PM_VHSS_data_format | 1.0.0 | CSV pipe delimited data format for VHSS PM files | sr229c | published | 2017-09-05 |
+ | Map_input | 1.0.0 | The input format for Mapper, that in 1707 is the UCSNMP Collector | an4828 | published | 2017-07-18 |
+ | | | output format, but will support more formats later | | | |
+ | Syslog Collector Parsed Json Message | 1.0.0 | Post processed/parsed collected syslog message | sh1986 | published | 2017-08-04 |
+ | Syslog Collector Syslog Message Input | 1.0.0 | The input message for the DCAE syslog collector is free/unstructured | sh1986 | published | 2017-08-04 |
+ | | | text | | | |
+ | myapp Alert Definition | 1.0.0 | The format of the output event from myapp | an4828 | published | 2017-08-10 |
+ | VES_specification | 5.28.4 | VES spec for 5.4 | vv770d | published | 2017-09-19 |
+
+ ...
+
+Show the contents of an item in the catalog
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ $ dcae_cli catalog show FOI_PM_VHSS_data_format
+
+ Data format
+ -----------
+ {
+ "dataformatversion": "1.0.0",
+ "delimitedschema": {
+ "delimiter": "|",
+ "fields": [
+ {
+ "description": "System ID",
+ "fieldtype": "string",
+ "name": "SYSTEM"
+ },
+ {
+ "description": "Date",
+ "fieldtype": "string",
+ "name": "DATE"
+ },
+ {
+ "description": "Time",
+ "fieldtype": "string",
+ "name": "TIME"
+ },
+
+ ...
+
+.. _dcae_cli_component_commands:
+
+--------------
+
+``component``
+-------------
+
+The ``component`` command is for validating (adding), listing, showing,
+verifying generated configuration, running, undeploying, and publishing
+components that YOU own.
+
+::
+
+ $ dcae_cli component --help
+ Usage: dcae_cli component [OPTIONS] COMMAND [ARGS]...
+
+ Options:
+ --help Show this message and exit.
+
+ Commands:
+ add
+ dev Set up component in development for...
+ list Lists components in the onboarding catalog.
+ publish Pushes COMPONENT to the onboarding catalog
+ run Runs the latest version of COMPONENT.
+ show Provides more information about COMPONENT
+ undeploy Undeploys the latest version of COMPONENT.
+
+--------------
+
+.. _dcae_cli_add_a_component:
+
+Add a Component
+~~~~~~~~~~~~~~~
+
+A component must be added to the onboarding catalog in order to be
+tested by the dcae_cli tool. The process of adding a component also
+validates it’s component specification. In order to add a component, the
+component docker image (or CDAP jar) must exist locally.
+
+Components in the onboarding catalog can be run by others, once they are
+``published.`` ``Published`` components cannot be modified or deleted.
+Rather a new version can be created instead.
+
+Validated component specs are used later to generate Tosca models and
+Cloudify Blueprints for the component, which makes them available for
+use in the SDC Tool for creating services.
+
+::
+
+ $ dcae_cli component add --help
+ Usage: dcae_cli component add [OPTIONS] COMPONENT-SPECIFICATION
+
+ Options:
+ --update Updates a previously added component if it has not been already
+ published
+ --help Show this message and exit.
+
+::
+
+ $ dcae_cli component add component-spec.json
+
+--------------
+
+List Components
+~~~~~~~~~~~~~~~
+
+List components in the onboarding catalog that owned by YOUR userid..
+
+::
+
+ $ dcae_cli component list
+ Active profile: solutioning
+ +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+ | Name | Version | Type | Description | Status | Modified | #Deployed |
+ +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+ | cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 |
+ | sandbox.platform.yourapp| 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 |
+ +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+
+The fields ``Name``, ``Version``, ``Type``, ``Description`` are
+referenced from the component specification’s ``self`` JSON. Use the
+“–deployed” argument to see more details on deploymed components
+
+--------------
+
+.. _dcae_cli_run_a_component:
+
+Run a Component
+~~~~~~~~~~~~~~~
+
+The ``run`` operation is to be used for running your application in its
+container remotely on the activated environment. Docker containers have
+the additional option to run locally on your development machine. If the
+component uses Dmaap, you can specify the Dmaap Connection Object as
+well. Refer to :doc:`Dmaap Connection Object <../component-specification/dmaap-connection-objects>`.
+
+When you run a component via the dcae_cli Tool, remember the blueprint
+has not been created and is not used for deployment.
+
+In order to run the component, the data formats and component must have
+been added to the onboarding catalog.
+
+**DOCKER NOTE:** Make sure the Docker image has been uploaded to the
+shared registry.
+
+A docker component can be run in either ``attached`` or ``unattached``
+mode. (Default is unattached).
+
++------------------+---------------------------------------------------+
+| Mode | Description |
++==================+===================================================+
+| attached | component is run in the ‘foreground’, container |
+| | logs are streamed to stdout. Ctrl-C is used to |
+| | terminate the dcae_cli session. |
++------------------+---------------------------------------------------+
+| unattached | component is run in the ‘background’, container |
+| | logs are viewed via ``docker logs`` command, |
+| | container runs until undeployed with dcae_cli |
+| | ``undeploy`` command. |
++------------------+---------------------------------------------------+
+
+Run a component in attached mode:
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+::
+
+ $ dcae_cli -v component run --attached sandbox.platform.yourapp:0.5.0
+
+ DCAE.Docker | INFO | Running image 'registry.proto.server.com/dcae-rework/yourapp:0.4.0' as 'user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp'
+ DCAE.Docker.user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp | INFO | Consul host: yourconsulhost.com
+
+ DCAE.Docker.user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp | INFO | service name: mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp
+
+ DCAE.Docker.user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp | INFO | Generated config: {'multiplier': 3}
+
+ DCAE.Docker.user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp | INFO | * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
+
+ DCAE.Docker.user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp | INFO | 135.205.226.140 - - [24/May/2017 03:37:57] "GET /health HTTP/1.1" 200 -
+
+ DCAE.Docker.user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp | INFO | 135.205.226.140 - - [24/May/2017 03:38:12] "GET /health HTTP/1.1" 200 -
+
+Hit Ctrl-C to terminate session.
+
+::
+
+ ^CDCAE.Docker | INFO | Stopping container 'user1.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-yourapp' and cleaning up...
+
+Run a component in unattached mode:
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+::
+
+ $ dcae_cli -v component run sandbox.platform.yourapp:0.5.0
+ DCAE.Docker | INFO | Running image 'registry.proto.server.com/dcae-rework/yourapp:0.4.0' as 'user1.4811da0e-08d5-429f-93bf-bf6814924577.0-5-0.sandbox-platform-yourapp'
+ DCAE.Run | INFO | Deployed /user1.4811da0e-08d5-429f-93bf-bf6814924577.0-5-0.sandbox-platform-yourapp
+
+**NOTE** You must undeploy this component when finished testing. This is
+important to conserve resources in the environment.
+
+Run a component that subscribes to Dmaap MR or DR
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+::
+
+ $ dcae_cli -v component run --attached --dmaap-file $dmaap-connection-file sandbox.platform.yourapp:0.5.0
+
+--------------
+
+Undeploy a Component
+~~~~~~~~~~~~~~~~~~~~
+
+The ``undeploy`` command is used to undeploy any instance of a specified
+component/version that you have deployed. This includes cleaning up the
+configuration.
+
+::
+
+ $ dcae_cli component undeploy sandbox.platform.yourapp:0.5.0
+ DCAE.Undeploy | WARNING | Undeploying components: 1
+ DCAE.Undeploy | WARNING | Undeployed components: 1
+
+--------------
+
+Publish a component
+~~~~~~~~~~~~~~~~~~~
+
+| Once a component has been tested, it should be published in the
+ onboarding catalog using the ``publish`` sub-command .
+| Publishing will change the status of the component (from ``staged`` to
+ ``published``), indicating that it has been tested, and making it
+ accessible for other developers to use.
+
+**Note** Before a component can be published, all data_formats that it
+references must be published.
+
+::
+
+ dcae_cli component publish sandbox.platform.yourapp:0.5.0
+
+--------------
+
+Show a Component
+~~~~~~~~~~~~~~~~
+
+This will print out the contents of a component and is useful to copy a
+component spec.
+
+::
+
+ $ dcae_cli component show
+
+--------------
+
+.. _dcae-cli-run-the-dev-command:
+
+Run the ``dev`` command
+~~~~~~~~~~~~~~~~~~~~~~~
+
+The ``dev`` command is used as part of a process to see the platform
+generated configuration. It established the environment variables and is
+best explained
+:any:`here <dcae-cli-view-the-platform>`.
+
+::
+
+ $ dcae_cli component dev component-spec.json
+ Ready for component development
+
+--------------
+
+.. _dcae_cli_data_format:
+
+``data_format``
+---------------
+
+The ``data_format`` command is for validating (adding), listing,
+showing, publishing data_formats that YOU own. data_formats can also be
+generated with this command.
+
+::
+
+ $ dcae_cli data_format --help
+ Usage: dcae_cli data_format [OPTIONS] COMMAND [ARGS]...
+
+ Options:
+ --help Show this message and exit.
+
+ Commands:
+ add Tracks a data format file DATA_FORMAT-SPECIFICATION...
+ generate Create schema from a file or directory...
+ list Lists all your data formats
+ publish Publishes data format to make available to others...
+ show Provides more information about FORMAT
+
+--------------
+
+.. _dcae_cli_add_a_data_format:
+
+Add a Data Format
+~~~~~~~~~~~~~~~~~
+
+A data_format must be in the onboarding catalog in order to be
+referenced in the component specification. The process of adding a
+data_format also validates it. Data_formats in the onboarding catalog
+can be run by others, once they are ``published.`` ``Published``
+data_formats cannot be modified or deleted. Rather a new version can be
+created instead.
+
+::
+
+ $ dcae_cli data_format add --help
+ Usage: dcae_cli data_format add [OPTIONS] DATA_FORMAT-SPECIFICATION
+
+ Options:
+ --update Updates a previously added data_format if it has not been already
+ published
+ --help Show this message and exit.
+
+::
+
+ dcae_cli data_format add health.json
+
+--------------
+
+List Data Formats
+~~~~~~~~~~~~~~~~~
+
+Only data_formats owned by YOUR userid will be shown.
+
+::
+
+ $ dcae_cli data_format list
+
+ Data formats for user1
+ +---------------------------------+---------+-------------------------------------------+--------+----------------------------+
+ | Name | Version | Description | Status | Modified |
+ +---------------------------------+---------+-------------------------------------------+--------+----------------------------+
+ | sandbox.platform.yourapp.health | 0.1.0 | Data format used for the /health endpoint | staged | 2017-05-23 04:02:38.952799 |
+ +---------------------------------+---------+-------------------------------------------+--------+----------------------------+
+
+The fields ``name``, ``version``, ``description`` are referenced from
+the data format specification’s ``self`` JSON. ``Status`` represents the
+status of the data format in the catalog. See `Publish a Data
+Format <#publish-a-data-format>`__ for more info.
+
+--------------
+
+Show a Data Format
+~~~~~~~~~~~~~~~~~~
+
+This will print out the contents of a data_format and is useful for
+copying a data_format.
+
+::
+
+ $ dcae_cli data_format show
+
+--------------
+
+Publish a Data Format
+~~~~~~~~~~~~~~~~~~~~~
+
+| Once a data_format has been tested (by referencing it in a component
+ spec that has been tested), it should be published in the onboarding
+ catalog using the ``publish`` sub-command .
+| Publishing will change the status of the data_format (from ``staged``
+ to ``published``), indicating that it has been tested, and making it
+ accessible for other developers to use.
+
+::
+
+ $ dcae_cli data_format publish data_format.json
+
+--------------
+
+Generate a Data Format
+~~~~~~~~~~~~~~~~~~~~~~
+
+If you already have a valid input or output file, you can use the
+generate command to create the it’s data format specification.
+
+::
+
+ $ dcae_cli data_format generate name:version file-or-dir-path
+
+--------------
+
+``profiles``
+------------
+
+The\ ``profiles`` command is for creating, deleting, listing, showing,
+activating, and updating (set) profiles. The profile contains
+environment variables used to connect to different environments. This is
+used in the running and deployment of a component using the
+``dcae_cli component run`` or ``dev`` command.
+
+::
+
+ $ dcae_cli profiles --help
+ Usage: dcae_cli profiles [OPTIONS] COMMAND [ARGS]...
+
+ Options:
+ --help Show this message and exit.
+
+ Commands:
+ activate Sets profile NAME as the active profile
+ create Creates a new profile NAME initialized with...
+ delete Deletes profile NAME
+ list Lists available profiles
+ set Updates profile NAME such that KEY=VALUE
+ show Prints the profile dictionary
+
+--------------
+
+List the available profiles
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ $ dcae_cli profiles list
+ * solutioning
+ 1710
+ 1802
+
+The \* identifies the active profile. ``dcae-cli`` is currently
+installed with profiles for the ``solutioning``, ``1710``, and ``1802``
+environments. They are intended for the following:
+
++-----------------------------------+-----------------------------------+
+| Environment | Description |
++===================================+===================================+
+| solutioning | default environment; used for |
+| | initial component developer |
+| | testing with the dcae_cli tool. |
++-----------------------------------+-----------------------------------+
+| 1710 | FTL3 (Functional Testing Lab 3) |
+| | environment, which represents the |
+| | 1710 release. |
++-----------------------------------+-----------------------------------+
+| 1802 | FTL3a (Functional Testing Lab 3a) |
+| | environment, which represents the |
+| | 1802 release. |
++-----------------------------------+-----------------------------------+
+
+--------------
+
+Show the details of a profile
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ $ dcae_cli profiles show solutioning
+ {
+ "cdap_broker": "cdap_broker",
+ "config_binding_service": "config_binding_service",
+ "consul_host": "yourconsulhost.com",
+ "docker_host": "yourdockerhost.com:2376"
+ }
+
+--------------
+
+.. _dcae_cli_activate_profile:
+
+Activate a profile
+~~~~~~~~~~~~~~~~~~
+
+To switch among profiles, use the activate sub-command. A subsequent
+``list`` will reveal the change made.
+
+::
+
+ $ dcae_cli profiles activate test
+
+--------------
+
+Create a new profile
+~~~~~~~~~~~~~~~~~~~~
+
+If you want to work in a different environment using the dcae_cli tool,
+you can make your own profile. (The environment must be a working DCAE
+Platform environment).
+
+::
+
+ $ dcae_cli profiles create new-profile
+
+After creating you would assign the variables with the ``set``
+sub-command. Then activate it to use.
+
+--------------
+
+Set variables in a profile
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+::
+
+ $ dcae_cli profiles set $profile $key $value
+
+--------------
+
+Delete a profile
+~~~~~~~~~~~~~~~~
+
+::
+
+ $ dcae_cli profiles delete new-profile
+
+
diff --git a/docs/sections/components/dcae-cli/dcae-cli-tool.rst b/docs/sections/components/dcae-cli/dcae-cli-tool.rst index 6f564cee..477fc341 100644 --- a/docs/sections/components/dcae-cli/dcae-cli-tool.rst +++ b/docs/sections/components/dcae-cli/dcae-cli-tool.rst @@ -8,6 +8,6 @@ dcae-cli Tool :maxdepth: 1 ./quickstart.rst + ./commands.rst ./walkthrough.rst - ./dmaap-connection-objects.rst diff --git a/docs/sections/components/dcae-cli/dmaap-connection-objects.rst b/docs/sections/components/dcae-cli/dmaap-connection-objects.rst deleted file mode 100755 index cad3b0c9..00000000 --- a/docs/sections/components/dcae-cli/dmaap-connection-objects.rst +++ /dev/null @@ -1,415 +0,0 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-
-DMaaP connection objects
-========================
-
-DMaaP connection objects are JSON objects that:
-
-1. Components should expect at runtime in their application
- configuration and is to be used to connect to the appropriate DMaaP
- feed or topic.
-2. Developers must provide through the command-line argument
- ``--dmaap-file`` to test their component with manually provisioned
- feeds and topics.
-
-This page is a reference to the specific structure that each type of
-DMaaP stream requires.
-
-Note for #1 that components should expect the entire object with all
-properties at runtime where the default will be ``null`` unless
-specified otherwise.
-
-Note for #2 that developers are not required to provide the entire
-object. The required properties will be labeled with “*required as
-input*”.
-
-.. _dmaap-message-router:
-
-Message router
---------------
-
-Publishers and subscribers both have the same JSON object structure.
-Here’s an example:
-
-.. code:: json
-
- {
- "type": "message_router",
- "aaf_username": "some-user",
- "aaf_password": "some-password",
- "dmaap_info": {
- "client_role": "com.dcae.member",
- "client_id": "1500462518108",
- "location": "mtc00",
- "topic_url": "https://we-are-message-router.us:3905/events/some-topic"
- }
- }
-
-
-At the top-level:
-
-+-------------+----+--------------------+
-| Property | Ty\| Descript\ |
-| Name | pe | ion |
-+=============+====+====================+
-| type | st\| *Require\ |
-| | ri\| d \ |
-| | ng | as \ |
-| | | input*. |
-| | | Must be |
-| | | ``message_router`` |
-| | | for |
-| | | message |
-| | | router |
-| | | topics |
-+-------------+----+--------------------+
-| aaf_usernam\| st\| AAF |
-| e | ri\| username |
-| | ng | message |
-| | | router |
-| | | clients |
-| | | use to |
-| | | authenti\ |
-| | | cate |
-| | | with |
-| | | secure |
-| | | topics |
-+-------------+----+--------------------+
-| aaf_passwor\| st\| AAF |
-| d | ri\| password |
-| | ng | message |
-| | | router |
-| | | clients |
-| | | use to |
-| | | authenti\ |
-| | | cate |
-| | | with |
-| | | secure |
-| | | topics |
-+-------------+----+--------------------+
-| dmaap_info | JS\| *Require\ |
-| | ON | d \ |
-| | ob\| as \ |
-| | je\| input*. |
-| | ct | Contains |
-| | | the |
-| | | topic |
-| | | connecti\ |
-| | | on |
-| | | details |
-+-------------+----+--------------------+
-
-The ``dmaap_info`` object contains:
-
-+-------------+----+----------+
-| Property | Ty\| Descript\|
-| Name | pe | ion |
-+=============+====+==========+
-| client_role | st\| AAF |
-| | ri\| client |
-| | ng | role |
-| | | that’s |
-| | | requesti\|
-| | | ng |
-| | | publish |
-| | | or |
-| | | subscrib\|
-| | | e |
-| | | access |
-| | | to the |
-| | | topic |
-+-------------+----+----------+
-| client_id | st\| Client |
-| | ri\| id for |
-| | ng | given |
-| | | AAF |
-| | | client |
-+-------------+----+----------+
-| location | st\| DCAE |
-| | ri\| location |
-| | ng | for the |
-| | | publishe\|
-| | | r |
-| | | or |
-| | | subscrib\|
-| | | er, |
-| | | used to |
-| | | set up |
-| | | routing |
-+-------------+----+----------+
-| topic_url | st\| *Require\|
-| | ri\| d \ |
-| | ng | as \ |
-| | | input*. |
-| | | URL for |
-| | | accessin\|
-| | | g |
-| | | the |
-| | | topic to |
-| | | publish |
-| | | or |
-| | | receive |
-| | | events |
-+-------------+----+----------+
-
-Here’s an example of the minimal JSON that must be provided as an input:
-
-.. code:: json
-
- {
- "type": "message_router",
- "dmaap_info": {
- "topic_url": "https://we-are-message-router.us:3905/events/some-topic"
- }
- }
-
-.. _dmaap-data-router:
-
-Data router
------------
-
-Publisher
-~~~~~~~~~
-
-Here’s an example of what the JSON object connection for data router
-publisher looks like:
-
-.. code:: json
-
- {
- "type": "data_router",
- "dmaap_info": {
- "location": "mtc00",
- "publish_url": "https://we-are-data-router.us/feed/xyz",
- "log_url": "https://we-are-data-router.us/feed/xyz/logs",
- "username": "some-user",
- "password": "some-password",
- "publisher_id": "123456"
- }
- }
-
-At the top-level:
-
-+-------------+----+----------------+
-| Property | Ty\| Descript\ |
-| Name | pe | ion |
-+=============+====+================+
-| type | st\| *Require\ |
-| | ri\| d \ |
-| | ng | as \ |
-| | | input*. |
-| | | Must be |
-| | | ``data_router``|
-| | | for data |
-| | | router |
-| | | feeds |
-+-------------+----+----------------+
-| dmaap_info | JS\| *Require\ |
-| | ON | d \ |
-| | ob\| as \ |
-| | je\| input*. |
-| | ct | Contains |
-| | | the |
-| | | topic |
-| | | connecti\ |
-| | | on |
-| | | details |
-+-------------+----+----------------+
-
-The ``dmaap_info`` object contains:
-
-+-------------+----+----------+
-| Property | Ty\| Descript\|
-| Name | pe | ion |
-+=============+====+==========+
-| location | st\| DCAE |
-| | ri\| location |
-| | ng | for the |
-| | | publishe\|
-| | | r, |
-| | | used to |
-| | | set up |
-| | | routing |
-+-------------+----+----------+
-| publish_url | st\| *Require\|
-| | ri\| d \ |
-| | ng | as \ |
-| | | input*. |
-| | | URL to |
-| | | which |
-| | | the |
-| | | publishe\|
-| | | r |
-| | | makes |
-| | | Data |
-| | | Router |
-| | | publish |
-| | | requests |
-+-------------+----+----------+
-| log_url | st\| URL from |
-| | ri\| which |
-| | ng | log data |
-| | | for the |
-| | | feed can |
-| | | be |
-| | | obtained |
-+-------------+----+----------+
-| username | st\| Username |
-| | ri\| the |
-| | ng | publishe\|
-| | | r |
-| | | uses to |
-| | | authenti\|
-| | | cate |
-| | | to Data |
-| | | Router |
-+-------------+----+----------+
-| password | st\| Password |
-| | ri\| the |
-| | ng | publishe\|
-| | | r |
-| | | uses to |
-| | | authenti\|
-| | | cate |
-| | | to Data |
-| | | Router |
-+-------------+----+----------+
-| publisher_i | st\| Publishe\|
-| d | ri\| r |
-| | ng | id in |
-| | | Data |
-| | | Router |
-+-------------+----+----------+
-
-Here’s an example of the minimal JSON that must be provided as an input:
-
-.. code:: json
-
- {
- "type": "data_router",
- "dmaap_info": {
- "publish_url": "https://we-are-data-router.us/feed/xyz"
- }
- }
-
-Subscriber
-~~~~~~~~~~
-
-Here’s an example of what the JSON object connection for data router
-subscriber looks like:
-
-.. code:: json
-
- {
- "type": "data_router",
- "dmaap_info": {
- "location": "mtc00",
- "delivery_url": "https://my-subscriber-app.dcae:8080/target-path",
- "username": "some-user",
- "password": "some-password",
- "subscriber_id": "789012"
- }
- }
-
-At the top-level:
-
-+-------------+----+----------------+
-| Property | Ty\| Descript\ |
-| Name | pe | ion |
-+=============+====+================+
-| type | st\| *Require\ |
-| | ri\| d |
-| | ng | as \ |
-| | | input*. |
-| | | Must be |
-| | | ``data_router``|
-| | | for data |
-| | | router |
-| | | feeds |
-+-------------+----+----------------+
-| dmaap_info | JS\| *Require\ |
-| | ON | d \ |
-| | ob\| as \ |
-| | je\| input*. |
-| | ct | Contains |
-| | | the |
-| | | topic |
-| | | connecti\ |
-| | | on |
-| | | details |
-+-------------+----+----------------+
-
-The ``dmaap_info`` object contains:
-
-+--------------+----+----------+
-| Property | Ty\| Descript\|
-| Name | pe | ion |
-+==============+====+==========+
-| location | st\| DCAE |
-| | ri\| location |
-| | ng | for the |
-| | | publishe\|
-| | | r, |
-| | | used to |
-| | | set up |
-| | | routing |
-+--------------+----+----------+
-| delivery_ur\ | st\| URL to |
-| l | ri\| which |
-| | ng | the Data |
-| | | Router |
-| | | should |
-| | | deliver |
-| | | files |
-+--------------+----+----------+
-| username | st\| Username |
-| | ri\| Data |
-| | ng | Router |
-| | | uses to |
-| | | authenti\|
-| | | cate |
-| | | to the |
-| | | subscrib\|
-| | | er |
-| | | when |
-| | | deliveri\|
-| | | ng |
-| | | files |
-+--------------+----+----------+
-| password | st\| Password |
-| | ri\| Data |
-| | ng | Router |
-| | | uses to |
-| | | authenti\|
-| | | cate |
-| | | to the |
-| | | subscrib\|
-| | | er |
-| | | when |
-| | | deliveri\|
-| | | ng |
-| | | files |
-+--------------+----+----------+
-| subscriber_i\| st | Subscrib\|
-| d | ri | er |
-| | ng | id in |
-| | | Data |
-| | | Router |
-+--------------+----+----------+
-
-Here’s an example of the minimal JSON that must be provided as an input:
-
-.. code:: json
-
- {
- "type": "data_router",
- "dmaap_info": {
- }
- }
-
-Developers are recommended to use ``username`` and ``password`` since
-this is the recommended security practice.
-
-Note that the dcae-cli will construct the ``delivery_url`` when
-deploying the component since this can only be known at deployment time.
diff --git a/docs/sections/components/dcae-cli/quickstart.rst b/docs/sections/components/dcae-cli/quickstart.rst index 60370330..47b9ac1b 100755 --- a/docs/sections/components/dcae-cli/quickstart.rst +++ b/docs/sections/components/dcae-cli/quickstart.rst @@ -3,86 +3,125 @@ .. _quickstart:
-Quickstart
-==========
+Overview
+========
-The ``dcae-cli`` is a Python command-line tool built to aide component
-developers with the development and testing of their micro-service
-component for the DCAE platform. It will help developers do functional
-and integration testing of their components locally and on remote
-environments as simple as possible.
+The ``dcae-cli`` is a Python command-line tool for component developers.
+With it, the developer can :
-The tool requires the component developers to create a valid component
-specification for their component which is used by the tool. This same
-component specification will be published in the :any:`onboarding catalog <glossary-onboarding-catalog>` at the end of development
-and testing.
+- validate the data formats and component specifications
+- publish the validated data formats and component specifications into
+ the ``onboarding catalog``
+- access the ``onboarding catalog`` to search for existing data formats
+ (for possible reuse) and component specs
+- deploy a component onto a local or remote DCAE platform for
+ functional and pair-wise testing (This is done without Cloudify)
-.. The git repository can be found `here <ONAP%20LINK%20TBD>`__
+The git repository for the dcae_cli tool can be found
+`here <https://gerrit.onap.org/r/gitweb?p=dcaegen2/platform/cli.git>`__
-Pre-requisite
--------------
+Pre-requisites
+--------------
For Docker
~~~~~~~~~~
-There are two options for development with Docker: developing locally on
-your machine which requires Docker to be installed and developing
-remotely by deploying onto remote infrastructure.
+There are two options for development with Docker:
For local development
^^^^^^^^^^^^^^^^^^^^^
-- You must install `Docker
- engine <https://docs.docker.com/engine/installation/>`__ locally on
+- Install `Docker engine <https://docs.docker.com/engine/installation/>`__ locally on
your machine.
-- You must know the *external ip* of where the Docker engine is
- running. The external ip is needed so that service discovery will
- wire up correctly.
+- Know the *external ip* of where the Docker engine is running. The
+ external ip is needed so that service discovery will connect to it.
- - For OSX users, this means making sure the VirtualBox VM that is
+ - *(For OSX users, this means making sure the VirtualBox VM that is
running your Docker engine has a bridged adapter and getting the
- ip of that adapter.
+ ip of that adapter).*
For remote development
^^^^^^^^^^^^^^^^^^^^^^
-You need access to a remote host with Docker engine installed with
-remote API access. You must have the associated connection information:
-domain name or IP and port (should be either 2375 or 2376). This
-information should be set in :any:`an active profile <setting-profile>`.
+- Have access to a remote host with Docker engine installed and with
+ remote API access.
+- Have the associated connection information:
+
+ - domain name or IP and port (port should be either 2375 or 2376).
+ Use this information to establish an active
+ :any:`profile <dcae_cli_activate_profile>`.
For CDAP
~~~~~~~~
-TBD
+None at this time.
+
+Python, Pip, Virtualenv
+~~~~~~~~~~~~~~~~~~~~~~~
+
+Install python, pip (9.0.1 or higher), and virtualenv if they are not
+installed. Do these when not in a VPN to avoid possible network issues.
+
+::
+
+ sudo apt-get -f install python
+ sudo apt-get -f install python-pip
+ sudo pip install virtualenv
+
+Set up a virtual environment and activate
+
+::
+
+ virtualenv cli_tool
+ source cli_tool/biin/activate
+
+Install dcae_cli
+----------------
+
+::
+
+ pip install onap-dcae-cli
-Install
--------
+Check dcae_cli version
+----------------------
+
+You can verify the version of the dcae-cli with the following command.
+To get the latest version of the dcae_cli tool,
::
- pip install --extra-index-url https://YOUR_NEXUS_PYPI_SERVER/simple dcae-cli
+ $ dcae_cli --version
-To do an upgrade, use the ``--upgrade`` flag.
+Upgrade dcae_cli
+----------------
+
+Periodically, upgrade the dcae_cli to get the latest version
+
+::
+
+ pip install --upgrade onap-dcae-cli
Configuration
-~~~~~~~~~~~~~
+-------------
-When you run the tool for the first time, the tool will create a
-`configuration
+When running the tool for the first time, a `configuration
directory <http://click.pocoo.org/5/api/#click.get_app_dir>`__ and
-generate a configuration file.
+configuration file will be created.
-Configuration is first sourced from an remote server that the platform
-team manages. This is overlaid with configuration details that you will
-be prompted to input particularly your user id.
+The configuration is first sourced from a remote server that is managed
+by the platform team. You will be prompted to enter your ATTUID to
+complete this process.
-``--reinit``
-^^^^^^^^^^^^
+Re-initializing Configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Configuration can be re-initialized or reset. There is a ``--reinit``
flag that is to be used to re-initialize your configuration and your
-environment profiles.
+environment profiles. You may be instructed to re-initialize after
+certain updates are made to the dcae_cli tool. When you re-initialize
+the configuration, your configuration will be added to or updated from
+the platform configuration and profiles. No profiles will be deleted via
+the reinit process.
To re-initialize:
@@ -90,10 +129,11 @@ To re-initialize: $ dcae_cli --reinit
-Verify
-~~~~~~
+Verify Installation
+-------------------
-Verify that its installed:
+To Verify that the dcae_cli tool is installed, run the following command
+and look for the output below.
::
@@ -112,11 +152,5 @@ Verify that its installed: data_format
profiles
-Version
--------
-
-You can verify the version of the dcae-cli with the following command:
+Refer to :doc:`dcae_cli Commands <./commands>`.
-::
-
- $ dcae_cli --version
diff --git a/docs/sections/components/dcae-cli/walkthrough.rst b/docs/sections/components/dcae-cli/walkthrough.rst index d33c35fb..559ba3ab 100755 --- a/docs/sections/components/dcae-cli/walkthrough.rst +++ b/docs/sections/components/dcae-cli/walkthrough.rst @@ -6,560 +6,389 @@ Walk-through
============
-The goal of this quickstart is to provide an overview of the
-functionalities of the ``dcae-cli`` and walk you through the
-capabilities:
+This section demonstrates the flow and usage of the dcae_cli tool to
+onboard a typical component to the DCAE platform. The commands are
+explained in more detail in :doc:`dcae_cli Commands <commands>`.
+
+- `Add (and validate) a data format <#add-a-data-format>`__
+- `Add (and validate) the component <#add-the-component>`__
+- `View the platform generated
+ configuration <#view-the-platform-generated-configuration>`__
+- `If needed, Create the dmaap file for Dmaap Testing <#create-the-input-file-for-dmaap-testing>`__
+- `If needed, Create the input file for *Sourced at Deployment* Testing <#create-the-input-file-for-sourced-at-deployment-testing>`__
+- `Run the component <#run-the-component>`__
+- :any:`Undeploy the component <dcae_cli_undeploy_the_component>`
+- :any:`Publish the component and data_format <dcae_cli_publish_the_component_and_data_format>` to let others
+ know its ready for reuse
+- `List the Catalog Contents <#list-the-catalog-contents>`__ to see
+ your published resources
-- `Adding data formats <#adding-data-formats>`__
-- `Adding component <#adding-component>`__
-- `Setting profile <#setting-profile>`__
-- `Development and testing <#development-and-testing>`__
-- `Publishing component <#publishing-component>`__
-- `Shared catalog <#shared-catalog>`__
-.. This walk-through uses example projects: COMMENTED OUT FOR NOW TBD
-..
-.. - `laika <ONAP%20URL%20TBD>`__
-.. - `CDAP examples <ONAP%20URL%20TBD>`__
-
-.. _adding-data-formats:
-
-Adding data formats
--------------------
+--------------
-``data_format`` is the sub-command that is used to execute operations
-that manage `data formats <../data-formats.md>`__.
+Add a Data Format
+-----------------
::
- $ dcae_cli data_format --help
- Usage: dcae_cli data_format [OPTIONS] COMMAND [ARGS]...
-
- Options:
- --help Show this message and exit.
-
- Commands:
- add Tracks a data format file SPECIFICATION...
- generate Generates a data format file from JSON input examples
- list Lists all your data formats
- publish Publishes data format to make publicly...
- show Provides more information about FORMAT
+ $ dcae_cli data_format add $HOME/yourapp/data-formats/health.json
-Your data format must be in the catalog in order to use in the component
-specification. Check the catalog using the ``data_format list``
-sub-command:
+Verify that the data_format was added
::
- $ dcae_cli data_format list
+ $ dcae_cli data_format list | grep yourapp
+ | sandbox.platform.yourapp.health | 0.1.0 | Data format used for the /health endpoint | staged | 2017-11-07 21:48:47.736518 |
- Data formats for mh677g
- +------+---------+-------------+--------+----------+
- | Name | Version | Description | Status | Modified |
- +------+---------+-------------+--------+----------+
- | | | | | |
- +------+---------+-------------+--------+----------+
+(Note: Each of the data formats for your component need to be added,
+unless already existing in the onboarding catalog )
-The fields ``name``, ``version``, ``description`` are referenced from
-the data format JSON from the ``self`` JSON.
-
-There are no data formats so you must add the data formats that your
-component specification references. Use the ``data_format add``
-sub-command:
+--------------
-Here’s an example command:
+Add the Component
+-----------------
::
- dcae_cli data_format add health.json
+ $ dcae_cli component add $HOME/yourapp/component-spec.json
-Verify that it was added:
+Verify that the component was added
::
- $ dcae_cli data_format list
-
- Data formats for mh677g
- +-------------------------------+---------+-------------------------------------------+--------+----------------------------+
- | Name | Version | Description | Status | Modified |
- +-------------------------------+---------+-------------------------------------------+--------+----------------------------+
- | sandbox.platform.laika.health | 0.1.0 | Data format used for the /health endpoint | staged | 2017-05-23 04:02:38.952799 |
- +-------------------------------+---------+-------------------------------------------+--------+----------------------------+
+ $ dcae_cli component list
+ Active profile: solutioning
-Go ahead and add other referenced data formats.
+ +-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+ | Name | Version | Type | Description | Status | Modified | #Deployed |
+ +-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+ | sandbox.platform.yourapp | 0.7.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-11-08 20:27:34.168854 | 0 |
+ +-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
-If you have JSON input you can generate a data format like this:
+--------------
-::
+.. _dcae-cli-view-the-platform:
- $ dcae_cli data_format --keywords generate myname:1.0.0 myjsoninputfile
+View the platform generated configuration
+-----------------------------------------
-where ``myname`` is the name of your data format, ``1.0.0`` is an
-example version, and ``myjsoninputfile`` is an example JSON input file
-(a directory of input JSON files can also be provided). The
-``--keywords`` option adds additional data attributes that can be
-completed to provide a more detailed data characterization. In any event
-the output should be reviewed for accuracy. The data format is written
-to stdout.
+The ``component dev`` command is useful during onboarding. Running this
+command is part of a multi-step process that sets up a temporary test
+environment, generates your application configuration, makes it
+available in that environment, and allows you to view that configuration
+to help with debugging.
-.. _adding-component:
+Here is a step-by-step example based on a component specification called
+``component-spec.json``.
-Adding component
-----------------
+Step 1 - Run the component dev command
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-``component`` is the sub-command that is used to work with operations
-for components:
+(This creates a file called env_$ENV (in the current directory)- where
+$ENV is the name of the active profile. Note: SERVICE_NAME and HOSTNAME
+always resolve to the same value).
::
- $ dcae_cli component --help
- Usage: dcae_cli component [OPTIONS] COMMAND [ARGS]...
-
- Options:
- --help Show this message and exit.
-
- Commands:
- add
- dev Set up component in development for...
- list Lists components in the public catalog.
- publish Pushes COMPONENT to the public catalog
- run Runs the latest version of COMPONENT.
- show Provides more information about COMPONENT
- undeploy Undeploys the latest version of COMPONENT.
-
-Your component must be accessible from the catalog in order for it to be
-used. Check the catalog using the ``component list`` sub-command:
-
-::
+ $ dcae_cli component dev component-spec.json
+ Ready for component development
- $ dcae_cli component list
- Active profile: solutioning
+ Setup these environment variables. Run "source env_solutioning":
- +------+---------+------+-------------+--------+----------+-----------+
- | Name | Version | Type | Description | Status | Modified | #Deployed |
- +------+---------+------+-------------+--------+----------+-----------+
- | | | | | | | |
- +------+---------+------+-------------+--------+----------+-----------+
+ export DOCKER_HOST=yourdockerhost.com:2376
+ export SERVICE_CHECK_INTERVAL=15s
+ export CONFIG_BINDING_SERVICE=config_binding_service
+ export HOSTNAME=user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp
+ export CONSUL_HOST=yourconsulhost.com
+ export CDAP_BROKER=cdap_broker
+ export SERVICE_NAME=user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp
+ export SERVICE_CHECK_TIMEOUT=1s
+ export SERVICE_CHECK_HTTP=/health
- Use the "--deployed" option to see more details on deployments
+ Press any key to stop and to clean up
-The fields ``name``, ``version``, ``type``, ``description`` are
-referenced from the component specification’s ``self`` JSON.
+Step 2 - Setup the environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-There are no components so you must add your component. Use the
-``component add`` sub-command. The command is the same for docker and
-cdap components:
+In another window, setup the temporary testing environment, by executing
+the environment file created above.
::
- $ dcae_cli component add --help
- Usage: dcae_cli component add [OPTIONS] SPECIFICATION
-
- Options:
- --update Updates a locally added component if it has not been already
- pushed
- --help Show this message and exit.
+ $ source env_solutioning
-*Note* use the ``--update`` flag to replace existing staged instances.
+(The application configuration is now available under the SERVICE_NAME
+shown above -
+``user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp``).
-The ``component dev`` sub-command can be useful in validating and
-experimenting when crafting your component specification. See details
-about ``dev`` under `Development and
-testing <#development-and-testing>`__.
+Step 3 - Query CONSUL
+~~~~~~~~~~~~~~~~~~~~~
-Once we add the components laika and helloworld, let’s verify that they
-got added ok:
+Query CONSUL to get the IP/PORT of CONFIG BINDING SERVICE
::
- $ dcae_cli component list
- Active profile: solutioning
+ $ curl http://$CONSUL_HOST:8500/v1/catalog/service/$CONFIG_BINDING_SERVICE
+ [
+ {
+ "ID": "bfbc220d-4603-7f90-ec2e-611d3c330f20",
+ "Node":"docker00",
+ "Address": "10.226.1.15",
+ "Datacenter":"solutioning-central",
+ "TaggedAddresses": {
+ "lan":"10.226.1.15",
+ "wan":"10.226.1.15"
+ },
+ "NodeMeta": {},
+ "ServiceID": "472b116f9035:config_binding_service:10000",
+ "ServiceName": "config_binding_service",
+ "ServiceTags": [],
+ "ServiceAddress":"135.205.226.126",
+ "ServicePort":10000,
+ "ServiceEnableTagOverride": false,
+ "CreateIndex":1078990,
+ "ModifyIndex":1078990
+ }
+ ]
- +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
- | Name | Version | Type | Description | Status | Modified | #Deployed |
- +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
- | cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 |
- | sandbox.platform.laika | 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 |
- +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+Fetch the generated configuration from CONFIG BINDING SERVICE using the
+‘serviceaddress’ and ‘serviceport’ from above along with $SERVICE_NAME
+from earlier.
- Use the "--deployed" option to see more details on deployments
+::
-.. _setting-profile:
+ $ curl http://135.205.226.126:10000/service_component/user12.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-yourapp
-Setting profile
----------------
+ {"streams_subscribes": {}, "services_calls": {}, "multiplier": 3, "streams_publishes": {}}
-``profile`` is the sub-command that is used to manage profiles. These
-profiles contain environment variables used to connect to different
-environments. This is used in the running and deployment of your
-component using the ``dcae_cli component run`` command. The ``dcae-cli``
-ships with profiles for ``solutioning`` and ``rework``.
+--------------
-::
+.. _dcae-cli-walkthrough-dmaap-testing:
- $ dcae_cli profiles --help
- Usage: dcae_cli profiles [OPTIONS] COMMAND [ARGS]...
+Create the input file for Dmaap Testing
+---------------------------------------
- Options:
- --help Show this message and exit.
+Currently, the dcae-cli tool does not have the capability to provision
+topics or feeds. Therefore, in order to test with ``message router`` or
+``data router`` feeds, the developer must manually provision the topic
+or feed and then provide the connection details in the form of a DMaap
+JSON file for testing. This file is then passed in on the
+``component run`` or ``component dev`` commands by using the argument
+``--dmaap-file``.
- Commands:
- activate Sets profile NAME as the active profile
- create Creates a new profile NAME initialized with...
- delete Deletes profile NAME
- list Lists available profiles
- set Updates profile NAME such that KEY=VALUE
- show Prints the profile dictionary
+The structure of the DMaaP JSON is an object of config keys with the
+topic or feed connection details. The config keys are the ``config_key``
+values specified in the component specification streams section where
+the streams must be type ``message router`` or ``data router``. This
+file corresponds to the ``Dmaap Connection Object`` which is generated
+by the platform and provided to the component at runtime. The exception
+is that ``delivery_url`` cannot be provided in the dmaap-file because it
+is not created until the component is deployed. Refer to :any:`Dmaap Connection Object <dmaap-connection-objects>`, for details on creating the dmaap-file for testing.
-To see what variables a profile contains, you can use the ``show``
-command, as in ``dcae_cli profiles show PROFILE_NAME``
+--------------
-Use the ``create`` sub-command to create your own profile and assign new
-values using the ``set`` command. Afterwards you will need to
-``activate`` the profile you wish to use. First take a look at which
-profile is active:
+Create the input file for *Sourced at Deployment* Testing
+---------------------------------------------------------
-::
+Components may have configuration parameters whose values are to be
+sourced at deployment time. This is established in the
+:any:`component specification <common-specification-parameters>`
+by setting the property ``sourced_at_deployment`` to ``true`` for each
+applicable parameter.
- $ dcae_cli profiles list
- rework
- * solutioning
+Then, use the ``--inputs-file`` command-line argument when running the
+component ``dev`` or ``run`` command for your component. This is to
+simulate providing the dynamic, deployment time values for those
+parameters marked as ``sourced_at_deployment``.
-The active profile is ``solutioning`` so to activate *rework* to use
-``rework``:
+For example, if your component specification has the following
+configuration parameters:
::
- $ dcae_cli profiles activate rework
+ "parameters": [{
+ "name": "vnf-ip",
+ "value": "",
+ "sourced_at_deployment": true
+ },
+ {
+ "name": "static-param",
+ "value": 5
+ }]
-Check
+Pass in an input file that looks like:
::
- $ dcae_cli profiles list
- * rework
- solutioning
+ {
+ "vnf-ip": "10.100.1.100"
+ }
-.. _development-and-testing:
+The application configuration would look like:
-Development and testing
------------------------
+::
-The following operations under the sub-command ``component`` are aimed
-to help developers with testing:
+ {
+ "vnf-ip": "10.100.1.100",
+ "static-param": 5
+ }
-- ``run``
-- ``undeploy``
-- ``dev``
+--------------
-``run``
-~~~~~~~
+Run the component
+-----------------
The ``run`` operation is to be used for running your application in its
container remotely on the activated environment. Docker containers have
-the additional option to run locally on your development machine.
+the additional option to run locally on your development machine. If the
+component uses Dmaap, you can specify the Dmaap Connection Object as
+well. Refer to :any:`Dmaap Connection Object <dmaap-connection-objects>`.
-In order to run your application, you must have added your data formats
-and your component to your catalog.
+In order to run the component, the data formats and component must have
+been added to the onboarding catalog.
-Let’s verify that your component is in the catalog:
+To verify what’s in the catalog:
::
- $ dcae_cli component list
+ $ dcae_cli catalog list --expanded
Active profile: solutioning
+ +---------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+ | Name | Version | Type | Description | Status | Modified | #Deployed |
+ +---------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
+ | sandbox.platform.yourapp | 0.7.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-11-08 20:27:34.168854 | 0 |
+ +---------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
- +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
- | Name | Version | Type | Description | Status | Modified | #Deployed |
- +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
- | cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 |
- | sandbox.platform.laika | 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 |
- +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+
-
- Use the "--deployed" option to see more details on deployments
-
-Docker
-^^^^^^
+For Docker
-**NOTE** Make sure your Docker image has been uploaded to the shared
+**NOTE** Make sure the Docker image has been uploaded to the shared
registry.
-For Docker containers, you can run either attached or unattached.
-Attached means that the dcae-cli tool will launch the container and not
-terminate. The dcae-cli while attached will stream in the logs of the
-Docker container. Doing a Ctrl-C will terminate the run session which
-means undeploy your container and force a clean up automatically.
-
-Running unattached means simply deploy your container. You will need to
-execute ``undeploy`` when you are done testing. #### CDAP
-
-**NOTE** Make sure your CDAP jar has been uploaded to Nexus.
-
-TODO
-
-``undeploy``
-~~~~~~~~~~~~
-
-The ``undeploy`` operation is to be used to undeploy any instances of a
-specified component, version that you have deployed. This includes
-cleaning up of configuration.
-
-Let’s undeploy ``sandbox.platform.laika`` that was deployed from the
-previous section:
+A docker component can be run in either ``attached`` or ``unattached``
+mode. (Default is unattached).
+
++------------------+---------------------------------------------------+
+| Mode | Description |
++==================+===================================================+
+| attached | component is run in the ‘foreground’, container |
+| | logs are streamed to stdout. Ctrl-C is used to |
+| | terminate the dcae_cli session. |
++------------------+---------------------------------------------------+
+| unattached | component is run in the ‘background’, container |
+| | logs are viewed via ``docker logs`` command, |
+| | container runs until undeployed with dcae_cli |
+| | ``undeploy`` command. |
++------------------+---------------------------------------------------+
+
+Run a component in attached mode:
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
::
- $ dcae_cli component undeploy sandbox.platform.laika:0.5.0
- DCAE.Undeploy | WARNING | Undeploying components: 1
- DCAE.Undeploy | WARNING | Undeployed components: 1
-
-.. _walkthrough-dev:
-
-``dev``
-~~~~~~~
-
-The ``dev`` operation is a convenient operation that can be useful for
-the development and testing of your component. It can be used to:
+ $ dcae_cli -v component run --attached sandbox.platform.yourapp:0.7.0
+ DCAE.Docker | INFO | Running image 'nexus01.server.com:18443/repository/solutioning01-mte2-docker/dcae-platform/yourapp:0.7.0' as 'user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp'
+ DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | Consul host: yourconsulhost.com
-- Help validate your experimental component specification before
- uploading to the catalog
-- Generate the application configuration from the component
- specification and make it available in a test environment. This
- allows you to view your resulting configuration for local development
- and to help debug potential related issues.
+ DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | service name: user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp
-Let’s say you have a component specification called
-``component-spec.json``:
-
-::
-
- $ dcae_cli component dev component-spec.json
- Ready for component development
+ DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | get_config returned the following configuration: {"streams_subscribes": {}, "multiplier": 3, "services_calls": {}, "streams_publishes": {}}
- Setup these environment varibles. Run "source env_solutioning":
+ DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
- export DOCKER_HOST=SOME_DOCKER_HOST:2376
- export SERVICE_CHECK_INTERVAL=15s
- export CONFIG_BINDING_SERVICE=config_binding_service
- export HOSTNAME=mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika
- export CONSUL_HOST=SOME_CONSUL_HOST
- export CDAP_BROKER=cdap_broker
- export SERVICE_NAME=mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika
- export SERVICE_CHECK_TIMEOUT=1s
- export SERVICE_CHECK_HTTP=/health
+ DCAE.Docker.user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp | INFO | 135.205.226.156 - - [08/Nov/2017 23:27:30] "GET /health HTTP/1.1" 200 -
- Press any key to stop and to clean up
-Your application configuration is now available under the name
-``mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika``.
+ Hit Ctrl-C to terminate session.
-To view the resulting configuration, you can ``curl`` a request to the
-config binding service or programmatically fetch your configuration
-within your application.
+ ^C
+ DCAE.Docker | INFO | Stopping container 'user12.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-yourapp' and cleaning up...
-You need to first query Consul to get the ip and port of config binding
-service:
+Run a component in unattached mode:
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
::
- curl http://$CONSUL_HOST:8500/v1/catalog/service/$CONFIG_BINDING_SERVICE
- [
- {
- "ID": "983d5c94-c508-4a8a-9be3-5912bd09786b",
- "Node": "realsolcnsl00",
- "Address": "10.226.1.22",
- "TaggedAddresses": {
- "lan": "10.226.1.22",
- "wan": "10.226.1.22"
- },
- "NodeMeta": {},
- "ServiceID": "5f371f295c90:config_binding_service:10000",
- "ServiceName": "config_binding_service",
- "ServiceTags": [],
- "ServiceAddress": "XXXX",
- "ServicePort": 32770,
- "ServiceEnableTagOverride": false,
- "CreateIndex": 487,
- "ModifyIndex": 487
- }
- ]
-
-.. _dmaap-testing:
-
-DMaaP testing
-~~~~~~~~~~~~~
-
-Currently, the dcae-cli does not have the capability of provisioning
-topics. In order to do testing with message router topics or with data
-router feeds, the developer must provision the topic or the feed
-manually and provide the connection details in the form of a JSON in a
-file to the dcae-cli. This file is to be passed in when using the
-``run`` and ``dev`` commands with the option ``--dmaap-file``.
-
-The structure of the DMaaP JSON is an object of config keys to matching
-topic or feed connection details. Config keys are the ``config_key``
-values specified in your component specification streams section where
-the streams must be type message router or data router. Information
-about the associated connection details can be found on `this
-page <dmaap-connection-objects.md>`__. Please check it out.
-
-For example, if you have a component specification that has the
-following streams entry:
-
-.. code:: json
-
- "streams": {
- "publishes": [{
- "format": "ves",
- "version": "1.0.0",
- "type": "message router",
- "config_key": "ves_connection"
- }]
- }
+ $ dcae_cli -v component run sandbox.platform.yourapp:0.7.0
+ DCAE.Docker | INFO | Running image 'nexus01.server.com:18443/repository/solutioning01-mte2-docker/dcae-platform/yourapp:0.7.0' as 'user12.22629ebd-417e-4e61-a9a0-f0cb16d4cef2.0-7-0.sandbox-platform-yourapp'
+ DCAE.Run | INFO | Deployed user12.22629ebd-417e-4e61-a9a0-f0cb16d4cef2.0-7-0.sandbox-platform-yourapp. Verifying..
+ DCAE.Run | INFO | Container is up and healthy
-Then to deploy and to run your component, you must use the
-``--dmaap-file`` command and pass in a JSON that looks like:
+**NOTE** You must undeploy this component when finished testing. This is
+important to conserve resources in the environment.
-.. code:: json
+Run a component that subscribes to Dmaap Message Router or Data Router
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
- {
- "ves_connection": {
- "type": "message_router",
- "dmaap_info": {
- "topic_url": "https://we-are-message-router.us:3905/events/some-topic"
- }
- }
- }
-
-The provided DMaaP JSON is used to simulate the output of provisioning
-and will be used to merge with the generated application configuration
-at runtime.
-
-Your final application config will look like:
-
-.. code:: json
-
- {
- "streams_publishes": {
- "ves_connection": {
- "type": "message_router",
- "dmaap_info": {
- "topic_url": "https://we-are-message-router.us:3905/events/some-topic"
- }
- }
- }
- }
+::
-Data router subscribers
-^^^^^^^^^^^^^^^^^^^^^^^
+ $ dcae_cli -v component run $component-that-uses-dmamp --dmaap-file $dmaap-connection-object
-Note for data router subscriber testing, you will need the delivery url
-in order to provision the subscriber to the feed. This is constructed at
-deployment time and will be provided by the dcae-cli after you deploy
-your component. The delivery urls will be displayed to the screen:
+Run a component that expects input that is ``sourced at deployment``
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
::
- DCAE.Run | WARNING | Your component is a data router subscriber. Here are the delivery urls:
+ $ dcae_cli -v component run $component-that-expects-dti --inputs-file $input-file-to-simulate-dti
- some-sub-dr: http://SOME_IP:32838/identity
+--------------
-*Sourced at deployment* testing
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Components may have configuration parameters whose values are to be
-sourced at deployment time. For example, there are components whose
-configuration parameters are to come from DTI events which are only
-available when the component is getting deployed. These configuration
-parameters must be setup correctly in the `component
-specification <http://localhost:8000/components/component-specification/docker-specification/#configuration-parameters>`__
-by setting the property ``sourced_at_deployment`` to ``true`` for each
-and every parameter that is expected to come in at deployment time.
+.. _dcae_cli_undeploy_the_component:
-Once your component specification has been updated correctly, you must
-use the ``--inputs-file`` command-line argument when running the
-commands ``dev`` or ``run`` with your component. This is to simulate
-providing the dynamic, deployment time values for those parameters
-marked as ``sourced_at_deployment``.
+Undeploy the component
+----------------------
-For example, if your component specification has the following
-configuration parameters:
-
-::
++-----------------------------------------------------------------+
+| The ``undeploy`` command is used to undeploy any instance of a |
+| specified component/version that you have deployed. This |
+| includes cleaning up the configuration. |
++-----------------------------------------------------------------+
+| Undeploy ``sandbox.platform.yourapp:0.7.0`` that was deployed |
+| above: |
++-----------------------------------------------------------------+
+| ``$ dcae_cli -v component undeploy sandbox.platform.yourapp:0.7.0 |
+| DCAE.Undeploy | WARNING | Undeploying components: 1 DCAE.Undep |
+| loy | WARNING | Undeployed components: 1`` |
++-----------------------------------------------------------------+
- "parameters": [{
- "name": "vnf-ip",
- "value": "",
- "sourced_at_deployment": true
- },
- {
- "name": "static-param",
- "value": 5
- }]
+.. _dcae_cli_publish_the_component_and_data_format:
-You would have to pass in an inputs file that looks like:
+Publish the component and data_format
+-------------------------------------
-::
+Once a component has been tested, it (and the data_format(s)) should be
+published in the onboarding catalog using the ``publish`` sub-command
+for both the ``data_format`` and ``component`` command.
- {
- "vnf-ip": "10.100.1.100"
- }
+**Note** Before a component can be published, all data_formats that it
+references must be published.
-Your application configuration would look like:
+Publishing will change the status of a component or data_format,
+indicating that it has been tested, make accessible for other developers
+to use.
::
- {
- "vnf-ip": "10.100.1.100",
- "static-param": 5
- }
-
-Publishing component
---------------------
+ $ dcae_cli data_format publish sandbox.platform.yourapp:0.7.0
+ Data format has been published
-Once components have their component specifications crafted and
-validated and have been tested, components should be published in the
-shared onboarding catalog using the ``publish`` sub-command for both
-data formats and components. You must publish all data formats of a
-component before publishing a component.
+ $dcae_cli component publish sandbox.platform.yourapp:0.7.0
+ Component has been published
-Publishing will change the status of a component, be made accessible for
-other developers to use, and will generate the associated TOSCA models
-for use in designing of compositions.
-
-::
-
- dcae_cli component publish sandbox.platform.laika:0.5.0
-
-Shared catalog
--------------
-``catalog`` is the sub-command used to access and to browse the shared
-onboarding catalog to view components and data formats that have been
-published and that are being worked on. Components and data formats have
-two statuses ``staged`` and ``published``.
-
-Staged means that the resource has been simply added and is under
-development. It is to be used only by the owner. Published means that
-the resource has been fully developed and tested and is ready to be
-shared.
-
-Published components can be deployed by non-owners and published data
-formats can be used in component specifications of non-owners.
-
-There are two available operations:
+List the catalog contents
+-------------------------
::
- $ dcae_cli catalog --help
- Usage: dcae_cli catalog [OPTIONS] COMMAND [ARGS]...
-
- Options:
- --help Show this message and exit.
+ $dcae_cli catalog list
- Commands:
- list
- show
+ $ dcae_cli data_format list | grep sandbox
+ | sandbox.platform.yourapp | 0.7.0 | docker | Web service used as a stand-alone test DCAE service compone.. | user12 | published | 2017-11-13 |
+ | sandbox.platform.yourapp.health | 0.1.0 | Data format used for the /health endpoint | published | 2017-11-13 17:48:10.121588 |
+ | sandbox.platform.any | 0.1.0 | Data format used when no data format is required. | published | 2017-11-13 17:47:51.622607 |
+ | sandbox.platform.yourapp.identity.response | 0.1.0 | Data format used for the /identity endpoint response which should | published | 2017-11-13 17:47:43.234715 |
+ | sandbox.platform.yourapp.identity.request | 0.1.0 | Data format used for the /identity endpoint request. This is | published | 2017-11-13 17:47:36.693643 |
+ | sandbox.platform.yourapp.rollcall.response | 0.1.0 | Data format used for the /rollcall endpoint respon.. | published | 2017-11-13 17:46:30.026846 |
-Staged components can be viewed under the ``list`` operation using the
-``--expanded`` flag.
|