diff options
Diffstat (limited to 'platformdoc/docs/components/dcae-cli')
| -rw-r--r-- | platformdoc/docs/components/dcae-cli/commands.md | 453 | ||||
| -rw-r--r-- | platformdoc/docs/components/dcae-cli/dmaap-connection-objects.md | 156 | ||||
| -rw-r--r-- | platformdoc/docs/components/dcae-cli/quickstart.md | 85 | ||||
| -rw-r--r-- | platformdoc/docs/components/dcae-cli/walkthrough.md | 494 |
4 files changed, 709 insertions, 479 deletions
diff --git a/platformdoc/docs/components/dcae-cli/commands.md b/platformdoc/docs/components/dcae-cli/commands.md new file mode 100644 index 00000000..d198f947 --- /dev/null +++ b/platformdoc/docs/components/dcae-cli/commands.md @@ -0,0 +1,453 @@ +# dcae_cli Commands + +The dcae_cli tool has four command groups. Each has several sub-commands. + +## `catalog` + +The `catalog` command lists and shows resources (components and data_formats) in the 'onboarding' catalog (regardless of the owner). A resource can have a status of `staged` or `published`. By default, only `published` resources are displayed. To see `staged` resources, add the --expanded argument. + +Catalog Status | Meaning +-------------- | ------- +staged | resource has be added (and validated), but is under development +| staged data_formats can only be referenced in their owners component specs +| staged components can only be deployed by their owners +published | resource has been tested and can be shared +| published data_formats can be used in anyone's component spec +| published components and be deployed by anyone + +``` +$ dcae_cli catalog --help +Usage: dcae_cli catalog [OPTIONS] COMMAND [ARGS]... + +Options: + --help Show this message and exit. + +Commands: + list Lists resources in the onboarding catalog + show Provides more information about resource +``` + +### List onboarding catalog contents + +``` +$ dcae_cli catalog list +Components: ++--------------------------------+---------+--------+---------------------------------------------------------------------+--------+-----------+------------+ +| Name | Version | Type | Description | Owner | Status | Published | ++--------------------------------+---------+--------+---------------------------------------------------------------------+--------+-----------+------------+ +| AAI_Broker | 3.1.0 | docker | DCAE Interface to DTI's AAI View | tc677g | published | 2017-06-15 | +| DcaeSyslogCollector | 2.0.0 | docker | DCAE Control Plane Syslog Collector | sh1986 | published | 2017-08-04 | +| cdap.dmaap.spec.example | 0.2.0 | cdap | dmaap spec example. Not a functioning application, only for showing | tc677g | published | 2017-07-24 | +| | | | how to pub/sub dmaap. Pretend this is like MAP with VES in and ou.. | | | | +| cdap.event.proc.enrich.app | 1.0.3 | cdap | CDAP Event Processing Enrich application | an4828 | published | 2017-09-20 | +| cdap.event.proc.map.app | 1.0.3 | cdap | CDAP Event Processing Map application | an4828 | published | 2017-09-20 | + +... + +Data formats: ++--------------------------------------------+---------+-----------------------------------------------------------------------+--------+-----------+------------+ +| Name | Version | Description | Owner | Status | Published | ++--------------------------------------------+---------+-----------------------------------------------------------------------+--------+-----------+------------+ +| FOI_PM_VHSS_data_format | 1.0.0 | CSV pipe delimited data format for VHSS PM files | sr229c | published | 2017-09-05 | +| Map_input | 1.0.0 | The input format for Mapper, that in 1707 is the UCSNMP Collector | an4828 | published | 2017-07-18 | +| | | output format, but will support more formats later | | | | +| Syslog Collector Parsed Json Message | 1.0.0 | Post processed/parsed collected syslog message | sh1986 | published | 2017-08-04 | +| Syslog Collector Syslog Message Input | 1.0.0 | The input message for the DCAE syslog collector is free/unstructured | sh1986 | published | 2017-08-04 | +| | | text | | | | +| TCA Alert Definition | 1.0.0 | The format of the output event from TCA | an4828 | published | 2017-08-10 | +| VES_specification | 5.28.4 | VES spec for 5.4 | vv770d | published | 2017-09-19 | + +... + +``` + +### Show the contents of an item in the catalog + +``` +$ dcae_cli catalog show FOI_PM_VHSS_data_format + +Data format +----------- +{ + "dataformatversion": "1.0.0", + "delimitedschema": { + "delimiter": "|", + "fields": [ + { + "description": "System ID", + "fieldtype": "string", + "name": "SYSTEM" + }, + { + "description": "Date", + "fieldtype": "string", + "name": "DATE" + }, + { + "description": "Time", + "fieldtype": "string", + "name": "TIME" + }, + +... + +``` + +--------------------------------------------------------------------------- + +## `component` + +The `component` command is for validating (adding), listing, showing, verifying generated configuration, running, undeploying, and publishing components that YOU own. + +``` +$ dcae_cli component --help +Usage: dcae_cli component [OPTIONS] COMMAND [ARGS]... + +Options: + --help Show this message and exit. + +Commands: + add + dev Set up component in development for... + list Lists components in the onboarding catalog. + publish Pushes COMPONENT to the onboarding catalog + run Runs the latest version of COMPONENT. + show Provides more information about COMPONENT + undeploy Undeploys the latest version of COMPONENT. +``` + +--------------------------------------------------------------------------- + +### Add a Component + +A component must be added to the onboarding catalog in order to be tested by the dcae_cli tool. The process of adding a component also validates it's component specification. In order to add a component, the component docker image (or CDAP jar) must exist locally. + +Components in the onboarding catalog can be run by others, once they are `published.` `Published` components cannot be modified or deleted. Rather a new version can be created instead. + +Validated component specs are used later to generate Tosca models and Cloudify Blueprints for the component, which makes them available for use in the SDC Tool for creating services. + +``` +$ dcae_cli component add --help +Usage: dcae_cli component add [OPTIONS] COMPONENT-SPECIFICATION + +Options: + --update Updates a previously added component if it has not been already + published + --help Show this message and exit. +``` + +``` +$ dcae_cli component add component-spec.json +``` + +--------------------------------------------------------------------------- + +### List Components + +List components in the onboarding catalog that owned by YOUR userid.. + +``` +$ dcae_cli component list +Active profile: solutioning ++-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ +| Name | Version | Type | Description | Status | Modified | #Deployed | ++-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ +| cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 | +| sandbox.platform.laika | 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 | ++-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ +``` + +The fields `Name`, `Version`, `Type`, `Description` are referenced from the component specification's `self` JSON. +Use the "--deployed" argument to see more details on deploymed components + +--------------------------------------------------------------------------- + +### Run a Component + +The `run` operation is to be used for running your application in its container remotely on the activated environment. Docker containers have the additional option to run locally on your development machine. If the component uses Dmaap, you can specify the Dmaap Connection Object as well. Refer to [Dmaap Connection Object](/components/component-specification/dmaap-connection-objects). + +When you run a component via the dcae_cli Tool, remember the blueprint has not been created and is not used for deployment. + +In order to run the component, the data formats and component must have been added to the onboarding catalog. + +**DOCKER NOTE:** Make sure the Docker image has been uploaded to the shared registry. + +A docker component can be run in either `attached` or `unattached` mode. (Default is unattached). + +Mode | Description +---- | ----------- +attached | component is run in the 'foreground', container logs are streamed to stdout. Ctrl-C is used to terminate the dcae_cli session. +unattached | component is run in the 'background', container logs are viewed via `docker logs` command, container runs until undeployed with dcae_cli `undeploy` command. + + +#### Run a component in attached mode: + +``` +$ dcae_cli -v component run --attached sandbox.platform.laika:0.5.0 + +DCAE.Docker | INFO | Running image 'registry.proto.research.att.com/dcae-rework/laika:0.4.0' as 'mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika' +DCAE.Docker.mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika | INFO | Consul host: realsolcnsl00.dcae.solutioning.homer.att.com + +DCAE.Docker.mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika | INFO | service name: mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika + +DCAE.Docker.mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika | INFO | Generated config: {'multiplier': 3} + +DCAE.Docker.mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika | INFO | * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit) + +DCAE.Docker.mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika | INFO | 135.205.226.140 - - [24/May/2017 03:37:57] "GET /health HTTP/1.1" 200 - + +DCAE.Docker.mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika | INFO | 135.205.226.140 - - [24/May/2017 03:38:12] "GET /health HTTP/1.1" 200 - +``` + +Hit Ctrl-C to terminate session. + +``` +^CDCAE.Docker | INFO | Stopping container 'mh677g.b7287639-37f5-4f25-8d54-8a2087f4c8da.0-5-0.sandbox-platform-laika' and cleaning up... +``` + +#### Run a component in unattached mode: + +``` +$ dcae_cli -v component run sandbox.platform.laika:0.5.0 +DCAE.Docker | INFO | Running image 'registry.proto.research.att.com/dcae-rework/laika:0.4.0' as 'mh677g.4811da0e-08d5-429f-93bf-bf6814924577.0-5-0.sandbox-platform-laika' +DCAE.Run | INFO | Deployed /mh677g.4811da0e-08d5-429f-93bf-bf6814924577.0-5-0.sandbox-platform-laika +``` + +**NOTE** You must undeploy this component when finished testing. This is important to conserve resources in the environment. + +#### Run a component that subscribes to Dmaap MR or DR + +``` +$ dcae_cli -v component run --attached --dmaap-file $dmaap-connection-file sandbox.platform.laika:0.5.0 +``` + +--------------------------------------------------------------------------- + +### Undeploy a Component + +The `undeploy` command is used to undeploy any instance of a specified component/version that you have deployed. This includes cleaning up the configuration. + +``` +$ dcae_cli component undeploy sandbox.platform.laika:0.5.0 +DCAE.Undeploy | WARNING | Undeploying components: 1 +DCAE.Undeploy | WARNING | Undeployed components: 1 +``` + +--------------------------------------------------------------------------- + +### Publish a component + +Once a component has been tested, it should be published in the onboarding catalog using the `publish` sub-command . +Publishing will change the status of the component (from `staged` to `published`), indicating that it has been tested, and making it accessible for other developers to use. + +**Note** Before a component can be published, all data_formats that it references must be published. + +``` +dcae_cli component publish sandbox.platform.laika:0.5.0 +``` + +--------------------------------------------------------------------------- + +### Show a Component + +This will print out the contents of a component and is useful to copy a component spec. + +``` +$ dcae_cli component show +``` + +--------------------------------------------------------------------------- + +### Run the `dev` command + +The `dev` command is used as part of a process to see the platform generated configuration. It established the environment variables and is best explained [here](/components/dcae-cli/walkthrough/#view-the-platform-generated-configuration). + +``` +$ dcae_cli component dev component-spec.json +Ready for component development +``` + +--------------------------------------------------------------------------- + +## `data_format` + +The `data_format` command is for validating (adding), listing, showing, publishing data_formats that YOU own. data_formats can also be generated with this command. + +``` +$ dcae_cli data_format --help +Usage: dcae_cli data_format [OPTIONS] COMMAND [ARGS]... + +Options: + --help Show this message and exit. + +Commands: + add Tracks a data format file DATA_FORMAT-SPECIFICATION... + generate Create schema from a file or directory... + list Lists all your data formats + publish Publishes data format to make available to others... + show Provides more information about FORMAT +``` + +--------------------------------------------------------------------------- + +### Add a Data Format + +A data_format must be in the onboarding catalog in order to be referenced in the component specification. The process of adding a data_format also validates it. +Data_formats in the onboarding catalog can be run by others, once they are `published.` `Published` data_formats cannot be modified or deleted. Rather a new version can be created instead. + +``` +$ dcae_cli data_format add --help +Usage: dcae_cli data_format add [OPTIONS] DATA_FORMAT-SPECIFICATION + +Options: + --update Updates a previously added data_format if it has not been already + published + --help Show this message and exit. +``` + +``` +dcae_cli data_format add health.json +``` + +--------------------------------------------------------------------------- + +### List Data Formats + +Only data_formats owned by YOUR userid will be shown. + +``` +$ dcae_cli data_format list + +Data formats for mh877g ++-------------------------------+---------+-------------------------------------------+--------+----------------------------+ +| Name | Version | Description | Status | Modified | ++-------------------------------+---------+-------------------------------------------+--------+----------------------------+ +| sandbox.platform.laika.health | 0.1.0 | Data format used for the /health endpoint | staged | 2017-05-23 04:02:38.952799 | ++-------------------------------+---------+-------------------------------------------+--------+----------------------------+ +``` + +The fields `name`, `version`, `description` are referenced from the data format specification's `self` JSON. `Status` represents the status of the data format in the catalog. See [Publish a Data Format](#publish-a-data-format) for more info. + +--------------------------------------------------------------------------- + +### Show a Data Format + +This will print out the contents of a data_format and is useful for copying a data_format. + +``` +$ dcae_cli data_format show +``` + +--------------------------------------------------------------------------- + +### Publish a Data Format + +Once a data_format has been tested (by referencing it in a component spec that has been tested), it should be published in the onboarding catalog using the `publish` sub-command . +Publishing will change the status of the data_format (from `staged` to `published`), indicating that it has been tested, and making it accessible for other developers to use. + +``` +$ dcae_cli data_format publish data_format.json +``` + +--------------------------------------------------------------------------- + +### Generate a Data Format + +If you already have a valid input or output file, you can use the generate command to create the it's data format specification. + +``` +$ dcae_cli data_format generate name:version file-or-dir-path +``` + +--------------------------------------------------------------------------- + +## `profiles` + +The`profiles` command is for creating, deleting, listing, showing, activating, and updating (set) profiles. The profile contains environment variables used to connect to different environments. This is used in the running and deployment of a component using the `dcae_cli component run` or `dev` command. + +``` +$ dcae_cli profiles --help +Usage: dcae_cli profiles [OPTIONS] COMMAND [ARGS]... + +Options: + --help Show this message and exit. + +Commands: + activate Sets profile NAME as the active profile + create Creates a new profile NAME initialized with... + delete Deletes profile NAME + list Lists available profiles + set Updates profile NAME such that KEY=VALUE + show Prints the profile dictionary +``` + +--------------------------------------------------------------------------- + +### List the available profiles + +``` +$ dcae_cli profiles list +* solutioning + 1710 + 1802 +``` + +The * identifies the active profile. `dcae-cli` is currently installed with profiles for the `solutioning`, `1710`, and `1802` environments. They are intended for the following: + +Environment | Description +----------- | ----------- +solutioning | default environment; used for initial component developer testing with the dcae_cli tool. +1710 | FTL3 (Functional Testing Lab 3) environment, which represents the 1710 release. +1802 | FTL3a (Functional Testing Lab 3a) environment, which represents the 1802 release. + +--------------------------------------------------------------------------- + +### Show the details of a profile + +``` +$ dcae_cli profiles show solutioning +{ + "cdap_broker": "cdap_broker", + "config_binding_service": "config_binding_service", + "consul_host": "realsolcnsl00.dcae.solutioning.homer.att.com", + "docker_host": "realsoldokr00.dcae.solutioning.homer.att.com:2376" +} +``` + +--------------------------------------------------------------------------- + +### Activate a profile + +To switch among profiles, use the activate sub-command. A subsequent `list` will reveal the change made. + +``` +$ dcae_cli profiles activate test +``` + +--------------------------------------------------------------------------- + +### Create a new profile + +If you want to work in a different environment using the dcae_cli tool, you can make your own profile. (The environment must be a working DCAE Platform environment). + +``` +$ dcae_cli profiles create new-profile +``` + +After creating you would assign the variables with the `set` sub-command. Then activate it to use. + +--------------------------------------------------------------------------- + +### Set variables in a profile + +``` +$ dcae_cli profiles set $profile $key $value +``` + +--------------------------------------------------------------------------- + +### Delete a profile + +``` +$ dcae_cli profiles delete new-profile +``` diff --git a/platformdoc/docs/components/dcae-cli/dmaap-connection-objects.md b/platformdoc/docs/components/dcae-cli/dmaap-connection-objects.md deleted file mode 100644 index cf03ccec..00000000 --- a/platformdoc/docs/components/dcae-cli/dmaap-connection-objects.md +++ /dev/null @@ -1,156 +0,0 @@ -# DMaaP connection objects - -DMaaP connection objects are JSON objects that: - -1. Components should expect at runtime in their application configuration and is to be used to connect to the appropriate DMaaP feed or topic. -2. Developers must provide through the command-line argument `--dmaap-file` to test their component with manually provisioned feeds and topics. - -This page is a reference to the specific structure that each type of DMaaP stream requires. - -Note for #1 that components should expect the entire object with all properties at runtime where the default will be `null` unless specified otherwise. - -Note for #2 that developers are not required to provide the entire object. The required properties will be labeled with "*required as input*". - -## Message router - -Publishers and subscribers both have the same JSON object structure. Here's an example: - -```json -{ - "type": "message_router", - "aaf_username": "some-user", - "aaf_password": "some-password", - "dmaap_info": { - "client_role": "com.dcae.member", - "client_id": "1500462518108", - "location": "mtc00", - "topic_url": "https://we-are-message-router.us:3905/events/some-topic" - } -} -``` - -At the top-level: - -Property Name | Type | Description -------------- | ---- | ----------- -type | string | *Required as input*. Must be `message_router` for message router topics -aaf_username | string | AAF username message router clients use to authenticate with secure topics -aaf_password | string | AAF password message router clients use to authenticate with secure topics -dmaap_info | JSON object | *Required as input*. Contains the topic connection details - -The `dmaap_info` object contains: - -Property Name | Type | Description -------------- | ---- | ----------- -client_role | string | AAF client role that's requesting publish or subscribe access to the topic -client_id | string | Client id for given AAF client -location | string | DCAE location for the publisher or subscriber, used to set up routing -topic_url | string | *Required as input*. URL for accessing the topic to publish or receive events - -Here's an example of the minimal JSON that must be provided as an input: - -```json -{ - "type": "message_router", - "dmaap_info": { - "topic_url": "https://we-are-message-router.us:3905/events/some-topic" - } -} -``` - -## Data router - -### Publisher - -Here's an example of what the JSON object connection for data router publisher looks like: - -```json -{ - "type": "data_router", - "dmaap_info": { - "location": "mtc00", - "publish_url": "https://we-are-data-router.us/feed/xyz", - "log_url": "https://we-are-data-router.us/feed/xyz/logs", - "username": "some-user", - "password": "some-password", - "publisher_id": "123456" - } -} -``` - -At the top-level: - -Property Name | Type | Description -------------- | ---- | ----------- -type | string | *Required as input*. Must be `data_router` for data router feeds -dmaap_info | JSON object | *Required as input*. Contains the topic connection details - -The `dmaap_info` object contains: - -Property Name | Type | Description -------------- | ---- | ----------- -location | string | DCAE location for the publisher, used to set up routing -publish_url | string | *Required as input*. URL to which the publisher makes Data Router publish requests -log_url | string | URL from which log data for the feed can be obtained -username | string | Username the publisher uses to authenticate to Data Router -password | string | Password the publisher uses to authenticate to Data Router -publisher_id | string | Publisher id in Data Router - -Here's an example of the minimal JSON that must be provided as an input: - -```json -{ - "type": "data_router", - "dmaap_info": { - "publish_url": "https://we-are-data-router.us/feed/xyz" - } -} -``` - -### Subscriber - -Here's an example of what the JSON object connection for data router subscriber looks like: - -```json -{ - "type": "data_router", - "dmaap_info": { - "location": "mtc00", - "delivery_url": "https://my-subscriber-app.dcae:8080/target-path", - "username": "some-user", - "password": "some-password", - "subscriber_id": "789012" - } -} -``` - -At the top-level: - -Property Name | Type | Description -------------- | ---- | ----------- -type | string | *Required as input*. Must be `data_router` for data router feeds -dmaap_info | JSON object | *Required as input*. Contains the topic connection details - -The `dmaap_info` object contains: - -Property Name | Type | Description -------------- | ---- | ----------- -location | string | DCAE location for the publisher, used to set up routing -delivery_url | string | URL to which the Data Router should deliver files -username | string | Username Data Router uses to authenticate to the subscriber when delivering files -password | string | Password Data Router uses to authenticate to the subscriber when delivering files -subscriber_id | string | Subscriber id in Data Router - -Here's an example of the minimal JSON that must be provided as an input: - -```json -{ - "type": "data_router", - "dmaap_info": { - } -} -``` - -Developers are recommended to use `username` and `password` since this is the recommended security practice. - -Note that the dcae-cli will construct the `delivery_url` when deploying the component since this can only be known at deployment time. diff --git a/platformdoc/docs/components/dcae-cli/quickstart.md b/platformdoc/docs/components/dcae-cli/quickstart.md index 905b1372..a70cd037 100644 --- a/platformdoc/docs/components/dcae-cli/quickstart.md +++ b/platformdoc/docs/components/dcae-cli/quickstart.md @@ -1,48 +1,84 @@ -# Quickstart +# Overview -The `dcae-cli` is a Python command-line tool built to aide component developers with the development and testing of their micro-service component for the DCAE platform. It will help developers do functional and integration testing of their components locally and on remote environments as simple as possible. +The `dcae-cli` is a Python command-line tool for component developers. With it, the developer can : -The tool requires the component developers to create a valid component specification for their component which is used by the tool. This same component specification will be published in the [onboarding catalog](../../glossary#onboarding-catalog) at the end of development and testing. +* validate the data formats and component specifications +* publish the validated data formats and component specifications into the `onboarding catalog` +* access the `onboarding catalog` to search for existing data formats (for possible reuse) and component specs +* deploy a component onto a local or remote DCAE platform for functional and pair-wise testing (This is done without Cloudify) -The git repository can be found [here](ONAP LINK TBD) +The git repository for the dcae_cli tool can be found [here](https://codecloud.web.att.com/projects/ST_DCAECNTR/repos/dcae-cli/browse) -## Pre-requisite +## Pre-requisites ### For Docker -There are two options for development with Docker: developing locally on your machine which requires Docker to be installed and developing remotely by deploying onto remote infrastructure. +There are two options for development with Docker: #### For local development -* You must install [Docker engine](https://docs.docker.com/engine/installation/) locally on your machine. -* You must know the *external ip* of where the Docker engine is running. The external ip is needed so that service discovery will wire up correctly. - - For OSX users, this means making sure the VirtualBox VM that is running your Docker engine has a bridged adapter and getting the ip of that adapter. +* Install [Docker engine](https://docs.docker.com/engine/installation/) locally on your machine. +* Know the *external ip* of where the Docker engine is running. The external ip is needed so that service discovery will connect to it. + - *(For OSX users, this means making sure the VirtualBox VM that is running your Docker engine has a bridged adapter and getting the ip of that adapter).* #### For remote development -You need access to a remote host with Docker engine installed with remote API access. You must have the associated connection information: domain name or IP and port (should be either 2375 or 2376). This information should be set in [an active profile](walkthrough.md#setting-profile). +* Have access to a remote host with Docker engine installed and with remote API access. +* Have the associated connection information: + - domain name or IP and port (port should be either 2375 or 2376). Use this information to establish an active [profile](/components/dcae-cli/commands#activate-a-profile). ### For CDAP -TBD +None at this time. -## Install +### Python, Pip, Virtualenv + +Install python, pip (9.0.1 or higher), and virtualenv if they are not installed. Do these when not in a VPN to avoid possible network issues. + +``` + sudo apt-get -f install python + sudo apt-get -f install python-pip + sudo pip install virtualenv +``` + +Set up a virtual environment and activate ``` -pip install --extra-index-url https://YOUR_NEXUS_PYPI_SERVER/simple dcae-cli + virtualenv cli_tool + source cli_tool/biin/activate ``` -To do an upgrade, use the `--upgrade` flag. +## Install dcae_cli + +``` +pip install --extra-index-url https://nexus01.research.att.com:8443/repository/solutioning01-mte2-pypi/simple dcae-cli +``` -### Configuration +## Check dcae_cli version -When you run the tool for the first time, the tool will create a [configuration directory](http://click.pocoo.org/5/api/#click.get_app_dir) and generate a configuration file. +You can verify the version of the dcae-cli with the following command. To get the latest version of the dcae_cli tool, -Configuration is first sourced from an remote server that the platform team manages. This is overlaid with configuration details that you will be prompted to input particularly your user id. +``` +$ dcae_cli --version +``` -#### `--reinit` +## Upgrade dcae_cli -Configuration can be re-initialized or reset. There is a `--reinit` flag that is to be used to re-initialize your configuration and your environment profiles. +Periodically, upgrade the dcae_cli to get the latest version + +``` +pip install --upgrade --extra-index-url https://nexus01.research.att.com:8443/repository/solutioning01-mte2-pypi/simple dcae-cli +``` + +## Configuration + +When running the tool for the first time, a [configuration directory](http://click.pocoo.org/5/api/#click.get_app_dir) and configuration file will be created. + +The configuration is first sourced from a remote server that is managed by the platform team. You will be prompted to enter your ATTUID to complete this process. + +### Re-initializing Configuration + +Configuration can be re-initialized or reset. There is a `--reinit` flag that is to be used to re-initialize your configuration and your environment profiles. You may be instructed to re-initialize after certain updates are made to the dcae_cli tool. When you re-initialize the configuration, your configuration will be added to or updated from the platform configuration and profiles. No profiles will be deleted via the reinit process. To re-initialize: @@ -50,9 +86,9 @@ To re-initialize: $ dcae_cli --reinit ``` -### Verify +## Verify Installation -Verify that its installed: +To Verify that the dcae_cli tool is installed, run the following command and look for the output below. ``` $ dcae_cli --help @@ -70,10 +106,5 @@ Commands: data_format profiles ``` -## Version - -You can verify the version of the dcae-cli with the following command: -``` -$ dcae_cli --version -``` +Refer to [dcae_cli Commands](/components/dcae-cli/commands). diff --git a/platformdoc/docs/components/dcae-cli/walkthrough.md b/platformdoc/docs/components/dcae-cli/walkthrough.md index da54e12e..683bfabd 100644 --- a/platformdoc/docs/components/dcae-cli/walkthrough.md +++ b/platformdoc/docs/components/dcae-cli/walkthrough.md @@ -1,438 +1,340 @@ # Walk-through -The goal of this quickstart is to provide an overview of the functionalities of the `dcae-cli` and walk you through the capabilities: +This section demonstrates the flow and usage of the dcae_cli tool to onboard a typical component to the DCAE platform. The commands are explained in more detail in [dcae_cli Commands](/components/dcae-cli/commands). -* [Adding data formats](#adding-data-formats) -* [Adding component](#adding-component) -* [Setting profile](#setting-profile) -* [Development and testing](#development-and-testing) -* [Publishing component](#publishing-component) -* [Shared catalog](#shared-catalog) +* [Add (and validate) a data format](#add-a-data-format) +* [Add (and validate) the component](#add-the-component) +* [View the platform generated configuration](#view-the-platform-generated-configuration) +* [If needed, Create the dmaap file for Dmaap Testing](#create-input-file-for-dmaap-testing) +* [If needed, Create the input file for *Sourced at Deployment* Testing](#create-input-file-for-sourced-at-deployment-testing) +* [Run the component](#run-the-component) +* [If needed, Create the DTI entry in CONSUL for DTI Reconfiguration Testing](#create-DTI-entry-for-reconfiguration) +* [Undeploy the component](#undeploy-the-component) +* [Publish the component and data_format](#publish-the-component-and-data_format) to let others know its ready for reuse +* [List the Catalog Contents](#list-the-catalog-contents) to see your published resources -This walk-through uses example projects: +This walk-through uses example projects that can be found in CodeCloud: -* [laika](ONAP URL TBD) -* [CDAP examples](ONAP URL TBD) +* [laika](https://codecloud.web.att.com/projects/ST_DCAECNTR/repos/laika/browse) +* [CDAP examples](https://codecloud.web.att.com/projects/ST_DCAECNTR/repos/dcae_cli_examples/browse) -## Adding data formats - -`data_format` is the sub-command that is used to execute operations that manage [data formats](../data-formats.md). +------------------------------------------------------------------- +## Add a Data Format ``` -$ dcae_cli data_format --help -Usage: dcae_cli data_format [OPTIONS] COMMAND [ARGS]... - -Options: - --help Show this message and exit. - -Commands: - add Tracks a data format file SPECIFICATION... - generate Generates a data format file from JSON input examples - list Lists all your data formats - publish Publishes data format to make publicly... - show Provides more information about FORMAT +$ dcae_cli data_format add $HOME/laika/data-formats/health.json ``` -Your data format must be in the catalog in order to use in the component specification. Check the catalog using the `data_format list` sub-command: - +Verify that the data_format was added ``` -$ dcae_cli data_format list - -Data formats for mh677g -+------+---------+-------------+--------+----------+ -| Name | Version | Description | Status | Modified | -+------+---------+-------------+--------+----------+ -| | | | | | -+------+---------+-------------+--------+----------+ +$ dcae_cli data_format list | grep laika +| sandbox.platform.laika.health | 0.1.0 | Data format used for the /health endpoint | staged | 2017-11-07 21:48:47.736518 | ``` -The fields `name`, `version`, `description` are referenced from the data format JSON from the `self` JSON. +(Note: Each of the data formats for your component need to be added, unless already existing in the onboarding catalog ) -There are no data formats so you must add the data formats that your component specification references. Use the `data_format add` sub-command: +------------------------------------------------------------------- -Here's an example command: +## Add the Component ``` -dcae_cli data_format add health.json +$ dcae_cli component add $HOME/laika/component-spec.json ``` -Verify that it was added: - +Verify that the component was added ``` -$ dcae_cli data_format list +$ dcae_cli component list +Active profile: solutioning -Data formats for mh677g -+-------------------------------+---------+-------------------------------------------+--------+----------------------------+ -| Name | Version | Description | Status | Modified | -+-------------------------------+---------+-------------------------------------------+--------+----------------------------+ -| sandbox.platform.laika.health | 0.1.0 | Data format used for the /health endpoint | staged | 2017-05-23 04:02:38.952799 | -+-------------------------------+---------+-------------------------------------------+--------+----------------------------+ ++-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ +| Name | Version | Type | Description | Status | Modified | #Deployed | ++-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ +| sandbox.platform.laika | 0.7.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-11-08 20:27:34.168854 | 0 | ++-------------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ ``` -Go ahead and add other referenced data formats. +------------------------------------------------------------------- -If you have JSON input you can generate a data format like this: +## View the platform generated configuration -``` -$ dcae_cli data_format --keywords generate myname:1.0.0 myjsoninputfile -``` +The `component dev` command is useful during onboarding. Running this command is part of a multi-step process that sets up a temporary test environment, generates your application configuration, makes it available in that environment, and allows you to view that configuration to help with debugging. -where `myname` is the name of your data format, `1.0.0` is an example version, and `myjsoninputfile` is an example JSON input file (a directory of input JSON files can also be provided). The `--keywords` option adds additional data attributes that can be completed to provide a more detailed data characterization. In any event the output should be reviewed for accuracy. The data format is written to stdout. +Here is a step-by-step example based on a component specification called `component-spec.json`. -## Adding component +### Step 1 - Run the component dev command -`component` is the sub-command that is used to work with operations for components: +(This creates a file called env_$ENV (in the current directory)- where $ENV is the name of the active profile. Note: SERVICE_NAME and HOSTNAME always resolve to the same value). ``` -$ dcae_cli component --help -Usage: dcae_cli component [OPTIONS] COMMAND [ARGS]... - -Options: - --help Show this message and exit. +$ dcae_cli component dev component-spec.json +Ready for component development -Commands: - add - dev Set up component in development for... - list Lists components in the public catalog. - publish Pushes COMPONENT to the public catalog - run Runs the latest version of COMPONENT. - show Provides more information about COMPONENT - undeploy Undeploys the latest version of COMPONENT. -``` +Setup these environment variables. Run "source env_solutioning": -Your component must be accessible from the catalog in order for it to be used. Check the catalog using the `component list` sub-command: +export DOCKER_HOST=realsoldokr00.dcae.solutioning.homer.att.com:2376 +export SERVICE_CHECK_INTERVAL=15s +export CONFIG_BINDING_SERVICE=config_binding_service +export HOSTNAME=ph8547.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-laika +export CONSUL_HOST=realsolcnsl00.dcae.solutioning.homer.att.com +export CDAP_BROKER=cdap_broker +export SERVICE_NAME=ph8547.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-laika +export SERVICE_CHECK_TIMEOUT=1s +export SERVICE_CHECK_HTTP=/health +Press any key to stop and to clean up ``` -$ dcae_cli component list -Active profile: solutioning -+------+---------+------+-------------+--------+----------+-----------+ -| Name | Version | Type | Description | Status | Modified | #Deployed | -+------+---------+------+-------------+--------+----------+-----------+ -| | | | | | | | -+------+---------+------+-------------+--------+----------+-----------+ +### Step 2 - Setup the environment +In another window, setup the temporary testing environment, by executing the environment file created above. -Use the "--deployed" option to see more details on deployments +``` +$ source env_solutioning ``` -The fields `name`, `version`, `type`, `description` are referenced from the component specification's `self` JSON. +(The application configuration is now available under the SERVICE_NAME shown above - `ph8547.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-laika`). -There are no components so you must add your component. Use the `component add` sub-command. The command is the same for docker and cdap components: -``` -$ dcae_cli component add --help -Usage: dcae_cli component add [OPTIONS] SPECIFICATION +### Step 3 - Query CONSUL +Query CONSUL to get the IP/PORT of CONFIG BINDING SERVICE -Options: - --update Updates a locally added component if it has not been already - pushed - --help Show this message and exit. +``` +$ curl http://$CONSUL_HOST:8500/v1/catalog/service/$CONFIG_BINDING_SERVICE +[ + { + "ID": "bfbc220d-4603-7f90-ec2e-611d3c330f20", + "Node":"realsoldokr00", + "Address": "10.226.1.15", + "Datacenter":"solutioning-central", + "TaggedAddresses": { + "lan":"10.226.1.15", + "wan":"10.226.1.15" + }, + "NodeMeta": {}, + "ServiceID": "472b116f9035:config_binding_service:10000", + "ServiceName": "config_binding_service", + "ServiceTags": [], + "ServiceAddress":"135.205.226.126", + "ServicePort":10000, + "ServiceEnableTagOverride": false, + "CreateIndex":1078990, + "ModifyIndex":1078990 + } +] ``` -*Note* use the `--update` flag to replace existing staged instances. - -The `component dev` sub-command can be useful in validating and experimenting when crafting your component specification. See details about `dev` under [Development and testing](#development-and-testing). +Fetch the generated configuration from CONFIG BINDING SERVICE using the 'serviceaddress' and 'serviceport' from above along with $SERVICE_NAME from earlier. -Once we add the components laika and helloworld, let's verify that they got added ok: +``` +$ curl http://135.205.226.126:10000/service_component/ph8547.b599cf0e-75e8-484b-b8e2-557576d77036.0-7-0.sandbox-platform-laika +{"streams_subscribes": {}, "services_calls": {}, "multiplier": 3, "streams_publishes": {}} ``` -$ dcae_cli component list -Active profile: solutioning -+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ -| Name | Version | Type | Description | Status | Modified | #Deployed | -+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ -| cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 | -| sandbox.platform.laika | 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 | -+-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ +------------------------------------------------------------------- -Use the "--deployed" option to see more details on deployments -``` +## Create the input file for Dmaap Testing -## Setting profile +Currently, the dcae-cli tool does not have the capability to provision topics or feeds. Therefore, in order to test with `message router` or `data router` feeds, the developer must manually provision the topic or feed and then provide the connection details in the form of a DMaap JSON file for testing. This file is then passed in on the `component run` or `component dev` commands by using the argument `--dmaap-file`. -`profile` is the sub-command that is used to manage profiles. These profiles contain environment variables used to connect to different environments. This is used in the running and deployment of your component using the `dcae_cli component run` command. The `dcae-cli` ships with profiles for `solutioning` and `rework`. +The structure of the DMaaP JSON is an object of config keys with the topic or feed connection details. The config keys are the `config_key` values specified in the component specification streams section where the streams must be type `message router` or `data router`. This file corresponds to the `Dmaap Connection Object` which is generated by the platform and provided to the component at runtime. The exception is that `delivery_url` cannot be provided in the dmaap-file because it is not created until the component is deployed. Refer to [Dmaap Connection Object](/components/component-specification/dmaap-connection-objects), for details on creating the dmaap-file for testing. -``` -$ dcae_cli profiles --help -Usage: dcae_cli profiles [OPTIONS] COMMAND [ARGS]... +------------------------------------------------------------------- -Options: - --help Show this message and exit. +## Create the input file for *Sourced at Deployment* Testing -Commands: - activate Sets profile NAME as the active profile - create Creates a new profile NAME initialized with... - delete Deletes profile NAME - list Lists available profiles - set Updates profile NAME such that KEY=VALUE - show Prints the profile dictionary -``` +Components may have configuration parameters whose values are to be sourced at deployment time. For example, there are components whose configuration parameters are to come from DTI events which are only available when the component is deployed. This is established in the [component specification](/components/component-specification/common-specification/#parameters) by setting the property `sourced_at_deployment` to `true` for each applicable parameter. -To see what variables a profile contains, you can use the `show` command, as in `dcae_cli profiles show PROFILE_NAME` +Then, use the `--inputs-file` command-line argument when running the component `dev` or `run` command for your component. This is to simulate providing the dynamic, deployment time values for those parameters marked as `sourced_at_deployment`. -Use the `create` sub-command to create your own profile and assign new values using the `set` command. Afterwards you will need to `activate` the profile you wish to use. First take a look at which profile is active: +For example, if your component specification has the following configuration parameters: ``` -$ dcae_cli profiles list - rework -* solutioning +"parameters": [{ + "name": "vnf-ip", + "value": "", + "sourced_at_deployment": true +}, +{ + "name": "static-param", + "value": 5 +}] ``` -The active profile is `solutioning` so to activate *rework* to use `rework`: +Pass in an input file that looks like: ``` -$ dcae_cli profiles activate rework +{ + "vnf-ip": "10.100.1.100" +} ``` -Check +The application configuration would look like: ``` -$ dcae_cli profiles list -* rework - solutioning +{ + "vnf-ip": "10.100.1.100", + "static-param": 5 +} ``` -## Development and testing - -The following operations under the sub-command `component` are aimed to help developers with testing: - -* `run` -* `undeploy` -* `dev` +------------------------------------------------------------------- -### `run` +## Run the component -The `run` operation is to be used for running your application in its container remotely on the activated environment. Docker containers have the additional option to run locally on your development machine. +The `run` operation is to be used for running your application in its container remotely on the activated environment. Docker containers have the additional option to run locally on your development machine. If the component uses Dmaap, you can specify the Dmaap Connection Object as well. Refer to [Dmaap Connection Object](/components/component-specification/dmaap-connection-objects). -In order to run your application, you must have added your data formats and your component to your catalog. +In order to run the component, the data formats and component must have been added to the onboarding catalog. -Let's verify that your component is in the catalog: +To verify what's in the catalog: ``` -$ dcae_cli component list +$ dcae_cli catalog list --expanded Active profile: solutioning - +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ | Name | Version | Type | Description | Status | Modified | #Deployed | +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ -| cdap.helloworld.endnode | 0.8.0 | cdap | cdap test component | staged | 2017-05-23 04:14:35.588075 | 0 | -| sandbox.platform.laika | 0.5.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-05-23 04:07:44.065610 | 0 | +| sandbox.platform.laika | 0.7.0 | docker | Web service used as a stand-alone test DCAE service compone.. | staged | 2017-11-08 20:27:34.168854 | 0 | +-------------------------+---------+--------+---------------------------------------------------------------+--------+----------------------------+-----------+ -Use the "--deployed" option to see more details on deployments ``` -#### Docker - -**NOTE** Make sure your Docker image has been uploaded to the shared registry. -For Docker containers, you can run either attached or unattached. Attached means that the dcae-cli tool will launch the container and not terminate. The dcae-cli while attached will stream in the logs of the Docker container. Doing a Ctrl-C will terminate the run session which means undeploy your container and force a clean up automatically. +For Docker -Running unattached means simply deploy your container. You will need to execute `undeploy` when you are done testing. -#### CDAP +**NOTE** Make sure the Docker image has been uploaded to the shared registry. -**NOTE** Make sure your CDAP jar has been uploaded to Nexus. +A docker component can be run in either `attached` or `unattached` mode. (Default is unattached). -TODO +Mode | Description +---- | ----------- +attached | component is run in the 'foreground', container logs are streamed to stdout. Ctrl-C is used to terminate the dcae_cli session. +unattached | component is run in the 'background', container logs are viewed via `docker logs` command, container runs until undeployed with dcae_cli `undeploy` command. -### `undeploy` -The `undeploy` operation is to be used to undeploy any instances of a specified component, version that you have deployed. This includes cleaning up of configuration. +#### Run a component in attached mode: -Let's undeploy `sandbox.platform.laika` that was deployed from the previous section: - -``` -$ dcae_cli component undeploy sandbox.platform.laika:0.5.0 -DCAE.Undeploy | WARNING | Undeploying components: 1 -DCAE.Undeploy | WARNING | Undeployed components: 1 ``` +$ dcae_cli -v component run --attached sandbox.platform.laika:0.7.0 +DCAE.Docker | INFO | Running image 'nexus01.research.att.com:18443/repository/solutioning01-mte2-docker/dcae-platform/laika:0.7.0' as 'ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika' +DCAE.Docker.ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika | INFO | Consul host: realsolcnsl00.dcae.solutioning.homer.att.com -### `dev` +DCAE.Docker.ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika | INFO | service name: ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika -The `dev` operation is a convenient operation that can be useful for the development and testing of your component. It can be used to: +DCAE.Docker.ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika | INFO | get_config returned the following configuration: {"streams_subscribes": {}, "multiplier": 3, "services_calls": {}, "streams_publishes": {}} -* Help validate your experimental component specification before uploading to the catalog -* Generate the application configuration from the component specification and make it available in a test environment. This allows you to view your resulting configuration for local development and to help debug potential related issues. +DCAE.Docker.ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika | INFO | * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit) -Let's say you have a component specification called `component-spec.json`: +DCAE.Docker.ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika | INFO | 135.205.226.156 - - [08/Nov/2017 23:27:30] "GET /health HTTP/1.1" 200 - -``` -$ dcae_cli component dev component-spec.json -Ready for component development -Setup these environment varibles. Run "source env_solutioning": +Hit Ctrl-C to terminate session. -export DOCKER_HOST=SOME_DOCKER_HOST:2376 -export SERVICE_CHECK_INTERVAL=15s -export CONFIG_BINDING_SERVICE=config_binding_service -export HOSTNAME=mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika -export CONSUL_HOST=SOME_CONSUL_HOST -export CDAP_BROKER=cdap_broker -export SERVICE_NAME=mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika -export SERVICE_CHECK_TIMEOUT=1s -export SERVICE_CHECK_HTTP=/health - -Press any key to stop and to clean up +^C +DCAE.Docker | INFO | Stopping container 'ph8547.dbb13a3c-d870-487e-b584-89929b856b5c.0-7-0.sandbox-platform-laika' and cleaning up... ``` -Your application configuration is now available under the name `mh677g.95740959-63d2-492a-b964-62a6dce2591d.0-6-0.sandbox-platform-laika`. - -To view the resulting configuration, you can `curl` a request to the config binding service or programmatically fetch your configuration within your application. - -You need to first query Consul to get the ip and port of config binding service: +#### Run a component in unattached mode: ``` -curl http://$CONSUL_HOST:8500/v1/catalog/service/$CONFIG_BINDING_SERVICE -[ - { - "ID": "983d5c94-c508-4a8a-9be3-5912bd09786b", - "Node": "realsolcnsl00", - "Address": "10.226.1.22", - "TaggedAddresses": { - "lan": "10.226.1.22", - "wan": "10.226.1.22" - }, - "NodeMeta": {}, - "ServiceID": "5f371f295c90:config_binding_service:10000", - "ServiceName": "config_binding_service", - "ServiceTags": [], - "ServiceAddress": "XXXX", - "ServicePort": 32770, - "ServiceEnableTagOverride": false, - "CreateIndex": 487, - "ModifyIndex": 487 - } -] +$ dcae_cli -v component run sandbox.platform.laika:0.7.0 +DCAE.Docker | INFO | Running image 'nexus01.research.att.com:18443/repository/solutioning01-mte2-docker/dcae-platform/laika:0.7.0' as 'ph8547.22629ebd-417e-4e61-a9a0-f0cb16d4cef2.0-7-0.sandbox-platform-laika' +DCAE.Run | INFO | Deployed ph8547.22629ebd-417e-4e61-a9a0-f0cb16d4cef2.0-7-0.sandbox-platform-laika. Verifying.. +DCAE.Run | INFO | Container is up and healthy ``` -### DMaaP testing - -Currently, the dcae-cli does not have the capability of provisioning topics. In order to do testing with message router topics or with data router feeds, the developer must provision the topic or the feed manually and provide the connection details in the form of a JSON in a file to the dcae-cli. This file is to be passed in when using the `run` and `dev` commands with the option `--dmaap-file`. - -The structure of the DMaaP JSON is an object of config keys to matching topic or feed connection details. Config keys are the `config_key` values specified in your component specification streams section where the streams must be type message router or data router. Information about the associated connection details can be found on [this page](dmaap-connection-objects.md). Please check it out. +**NOTE** You must undeploy this component when finished testing. This is important to conserve resources in the environment. -For example, if you have a component specification that has the following streams entry: +#### Run a component that subscribes to Dmaap Message Router or Data Router -```json -"streams": { - "publishes": [{ - "format": "ves", - "version": "1.0.0", - "type": "message router", - "config_key": "ves_connection" - }] -} ``` - -Then to deploy and to run your component, you must use the `--dmaap-file` command and pass in a JSON that looks like: - -```json -{ - "ves_connection": { - "type": "message_router", - "dmaap_info": { - "topic_url": "https://we-are-message-router.us:3905/events/some-topic" - } - } -} +$ dcae_cli -v component run $component-that-uses-dmamp --dmaap-file $dmaap-connection-object ``` -The provided DMaaP JSON is used to simulate the output of provisioning and will be used to merge with the generated application configuration at runtime. - -Your final application config will look like: +#### Run a component that expects input that is `sourced at deployment` -```json -{ - "streams_publishes": { - "ves_connection": { - "type": "message_router", - "dmaap_info": { - "topic_url": "https://we-are-message-router.us:3905/events/some-topic" - } - } - } -} ``` - -#### Data router subscribers - -Note for data router subscriber testing, you will need the delivery url in order to provision the subscriber to the feed. This is constructed at deployment time and will be provided by the dcae-cli after you deploy your component. The delivery urls will be displayed to the screen: - +$ dcae_cli -v component run $component-that-expects-dti --inputs-file $input-file-to-simulate-dti ``` -DCAE.Run | WARNING | Your component is a data router subscriber. Here are the delivery urls: - some-sub-dr: http://SOME_IP:32838/identity +------------------------------------------------------------------- -``` +## Create the DTI Entry for Reconfiguration -### *Sourced at deployment* testing +Go the the CONSUL UI for the environment that you are working in. Add a `dti` entry to represent one or more instances of `vnfType-vnfFuncId` for your component. -Components may have configuration parameters whose values are to be sourced at deployment time. For example, there are components whose configuration parameters are to come from DTI events which are only available when the component is getting deployed. These configuration parameters must be setup correctly in the [component specification](http://localhost:8000/components/component-specification/docker-specification/#configuration-parameters) by setting the property `sourced_at_deployment` to `true` for each and every parameter that is expected to come in at deployment time. +For example, in 1802, go (here)[http://zldcrdm5bdcc2cnsl00.2f3fb3.rdm5b.tci.att.com:8500/ui/#/zldcrdm5bdcc2/kv/). -Once your component specification has been updated correctly, you must use the `--inputs-file` command-line argument when running the commands `dev` or `run` with your component. This is to simulate providing the dynamic, deployment time values for those parameters marked as `sourced_at_deployment`. +Do CNTL-F to find your running MS +Click on + to add your entry +Enter your $SERVICE_NAME:dti as the Key +Paste your JSON into the box, remember to `check` the VALIDATE JSON box +Click on CREATE -For example, if your component specification has the following configuration parameters: +Verify that you can retrieve the dti entry you just created as in this example: (Remember to use the 'serviceaddress' and 'serviceport' from above for CONFIG BINDING SERVICE). ``` -"parameters": [{ - "name": "vnf-ip", - "value": "", - "sourced_at_deployment": true -}, -{ - "name": "static-param", - "value": 5 -}] +http://135.203.226.126:10000/dti/<service name>`` ``` +(You should see the entry you created above) -You would have to pass in an inputs file that looks like: +------------------------------------------------------------------- -``` -{ - "vnf-ip": "10.100.1.100" -} -``` +## Run the reconfigure script -Your application configuration would look like: +Execute the components reconfigure script as defined in the Auxilary section of the component spec, such as in this example: ``` -{ - "vnf-ip": "10.100.1.100", - "static-param": 5 -} +/opt/app/reconfigure.sh dti $updated_dti ``` -## Publishing component +(Refer to [DTI Reconfiguration](/components/component-specification/docker-specification/#dti-reconfiguration) + +Verify that your component received and is processed the updated set of vnfType-vnfFuncId instances. -Once components have their component specifications crafted and validated and have been tested, components should be published in the shared onboarding catalog using the `publish` sub-command for both data formats and components. You must publish all data formats of a component before publishing a component. +------------------------------------------------------------------ +## Undeploy the component -Publishing will change the status of a component, be made accessible for other developers to use, and will generate the associated TOSCA models for use in designing of compositions. +The `undeploy` command is used to undeploy any instance of a specified component/version that you have deployed. This includes cleaning up the configuration. + +Undeploy `sandbox.platform.laika:0.7.0` that was deployed above: ``` -dcae_cli component publish sandbox.platform.laika:0.5.0 +$ dcae_cli -v component undeploy sandbox.platform.laika:0.7.0 +DCAE.Undeploy | WARNING | Undeploying components: 1 +DCAE.Undeploy | WARNING | Undeployed components: 1 ``` -## Shared catalog +------------------------------------------------------------------- + +## Publish the component and data_format -`catalog` is the sub-command used to access and to browse the shared onboarding catalog to view components and data formats that have been published and that are being worked on. Components and data formats have two statuses `staged` and `published`. +Once a component has been tested, it (and the data_format(s)) should be published in the onboarding catalog using the `publish` sub-command for both the `data_format` and `component` command. -Staged means that the resource has been simply added and is under development. It is to be used only by the owner. Published means that the resource has been fully developed and tested and is ready to be shared. +**Note** Before a component can be published, all data_formats that it references must be published. -Published components can be deployed by non-owners and published data formats can be used in component specifications of non-owners. +Publishing will change the status of a component or data_format, indicating that it has been tested, make accessible for other developers to use. -There are two available operations: +``` +$ dcae_cli data_format publish sandbox.platform.laika:0.7.0 +Data format has been published + +$dcae_cli component publish sandbox.platform.laika:0.7.0 +Component has been published ``` -$ dcae_cli catalog --help -Usage: dcae_cli catalog [OPTIONS] COMMAND [ARGS]... +------------------------------------------------------------------- -Options: - --help Show this message and exit. +## List the catalog contents + +``` +$dcae_cli catalog list -Commands: - list - show +$ dcae_cli data_format list | grep sandbox +| sandbox.platform.laika | 0.7.0 | docker | Web service used as a stand-alone test DCAE service compone.. | ph8547 | published | 2017-11-13 | +| sandbox.platform.laika.health | 0.1.0 | Data format used for the /health endpoint | published | 2017-11-13 17:48:10.121588 | +| sandbox.platform.any | 0.1.0 | Data format used when no data format is required. | published | 2017-11-13 17:47:51.622607 | +| sandbox.platform.laika.identity.response | 0.1.0 | Data format used for the /identity endpoint response which should | published | 2017-11-13 17:47:43.234715 | +| sandbox.platform.laika.identity.request | 0.1.0 | Data format used for the /identity endpoint request. This is | published | 2017-11-13 17:47:36.693643 | +| sandbox.platform.laika.rollcall.response | 0.1.0 | Data format used for the /rollcall endpoint respon.. | published | 2017-11-13 17:46:30.026846 | ``` -Staged components can be viewed under the `list` operation using the `--expanded` flag. |