dcae design doc updates

Change-Id: I7145f840d9a7de34bb6a615fe992ba22e1ff0380
Signed-off-by: VENKATESH KUMAR <vv770d@att.com>
Issue-ID: DCAEGEN2-2024
Issue-ID: DCAEGEN2-1865
Signed-off-by: VENKATESH KUMAR <vv770d@att.com>

1 files changed, 0 insertions, 208 deletions

diff --git a/docs/sections/components/component-specification/cdap-specification.rst b/docs/sections/components/component-specification/cdap-specification.rst
deleted file mode 100755
index 2b26442a..00000000
--- a/docs/sections/components/component-specification/cdap-specification.rst
+++ /dev/null
@@ -1,208 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.

-.. http://creativecommons.org/licenses/by/4.0

-

-.. _cdap-specification:

-

-Component specification (CDAP)

-==============================

-

-The CDAP component specification contains the following groups of

-information. Many of these are common to both CDAP and Docker components

-and are therefore described in the common specification.

-

--  :any:`Metadata <metadata>`

--  :any:`Interfaces <interfaces>` including the

-   associated :any:`Data Formats <data-formats>`

--  :any:`Parameters <parameters>` - for specifying parameters in your

-   AppConfig, AppPreferences, and ProgramPreferences to the Designer and

-   Policy. This of course is CDAP-specific and is described below.

--  :any:`Auxiliary Details <auxiliary-details>`

--  :any:`List of artifacts <artifacts>`

-

-Current Limitations and TODOs

------------------------------

-

--  The integration of DMD is likely to significantly change the

-   :any:`Interfaces <interfaces>` section in this specification.

-

-.. _parameters:

-

-Parameters

-----------

-

-There is a ``parameters`` section in your component specification. This

-section contains three optional keys: `app_config <#appconfig>`__,

-`app_preferences <#apppreferences>`__, and

-`program_preferences <#programpreferences>`__:

-

-::

-

-    "parameters" : {

-        "app_config" : [ ...],               

-        "app_preferences" : [ ...],          

-        "program_preferences" : [...]

-        // any additional keys are ignored

-    }

-

--  Each section details the parameters that are a part of each of these

-   CDAP constructs (see below).

--  All such parameters will be exposed to the designer and to policy for

-   override.

--  These parameters should have default values specified by the

-   component developer where necessary, i.e., parameters that *must*

-   come from the designer/policy should not have defaults.

--  All of these keys are optional because not every CDAP application

-   uses preferences and not every application uses the AppConfig.

-   However, you should specify at least one, or else your application

-   will have no parameters exposed to policy or to the DCAE designer,

-   which means it would be non-configurable.

--  Despite the AppConfig being optional to *specify* in the case that

-   you have no parameters in your AppConfig, it is *required for

-   processing* in your application. That is because the DCAE platform

-   will place important information into your AppConfig as discussed

-   below.

-

-Parameter

-~~~~~~~~~

-

-The following CDAP specific definitions use ``param1`` to refer to the

-common parameter layout in

-:any:`Parameter <parameters>`

-

-AppConfig

-~~~~~~~~~

-

-The ``app_config`` key refers to the `CDAP AppConfig <http://docs.cask.co/cdap/current/en/reference-manual/http-restful-api/configuration.html>`_.

-It is expected to be a JSON:

-

-::

-

-    "app_config" : [ // list of JSON

-        param1,      // common parameter layout

-        ...

-    ]

-

-Unfortunately, at the time of writing, the AppConfig is a Java map of

-``string:string``, which means you cannot have more complex structures

-(than string) as any value in your AppConfig. However, there is a way to

-bypass this constraint: you can pass a JSON by encoding the JSON as a

-string. E.g., the ``json.dumps()`` and it’s converse ``loads`` methods

-in Python:

-

-::

-

-    >>> import json

-    >>> json.dumps({"foo" : "bar"})     # This is a real JSON

-    '{"foo": "bar"}'                    # It is now a string: pass this in as your parameter value

-    >>> json.loads('{"foo": "bar"}')    # Do the equivelent of this in your application

-    {u'foo': u'bar'}                    # ...and you'll get back a real JSON 

-    >>>

-

-The final AppConfig (after the designer and policy override parameter

-values) is passed into CDAP’s AppConfig API when starting the

-application.

-

-

-AppPreferences

-~~~~~~~~~~~~~~

-

-In addition to the CDAP AppConfig, the platform supports `Application Preferences <http://docs.cask.co/cdap/current/en/reference-manual/http-restful-api/preferences.html#set-preferences>`_.

-The format of the ``app_preferences`` value is the same as the above:

-

-::

-

-    "app_preferences" : [   // list of JSON

-        param1,      // common parameter layout

-        ...

-    ]

-

-The final Application Preferences JSON (after the designer and policy

-override parameter values) is passed into CDAP’s Preferences API when

-starting your application.

-

-

-ProgramPreferences

-~~~~~~~~~~~~~~~~~~

-

-Preferences can also be specified `per program <http://docs.cask.co/cdap/current/en/reference-manual/http-restful-api/lifecycle.html#program-lifecycle>`_

-in CDAP. This key’s value is a list of JSON with the following format:

-

-::

-

-    "program_preferences" : [                // note: this is a list of JSON 

-        {

-          "program_id" :   "program name 1",  // the name of this CDAP program

-          "program_type" : "e.g., flows",     // "must be one of flows, mapreduce, schedules, spark, workflows, workers, or services",

-          "program_pref" : [                  // list of JSON

-          param1,      // common parameter layout

-              ...

-          ]

-        },

-        // repeat for each program that receives a program_preferences JSON 

-    ]

-

-Each ``program_pref`` JSON is passed into the CDAP API as the preference

-for ``program_id``.

-

-NOTE: for CDAP, this section is very likely to change when DMD is

-available. The *future* vision is

-that you would publish your data as a series of files on HDFS, and DMD

-will pick them up and send them to the appropriate DMaaP feeds or

-directly when needed.

-

-.. _auxiliary-details:

-

-Auxiliary Details

------------------

-

-*auxiliary* contains details about CDAP specific parameters.

-

-+----------------------+----------------------+----------------------+

-| Property Name        | Type                 | Description          |

-+======================+======================+======================+

-| streamname           | string               | *Required*.          |

-+----------------------+----------------------+----------------------+

-| artifact_name        | string               |                      |

-+----------------------+----------------------+----------------------+

-| artifact_version     | string               | the version of your  |

-|                      |                      | CDAP JAR artifact    |

-+----------------------+----------------------+----------------------+

-| namespace            | string               | the CDAP namespace   |

-|                      |                      | to deploy into,      |

-|                      |                      | default is ‘default’ |

-+----------------------+----------------------+----------------------+

-| programs             | array                | contains each CDAP   |

-|                      |                      | entity represented   |

-|                      |                      | in the artifact      |

-+----------------------+----------------------+----------------------+

-| program_type         | string               | CDAP entity (eg      |

-|                      |                      | “flows”)             |

-+----------------------+----------------------+----------------------+

-| program_id           | string               | name of CDAP entity  |

-|                      |                      | (eg “WhoFlow”)       |

-+----------------------+----------------------+----------------------+

-

-Example:

-

-.. code:: json

-

-    "auxiliary": {

-        "streamname" : "who",

-        "artifact_name" : "HelloWorld",

-        "artifact_version" : "3.4.3",

-        "namespace" : "hw",

-        "programs" : [

-                        {"program_type" : "flows", "program_id" : "WhoFlow"}, 

-                        {"program_type" : "services", "program_id" : "Greeting"},

-                        ...

-                      ],

-    }

-

-The ``programs`` key is identical to the ``program_preferences`` key

-discussed `above <#programpreferences>`__ except:

-

--  each JSON in the list does not contain ``program_pref``

--  this is required! You must include all of your programs in this, as

-   it is used to start each program as well as for DCAE to perform

-   periodic healthchecks on your application. Don’t forget about your

-   services; they are programs too.