summaryrefslogtreecommitdiffstats
path: root/docs
diff options
context:
space:
mode:
authorLovett, Trevor <trevor.lovett@att.com>2019-09-04 12:14:04 -0500
committerLovett, Trevor (tl2972) <tl2972@att.com>2019-09-04 12:46:22 -0500
commit3326ce230c992382896905d9b7c642faa602319c (patch)
tree37f2121b8f36a2fc0bd7c89b5b98d68779b1c79b /docs
parent7766554b3057c8955ec8c0c15e4d9c62a83eb126 (diff)
[VVP] Adding documenation for preload generation
Issue-ID: VVP-279 Change-Id: Ibc30244603ddb2588348fecd51721ab11e955618 Signed-off-by: Lovett, Trevor <trevor.lovett@att.com> Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>
Diffstat (limited to 'docs')
-rw-r--r--docs/contributing.rst10
-rw-r--r--docs/humaninterfaces.rst35
-rw-r--r--docs/index.rst1
-rw-r--r--docs/preload-generation.rst363
-rw-r--r--docs/release-notes.rst1
-rw-r--r--docs/validation-scripts.rst38
-rw-r--r--docs/vvp_app.pngbin38936 -> 41675 bytes
7 files changed, 414 insertions, 34 deletions
diff --git a/docs/contributing.rst b/docs/contributing.rst
index 8b561af..2a3af02 100644
--- a/docs/contributing.rst
+++ b/docs/contributing.rst
@@ -6,7 +6,7 @@ How to Contribute
#################
Overview
-~~~~~~~~
+--------
This section will provide details on how to contribute changes to the project
covering both the mechanics of how to contribute new code as well as
@@ -14,7 +14,7 @@ how to adhere to the code quality and coding practices of the project.
Prerequisites
-~~~~~~~~~~~~~
+-------------
As this project is part of the overall ONAP project, there are some common
guidelines and activities you will need to adhere to:
@@ -41,7 +41,7 @@ Other useful links:
for scheduling details
Objective
-~~~~~~~~~
+---------
The primary focus of VVP is ensuring that a VNF that is described using Openstack
Heat complies with the ONAP Heat requirements specified in the `VNF Requirements (VNFRQTS) <https://onap.readthedocs.io/en/latest/submodules/vnfrqts/requirements.git/docs/index.html>`__
@@ -63,7 +63,7 @@ requirement, then please consider making a contribution to that project first.
Writing Tests
-#############
+-------------
Coding Conventions
~~~~~~~~~~~~~~~~~~
@@ -305,7 +305,7 @@ The tests should be compatible with ``pytest``, but these tests
do not use the fixtures mechanism.
Submitting Your Changes For Review
-##################################
+----------------------------------
Once you have completed your changes and tested they work as expected, then the
next step is to validate they are ready for submission. The ``checks.py``
diff --git a/docs/humaninterfaces.rst b/docs/humaninterfaces.rst
index 8f8ccc3..63e148a 100644
--- a/docs/humaninterfaces.rst
+++ b/docs/humaninterfaces.rst
@@ -64,13 +64,22 @@ This maps to the ``--category`` command-line option.
| | specifying ``--category=environment_file`` from the |
| | command-line. |
+----------------------+-------------------------------------------------------+
+| OpenStack Heat | When selected, will validate the Heat templates are |
+| Testing | valid per OpenStack specifications (using the |
+| | latest version of OpenStack available). If not |
+| | selected, then VVP will only validate that the Heat |
+| | is compliant with ONAP rules. |
++----------------------+-------------------------------------------------------+
+
Settings
~~~~~~~~
-**Verbosity** - Controls the level of comparison output displayed in the
-underlying pytest validation output. Default maps to pytest ``-v`` level of
-verbosity.
+**Preload Template** - Determines the format of the preload template that will
+be generated by VVP. The preload template can be completed to load per-instance
+values into SDNC. This allows the SDC model to be generic and re-used across
+environments. There are currently two formats supported by ONAP (VNF-API and
+GR-API)
**Report Format** - Controls the format of the output report generated after
validation. The options are: HTML (the default), Excel, and CSV. This is
@@ -90,22 +99,28 @@ a report of this single failure. This can be useful in reducing the number
of errors to sift through in these situations. De-selecting this option
is the equivalent of specifying ``--continue-on-failure`` as a command-line option.
+**Create Preloads from Env Files** - When selected, the Env Files file selection
+box will be enabled allow the user to select a directory of .env file definitions
+that can be used to populate a preload template. See
+:ref:`Preload Generation <vvp-preload-gen>` for more details
+
Running Validations
~~~~~~~~~~~~~~~~~~~
-1. Select the desired input format in the settings (ZIP or Directory)
-2. Select the [...] button next to the Template Location input box
-3. Select the directory or ZIP file containing the Heat templates, and then
+1. Select the desired preload format (VNF-API or GR-API)
+2. Select the desired input format in the settings (ZIP or Directory)
+3. Select the [...] button next to the Template Location input box
+4. Select the directory or ZIP file containing the Heat templates, and then
click Open
-4. Once the input is selected, select the "Validate Templates" button to
+5. Once the input is selected, select the "Validate Templates" button to
start the validation process. The white box to the right will
display output as the validations are executed.
-5. Once validation is complete a summary of pass and fail will be written
+6. Once validation is complete a summary of pass and fail will be written
to the output window, and a "View Report" option will appear on the
left-hand control panel.
-6. Select the "View Report" option, and the report will be opened in the
+7. Select the "View Report" option, and the report will be opened in the
the appropriate application based on report format.
-7. If you have questions about report output, please refer to the
+8. If you have questions about report output, please refer to the
:ref:`Validation Report reference material <vvp-report>` for more
information.
diff --git a/docs/index.rst b/docs/index.rst
index b381d9f..3683412 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -29,5 +29,6 @@ setup, and execute the validations.
installation.rst
validation-scripts.rst
humaninterfaces.rst
+ preload-generation.rst
contributing.rst
release-notes.rst
diff --git a/docs/preload-generation.rst b/docs/preload-generation.rst
new file mode 100644
index 0000000..85f67c8
--- /dev/null
+++ b/docs/preload-generation.rst
@@ -0,0 +1,363 @@
+.. This work is licensed under a Creative Commons Attribution 4.0 International License.
+.. http://creativecommons.org/licenses/by/4.0
+.. Copyright 2019 AT&T Intellectual Property. All rights reserved.
+
+.. _vvp-preload-gen:
+
+Preload Generation
+##################
+
+Overview
+--------
+
+To maximize the value of ONAP, the `ONAP Heat requirements <https://onap.readthedocs.io/en/latest/submodules/vnfrqts/requirements.git/docs/Chapter5/Heat/index.html>`__
+are defined to ensure that configuration that will vary per environment/instantiation
+should be defined as inputs into the Heat templates via parameters
+and excluded from ``.env`` files. This ensures that when a VNF described by
+a set of Heat templates is onboarded into SDC for modeling, the model remains
+generic and reusable across multiple environments.
+
+In order to instantiate a VNF and its VF Modules, ONAP must be able to
+supply the instance specific configurations when a module is instantiated. The
+most common method to achieve this is to register a **preload** for
+your VF module into SDNC. The preload is a JSON artifact that:
+
+1. Provides identifiers that map it to a specific VF Module in SDC
+2. Provides values that will be passed into the Heat templates' ``parameters``
+
+When ONAP instantiates the VF Module, SDNC will find the preload based on the
+**SDC Model Identifiers**, and then map the parameter values in the preload to
+the appropriate ``parameters`` in the Heat template.
+
+The validation tool/scripts will generate a "blank" preload template in the
+requested format (VNF-API or GR-API) that the end user will complete prior
+to :ref:`registering the preload to SDNC <vvp_register_preload>`.
+
+Optionally, the end user can specify a directory containing ``.env`` files
+and other special files (``defaults.yaml``, CSAR package) that can generate
+not just blank templates, but fully or partially completed preload templates.
+
+This section will describe how to perform these actions.
+
+Blank Preload Template Generation
+---------------------------------
+
+As part of normal validation, VVP will generate a blank preload template
+for the user to complete.
+
+If validating via the command line, then the
+resulting blank templates will be in the
+``<output-directory>/preloads/<preload-format>/`` directory. Where ``output-directory``
+is either the default output directory under ``ice_validator`` or a custom
+directory if the default was overridden by the ``--output-directory``
+parameter. If ``--preload-format`` was specified, then only that format will
+be produced; otherwise all available formats will be produced.
+
+If validating via the GUI, then a link to the preload templates will be
+provided at the end of the validation. The preload format will be in the
+format selected in the GUI's settings.
+
+**Populating the Preload Template**
+
+.. note::
+
+ The resulting JSON file(s) should be copied to a new directory to avoid
+ being over-written by subsequent validations.
+
+The preload template will be pre-populated based on the VMs, networks,
+sub-networks, IPs, availability zones, and parameters in the Heat templates.
+Every value that must be provided will be prefixed by ``VALUE FOR:`` and will
+include either the parameter name that the value will map to or will describe
+the value that should be provided if it is not related to a Heat parameter.
+
+.. code-block:: json
+ :caption: Example Partial VNF-API Preload
+
+ {
+ "availability-zones": [
+ {
+ "availability-zone": "VALUE FOR: availability_zone_0"
+ },
+ {
+ "availability-zone": "VALUE FOR: availability_zone_1"
+ }
+ ],
+ "vnf-networks": [
+ {
+ "network-role": "private",
+ "network-name": "VALUE FOR: network name for private_net_name"
+ },
+ {
+ "network-role": "oam",
+ "network-name": "VALUE FOR: network name for oam_net_id"
+ }
+ ],
+ "vnf-vms": ["OMITTED FOR THIS EXAMPLE"]
+ }
+
+There are instances where a parameter is defined ``comma-delimited-list`` in
+Heat. In these instances, the parameter name will be repeated for each value
+that needs to be provided. For example, if there are three virtual machines
+defined in the template and the names are pulled from a parameter called
+``{vm-type}_names``, then ``VALUE_FOR: {vm-type}_names`` will be repeated
+three times in the template.
+
+.. code-block:: json
+ :caption: Example: comma-delimited-list parameters
+ :emphasize-lines: 6-8
+
+ {
+ "vm-type": "admin",
+ "vm-count": 3,
+ "vm-names": {
+ "vm-name": [
+ "VALUE FOR: admin_names",
+ "VALUE FOR: admin_names",
+ "VALUE FOR: admin_names"
+ ]
+ }
+ }
+
+.. _vvp_preload_special_values:
+
+Special Values
+~~~~~~~~~~~~~~~~~~~~~
+
+Some values are not supplied to the the Heat template directly, but instead
+are used to either map the preload template to the SDC model **or** provide
+instance specific identifiers such as a module or VNF name. These will
+still be prefixed with ``VALUE FOR:``.
+
+.. note::
+
+ Refer to the section below on :ref:`Preload Template Population <vvp_preload_population>`
+ for alternate ways to populate this information.
+
+The values are:
+
++-----------------+-------------+--------------+-------------------------------+
+| Value | VNF-API | GR-API | Source & Description |
+| | Name | Name | |
++=================+=============+==============+===============================+
+| VNF Name | vnf-name | vnf-name | Name of the VNF as it will |
+| | | | appear in A&AI. This is user |
+| | | | defined and does not need to |
+| | | | match a value in SDC. |
++-----------------+-------------+--------------+-----------+-------------------+
+| Virtual | generic-vnf-| vnf-type | Maps preload to a specific |
+| Function | type | | instance of the VF model from |
+| Instance | | | SDC. This field is must be in |
+| | | | the format of: |
+| | | | <Service Name>/<VF Instance |
+| | | | Name> and must match SDC |
++-----------------+-------------+--------------+-----------+-------------------+
+| VF | vnf-type | vf-module- | Maps preload to the |
+| Module | | type | ``vfModuleModelName`` from |
+| Model | | | the VSP CSAR or the value |
+| | | | from SDC |
++-----------------+-------------+--------------+-----------+-------------------+
+
+
+.. _vvp_preload_population:
+
+Preload Template Population
+---------------------------
+
+Basic Usage - Single Environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+VVP can also generate fully or partially populated preload templates if a
+an optional environment directory is provided as a source for the parameter
+values.
+
+When using the command line, the environment directory is provided using the
+``--env-directory`` parameter.
+
+When using the GUI, the environment directory is provided by first selecting
+the **Create Preloads from Env Files** option in **Settings** and then providing
+the directory in the **Env Files** field.
+
+A template for the environment directory is created in the output directory
+under ``preloads/<preload-format>/preload_env``. There will be an ``*.env`` file
+for every VF module in the Heat template and a ``defaults.yaml``. This
+directory can be copied and updated to serve as a data source for populating
+the preload templates. Every value that needs to be updated will be set to
+``CHANGEME``.
+
+.. code-block:: yaml
+ :caption: Example Partial env file (``base.env``)
+
+ parameters:
+ ctrl_net_id: CHANGEME
+ ctrl_subnet_id: CHANGEME
+ db_ha_floating_ip: CHANGEME
+ db_ha_floating_v6_ip: CHANGEME
+ svc_flavor_name: svc_flavor
+ svc_image_name: svc_image
+ db_name_0: CHANGEME
+
+Each environment directory consists of three file types:
+
+1. **Environment files (.env)** - One file per VF module with the base name of
+ the env file matching the base name of the heat template it corresponds to.
+2. **defaults.yaml** - Values specified in here will be used in all modules.
+ This is a useful place to put values that will be the same in every VF
+ module. It is important to note that the environment files take precedence
+ so if you specify a value in ``defaults.yaml``, then that value should be
+ removed from the environment files.
+3. **VSP CSAR** - Optionally the CSAR can be downloaded from the SDC Artifacts
+ section, and put into the directory. If present, then the
+ :ref:`SDC Model Identifiers <vvp_preload_special_values>` will be pulled from
+ the CSAR instead of them being manually specified
+
+After template processing completes, the blank preload templates will still
+be generated to the output directory. The **populated** templates will be
+generated the ``<env-directory>/preloads/<preload-format>`` directory. If a
+template was not fully populated, then it will be suffixed with ``_incomplete``
+(ex: ``base_incomplete.json``)
+
+
+Advanced Usage - Multiple Environments
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Using a single environment directory is the most basic use case, but you may
+need to generate preloads for multiple environments where some values do not
+change by environment while others do.
+
+To enable this capability, environment directories can be nested and inherit
+values from their parent directories. The parent directory's environment files
+and ``defaults.yaml`` files should contain the default values for every
+environment.
+
+These can be overridden in the child environment directories so
+if you want to force each environment to provide a value, then do not specify
+that value in any parent directory.
+
+.. code-block:: none
+ :caption: Example Nested Environment Directory
+
+ env_directory/
+ |--- vsp_name.csar <-- Global CSAR for all enviroments (assumes shared SDC instance)
+ |--- defaults.yaml <-- Global defaults for all env and modules
+ |--- base.env <-- Global defaults for base modules
+ |--- mod_one.env <-- Global defaults for mod_one modules
+ |--- env_one/
+ | |--- defaults.yaml <-- env_one specific defaults for all modules
+ | |--- base.env <-- env_one values for the base module
+ | |--- mod_one.env <-- env_one values for mod_one
+ |--- env_two/
+ |--- defaults.yaml <-- env_two specific defaults for all modules
+ |--- base.env <-- env_two values for the base module
+ |--- mod_one.env <-- env_two values for mod_one
+
+
+Preload templates will be generated in the **leaf** directories of the environment
+directory. In this example, preload templates will be generated in ``env_one``
+and ``env_two``, but **not** ``env_directory``.
+
+The value supplied to the preload will follow the following order of precedence
+with the value being looked up from each of the following in turn and halting
+once the value is found:
+
+1. Corresponding ``.env`` file for the module in the environment specific
+ directory (ex: ``env_one/base.env``)
+2. The ``defaults.yaml`` file in the environment specific directory (ex:
+ ``env_one/defaults.yaml``)
+3. The CSAR file in the environment specific directory (NOTE: only the
+ **special values** are looked up from the CSAR
+4. If not found in the environment specific, then the parent directory will be
+ searched using the same precedence (1-3).
+
+This lookup chain will continue until the root environment directory is reached.
+If no value is found, then the template will continue revert the value from
+the blank template (i.e. ``VALUE FOR: {parameter-name}``)
+
+The environment directories can be nested as deeply as needed to map your needs.
+For example you could have a global directory, production and test environments,
+and then multiple sub-environments in each test nd production directory.
+
+**Example:**
+
+.. code-block:: none
+ :caption: Example Nested Environment Directory
+
+ env_directory/
+ |--- vsp_name.csar
+ |--- defaults.yaml
+ |--- base.env
+ |--- mod_one.env
+ |--- production/
+ | |--- defaults.yaml
+ | |--- base.env
+ | |--- mod_one.env
+ | |--- prod_one/
+ | | |--- defaults.yaml
+ | | |--- base.env
+ | | |--- mod_one.env
+ | |--- prod_two/
+ | |--- defaults.yaml
+ | |--- base.env
+ | |--- mod_one.env
+ |--- test/
+ |--- defaults.yaml
+ |--- base.env
+ |--- mod_one.env
+ |--- test_one/
+ | |--- defaults.yaml
+ | |--- base.env
+ | |--- mod_one.env
+ |--- test_two/
+ |--- defaults.yaml
+ |--- base.env
+ |--- mod_one.env
+
+Alternate Method for Specifying Special Values
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If you wish to populate the special values without providing a CSAR, then add
+the following parameters and values to either your ``defaults.yaml`` file or
+the appropriate ``.env`` file.
+
+* ``vnf-name`` - Corresponds to VNF Name in :ref:`special values <vvp_preload_special_values>`
+* ``vnf-type`` - Corresponds to Virtual Function Instance in :ref:`special values <vvp_preload_special_values>`
+* ``vf-module-model-name`` - Corresponds to VF Module Model Name in :ref:`special values <vvp_preload_special_values>`
+
+
+.. _vvp_register_preload:
+
+Registering Preload with SDNC
+-----------------------------
+
+At the time of this writing, the ONAP documentation does not provide a good
+source of documentation on how to provide preloads to SDNC. The options known
+are to either use a REST ``POST`` call to the following APIs based on format:
+
+.. note::
+
+ Host and port numbers may vary in your ONAP environment
+
+* **GR-API** - https://sdnc.api.simpledemo.onap.org:30267/restconf/operations/GENERIC-RESOURCE-API:preload-vf-module-topology-operation
+* **VNF-API** - https://sdnc.api.simpledemo.onap.org:30267/restconf/operations/GENERIC-RESOURCE-API:preload-vf-module-topology-operation
+
+The ONAP-CLI project could also be used, but it only supports the **VNF-API**
+as of the writing of this document.
+
+Customizing Preload Generation
+------------------------------
+
+VVP's preload generation capability leverages a plugin mechanism to enable additional
+preload formats to be added. This can be useful if you define an intermediary
+format such as a spreadsheet to capture the preload information.
+
+Preload plugins are discovered using the following method.
+
+1. The ``sys.path`` is scanned to find any top level modules that begin with
+ ``preload_``
+2. If found, then any implementations of ``AbstractPreloadGenerator`` are
+ registered as available formats
+3. You can selectively disable formats by specifying them as excluded in the
+ ``vvp-config.yaml`` file.
+
+Please refer to the ``preload_vnfapi.VnfApiPreloadGenerator`` and
+``preload_grapi.GrApiPreloadGenerator`` for examples of how to implement a
+generator.
diff --git a/docs/release-notes.rst b/docs/release-notes.rst
index 2fdb5fa..24ae5ea 100644
--- a/docs/release-notes.rst
+++ b/docs/release-notes.rst
@@ -49,6 +49,7 @@ Version: 5.1.0
**Bug Fixes**
+
- Fixed errors in ``test_environment_file_parameters`` where wrong variables
were being checked (``VVP-267``)
- VVP GUI fails to open reports when tools is launched from a network share
diff --git a/docs/validation-scripts.rst b/docs/validation-scripts.rst
index b327ad4..9de4502 100644
--- a/docs/validation-scripts.rst
+++ b/docs/validation-scripts.rst
@@ -65,6 +65,18 @@ that are used by VVP::
--output-directory=OUTPUT_DIR
Alternate directory for report output.
+ --category=TEST_CATEGORIES
+ optional category of test to execute
+
+ --env-directory=ENV_DIR
+ optional directory of .env files for preload
+ generation
+
+ --preload-format=PRELOAD_FORMATS
+ Preload format to create (multiple allowed). If not
+ provided then all available formats will be created:
+ GR-API, VNF-API
+
.. _vvp-reports:
VVP Reports
@@ -99,6 +111,13 @@ ouput directory (specified via ``--output-directory`` or
| ``failures`` | **Deprecated** JSON version of test failures. Use |
| | ``report.json`` instead. |
+-----------------------+------------------------------------------------------+
+|``preloads/<format>/*``| A blank preload will be created for every VF Module |
+| | in the template directory. The ``<format>`` |
+| | will be based on the preload format(s) selected. |
+| | See :ref:`Preload Generation <vvp-preload-gen>` for |
+| | for more detail. |
++-----------------------+------------------------------------------------------+
+
.. _vvp-report:
@@ -341,22 +360,3 @@ written on your local machine::
The same :ref:`command line options <vvp-cmd-options>` can be used with the
Docker image that are used with the version from source.
-
-Self-Test Suite
----------------
-
-The ``ice_validator`` includes an extensive self-test suite. It is a
-**requirement** for any additions or changes to the test suite to
-successfully and cleanly complete a tox run. Simply run ``tox`` from
-the project root as:
-
-``$ tox``
-
-You can also run it under the folder ``ice_validator``:
-
-``$ pytest --self-test``
-
-
-
-
-
diff --git a/docs/vvp_app.png b/docs/vvp_app.png
index 8a27249..b33f840 100644
--- a/docs/vvp_app.png
+++ b/docs/vvp_app.png
Binary files differ