diff options
| -rw-r--r-- | docs/contributing.rst | 10 | ||||
| -rw-r--r-- | docs/humaninterfaces.rst | 35 | ||||
| -rw-r--r-- | docs/index.rst | 1 | ||||
| -rw-r--r-- | docs/preload-generation.rst | 363 | ||||
| -rw-r--r-- | docs/release-notes.rst | 1 | ||||
| -rw-r--r-- | docs/validation-scripts.rst | 38 | ||||
| -rw-r--r-- | docs/vvp_app.png | bin | 38936 -> 41675 bytes |
7 files changed, 414 insertions, 34 deletions
diff --git a/docs/contributing.rst b/docs/contributing.rst index 8b561af..2a3af02 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -6,7 +6,7 @@ How to Contribute ################# Overview -~~~~~~~~ +-------- This section will provide details on how to contribute changes to the project covering both the mechanics of how to contribute new code as well as @@ -14,7 +14,7 @@ how to adhere to the code quality and coding practices of the project. Prerequisites -~~~~~~~~~~~~~ +------------- As this project is part of the overall ONAP project, there are some common guidelines and activities you will need to adhere to: @@ -41,7 +41,7 @@ Other useful links: for scheduling details Objective -~~~~~~~~~ +--------- The primary focus of VVP is ensuring that a VNF that is described using Openstack Heat complies with the ONAP Heat requirements specified in the `VNF Requirements (VNFRQTS) <https://onap.readthedocs.io/en/latest/submodules/vnfrqts/requirements.git/docs/index.html>`__ @@ -63,7 +63,7 @@ requirement, then please consider making a contribution to that project first. Writing Tests -############# +------------- Coding Conventions ~~~~~~~~~~~~~~~~~~ @@ -305,7 +305,7 @@ The tests should be compatible with ``pytest``, but these tests do not use the fixtures mechanism. Submitting Your Changes For Review -################################## +---------------------------------- Once you have completed your changes and tested they work as expected, then the next step is to validate they are ready for submission. The ``checks.py`` diff --git a/docs/humaninterfaces.rst b/docs/humaninterfaces.rst index 8f8ccc3..63e148a 100644 --- a/docs/humaninterfaces.rst +++ b/docs/humaninterfaces.rst @@ -64,13 +64,22 @@ This maps to the ``--category`` command-line option. | | specifying ``--category=environment_file`` from the | | | command-line. | +----------------------+-------------------------------------------------------+ +| OpenStack Heat | When selected, will validate the Heat templates are | +| Testing | valid per OpenStack specifications (using the | +| | latest version of OpenStack available). If not | +| | selected, then VVP will only validate that the Heat | +| | is compliant with ONAP rules. | ++----------------------+-------------------------------------------------------+ + Settings ~~~~~~~~ -**Verbosity** - Controls the level of comparison output displayed in the -underlying pytest validation output. Default maps to pytest ``-v`` level of -verbosity. +**Preload Template** - Determines the format of the preload template that will +be generated by VVP. The preload template can be completed to load per-instance +values into SDNC. This allows the SDC model to be generic and re-used across +environments. There are currently two formats supported by ONAP (VNF-API and +GR-API) **Report Format** - Controls the format of the output report generated after validation. The options are: HTML (the default), Excel, and CSV. This is @@ -90,22 +99,28 @@ a report of this single failure. This can be useful in reducing the number of errors to sift through in these situations. De-selecting this option is the equivalent of specifying ``--continue-on-failure`` as a command-line option. +**Create Preloads from Env Files** - When selected, the Env Files file selection +box will be enabled allow the user to select a directory of .env file definitions +that can be used to populate a preload template. See +:ref:`Preload Generation <vvp-preload-gen>` for more details + Running Validations ~~~~~~~~~~~~~~~~~~~ -1. Select the desired input format in the settings (ZIP or Directory) -2. Select the [...] button next to the Template Location input box -3. Select the directory or ZIP file containing the Heat templates, and then +1. Select the desired preload format (VNF-API or GR-API) +2. Select the desired input format in the settings (ZIP or Directory) +3. Select the [...] button next to the Template Location input box +4. Select the directory or ZIP file containing the Heat templates, and then click Open -4. Once the input is selected, select the "Validate Templates" button to +5. Once the input is selected, select the "Validate Templates" button to start the validation process. The white box to the right will display output as the validations are executed. -5. Once validation is complete a summary of pass and fail will be written +6. Once validation is complete a summary of pass and fail will be written to the output window, and a "View Report" option will appear on the left-hand control panel. -6. Select the "View Report" option, and the report will be opened in the +7. Select the "View Report" option, and the report will be opened in the the appropriate application based on report format. -7. If you have questions about report output, please refer to the +8. If you have questions about report output, please refer to the :ref:`Validation Report reference material <vvp-report>` for more information. diff --git a/docs/index.rst b/docs/index.rst index b381d9f..3683412 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -29,5 +29,6 @@ setup, and execute the validations. installation.rst validation-scripts.rst humaninterfaces.rst + preload-generation.rst contributing.rst release-notes.rst diff --git a/docs/preload-generation.rst b/docs/preload-generation.rst new file mode 100644 index 0000000..85f67c8 --- /dev/null +++ b/docs/preload-generation.rst @@ -0,0 +1,363 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. Copyright 2019 AT&T Intellectual Property. All rights reserved. + +.. _vvp-preload-gen: + +Preload Generation +################## + +Overview +-------- + +To maximize the value of ONAP, the `ONAP Heat requirements <https://onap.readthedocs.io/en/latest/submodules/vnfrqts/requirements.git/docs/Chapter5/Heat/index.html>`__ +are defined to ensure that configuration that will vary per environment/instantiation +should be defined as inputs into the Heat templates via parameters +and excluded from ``.env`` files. This ensures that when a VNF described by +a set of Heat templates is onboarded into SDC for modeling, the model remains +generic and reusable across multiple environments. + +In order to instantiate a VNF and its VF Modules, ONAP must be able to +supply the instance specific configurations when a module is instantiated. The +most common method to achieve this is to register a **preload** for +your VF module into SDNC. The preload is a JSON artifact that: + +1. Provides identifiers that map it to a specific VF Module in SDC +2. Provides values that will be passed into the Heat templates' ``parameters`` + +When ONAP instantiates the VF Module, SDNC will find the preload based on the +**SDC Model Identifiers**, and then map the parameter values in the preload to +the appropriate ``parameters`` in the Heat template. + +The validation tool/scripts will generate a "blank" preload template in the +requested format (VNF-API or GR-API) that the end user will complete prior +to :ref:`registering the preload to SDNC <vvp_register_preload>`. + +Optionally, the end user can specify a directory containing ``.env`` files +and other special files (``defaults.yaml``, CSAR package) that can generate +not just blank templates, but fully or partially completed preload templates. + +This section will describe how to perform these actions. + +Blank Preload Template Generation +--------------------------------- + +As part of normal validation, VVP will generate a blank preload template +for the user to complete. + +If validating via the command line, then the +resulting blank templates will be in the +``<output-directory>/preloads/<preload-format>/`` directory. Where ``output-directory`` +is either the default output directory under ``ice_validator`` or a custom +directory if the default was overridden by the ``--output-directory`` +parameter. If ``--preload-format`` was specified, then only that format will +be produced; otherwise all available formats will be produced. + +If validating via the GUI, then a link to the preload templates will be +provided at the end of the validation. The preload format will be in the +format selected in the GUI's settings. + +**Populating the Preload Template** + +.. note:: + + The resulting JSON file(s) should be copied to a new directory to avoid + being over-written by subsequent validations. + +The preload template will be pre-populated based on the VMs, networks, +sub-networks, IPs, availability zones, and parameters in the Heat templates. +Every value that must be provided will be prefixed by ``VALUE FOR:`` and will +include either the parameter name that the value will map to or will describe +the value that should be provided if it is not related to a Heat parameter. + +.. code-block:: json + :caption: Example Partial VNF-API Preload + + { + "availability-zones": [ + { + "availability-zone": "VALUE FOR: availability_zone_0" + }, + { + "availability-zone": "VALUE FOR: availability_zone_1" + } + ], + "vnf-networks": [ + { + "network-role": "private", + "network-name": "VALUE FOR: network name for private_net_name" + }, + { + "network-role": "oam", + "network-name": "VALUE FOR: network name for oam_net_id" + } + ], + "vnf-vms": ["OMITTED FOR THIS EXAMPLE"] + } + +There are instances where a parameter is defined ``comma-delimited-list`` in +Heat. In these instances, the parameter name will be repeated for each value +that needs to be provided. For example, if there are three virtual machines +defined in the template and the names are pulled from a parameter called +``{vm-type}_names``, then ``VALUE_FOR: {vm-type}_names`` will be repeated +three times in the template. + +.. code-block:: json + :caption: Example: comma-delimited-list parameters + :emphasize-lines: 6-8 + + { + "vm-type": "admin", + "vm-count": 3, + "vm-names": { + "vm-name": [ + "VALUE FOR: admin_names", + "VALUE FOR: admin_names", + "VALUE FOR: admin_names" + ] + } + } + +.. _vvp_preload_special_values: + +Special Values +~~~~~~~~~~~~~~~~~~~~~ + +Some values are not supplied to the the Heat template directly, but instead +are used to either map the preload template to the SDC model **or** provide +instance specific identifiers such as a module or VNF name. These will +still be prefixed with ``VALUE FOR:``. + +.. note:: + + Refer to the section below on :ref:`Preload Template Population <vvp_preload_population>` + for alternate ways to populate this information. + +The values are: + ++-----------------+-------------+--------------+-------------------------------+ +| Value | VNF-API | GR-API | Source & Description | +| | Name | Name | | ++=================+=============+==============+===============================+ +| VNF Name | vnf-name | vnf-name | Name of the VNF as it will | +| | | | appear in A&AI. This is user | +| | | | defined and does not need to | +| | | | match a value in SDC. | ++-----------------+-------------+--------------+-----------+-------------------+ +| Virtual | generic-vnf-| vnf-type | Maps preload to a specific | +| Function | type | | instance of the VF model from | +| Instance | | | SDC. This field is must be in | +| | | | the format of: | +| | | | <Service Name>/<VF Instance | +| | | | Name> and must match SDC | ++-----------------+-------------+--------------+-----------+-------------------+ +| VF | vnf-type | vf-module- | Maps preload to the | +| Module | | type | ``vfModuleModelName`` from | +| Model | | | the VSP CSAR or the value | +| | | | from SDC | ++-----------------+-------------+--------------+-----------+-------------------+ + + +.. _vvp_preload_population: + +Preload Template Population +--------------------------- + +Basic Usage - Single Environment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +VVP can also generate fully or partially populated preload templates if a +an optional environment directory is provided as a source for the parameter +values. + +When using the command line, the environment directory is provided using the +``--env-directory`` parameter. + +When using the GUI, the environment directory is provided by first selecting +the **Create Preloads from Env Files** option in **Settings** and then providing +the directory in the **Env Files** field. + +A template for the environment directory is created in the output directory +under ``preloads/<preload-format>/preload_env``. There will be an ``*.env`` file +for every VF module in the Heat template and a ``defaults.yaml``. This +directory can be copied and updated to serve as a data source for populating +the preload templates. Every value that needs to be updated will be set to +``CHANGEME``. + +.. code-block:: yaml + :caption: Example Partial env file (``base.env``) + + parameters: + ctrl_net_id: CHANGEME + ctrl_subnet_id: CHANGEME + db_ha_floating_ip: CHANGEME + db_ha_floating_v6_ip: CHANGEME + svc_flavor_name: svc_flavor + svc_image_name: svc_image + db_name_0: CHANGEME + +Each environment directory consists of three file types: + +1. **Environment files (.env)** - One file per VF module with the base name of + the env file matching the base name of the heat template it corresponds to. +2. **defaults.yaml** - Values specified in here will be used in all modules. + This is a useful place to put values that will be the same in every VF + module. It is important to note that the environment files take precedence + so if you specify a value in ``defaults.yaml``, then that value should be + removed from the environment files. +3. **VSP CSAR** - Optionally the CSAR can be downloaded from the SDC Artifacts + section, and put into the directory. If present, then the + :ref:`SDC Model Identifiers <vvp_preload_special_values>` will be pulled from + the CSAR instead of them being manually specified + +After template processing completes, the blank preload templates will still +be generated to the output directory. The **populated** templates will be +generated the ``<env-directory>/preloads/<preload-format>`` directory. If a +template was not fully populated, then it will be suffixed with ``_incomplete`` +(ex: ``base_incomplete.json``) + + +Advanced Usage - Multiple Environments +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Using a single environment directory is the most basic use case, but you may +need to generate preloads for multiple environments where some values do not +change by environment while others do. + +To enable this capability, environment directories can be nested and inherit +values from their parent directories. The parent directory's environment files +and ``defaults.yaml`` files should contain the default values for every +environment. + +These can be overridden in the child environment directories so +if you want to force each environment to provide a value, then do not specify +that value in any parent directory. + +.. code-block:: none + :caption: Example Nested Environment Directory + + env_directory/ + |--- vsp_name.csar <-- Global CSAR for all enviroments (assumes shared SDC instance) + |--- defaults.yaml <-- Global defaults for all env and modules + |--- base.env <-- Global defaults for base modules + |--- mod_one.env <-- Global defaults for mod_one modules + |--- env_one/ + | |--- defaults.yaml <-- env_one specific defaults for all modules + | |--- base.env <-- env_one values for the base module + | |--- mod_one.env <-- env_one values for mod_one + |--- env_two/ + |--- defaults.yaml <-- env_two specific defaults for all modules + |--- base.env <-- env_two values for the base module + |--- mod_one.env <-- env_two values for mod_one + + +Preload templates will be generated in the **leaf** directories of the environment +directory. In this example, preload templates will be generated in ``env_one`` +and ``env_two``, but **not** ``env_directory``. + +The value supplied to the preload will follow the following order of precedence +with the value being looked up from each of the following in turn and halting +once the value is found: + +1. Corresponding ``.env`` file for the module in the environment specific + directory (ex: ``env_one/base.env``) +2. The ``defaults.yaml`` file in the environment specific directory (ex: + ``env_one/defaults.yaml``) +3. The CSAR file in the environment specific directory (NOTE: only the + **special values** are looked up from the CSAR +4. If not found in the environment specific, then the parent directory will be + searched using the same precedence (1-3). + +This lookup chain will continue until the root environment directory is reached. +If no value is found, then the template will continue revert the value from +the blank template (i.e. ``VALUE FOR: {parameter-name}``) + +The environment directories can be nested as deeply as needed to map your needs. +For example you could have a global directory, production and test environments, +and then multiple sub-environments in each test nd production directory. + +**Example:** + +.. code-block:: none + :caption: Example Nested Environment Directory + + env_directory/ + |--- vsp_name.csar + |--- defaults.yaml + |--- base.env + |--- mod_one.env + |--- production/ + | |--- defaults.yaml + | |--- base.env + | |--- mod_one.env + | |--- prod_one/ + | | |--- defaults.yaml + | | |--- base.env + | | |--- mod_one.env + | |--- prod_two/ + | |--- defaults.yaml + | |--- base.env + | |--- mod_one.env + |--- test/ + |--- defaults.yaml + |--- base.env + |--- mod_one.env + |--- test_one/ + | |--- defaults.yaml + | |--- base.env + | |--- mod_one.env + |--- test_two/ + |--- defaults.yaml + |--- base.env + |--- mod_one.env + +Alternate Method for Specifying Special Values +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you wish to populate the special values without providing a CSAR, then add +the following parameters and values to either your ``defaults.yaml`` file or +the appropriate ``.env`` file. + +* ``vnf-name`` - Corresponds to VNF Name in :ref:`special values <vvp_preload_special_values>` +* ``vnf-type`` - Corresponds to Virtual Function Instance in :ref:`special values <vvp_preload_special_values>` +* ``vf-module-model-name`` - Corresponds to VF Module Model Name in :ref:`special values <vvp_preload_special_values>` + + +.. _vvp_register_preload: + +Registering Preload with SDNC +----------------------------- + +At the time of this writing, the ONAP documentation does not provide a good +source of documentation on how to provide preloads to SDNC. The options known +are to either use a REST ``POST`` call to the following APIs based on format: + +.. note:: + + Host and port numbers may vary in your ONAP environment + +* **GR-API** - https://sdnc.api.simpledemo.onap.org:30267/restconf/operations/GENERIC-RESOURCE-API:preload-vf-module-topology-operation +* **VNF-API** - https://sdnc.api.simpledemo.onap.org:30267/restconf/operations/GENERIC-RESOURCE-API:preload-vf-module-topology-operation + +The ONAP-CLI project could also be used, but it only supports the **VNF-API** +as of the writing of this document. + +Customizing Preload Generation +------------------------------ + +VVP's preload generation capability leverages a plugin mechanism to enable additional +preload formats to be added. This can be useful if you define an intermediary +format such as a spreadsheet to capture the preload information. + +Preload plugins are discovered using the following method. + +1. The ``sys.path`` is scanned to find any top level modules that begin with + ``preload_`` +2. If found, then any implementations of ``AbstractPreloadGenerator`` are + registered as available formats +3. You can selectively disable formats by specifying them as excluded in the + ``vvp-config.yaml`` file. + +Please refer to the ``preload_vnfapi.VnfApiPreloadGenerator`` and +``preload_grapi.GrApiPreloadGenerator`` for examples of how to implement a +generator. diff --git a/docs/release-notes.rst b/docs/release-notes.rst index 2fdb5fa..24ae5ea 100644 --- a/docs/release-notes.rst +++ b/docs/release-notes.rst @@ -49,6 +49,7 @@ Version: 5.1.0 **Bug Fixes** + - Fixed errors in ``test_environment_file_parameters`` where wrong variables were being checked (``VVP-267``) - VVP GUI fails to open reports when tools is launched from a network share diff --git a/docs/validation-scripts.rst b/docs/validation-scripts.rst index b327ad4..9de4502 100644 --- a/docs/validation-scripts.rst +++ b/docs/validation-scripts.rst @@ -65,6 +65,18 @@ that are used by VVP:: --output-directory=OUTPUT_DIR Alternate directory for report output. + --category=TEST_CATEGORIES + optional category of test to execute + + --env-directory=ENV_DIR + optional directory of .env files for preload + generation + + --preload-format=PRELOAD_FORMATS + Preload format to create (multiple allowed). If not + provided then all available formats will be created: + GR-API, VNF-API + .. _vvp-reports: VVP Reports @@ -99,6 +111,13 @@ ouput directory (specified via ``--output-directory`` or | ``failures`` | **Deprecated** JSON version of test failures. Use | | | ``report.json`` instead. | +-----------------------+------------------------------------------------------+ +|``preloads/<format>/*``| A blank preload will be created for every VF Module | +| | in the template directory. The ``<format>`` | +| | will be based on the preload format(s) selected. | +| | See :ref:`Preload Generation <vvp-preload-gen>` for | +| | for more detail. | ++-----------------------+------------------------------------------------------+ + .. _vvp-report: @@ -341,22 +360,3 @@ written on your local machine:: The same :ref:`command line options <vvp-cmd-options>` can be used with the Docker image that are used with the version from source. - -Self-Test Suite ---------------- - -The ``ice_validator`` includes an extensive self-test suite. It is a -**requirement** for any additions or changes to the test suite to -successfully and cleanly complete a tox run. Simply run ``tox`` from -the project root as: - -``$ tox`` - -You can also run it under the folder ``ice_validator``: - -``$ pytest --self-test`` - - - - - diff --git a/docs/vvp_app.png b/docs/vvp_app.png Binary files differindex 8a27249..b33f840 100644 --- a/docs/vvp_app.png +++ b/docs/vvp_app.png |
