diff options
Diffstat (limited to 'docs/preload-generation.rst')
| -rw-r--r-- | docs/preload-generation.rst | 363 |
1 files changed, 363 insertions, 0 deletions
diff --git a/docs/preload-generation.rst b/docs/preload-generation.rst new file mode 100644 index 0000000..85f67c8 --- /dev/null +++ b/docs/preload-generation.rst @@ -0,0 +1,363 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 +.. Copyright 2019 AT&T Intellectual Property. All rights reserved. + +.. _vvp-preload-gen: + +Preload Generation +################## + +Overview +-------- + +To maximize the value of ONAP, the `ONAP Heat requirements <https://onap.readthedocs.io/en/latest/submodules/vnfrqts/requirements.git/docs/Chapter5/Heat/index.html>`__ +are defined to ensure that configuration that will vary per environment/instantiation +should be defined as inputs into the Heat templates via parameters +and excluded from ``.env`` files. This ensures that when a VNF described by +a set of Heat templates is onboarded into SDC for modeling, the model remains +generic and reusable across multiple environments. + +In order to instantiate a VNF and its VF Modules, ONAP must be able to +supply the instance specific configurations when a module is instantiated. The +most common method to achieve this is to register a **preload** for +your VF module into SDNC. The preload is a JSON artifact that: + +1. Provides identifiers that map it to a specific VF Module in SDC +2. Provides values that will be passed into the Heat templates' ``parameters`` + +When ONAP instantiates the VF Module, SDNC will find the preload based on the +**SDC Model Identifiers**, and then map the parameter values in the preload to +the appropriate ``parameters`` in the Heat template. + +The validation tool/scripts will generate a "blank" preload template in the +requested format (VNF-API or GR-API) that the end user will complete prior +to :ref:`registering the preload to SDNC <vvp_register_preload>`. + +Optionally, the end user can specify a directory containing ``.env`` files +and other special files (``defaults.yaml``, CSAR package) that can generate +not just blank templates, but fully or partially completed preload templates. + +This section will describe how to perform these actions. + +Blank Preload Template Generation +--------------------------------- + +As part of normal validation, VVP will generate a blank preload template +for the user to complete. + +If validating via the command line, then the +resulting blank templates will be in the +``<output-directory>/preloads/<preload-format>/`` directory. Where ``output-directory`` +is either the default output directory under ``ice_validator`` or a custom +directory if the default was overridden by the ``--output-directory`` +parameter. If ``--preload-format`` was specified, then only that format will +be produced; otherwise all available formats will be produced. + +If validating via the GUI, then a link to the preload templates will be +provided at the end of the validation. The preload format will be in the +format selected in the GUI's settings. + +**Populating the Preload Template** + +.. note:: + + The resulting JSON file(s) should be copied to a new directory to avoid + being over-written by subsequent validations. + +The preload template will be pre-populated based on the VMs, networks, +sub-networks, IPs, availability zones, and parameters in the Heat templates. +Every value that must be provided will be prefixed by ``VALUE FOR:`` and will +include either the parameter name that the value will map to or will describe +the value that should be provided if it is not related to a Heat parameter. + +.. code-block:: json + :caption: Example Partial VNF-API Preload + + { + "availability-zones": [ + { + "availability-zone": "VALUE FOR: availability_zone_0" + }, + { + "availability-zone": "VALUE FOR: availability_zone_1" + } + ], + "vnf-networks": [ + { + "network-role": "private", + "network-name": "VALUE FOR: network name for private_net_name" + }, + { + "network-role": "oam", + "network-name": "VALUE FOR: network name for oam_net_id" + } + ], + "vnf-vms": ["OMITTED FOR THIS EXAMPLE"] + } + +There are instances where a parameter is defined ``comma-delimited-list`` in +Heat. In these instances, the parameter name will be repeated for each value +that needs to be provided. For example, if there are three virtual machines +defined in the template and the names are pulled from a parameter called +``{vm-type}_names``, then ``VALUE_FOR: {vm-type}_names`` will be repeated +three times in the template. + +.. code-block:: json + :caption: Example: comma-delimited-list parameters + :emphasize-lines: 6-8 + + { + "vm-type": "admin", + "vm-count": 3, + "vm-names": { + "vm-name": [ + "VALUE FOR: admin_names", + "VALUE FOR: admin_names", + "VALUE FOR: admin_names" + ] + } + } + +.. _vvp_preload_special_values: + +Special Values +~~~~~~~~~~~~~~~~~~~~~ + +Some values are not supplied to the the Heat template directly, but instead +are used to either map the preload template to the SDC model **or** provide +instance specific identifiers such as a module or VNF name. These will +still be prefixed with ``VALUE FOR:``. + +.. note:: + + Refer to the section below on :ref:`Preload Template Population <vvp_preload_population>` + for alternate ways to populate this information. + +The values are: + ++-----------------+-------------+--------------+-------------------------------+ +| Value | VNF-API | GR-API | Source & Description | +| | Name | Name | | ++=================+=============+==============+===============================+ +| VNF Name | vnf-name | vnf-name | Name of the VNF as it will | +| | | | appear in A&AI. This is user | +| | | | defined and does not need to | +| | | | match a value in SDC. | ++-----------------+-------------+--------------+-----------+-------------------+ +| Virtual | generic-vnf-| vnf-type | Maps preload to a specific | +| Function | type | | instance of the VF model from | +| Instance | | | SDC. This field is must be in | +| | | | the format of: | +| | | | <Service Name>/<VF Instance | +| | | | Name> and must match SDC | ++-----------------+-------------+--------------+-----------+-------------------+ +| VF | vnf-type | vf-module- | Maps preload to the | +| Module | | type | ``vfModuleModelName`` from | +| Model | | | the VSP CSAR or the value | +| | | | from SDC | ++-----------------+-------------+--------------+-----------+-------------------+ + + +.. _vvp_preload_population: + +Preload Template Population +--------------------------- + +Basic Usage - Single Environment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +VVP can also generate fully or partially populated preload templates if a +an optional environment directory is provided as a source for the parameter +values. + +When using the command line, the environment directory is provided using the +``--env-directory`` parameter. + +When using the GUI, the environment directory is provided by first selecting +the **Create Preloads from Env Files** option in **Settings** and then providing +the directory in the **Env Files** field. + +A template for the environment directory is created in the output directory +under ``preloads/<preload-format>/preload_env``. There will be an ``*.env`` file +for every VF module in the Heat template and a ``defaults.yaml``. This +directory can be copied and updated to serve as a data source for populating +the preload templates. Every value that needs to be updated will be set to +``CHANGEME``. + +.. code-block:: yaml + :caption: Example Partial env file (``base.env``) + + parameters: + ctrl_net_id: CHANGEME + ctrl_subnet_id: CHANGEME + db_ha_floating_ip: CHANGEME + db_ha_floating_v6_ip: CHANGEME + svc_flavor_name: svc_flavor + svc_image_name: svc_image + db_name_0: CHANGEME + +Each environment directory consists of three file types: + +1. **Environment files (.env)** - One file per VF module with the base name of + the env file matching the base name of the heat template it corresponds to. +2. **defaults.yaml** - Values specified in here will be used in all modules. + This is a useful place to put values that will be the same in every VF + module. It is important to note that the environment files take precedence + so if you specify a value in ``defaults.yaml``, then that value should be + removed from the environment files. +3. **VSP CSAR** - Optionally the CSAR can be downloaded from the SDC Artifacts + section, and put into the directory. If present, then the + :ref:`SDC Model Identifiers <vvp_preload_special_values>` will be pulled from + the CSAR instead of them being manually specified + +After template processing completes, the blank preload templates will still +be generated to the output directory. The **populated** templates will be +generated the ``<env-directory>/preloads/<preload-format>`` directory. If a +template was not fully populated, then it will be suffixed with ``_incomplete`` +(ex: ``base_incomplete.json``) + + +Advanced Usage - Multiple Environments +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Using a single environment directory is the most basic use case, but you may +need to generate preloads for multiple environments where some values do not +change by environment while others do. + +To enable this capability, environment directories can be nested and inherit +values from their parent directories. The parent directory's environment files +and ``defaults.yaml`` files should contain the default values for every +environment. + +These can be overridden in the child environment directories so +if you want to force each environment to provide a value, then do not specify +that value in any parent directory. + +.. code-block:: none + :caption: Example Nested Environment Directory + + env_directory/ + |--- vsp_name.csar <-- Global CSAR for all enviroments (assumes shared SDC instance) + |--- defaults.yaml <-- Global defaults for all env and modules + |--- base.env <-- Global defaults for base modules + |--- mod_one.env <-- Global defaults for mod_one modules + |--- env_one/ + | |--- defaults.yaml <-- env_one specific defaults for all modules + | |--- base.env <-- env_one values for the base module + | |--- mod_one.env <-- env_one values for mod_one + |--- env_two/ + |--- defaults.yaml <-- env_two specific defaults for all modules + |--- base.env <-- env_two values for the base module + |--- mod_one.env <-- env_two values for mod_one + + +Preload templates will be generated in the **leaf** directories of the environment +directory. In this example, preload templates will be generated in ``env_one`` +and ``env_two``, but **not** ``env_directory``. + +The value supplied to the preload will follow the following order of precedence +with the value being looked up from each of the following in turn and halting +once the value is found: + +1. Corresponding ``.env`` file for the module in the environment specific + directory (ex: ``env_one/base.env``) +2. The ``defaults.yaml`` file in the environment specific directory (ex: + ``env_one/defaults.yaml``) +3. The CSAR file in the environment specific directory (NOTE: only the + **special values** are looked up from the CSAR +4. If not found in the environment specific, then the parent directory will be + searched using the same precedence (1-3). + +This lookup chain will continue until the root environment directory is reached. +If no value is found, then the template will continue revert the value from +the blank template (i.e. ``VALUE FOR: {parameter-name}``) + +The environment directories can be nested as deeply as needed to map your needs. +For example you could have a global directory, production and test environments, +and then multiple sub-environments in each test nd production directory. + +**Example:** + +.. code-block:: none + :caption: Example Nested Environment Directory + + env_directory/ + |--- vsp_name.csar + |--- defaults.yaml + |--- base.env + |--- mod_one.env + |--- production/ + | |--- defaults.yaml + | |--- base.env + | |--- mod_one.env + | |--- prod_one/ + | | |--- defaults.yaml + | | |--- base.env + | | |--- mod_one.env + | |--- prod_two/ + | |--- defaults.yaml + | |--- base.env + | |--- mod_one.env + |--- test/ + |--- defaults.yaml + |--- base.env + |--- mod_one.env + |--- test_one/ + | |--- defaults.yaml + | |--- base.env + | |--- mod_one.env + |--- test_two/ + |--- defaults.yaml + |--- base.env + |--- mod_one.env + +Alternate Method for Specifying Special Values +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If you wish to populate the special values without providing a CSAR, then add +the following parameters and values to either your ``defaults.yaml`` file or +the appropriate ``.env`` file. + +* ``vnf-name`` - Corresponds to VNF Name in :ref:`special values <vvp_preload_special_values>` +* ``vnf-type`` - Corresponds to Virtual Function Instance in :ref:`special values <vvp_preload_special_values>` +* ``vf-module-model-name`` - Corresponds to VF Module Model Name in :ref:`special values <vvp_preload_special_values>` + + +.. _vvp_register_preload: + +Registering Preload with SDNC +----------------------------- + +At the time of this writing, the ONAP documentation does not provide a good +source of documentation on how to provide preloads to SDNC. The options known +are to either use a REST ``POST`` call to the following APIs based on format: + +.. note:: + + Host and port numbers may vary in your ONAP environment + +* **GR-API** - https://sdnc.api.simpledemo.onap.org:30267/restconf/operations/GENERIC-RESOURCE-API:preload-vf-module-topology-operation +* **VNF-API** - https://sdnc.api.simpledemo.onap.org:30267/restconf/operations/GENERIC-RESOURCE-API:preload-vf-module-topology-operation + +The ONAP-CLI project could also be used, but it only supports the **VNF-API** +as of the writing of this document. + +Customizing Preload Generation +------------------------------ + +VVP's preload generation capability leverages a plugin mechanism to enable additional +preload formats to be added. This can be useful if you define an intermediary +format such as a spreadsheet to capture the preload information. + +Preload plugins are discovered using the following method. + +1. The ``sys.path`` is scanned to find any top level modules that begin with + ``preload_`` +2. If found, then any implementations of ``AbstractPreloadGenerator`` are + registered as available formats +3. You can selectively disable formats by specifying them as excluded in the + ``vvp-config.yaml`` file. + +Please refer to the ``preload_vnfapi.VnfApiPreloadGenerator`` and +``preload_grapi.GrApiPreloadGenerator`` for examples of how to implement a +generator. |