[VVP] Doc for GUI, Docker, and Deprecation

Revising documentation to reflect the deprecation
of the VVP web application as well as adding
documentation for the new additions such as:

* Alternate report options
* Native GUI tool
* Docker execution

Change-Id: Ia1b1e99e7d5fb657286efb281049e4550a80d1b6
Issue-ID: VVP-137
Signed-off-by: Lovett, Trevor (tl2972) <tl2972@att.com>

17 files changed, 574 insertions, 246 deletions

diff --git a/docs/administration.rst b/docs/administration.rst
deleted file mode 100644
index 0fb3fe5..0000000
--- a/docs/administration.rst
+++ /dev/null
@@ -1,27 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
-
-Administration
-================
-
-There is a functionality which part of Django framework to manage all
-models and tables located on our application's db.
-The django based containers are: engagement-manager, cms, ci tests
-
-Usually, the admin dashboard url is http(s)://{{domain}}/admin/
-Throght there you will see all models exists in the related application
-(in the shape of tables) and will be able to create/update/delete there rows.
-
-The VVP application allow you to login into portal as admin (usually created
-under admin@{{your-domain}}.com). The admin allow you to manage VVP entities,
-update templates and display all availiable engagements and VFs.
-You can reach the screen by clicking the user bubble on top-right
-of the screen, and from the drop-down menu - click the 'Admin' button.
-
-Processes
------------
-
-
-Actions
----------
diff --git a/docs/architecture.png b/docs/architecture.png
deleted file mode 100755
index f65e1ae..0000000
--- a/docs/architecture.png
+++ /dev/null
diff --git a/docs/architecture.rst b/docs/architecture.rst
deleted file mode 100644
index 671abc8..0000000
--- a/docs/architecture.rst
+++ /dev/null
@@ -1,77 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
-
-
-Architecture
-===============
-
-The architecture of VVP based on micro-services. Each element has
-dedicated container.
-VVP uses rocket and kubernetes to manage the VVP system.
-There is a devkit component which allows you to render and deploy VVP.
-It's render and deploy the rockets with kuberenetes management (via quay.io).
-
-We store files on Ceph distributed storage system that includes 'RADOS Gateway'
-which allows us to store files in AWS S3 based API (buckets etc..).
-
-|image0|
-
-Architecture Alignment
-----------------------------
-
-How does this project fit into the rest of the ONAP Architecture?
-
-    * The VNF Validation Program will utilize the architecture to
-      validate VNFs against it.
-
-What other ONAP projects does this project depend on?
-
-    * This project depends on SDC, VNF Validation Program, and VNF Requirements
-
-How does this align with external standards/specifications?
-
-    * Are there dependencies with other open source projects?
-        * OpenStack
-
-Interactions
--------------------
-
-This is the list of containers part of VVP application:
-
-    * ci-uwsgi - end-to-end flow tests based on Seleniunm
-    * cms-nginx - webserver of CMS
-    * cms-uwsgi - backend uwsgi server which hosts django application
-    * em-nginx - webserver of engagement manager
-    * em-uwsgi - backend uwsgi server which hosts django application
-    * ext-haproxy - load balancer for external transport
-    * int-haproxy - load balancer for internal (container to container)
-      transport
-    * gitlab - holds all customers files in repos
-    * imagescanner - scan for validity and viruses on users files
-    * jenkins - run validation tasks
-    * portal - run the UI/UX application AKA VVP-Portal
-    * postgresql - store all data of engagement manager
-    * redis - in memory key-value store for all project
-    * celery - task queue manager which manage all validation tasks
-
-Repo name:
-
-    * org.onap.vvp/devkit
-    * org.onap.vvp/ansible-ice-bootstrap
-    * org.onap.vvp/portal
-    * org.onap.vvp/engagementmgr
-    * org.onap.vvp/cms
-    * org.onap.vvp/jenkins
-    * org.onap.vvp/haproxy
-    * org.onap.vvp/postgresql
-    * org.onap.vvp/gitlab
-    * org.onap.vvp/jeeves
-    * org.onap.vvp/test-engine
-    * org.onap.vvp/validation-scripts
-    * org.onap.vvp/documentation
-    * org.onap.vvp/image-scanner
-
-.. |image0| image:: architecture.png
-   :width: 11in
-   :height: 5in
diff --git a/docs/conf.py b/docs/conf.py
index 38b57a5..7e484bc 100644
--- a/docs/conf.py
+++ b/docs/conf.py
@@ -51,7 +51,6 @@ extensions = [
     'sphinx.ext.doctest',
     'sphinx.ext.graphviz',
     'sphinx.ext.todo',
-    'sphinx.ext.pngmath',
     'sphinx.ext.viewcode',
 ]
 
diff --git a/docs/configuration.rst b/docs/configuration.rst
deleted file mode 100644
index 03a4ae4..0000000
--- a/docs/configuration.rst
+++ /dev/null
@@ -1,9 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
-
-Configuration
-================
-
-All the configuration is happening on vvp-devkit component.
-There you will define all related fields for each component in a dedicated .yaml file.
diff --git a/docs/delivery.rst b/docs/delivery.rst
deleted file mode 100644
index e722662..0000000
--- a/docs/delivery.rst
+++ /dev/null
@@ -1,11 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
-
-Delivery
-==========
-
-Describe how functions are packaged into run-time components.
-For some components a block diagram may be useful.
-
-All the project deploy is under ansible files which rendered and deployed in devkit component.
diff --git a/docs/humaninterfaces.rst b/docs/humaninterfaces.rst
index 8f609f1..522c713 100644
--- a/docs/humaninterfaces.rst
+++ b/docs/humaninterfaces.rst
@@ -1,11 +1,110 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
+.. Copyright 2019 AT&T Intellectual Property.  All rights reserved.
 
-Human Interfaces
-======================
+Graphical User Interface
+========================
 
-The project is to develop a validation program to provide assurance of VNF
-interoperability with ONAP. Obtaining a validation shall be a self-service
-activity and should be against a reference release of ONAP for use by the
-VNF provider & any other validation authority.
+If desired, a graphical user interface is also provided when the application
+has been :doc:`installed from source <installation>`.  This can provide a
+convenient wrapper that enables users to perform validations without using
+extensive command-line usage.
+
+At this time the application can only be run from source, but in the future
+a packaged version may be provided.
+
+
+How to Start the Application
+----------------------------
+
+1.  Ensure you have installed the application and its dependencies from
+    source as described in the :doc:`Installation chapter <installation>`
+
+2.  Navigate to the ``ice_validator`` directory::
+
+    > cd <vvp-directory>/ice_validator
+
+3.  Launch the gui using the ``vvp.py`` command::
+
+    > python vvp.py
+
+
+How to Use the Tool
+-------------------
+
+.. note::
+    The look-and-feel of the application will vary slightly depending
+    on the Operating System of the host machine.  The screenshot below is how
+    the application looks on a Windows machine.
+
+**Sample Screenshot of VVP GUI Application**
+
+.. image:: vvp_app.png
+
+
+All configuration options available to the command-line version of the
+application are exposed as options on the left-hand side of the GUI.
+
+Additional Validation Categories
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+This allows the base set of tests to be extended by selecting additional
+categories of tests.  At this time, only one additional category is supported.
+
+This maps to the ``--category`` command-line option.
+
++----------------------+-------------------------------------------------------+
+| Category             | Description                                           |
++======================+=======================================================+
+| Environment          | When selected, VVP will flag parameters in environment|
+| File Compliance      | files that should be excluded per the ONAP Heat       |
+|                      | requirements.  De-selecting this can be useful when   |
+|                      | when testing instantiation directly in OpenStack      |
+|                      | without ONAP orchestration.  This is equivalent to    |
+|                      | specifying ``--category=environment_file`` from the   |
+|                      | command-line.                                         |
++----------------------+-------------------------------------------------------+
+
+Settings
+~~~~~~~~
+
+**Verbosity** - Controls the level of comparison output displayed in the
+underlying pytest validation output.  Default maps to pytest ``-v`` level of
+verbosity.
+
+**Report Format** - Controls the format of the output report generated after
+validation.  The options are: HTML (the default), Excel, and CSV.  This is
+equivalent the ``--report-format`` command-line option with the exception,
+that the JSON format is not supported via the GUI.
+
+**Input Format** - Controls the expected format of the template files.  This
+can either be a ZIP file containing the Heat templates or a directory
+containing the Heat templates.  There is no ZIP file option for the command-line
+script at this time.
+
+**Halt on Basic Failures** - VVP deems certain tests as "base tests" which if
+failed have the potential to generate a large number of other errors. This
+would include tests such as validating that the Heat templates are valid
+YAML.  If checked, the tool will immediately stop all other tests and show
+a report of this single failure. This can be useful in reducing the number
+of errors to sift through in these situations.  De-selecting this option
+is the equivalent of specifying ``--continue-on-failure`` as a command-line option.
+
+Running Validations
+~~~~~~~~~~~~~~~~~~~
+
+1. Select the desired input format in the settings (ZIP or Directory)
+2. Select the [...] button next to the Template Location input box
+3. Select the directory or ZIP file containing the Heat templates, and then
+   click Open
+4. Once the input is selected, select the "Validate Templates" button to
+   start the validation process.  The white box to the right will
+   display output as the validations are executed.
+5. Once validation is complete a summary of pass and fail will be written
+   to the output window, and a "View Report" option will appear on the
+   left-hand control panel.
+6. Select the "View Report" option, and the report will be opened in the
+   the appropriate application based on report format.
+7. If you have questions about report output, please refer to the
+   :ref:`Validation Report reference material <vvp-report>` for more
+   information.
diff --git a/docs/index.rst b/docs/index.rst
index e139e9d..906a352 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -1,26 +1,32 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
+.. Copyright 2019 AT&T Intellectual Property.  All rights reserved.
 
 VVP Documentation
-====================
+=================
 
-The project is to develop a validation program to provide assurance of
-VNF interoperability with ONAP. Obtaining a validation shall be a
-self-service activity and should be against a reference release of
-ONAP for use by the VNF provider & any other validation authority.
+The VNF Validation Platform (VVP) is an application to validate that `OpenStack
+Heat Templates <https://wiki.openstack.org/wiki/Heat>`__ comply with the
+ONAP requirements and guidelines documented in the :doc:`Heat section <../../../../../submodules/vnfrqts/requirements.git/docs/Chapter5/Heat/index>` of
+the ONAP's :doc:`VNF Requirements and Guidelines documentation <../../../../../guides/onap-provider/index>`.
 
+Adherence to these guidelines ensures that a VNF can be successfully
+onboarded, modeled, instantiated, and orchestrated by ONAP to the fullest
+extent possible.
+
+VVP is a utility written in Python that can be executed via a
+:doc:`command-line script <validation-scripts>`,
+:ref:`Docker container <vvp-docker-execution>`, or a
+native :doc:`Desktop GUI application <humaninterfaces>` to analyze and report
+on the compliance of a given set of Heat templates to the ONAP requirements.
+
+This guide will provide the user with instructions on how to acquire,
+setup, and execute the validations.
 
 .. toctree::
    :maxdepth: 1
 
+   installation.rst
    validation-scripts.rst
-   administration.rst
-   architecture.rst
-   configuration.rst
-   delivery.rst
    humaninterfaces.rst
-   installation.rst
-   logging.rst
-   offeredapis.rst
    release-notes.rst
diff --git a/docs/installation.rst b/docs/installation.rst
index 0353ec4..a20bbed 100644
--- a/docs/installation.rst
+++ b/docs/installation.rst
@@ -1,14 +1,54 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
+.. Copyright 2019 AT&T Intellectual Property.  All rights reserved.
+
+.. _vvp-installation:
 
 Installation
-==================
+============
+
+The VNF Validation Platform (VVP) can be run from source as a normal
+Python executable or a Docker image is provided.  This section will
+describe how to setup and install the ``vvp/validation-scripts`` and
+run from source.
 
-Installation happens through the vvp-devkit project.
+Installation and configuration of Docker is beyond the scope of this document,
+but you can refer to the :ref:`Docker Execution <vvp-docker-execution>`
+instructions for more details on running the validations as from
+the Docker image.
 
-Environment
+Pre-requisites
 --------------
 
-Steps
-----------
+This document assumes you have the following system-level utilities
+installed.
+
+Please refer to the respective sites for these tools for the appropriate
+installation instructions for your given operating system.
+
+* `Python 3.6+ <https://www.python.org/downloads/>`__
+* `Git <https://git-scm.com/>`__
+
+Setup
+------
+
+The source code for VVP can be obtained from the `ONAP Gerrit site <https://gerrit.onap.org/r/#/admin/projects/vvp/validation-scripts>`__
+or its `GitHub mirror <https://github.com/onap/vvp-validation-scripts>`__.
+
+1.  Clone the source from your desired repository host:
+
+    Choose **one** of the following ``git clone`` commands::
+
+    > git clone https://github.com/onap/vvp-validation-scripts.git
+    > git clone https://gerrit.onap.org/r/vvp/validation-scripts
+
+2.  (*Optional*) If desired, you can create a virtual Python environment to
+    avoid installing VVP's dependencies in your system level installation
+    of Python::
+
+    > python -m venv vvp
+    > source vvp/activate
+
+3. Install the required dependencies with the following command::
+
+    > python pip install -r requirements.txt
diff --git a/docs/logging.rst b/docs/logging.rst
deleted file mode 100644
index 4c795b2..0000000
--- a/docs/logging.rst
+++ /dev/null
@@ -1,14 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
-
-Logging
-==========
-Login into portal/em admin/cms admin
-
-Where to Access Information
---------------------------------
-
-
-Error / Warning Messages
-----------------------------
diff --git a/docs/offeredapis.rst b/docs/offeredapis.rst
deleted file mode 100644
index 20a696b..0000000
--- a/docs/offeredapis.rst
+++ /dev/null
@@ -1,10 +0,0 @@
-.. This work is licensed under a Creative Commons Attribution 4.0 International License.
-.. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
-
-
-Offered APIs
-==============
-
-
-None Available at this time
diff --git a/docs/release-notes.rst b/docs/release-notes.rst
index 2e9c394..e5876a9 100644
--- a/docs/release-notes.rst
+++ b/docs/release-notes.rst
@@ -1,16 +1,47 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2017 AT&T Intellectual Property.  All rights reserved.
+.. Copyright 2019 AT&T Intellectual Property.  All rights reserved.
 
 
 Release Notes
 ===============================
 
-.. note::
-   VNF Onboarding is a challenge across the industry because of a lack of a
-   standard form for VNFs.
-   The VVP project was created to provide validation scripts for
-   VNFs to test validity against the ONAP Requirements.
+
+Version: 4.0.0 (Dublin)
+-----------------------
+
+:Release Date: TBD
+
+**Removed Features**
+
+- The VVP web application has been deprecated and is no longer supported
+  as of the Dublin release.  The validation scripts continue to be supported
+  and enhanced, but contributions to the web-related repositories are now locked
+  and VVP will no longer be supported for deployment via ONAP Operations
+  Manager (OOM).
+
+  The following repositories are now locked as of this release:
+
+    - ``vvp/ansibile-ice-bootstrap``
+    - ``vvp/cms``
+    - ``vvp/devkit``
+    - ``vvp/engagementmgr``
+    - ``vvp/gitlab``
+    - ``vvp/image-scanner``
+    - ``vvp/jenkins``
+    - ``vvp/portal``
+    - ``vvp/postgresql``
+    - ``vvp/test-engine``
+
+**New Features**
+
+- A new :doc:`GUI application <humaninterfaces>` has been contributed and can
+  be used to execute validations in a user-friendly way without using complex
+  command line options.
+- VVP is now packaged as a Docker container eliminating the need to run the
+  application from source code.  See the :ref:`Docker Execution <vvp-docker-execution>`
+  instructions for more details.
+
 
 Version: 3.0.0
 --------------
@@ -70,6 +101,7 @@ Quick Links:
 - Initial release - none
 
 **Other**
+
 	NA
 
 ===========
diff --git a/docs/validation-scripts.rst b/docs/validation-scripts.rst
index 0ff6e32..b327ad4 100644
--- a/docs/validation-scripts.rst
+++ b/docs/validation-scripts.rst
@@ -1,105 +1,349 @@
 .. This work is licensed under a Creative Commons Attribution 4.0 International License.
 .. http://creativecommons.org/licenses/by/4.0
-.. Copyright 2018 AT&T Intellectual Property.  All rights reserved.
+.. Copyright 2019 AT&T Intellectual Property.  All rights reserved.
 
-Manual Heat Template Validation
-===============================
+Heat Validation via Command Line
+================================
 
-validation-scripts
-------------------
+.. note::
+    This section assumes you have already acquired and setup the validation
+    scripts as described in the :ref:`VVP Installation <vvp-installation>`
+    section.
 
-This project contains validation scripts to test
-that a set of Heat Templates adheres to
-the ONAP VNF Heat Orchestration Template guidelines.
+Basic Script Execution
+----------------------
 
-For more information on the ONAP Heat Orchestration
-Template Guidelines, vist the `Heat Guidelines <https://onap.readthedocs.io/en/latest/submodules/vnfrqts/requirements.git/docs/Chapter5/Heat/index.html>`__
+1.  The scripts must be executed from the ``ice_validator`` directory.
+    Navigate to the directory first::
 
-About
-_____
+    > cd <vvp-directory>/ice_validator
 
-The validation scripts project allows performing heat template
-validation without installing the full VVP platform. The following
-instructions apply to running these validation scripts in that manner.
+2.  To run the script with default options, issue the following command where
+    ``<Directory>`` is the directory containing the Heat templates::
+
+    /path/to/validation-scripts/ice_validator>$ pytest tests --template-directory=<Directory>``
+
+3.  Test results will be written to the consoles describing any failures that
+    are encountered.  If no failures are found, then a success message will
+    be displayed.  Additional reports can be found in the
+    ``<VVP Directory>/ice_validator/output`` directory.  See
+    :ref:`Reports <vvp-reports>` for more details.
+
+
+.. _vvp-cmd-options:
+
+Command Line Options
+--------------------
+
+Additional options can be specified at the command line to customize output
+and execution.  VVP uses the `pytest <https://docs.pytest.org>`__
+framework for test execution, and therefore all pytest options are available.
+
+Please refer to the pytest documentation for information on the command-line
+options that framework provides.  We will only document the specific options
+that are used by VVP::
+
+      --template-directory=TEMPLATE_DIR
+                            Directory which holds the templates for validation
+
+      --category=CATEGORY_NAME
+                            Additional validations that can be selected.
+                            The only supported value at this time is
+                            environment_file which will enforce that specific
+                            parameter values are excluded from the environment
+                            file per ONAP Heat requirements.
+
+      --self-test           Test the unit tests against their fixtured data
+
+      --report-format=REPORT_FORMAT
+                            Format of output report (html, csv, excel, json)
+
+      --continue-on-failure
+                            Continue validation even when structural errors exist
+                            in input files
+
+      --output-directory=OUTPUT_DIR
+                            Alternate directory for report output.
+
+.. _vvp-reports:
+
+VVP Reports
+-----------
+
+After completion of the validations, several reports will be written to the
+ouput directory (specified via ``--output-directory`` or
+``<vvp-directory>/ice_validator/output`` by default).
+
+
++-----------------------+------------------------------------------------------+
+| Report                | Description                                          |
++=======================+======================================================+
+| ``report.{html,csv,`` | Full report of validation results showing the pass   |
+|         ``xlsx,json}``| or failure of the validation and details on all      |
+|                       | errors.  See :ref:`Validation Report <vvp-report>`   |
+|                       | for information on the HTML, CSV, and Excel versions |
+|                       | of the report, or                                    |
+|                       | :ref:`JSON Report <vvp-json-report>` for details on  |
+|                       | the machine-readable results.                        |
++-----------------------+------------------------------------------------------+
+| ``traceability.csv``  | This shows a mapping from Heat requirement to        |
+|                       | test case.                                           |
++-----------------------+------------------------------------------------------+
+| ``mapping_errors.csv``| File should be empty, but if present shows tests     |
+|                       | that are mapped to requirements that no longer exist |
++-----------------------+------------------------------------------------------+
+| ``traceability.rst``  | Similar information to ``traceability.csv``, but     |
+|                       | in reStructuredText for use in the VNF Requirements  |
+|                       | documentation.                                       |
++-----------------------+------------------------------------------------------+
+| ``failures``          | **Deprecated** JSON version of test failures.  Use   |
+|                       | ``report.json`` instead.                             |
++-----------------------+------------------------------------------------------+
+
+.. _vvp-report:
+
+Validation Report
+~~~~~~~~~~~~~~~~~
+
+If the report format of ``html`` (*default*), ``excel``, or ``csv`` are
+requested via the ``--report-format`` option, then the a report file will
+be written to the output directory.  Regardless of format, the file will contain
+a header section that summarizes the results and files scanned, and an error
+section that has a row for each failure with four columns.
+
+Header Information
+^^^^^^^^^^^^^^^^^^
+
++-----------------------+------------------------------------------------------+
+| Header Element        | Description                                          |
++=======================+======================================================+
+| Categories Selected   | Any additional categories selected via ``--category``|
++-----------------------+------------------------------------------------------+
+| Tool Version          | Version of the tool that produced the report         |
++-----------------------+------------------------------------------------------+
+| Report Generated At   | The timestamp of when the report was generated       |
++-----------------------+------------------------------------------------------+
+| Directory Validated   | Absolute path to directory validated                 |
++-----------------------+------------------------------------------------------+
+| Checksum              | Unique MD5 has of the contents of template directory |
++-----------------------+------------------------------------------------------+
+| Total Errors          | Number of errors/violations found                    |
++-----------------------+------------------------------------------------------+
+
+
+Error Information
+^^^^^^^^^^^^^^^^^
+
+If any violations are found, then there will be one row for each violation
+with the following columns:
+
++-----------------------+------------------------------------------------------+
+| Column Name           | Description                                          |
++=======================+======================================================+
+| Files                 | The file or files that were scanned as part of the   |
+|                       | test.                                                |
++-----------------------+------------------------------------------------------+
+| Tests                 | Name of the test case (not shown in HTML version)    |
++-----------------------+------------------------------------------------------+
+| Error Message         | This shows the test and brief error message from the |
+|                       | test that failed.  This will contain details about   |
+|                       | the element that triggered the violation such as the |
+|                       | parameter name, resource ID, etc.                    |
+|                       |                                                      |
+|                       | In the HTML version of the report this column will   |
+|                       | also show the test case name, and provide a link to  |
+|                       | ``Full Details`` the raw output of the test          |
++-----------------------+------------------------------------------------------+
+| Requirements          | The requirement ID and text that was violated        |
++-----------------------+------------------------------------------------------+
+| Resolution Steps      | For some violations, there are pre-defined resolution|
+|                       | steps that indicate what action the user should take |
+|                       | to resolve the violation.                            |
+|                       |                                                      |
+|                       | **Note**: Not all violations will have resolution    |
+|                       | steps, rather the error message and requirement is   |
+|                       | sufficient.                                          |
++-----------------------+------------------------------------------------------+
+| Raw Test Output       | Full output from the pytest test case. This not a    |
+|                       | dedicated column in the HTML version of the report.  |
++-----------------------+------------------------------------------------------+
+
+
+.. _vvp-json-report:
+
+JSON Report
+~~~~~~~~~~~
+
+This report is intended to provide a machine-readable version of the test
+execution, and provides the most comprehensive summary of the test execution
+and results.
+
+File Header/Top Level
+^^^^^^^^^^^^^^^^^^^^^
+
+JSON Report <vvp-json-report>
+The top level will include a summary of available execution metadata.
+
+NOTE:  The ``tests`` and ``requirements`` entries are elided in the
+example below.
+
+**Example Header**:
+
+.. code-block:: javascript
+
+    {
+      "version": "dublin",
+      "template_directory": "/path/to/template",
+      "timestamp": "2019-01-21T02:11:07.305000",
+      "checksum": "6296aa211870634f9b4a23477c5eab28",
+      "profile": "",
+      "outcome": "FAIL",
+      "tests": [],
+      "requirements": [],
+    }
+
+**Header Definition**:
+
+.. csv-table::
+    :header-rows: 1
+    :file: vvp_json_header.csv
+
+.. _vvp-test-result:
+
+Test Result
+^^^^^^^^^^^
+
+For each test result a JSON object will be provided that informs the consumer
+what tests was run, its result, and the requirements it validated.
+
+**Example Test Result**:
+
+.. code-block:: javascript
+
+    {
+      "files": [
+        "/Users/username/Desktop/stark_template2/STARKDB-nested-1.yaml",
+        "/Users/username/Desktop/stark_template2/base_starkdb.yaml",
+     ],
+      "test_module": "test_resource_indices",
+      "test_case": "test_indices_start_at_0_increment",
+      "result": "FAIL",
+      "error": " Index values associated with resource ID prefix STARKDB_server_ do not start at 0\n",
+      "requirements": [
+        {
+          "id": "R-11690",
+          "text": "When a VNF's Heat Orchestration Template's Resource ID contains an\n``{index}``, the ``{index}`` is a numeric value that **MUST** start at\nzero and **MUST** increment by one.\n\nAs stated in R-16447,\n*a VNF's <resource ID> MUST be unique across all Heat\nOrchestration Templates and all HEAT Orchestration Template\nNested YAML files that are used to create the VNF*.  While the ``{index}``\nwill start at zero in the VNF, the ``{index}`` may not start at zero\nin a given Heat Orchestration Template or HEAT Orchestration Template\nNested YAML file.",
+          "keyword": "MUST"
+        }
+      ]
+    }
+
+
+**Test Result Definition**:
+
+.. csv-table::
+    :header-rows: 1
+    :file: vvp_json_test_result.csv
 
 
-Installation
-____________
+.. _vvp-req-metadata:
 
-This software is not platform dependent and can be run in a Windows, Unix or
-OS X environment.
+Requirement Metadata
+^^^^^^^^^^^^^^^^^^^^
 
-Satisfy Dependencies
-####################
+For each test case, the following requirement metadata will be reported.
 
+**Example Requirement Metadata**:
 
- These can be installed using pip (assuming pip is installed) with the command:
+.. code-block:: javascript
 
-``$ pip install -r requirements.txt``
+    {
+      "id": "R-11690",
+      "text": "When a VNF's Heat Orchestration Template's Resource ID contains an\n``{index}``, the ``{index}`` is a numeric value that **MUST** start at\nzero and **MUST** increment by one.\n\nAs stated in R-16447,\n*a VNF's <resource ID> MUST be unique across all Heat\nOrchestration Templates and all HEAT Orchestration Template\nNested YAML files that are used to create the VNF*.  While the ``{index}``\nwill start at zero in the VNF, the ``{index}`` may not start at zero\nin a given Heat Orchestration Template or HEAT Orchestration Template\nNested YAML file.",
+      "keyword": "MUST"
+    }
 
-Use
-___
 
-Clone this project.
+**Requirement Metadata Definition**:
 
-To validate Heat templates just run this the command under the folder ``ice_validator``:
++-----------------+-------------+--------------+-----------+-------------------+
+| Field Name      | Required    | Data Type    | Valid     | Description       |
+|                 | Optional    |              | Values    |                   |
+|                 | Conditional |              |           |                   |
++=================+=============+==============+===========+===================+
+| ``id``          | Required    | ``string``   |           | Requirement ID    |
+|                 |             |              |           | from the VNFRQTS  |
+|                 |             |              |           | project           |
++-----------------+-------------+--------------+-----------+-------------------+
+| ``text``        | Required    | ``string``   |           | Full text of      |
+|                 |             |              |           | requirement       |
++-----------------+-------------+--------------+-----------+-------------------+
+| ``keyword``     | Required    | ``string``   | MUST,     | RFC 2119 keyword  |
+|                 |             |              | MUST NOT, | of the requirement|
+|                 |             |              | MAY,      |                   |
+|                 |             |              | SHOULD,   |                   |
+|                 |             |              | SHOULD NOT|                   |
++-----------------+-------------+--------------+-----------+-------------------+
 
-``</path/to/validation-scripts/ice_validator>$ pytest --tap-stream --template-directory=<Directory>``
 
-where ``<Directory>`` is the full path to a folder containing heat templates.
+.. _vvp-req-result:
 
-Output
-______
+Requirement Result
+^^^^^^^^^^^^^^^^^^
 
-After performing a validation, an output folder will be created.
+The file also includes an aggregated view of adherence to the VNF Requirements
+validated by the validation scripts.  Since some requirements have multiple
+test cases, these results roll-up the result to an aggregated result for each
+requirement. This section does not include detailed test results.  If you
+require detailed error information, then refer to the tests section of the
+results.
 
-``/path/to/validation-scripts/ice_validator/output/``
+**Example Requirement Result**:
 
-This folder will contain a file ``report.html`` which contains a list of all
-of the ONAP VNF Heat Template Guideline violations. If there are no violations,
-the report will say ``No validation errors found.``
+.. code-block:: javascript
 
-Interpreting the Output
-_______________________
+    {
+      "id": "R-16447",
+      "text": "A VNF's <resource ID> **MUST** be unique across all Heat\nOrchestration Templates and all HEAT Orchestration Template\nNested YAML files that are used to create the VNF.",
+      "keyword": "MUST",
+      "result": "FAIL"
+      "errors": [
+         "The error message"
+      ]
+    }
 
-The report file will have 4 columns for details about a violation, and one
-row for each violation. Below contains details about each column.
 
-File
-####
+**Requirement Result Definition**:
 
-This is the file(s) that contained the violation
+.. csv-table::
+    :header-rows: 1
+    :file: vvp_json_req_result.csv
 
-Error Message
-#############
 
-This shows the test and brief error message from the validation script that
-contained the violation. There is a ``Full Details`` button to show the
-complete raw test output. The error message will also contain details
-about what element is involved with the violation (such as the parameter
-name, resource id, etc...).
+.. _vvp-docker-execution:
 
-Requirement(s)
-##############
+Docker Execution
+----------------
 
-This column contains the requirement(s) that each test/violation is
-mapped to. These requirements are taken directly from the VNF Requirements
-project Heat Orchestration Template Guidelines section.
+A version of VVP is also provided as a Docker image.  If your environment
+supports Docker, then this eliminates the need to setup and install
+the application from source code.
 
+To execute from Docker, issue the following command where
+``<Local Template Directory>`` is where the Heat templates are located and
+``<Local Report Directory>`` is where you would like the reports to be
+written on your local machine::
 
-Resolution Steps
-################
-
-For some violations, there are pre-defined resolution steps that
-indicate what action the user should take to resolve the violation.
-
-**Note**: Not all violations will have resolution steps. Most violations
-can be resolved simply by reviewing the requirements that have been violated
-in the previous column.
+    docker run --rm -i -v ~/<Local Template Directory>/:/template \
+    -v ~/<Local Report Directory>:/reports \
+    onap/vvp/validation-scripts --template-directory=/template \
+    --output-directory=/reports
 
+The same :ref:`command line options <vvp-cmd-options>` can be used with the
+Docker image that are used with the version from source.
 
 Self-Test Suite
-_______________
+---------------
 
 The ``ice_validator`` includes an extensive self-test suite. It is a
 **requirement** for any additions or changes to the test suite to
@@ -111,3 +355,8 @@ the project root as:
 You can also run it under the folder ``ice_validator``:
 
 ``$ pytest --self-test``
+
+
+
+
+
diff --git a/docs/vvp_app.png b/docs/vvp_app.png
new file mode 100644
index 0000000..8a27249
--- /dev/null
+++ b/docs/vvp_app.png
diff --git a/docs/vvp_json_header.csv b/docs/vvp_json_header.csv
new file mode 100644
index 0000000..5827f8b
--- /dev/null
+++ b/docs/vvp_json_header.csv
@@ -0,0 +1,17 @@
+Field Name,"Required/
+Optional/
+Conditional",Data Type,Valid Values,Description
+``checksum``,Required,``string``,,MD5 hash of all file contents in the ``template_directory``
+``outcome``,Required,``string``,"``PASS``
+``FAIL``
+``ERROR``","One of the valid values:
+
+* ``PASS`` - All tests passed successfully (some may have been skipped as not applicable based on the contents of the template)
+* ``FAIL`` - At least one test failed.  In this scenario the templates will need to be corrected to comply with the requirements
+* ``ERROR`` - An unexpected error occurred during test setup.  Some or all tests may have not executed.  Issue should be referred to the VVP team for investigation."
+``categories``,Optional,array of ``string``,,Categories selected via the ``--category`` option.
+``template_directory``,Required,``string``,,Absolute path of the directory containing the Heat templates that were validated
+``version``,Required,``string``,,Version of the validation scripts that produced the report
+``timestamp``,Required,``string``,,ISO 8601 Timestamp in UTC of when the report was generated
+``tests``,Required,List of :ref:`Test Result <vvp-test-result>`,,See :ref:`Test Result <vvp-test-result>`
+``requirements``,Required,List of :ref:`Requirement Result <vvp-req-result>`,,See :ref:`Requirement Result <vvp-req-result>`
diff --git a/docs/vvp_json_req_result.csv b/docs/vvp_json_req_result.csv
new file mode 100644
index 0000000..42b96b0
--- /dev/null
+++ b/docs/vvp_json_req_result.csv
@@ -0,0 +1,17 @@
+Field Name,Required/Optional/Conditional,Data Type,Valid Values,Description
+``id``,Required,``string``,,"Requirement ID from the VNFRQTS project.
+
+**NOTE**:a requirement ID of ""Unmapped"" may be included if one or more tests are not mapped to a requirement."
+``text``,Required,``string``,,Full text of the requirement.
+``keyword``,Required,``string``,"``MUST``,
+``MUST NOT``,
+``MAY``,
+``""SHOULD""``,
+``""SHOULD NOT""``",RFC 2119 keyword associated with the requirement
+``result``,Required,``string``,"``PASS``, ``SKIP``, ``FAIL``, ``ERROR``","One of the valid values:
+
+* ``PASS`` - The test case passed with no violations
+* ``SKIP`` - The test case was skipped because it was deemed not applicable
+* ``FAIL`` - The test case completed, but it found a violation
+* ``ERROR`` - An unexpected error was found while setting up the test case"
+``errors``,Required,List of ``string``,,Error messages associated with this requirement.  This will be an empty string if the result is ``PASS`` or ``SKIP``
diff --git a/docs/vvp_json_test_result.csv b/docs/vvp_json_test_result.csv
new file mode 100644
index 0000000..14982c0
--- /dev/null
+++ b/docs/vvp_json_test_result.csv
@@ -0,0 +1,17 @@
+Field Name,Required/Optional/Conditional,Data Type,Valid Values,Description
+``files``,Required,list of ``string``,,"List of files that were passed to the test case.
+
+**NOTE**: If ``result`` is ``ERROR`` this may be an empty list"
+``test_module``,Required,``string``,,Name of module/file anme that contains the test case
+``test_case``,Required,``string``,,Name of the test case
+``result``,Required,``string``,"* ``PASS``
+* ``SKIP``
+* ``FAIL``
+* ``ERROR``","One of the valid values:
+
+* ``PASS`` - The test case passed with no violations
+* ``SKIP`` - The test case was deemed not applicable
+* ``FAIL`` - The test case completed, but a violation was found
+* ``ERROR`` - An unexpected error was found while setting up the test case"
+``error``,Required,``string``,,"If the test failed or encountered an error, then ths will be a message summarizing the error.  If the test passed or was skipped, then this will be an empty string"
+``requirements``,Required,List of :ref:`Requirement Metadata <vvp-req-metadata>`,,See :ref:`Requirement Metadata <vvp-req-metadata>`