summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorkrishnaa96 <krishna.moorthy6@wipro.com>2021-06-01 09:57:40 +0530
committerkrishnaa96 <krishna.moorthy6@wipro.com>2021-06-02 12:35:06 +0530
commit211b609bd426b22fe7a0fc60a970a787acc006ca (patch)
tree2ed0a472d5d552fbf2c39a9b85449f7850b440ae
parent5034a07fa6174039e8fb88e2d508932ed05a120c (diff)
Update docs for music migration
- Update the Architecture diagram - Split Installation page into configuration and advanced options Issue-ID: OPTFRA-964 Signed-off-by: krishnaa96 <krishna.moorthy6@wipro.com> Change-Id: I214b05ef0f88cb11f50aa6156450506074ed06c0
-rw-r--r--docs/index.rst2
-rw-r--r--docs/sections/architecture.rst57
-rw-r--r--docs/sections/configuration.rst159
-rw-r--r--docs/sections/diagrams/HAS_SeedCode_Architecture_R2.jpgbin321115 -> 0 bytes
-rw-r--r--docs/sections/diagrams/HAS_SeedCode_Architecture_R9.jpgbin0 -> 346149 bytes
-rw-r--r--docs/sections/installation-advanced.rst150
-rw-r--r--docs/sections/installation.rst744
-rw-r--r--docs/sections/logging.rst22
8 files changed, 468 insertions, 666 deletions
diff --git a/docs/index.rst b/docs/index.rst
index 59759d4..86c78f1 100644
--- a/docs/index.rst
+++ b/docs/index.rst
@@ -42,6 +42,8 @@ is an orchestration service, or Nova is a compute service.
./sections/offeredapis.rst
./sections/consumedapis.rst
./sections/installation.rst
+ ./sections/configuration.rst
+ ./sections/installation-advanced.rst
./sections/logging.rst
./sections/homingspecification.rst
Example Homing Templates <./sections/example.rst>
diff --git a/docs/sections/architecture.rst b/docs/sections/architecture.rst
index cfbd535..5588c75 100644
--- a/docs/sections/architecture.rst
+++ b/docs/sections/architecture.rst
@@ -39,7 +39,7 @@ sample homing template has been drawn up for residential vCPE Homing Use Case.
HAS Architecture (R2)
----------------------
-.. image:: ./diagrams/HAS_SeedCode_Architecture_R2.jpg
+.. image:: ./diagrams/HAS_SeedCode_Architecture_R9.jpg
Lifecycle of a Homing request in HAS
--------------------------------------------
@@ -59,3 +59,58 @@ A sample heuristic greedy algorithm of HAS (using a vCPE as example)
------------------------------------------------------------------------
.. image:: ./diagrams/HASHeuristicGreedyAlgorithm.jpg
+
+Components
+----------
+
+Conductor consists of five services that work together:
+
+- **``conductor-api``**: An HTTP REST API
+- **``conductor-controller``**: Validation, translation, and
+ status/results
+- **``conductor-data``**: Inventory provider and service controller
+ gateway
+- **``conductor-solver``**: Processing and solution calculation
+- **``conductor-reservation``**: Reserves the suggested solution solved
+ by Solver component.
+
+Workflow
+--------
+
+- Deployment **plans** are created, viewed, and deleted via
+ ``conductor-api`` and its `REST API <doc/api/README.md>`__.
+- Included within each ``conductor-api`` plan request is a `Homing
+ Template <doc/template/README.md>`__.
+- Homing Templates describe a set of inventory demands and constraints
+ to be solved against.
+- ``conductor-api`` hands off all API requests to
+ ``conductor-controller`` for handling.
+- All deployment plans are assigned a unique identifier (UUID-4), which
+ can be used to check for solution status asynchronously. (Conductor
+ does not support callbacks at this time.)
+- ``conductor-controller`` ensures templates are well-formed and valid.
+ Errors and remediation are made visible through ``conductor-api``.
+ When running in debug mode, the API will also include a python
+ traceback in the response body, if available.
+- ``conductor-controller`` uses ``conductor-data`` to resolve demands
+ against a particular **inventory provider** (e.g., A&AI).
+- ``conductor-controller`` translates the template into a format
+ suitable for solving.
+- As each template is translated, ``conductor-solver`` begins working
+ on it.
+- ``conductor-solver`` uses ``conductor-data`` to resolve constraints
+ against a particular **service controller** (e.g., SDN-C).
+- ``conductor-solver`` determines the most suitable inventory to
+ recommend.
+- ``conductor-reservation`` attempts to reserve the solved solution in
+ SDN-GC
+
+**NOTE**: There is no Command Line Interface or Python API Library at
+this time.
+
+DB Backend
+----------
+
+All Conductor services use a DB backend for data storage/persistence and/or
+as a RPC transport mechanism. The current implementation supports two services which can be used as backend - `Music <https://github.com/att/music>`__ and
+`ETCD <https://github.com/etcd-io/etcd>`__.
diff --git a/docs/sections/configuration.rst b/docs/sections/configuration.rst
new file mode 100644
index 0000000..968cece
--- /dev/null
+++ b/docs/sections/configuration.rst
@@ -0,0 +1,159 @@
+Configuration
+=============
+
+Configuration files are located in ``etc/conductor`` relative to the
+python environment Conductor is installed in.
+
+To generate a sample configuration file, change to the directory just
+above where ``etc/conductor`` is located (e.g., ``/`` for the default
+environment, or the virtual environment root directory). Then:
+
+.. code:: bash
+
+ $ oslo-config-generator --config-file=etc/conductor/conductor-config-generator.conf
+
+This will generate ``etc/conductor/conductor.conf.sample``.
+
+Because the configuration directory and files will include credentials,
+consider removing world permissions:
+
+.. code:: bash
+
+ $ find etc/conductor -type f -exec chmod 640 {} +
+ $ find etc/conductor -type d -exec chmod 750 {} +
+
+The sample config may then be copied and edited. Be sure to backup any
+previous ``conductor.conf`` if necessary.
+
+.. code:: bash
+
+ $ cd etc/conductor
+ $ cp -p conductor.conf.sample conductor.conf
+
+``conductor.conf`` is fully annotated with descriptions of all options.
+Defaults are included, with all options commented out. Conductor will
+use defaults even if an option is not present in the file. To change an
+option, simply uncomment it and edit its value.
+
+With the exception of the ``DEFAULT`` section, it's best to restart the
+Conductor services after making any config changes. In some cases, only
+one particular service actually needs to be restarted. When in doubt,
+however, it's best to restart all of them.
+
+A few options in particular warrant special attention:
+
+::
+
+ [DEFAULT]
+
+ # If set to true, the logging level will be set to DEBUG instead of the default
+ # INFO level. (boolean value)
+ # Note: This option can be changed without restarting.
+ #debug = false
+
+For more verbose logging across all Conductor services, set ``debug`` to
+true.
+
+::
+
+ [db_options]
+
+ # db_backend to use
+ db_backend = etcd
+
+ # Use music mock api
+ music_mock = False
+
+Set ``db_backend`` to the db(music/etcd) which is being deployed. Based on this
+options, conductor will decide on using the corresponding client to access the
+backend.
+
+::
+
+ [aai]
+
+ # Base URL for A&AI, up to and not including the version, and without a
+ # trailing slash. (string value)
+ #server_url = https://controller:8443/aai
+
+ # SSL/TLS certificate file in pem format. This certificate must be registered
+ # with the A&AI endpoint. (string value)
+ #certificate_file = certificate.pem
+
+ # Private Certificate Key file in pem format. (string value)
+ #certificate_key_file = certificate_key.pem
+
+ # Certificate Authority Bundle file in pem format. Must contain the appropriate
+ # trust chain for the Certificate file. (string value)
+ #certificate_authority_bundle_file = certificate_authority_bundle.pem
+
+Set ``server_url`` to the A&AI server URL, to but not including the
+version, omitting any trailing slash. Conductor supports A&AI API v9 at
+a minimum.
+
+Set the ``certificate`` prefixed keys to the appropriate SSL/TLS-related
+files.
+
+**IMPORTANT**: The A&AI server may have a mismatched host/domain name
+and SSL/TLS certificate. In such cases, certificate verification will
+fail. To mitigate this, ``certificate_authority_bundle_file`` may be set
+to an empty value. While Conductor normally requires a CA Bundle
+(otherwise why bother using SSL/TLS), this requirement has been
+temporarily relaxed so that development and testing may continue.
+
+::
+
+ [messaging_server]
+
+ # Log debug messages. Default value is False. (boolean value)
+ #debug = false
+
+When the ``DEFAULT`` section's ``debug`` option is ``true``, set this
+section's ``debug`` option to ``true`` to enable detailed Conductor-side
+RPC-over-Music debug messages.
+
+Be aware, it is voluminous. "You have been warned." ``:)``
+
+::
+
+ [music_api]
+
+ # List of hostnames (round-robin access) (list value)
+ #hostnames = localhost
+
+ # Log debug messages. Default value is False. (boolean value)
+ #debug = false
+
+Set ``hostnames`` to match wherever the Music REST API is being hosted
+(wherever Apache Tomcat and ``MUSIC.war`` are located).
+
+When the ``DEFAULT`` section's ``debug`` option is ``true``, set this
+section's ``debug`` option to ``true`` to enable detailed Conductor-side
+MUSIC API debug messages.
+
+The previous comment around the volume of log lines applies even more so
+here. (Srsly. We're not kidding.)
+
+**IMPORTANT**: Conductor does not presently use Music's atomic
+consistency features due to concern around lock creation/acquisition.
+Instead, Conductor uses eventual consistency. For this reason,
+consistency issues may occur when using Music in a multi-server, High
+Availability configuration.
+
+::
+
+ [sdnc]
+
+ # Base URL for SDN-C. (string value)
+ #server_url = https://controller:8443/restconf
+
+ # Basic Authentication Username (string value)
+ #username = <None>
+
+ # Basic Authentication Password (string value)
+ #password = <None>
+
+Set ``server_url`` to the SDN-C server URL, omitting any trailing slash.
+
+Set ``username`` and ``password`` to the appropriate values as directed
+by SDN-C.
diff --git a/docs/sections/diagrams/HAS_SeedCode_Architecture_R2.jpg b/docs/sections/diagrams/HAS_SeedCode_Architecture_R2.jpg
deleted file mode 100644
index 1d247ca..0000000
--- a/docs/sections/diagrams/HAS_SeedCode_Architecture_R2.jpg
+++ /dev/null
Binary files differ
diff --git a/docs/sections/diagrams/HAS_SeedCode_Architecture_R9.jpg b/docs/sections/diagrams/HAS_SeedCode_Architecture_R9.jpg
new file mode 100644
index 0000000..4141cd9
--- /dev/null
+++ b/docs/sections/diagrams/HAS_SeedCode_Architecture_R9.jpg
Binary files differ
diff --git a/docs/sections/installation-advanced.rst b/docs/sections/installation-advanced.rst
new file mode 100644
index 0000000..c0cf62e
--- /dev/null
+++ b/docs/sections/installation-advanced.rst
@@ -0,0 +1,150 @@
+Installation - Advanced Options
+===============================
+
+Running conductor-api Under apache2 httpd and mod\_wsgi
+-------------------------------------------------------
+
+``conductor-api`` may be run as-is for development and test purposes.
+When used in a production environment, it is recommended that
+``conductor-api`` run under a multithreaded httpd service supporting
+`WSGI <https://www.wikipedia.org/wiki/Web_Server_Gateway_Interface>`__,
+tuned as appropriate.
+
+Configuration instructions for **apache2 httpd** and **nginx** are
+included herein. Respective package requirements are:
+
+- `apache2 <http://packages.ubuntu.com/focal/apache2>`__ and
+ `libapache2-mod-wsgi <http://packages.ubuntu.com/focal/libapache2-mod-wsgi>`__
+- `nginx <http://packages.ubuntu.com/focal/nginx>`__ and
+ `uwsgi <http://packages.ubuntu.com/focal/uwsgi>`__
+
+
+Sample configuration files have been provided in the repository.
+
+These instructions presume a ``conductor`` user exists. See the
+**Service Scripts** section for details.
+
+First, set up a few directories:
+
+.. code:: bash
+
+ $ sudo mkdir -p /var/www/conductor
+ $ sudo mkdir /var/log/apache2/conductor
+
+To install, place the Conductor `WSGI application
+file </conductor/api/app.wsgi>`__ in ``/var/www/conductor``.
+
+Set the owner/group of both directories/files to ``conductor``:
+
+.. code:: bash
+
+ $ sudo chown -R conductor:conductor /var/log/apache2/conductor /var/www/conductor
+
+Next, place the Conductor `apache2 httpd site config
+file </examples/apache2/conductor.conf>`__ in
+``/etc/apache2/sites-available``.
+
+Set the owner/group to ``root``:
+
+.. code:: bash
+
+ $ sudo chown -R root:root /etc/apache2/sites-available/conductor.conf
+
+If Conductor was installed in a python virtual environment, append
+``python-home=VENV`` to ``WSGIDaemonProcess``, where ``VENV`` is the
+python virtual environment root directory.
+
+**IMPORTANT**: Before proceeding, disable the ``conductor-api`` sysvinit
+and upstart services, as the REST API will now be handled by apache2
+httpd. Otherwise there will be a port conflict, and you will be sad.
+
+Enable the Conductor site, ensure the configuration syntax is valid, and
+gracefully restart apache2 httpd.
+
+.. code:: bash
+
+ $ sudo a2ensite conductor
+ $ sudo apachectl -t
+ Syntax OK
+ $ sudo apachectl graceful
+
+To disable the Conductor site, run ``sudo a2dissite conductor``, then
+gracefully restart once again. Optionally, re-enable the
+``conductor-api`` sysvinit and upstart services.
+
+Running conductor-api Under nginx and uWSGI
+-------------------------------------------
+
+Sample configuration files have been provided in the repository.
+
+These instructions presume a ``conductor`` user exists. See the
+**Service Scripts** section for details.
+
+To install, place the Conductor `nginx config
+files </examples/nginx/>`__ and `WSGI application
+file </conductor/api/app.wsgi>`__ in ``/etc/nginx`` (taking care to
+backup any prior configuration files). It may be desirable to
+incorporate Conductor's ``nginx.conf`` into the existing config.
+
+Rename ``app.wsgi`` to ``conductor.wsgi``:
+
+.. code:: bash
+
+ $ cd /etc/nginx
+ $ sudo mv app.wsgi conductor.wsgi
+
+In ``nginx.conf``, set ``CONDUCTOR_API_FQDN`` to the server name.
+
+**IMPORTANT**: Before proceeding, disable the ``conductor-api`` sysvinit
+and upstart services, as the REST API will now be handled by nginx.
+Otherwise there will be a port conflict, and you will be sad.
+
+Restart nginx:
+
+.. code:: bash
+
+ $ sudo service nginx restart
+
+Then, run ``conductor-api`` under nginx using uWSGI:
+
+.. code:: bash
+
+ $ sudo uwsgi -s /tmp/uwsgi.sock --chmod-socket=777 --wsgi-file /etc/nginx/conductor.wsgi --callable application --set port=8091
+
+To use a python virtual environment, add ``--venv VENV`` to the
+``uwsgi`` command, where ``VENV`` is the python virtual environment root
+directory.
+
+Networking
+----------
+
+All conductor services require line-of-sight access to all Music/ETCD
+servers/ports.
+
+The ``conductor-api`` service uses TCP port 8091.
+
+Security
+--------
+
+``conductor-api`` is accessed via HTTP. SSL/TLS certificates and
+AuthN/AuthZ (e.g., AAF) are supported at this time in kubernetes
+environment.
+
+Conductor makes use of plugins that act as gateways to *inventory
+providers* and *service controllers*. At present, two plugins are
+supported out-of-the-box: **A&AI** and **SDN-C**, respectively.
+
+A&AI requires two-way SSL/TLS. Certificates must be registered and
+whitelisted with A&AI. SDN-C uses HTTP Basic Authentication. Consult
+with each respective service for official information on how to obtain
+access.
+
+Storage
+-------
+
+For a cloud environment in particular, it may be desirable to use a
+separate block storage device (e.g., an OpenStack Cinder volume) for
+logs, configuration, and other data persistence. In this way, it becomes
+a trivial matter to replace the entire VM if necessary, followed by
+reinstallation of the app and any supplemental configuration. Take this
+into consideration when setting various Conductor config options.
diff --git a/docs/sections/installation.rst b/docs/sections/installation.rst
index d4b93d6..217a6dc 100644
--- a/docs/sections/installation.rst
+++ b/docs/sections/installation.rst
@@ -1,239 +1,112 @@
-Python/Linux Distribution Notes
-===============================
-
-*Updated: 10 Nov 2017 23:30 GMT*
-
-This document exists to help bridge the gap between the Conductor python
-package and any downstream distribution. The steps outlined herein may
-be taken into consideration when creating an AT&T SWM package,
-Ubuntu/Debian package, Chef cookbook, or Ansible playbook.
-
-Components
-----------
-
-Conductor consists of five services that work together:
-
-- **``conductor-api``**: An HTTP REST API
-- **``conductor-controller``**: Validation, translation, and
- status/results
-- **``conductor-data``**: Inventory provider and service controller
- gateway
-- **``conductor-solver``**: Processing and solution calculation
-- **``conductor-reservation``**: Reserves the suggested solution solved
- by Solver component.
-
-Workflow
---------
-
-- Deployment **plans** are created, viewed, and deleted via
- ``conductor-api`` and its `REST API <doc/api/README.md>`__.
-- Included within each ``conductor-api`` plan request is a `Homing
- Template <doc/template/README.md>`__.
-- Homing Templates describe a set of inventory demands and constraints
- to be solved against.
-- ``conductor-api`` hands off all API requests to
- ``conductor-controller`` for handling.
-- All deployment plans are assigned a unique identifier (UUID-4), which
- can be used to check for solution status asynchronously. (Conductor
- does not support callbacks at this time.)
-- ``conductor-controller`` ensures templates are well-formed and valid.
- Errors and remediation are made visible through ``conductor-api``.
- When running in debug mode, the API will also include a python
- traceback in the response body, if available.
-- ``conductor-controller`` uses ``conductor-data`` to resolve demands
- against a particular **inventory provider** (e.g., A&AI).
-- ``conductor-controller`` translates the template into a format
- suitable for solving.
-- As each template is translated, ``conductor-solver`` begins working
- on it.
-- ``conductor-solver`` uses ``conductor-data`` to resolve constraints
- against a particular **service controller** (e.g., SDN-C).
-- ``conductor-solver`` determines the most suitable inventory to
- recommend.
-- ``conductor-reservation`` attempts to reserve the solved solution in
- SDN-GC
-
-**NOTE**: There is no Command Line Interface or Python API Library at
-this time.
-
-Pre-Flight and Pre-Installation Considerations
-----------------------------------------------
-
-AT&T Application Identifiers and Roles
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-
-- App/Tool Name: ECOMP Conductor
-- MOTS Application ID: 26213
-- MechID: m04308
-- ECOMP Feature ID: F13704
-- PMT: 461306
-- UAM Role Name: Conductor Production Support
-- UAM Role id: 0000025248
-
-Root
-~~~~
-
-Be aware that some commands may require ``sudo``, depending on the
-account being used to perform the installation.
-
-Proxy
-~~~~~
-
-If line-of-sight to internet-facing repositories is permitted and
-available, set the following shell environment variables if AT&T proxy
-services are required:
+Installation
+============
-.. code:: bash
-
- $ export http_proxy="http://one.proxy.att.com:8080/"
- $ export https_proxy="http://one.proxy.att.com:8080/"
-
-Requirements
-~~~~~~~~~~~~
-
-Conductor is officially supported on `Ubuntu 14.04 LTS (Trusty
-Tahr) <http://releases.ubuntu.com/14.04/>`__, though it should also work
-on newer releases.
+OOF-HAS OOM Charts
+------------------
-Ensure the following Ubuntu packages are present, as they may not be
-included by default:
-
-- `libffi-dev <http://packages.ubuntu.com/trusty/libffi-dev>`__
-- `postgresql-server-dev-9.3 <http://packages.ubuntu.com/trusty/postgresql-server-dev-9.3>`__
-- `python2.7 <http://packages.ubuntu.com/trusty/python2.7>`__
-
-``conductor-api`` may be run as-is for development and test purposes.
-When used in a production environment, it is recommended that
-``conductor-api`` run under a multithreaded httpd service supporting
-`WSGI <https://www.wikipedia.org/wiki/Web_Server_Gateway_Interface>`__,
-tuned as appropriate.
+HAS charts are located in the `OOM repository <https://git.onap.org/oom/>`__
-Configuration instructions for **apache2 httpd** and **nginx** are
-included herein. Respective package requirements are:
+Please refer OOM documentation for deploying/undeploying the OOF compoenents
+via helm charts in the k8s environment.
-- `apache2 <http://packages.ubuntu.com/trusty/apache2>`__ and
- `libapache2-mod-wsgi <http://packages.ubuntu.com/trusty/libapache2-mod-wsgi>`__
-- `nginx <http://packages.ubuntu.com/trusty/nginx>`__ and
- `uwsgi <http://packages.ubuntu.com/trusty/uwsgi>`__
+Local Installation
+------------------
-All Conductor services use AT&T `Music <https://github.com/att/music>`__
-for data storage/persistence and/or as a RPC transport mechanism.
-Consult the `Music Local Installation
-Guide <https://github.com/att/music/blob/master/README.md>`__ for
-installation/configuration steps.
+HAS components can be deployed in two ways in a local environment for development
+and testing.
-Networking
-~~~~~~~~~~
+Docker Installation
+-------------------
-All conductor services require line-of-sight access to all Music
-servers/ports.
-
-The ``conductor-api`` service uses TCP port 8091.
-
-Security
-~~~~~~~~
+Building Docker Images
+~~~~~~~~~~~~~~~~~~~~~~
-``conductor-api`` is accessed via HTTP. SSL/TLS certificates and
-AuthN/AuthZ (e.g., AAF) are not supported at this time.
+Build the HAS docker images using the maven build from the root of the project
-Conductor makes use of plugins that act as gateways to *inventory
-providers* and *service controllers*. At present, two plugins are
-supported out-of-the-box: **A&AI** and **SDN-C**, respectively.
+.. code:: bash
-A&AI requires two-way SSL/TLS. Certificates must be registered and
-whitelisted with A&AI. SDN-C uses HTTP Basic Authentication. Consult
-with each respective service for official information on how to obtain
-access.
+ git clone --depth 1 https://gerrit.onap.org/r/optf/has
+ cd has
+ mvn clean install
-Storage
-~~~~~~~
+Installing the components and simulators
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-For a cloud environment in particular, it may be desirable to use a
-separate block storage device (e.g., an OpenStack Cinder volume) for
-logs, configuration, and other data persistence. In this way, it becomes
-a trivial matter to replace the entire VM if necessary, followed by
-reinstallation of the app and any supplemental configuration. Take this
-into consideration when setting various Conductor config options.
+HAS docker containers can be installed using the shell scripts in the CSIT directory
+which includes the script to deploy the startup dependencies(SMS, ETCD) and a
+few simulators.
-Python Virtual Environments
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+.. code:: bash
-At present, Conductor installation is only supported at the (upstream)
-python package level and not the (downstream) Ubuntu distribution or SWM
-package levels.
+ export WORKSPACE=$(pwd)/csit
+ ./csit/plans/default/setup.sh
-To mitigate/eliminate the risk of introducing conflicts with other
-python applications or Ubuntu/SWM package dependencies, consider
-installing Conductor in a `python virtual
-environment <http://docs.python-guide.org/en/latest/dev/virtualenvs/>`__
-(or *venv* for short).
+Similarly the installed components can be deleted using the teardown script.
-Example venv-aware WSGI app configurations, sysvinit scripts, and
-upstart scripts can be found in the Conductor repository under
-`examples </examples/>`__.
+.. code:: bash
-Python Package Dependencies
-~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ export WORKSPACE=$(pwd)/csit
+ ./csit/plans/default/teardown.sh
-Conductor is installed using the python ``pip`` command. ``pip`` uses a
-python project's `requirements manifest </requirements.txt>`__ to
-install all python module dependencies.
+Note: The simulator setup can be disabled by the commenting out the commands from
+the setup scripts.
-**NOTE**: When line-of-sight access to a PyPI-compatible package index
-is not available, advance installation of Conductor's python package
-dependencies is required *before* installation.
+Installation from the source
+----------------------------
-Other Production Environment Considerations
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+HAS components can be installed locally by directly in a linux based environment.
+This will be significant for testing and debugging during developme
-TBD. ``:)``
+Requirements
+~~~~~~~~~~~~
-Over time, considerations may include services such as:
+Conductor is officially supported on most of the linux based environment, but of
+the development and testing were done on Ubuntu based machines.
-- AAF
-- AppMetrics
-- Introscope
-- Nagios
-- Splunk
-- UAM
+Ensure the following packages are present, as they may not be
+included by default:
-Installation and Configuration
-------------------------------
+- libffi-dev
+- python3.8
-**IMPORTANT**: Perform the steps in this section after *optionally*
-configuring and activating a python virtual environment.
+Installing Dependent Components(AAF-SMS, ETCD/MUSIC)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Installing From a PyPI Repository
-~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+The scripts to install and uninstall the components are present in the
+CSIT directory.
-In ONAP, the ``conductor`` package can be found on \`\`\`\`.
+**Note**: For setting up SMS, ETCD and MUSIC, docker must be present in
+the machine.
-Installation is via the ``pip`` command. Here is an example ``pip.conf``
-file that uses both the internet and intranet-facing PyPI repositories:
+For installing/uninstalling AAF-SMS,
-.. code:: ini
+.. code:: bash
- [global]
- index = https://pypi.python.org/pypi
- index-url = https://pypi.python.org/simple
- extra-index-url =
- trusted-host =
+ cd csit/scripts
+ # install SMS
+ source setup-sms.sh
+ # uninstall SMS
+ docker stop sms
+ docker stop vault
+ docker rm sms
+ docker rm vault
-Once the configuration is in place, installation is simple:
+For installing/uninstalling ETCD
.. code:: bash
- $ pip install --no-cache-dir of-has
-
-To upgrade or downgrade, simply re-run ``pip install --no-cache-dir`` using the
-appropriate ``pip`` command line options.
-
-**NOTE**: Be sure proxy settings are in place if they're required to
-access ``pypi.python.org``.
+ cd csit/scripts
+ # install etcd
+ source etcd_Script.sh
+ # uninstall etcd
+ source etcd_teardown_script.sh
Installing From Source
~~~~~~~~~~~~~~~~~~~~~~
+**IMPORTANT**: Perform the steps in this section after *optionally*
+configuring and activating a python virtual environment.
+
Conductor source in ONAP is maintained in
https://gerrit.onap.org/r/optf/has.
@@ -242,23 +115,9 @@ directory:
.. code:: bash
- $ git clone --depth 1 https://gerrit.onap.org/r/optf/has
- Cloning into 'conductor'...
- remote: Counting objects: 2291, done.
- remote: Compressing objects: 88% (1918/2179)
- remote: Compressing objects: 100% (2179/2179), done.
- remote: Total 2291 (delta 1422), reused 0 (delta 0)
- Receiving objects: 100% (2291/2291), 477.59 KiB | 0 bytes/s, done.
- Resolving deltas: 100% (1422/1422), done.
- $ cd conductor
- $ pip install --no-cache-dir .
-
-The latest source can be pulled from ONAP at any time and reinstalled:
-
-.. code:: bash
-
- $ git pull
- $ pip install --no-cache-dir .
+ git clone --depth 1 https://gerrit.onap.org/r/optf/has
+ cd conductor
+ pip install --no-cache-dir -e .
Verifying Installation
~~~~~~~~~~~~~~~~~~~~~~
@@ -268,227 +127,16 @@ option:
.. code:: bash
- $ conductor-api -- --help
- $ conductor-controller --help
- $ conductor-data --help
- $ conductor-solver --help
- $ conductor-reservation --help
+ conductor-api -- --help
+ conductor-controller --help
+ conductor-data --help
+ conductor-solver --help
+ conductor-reservation --help
**NOTE**: The ``conductor-api`` command is deliberate. ``--`` is used as
as separator between the arguments used to start the WSGI server and the
arguments passed to the WSGI application.
-Post-Flight and Post-Installation Considerations
-------------------------------------------------
-
-User and Group
-~~~~~~~~~~~~~~
-
-It's good practice to create an unprivileged account (e.g., a user/group
-named ``conductor``) and run all Conductor services as that user:
-
-.. code:: bash
-
- $ sudo addgroup --system conductor
- $ sudo adduser --system --home /var/lib/conductor --ingroup conductor --no-create-home --shell /bin/false conductor
-
-SSL/TLS Certificates
-~~~~~~~~~~~~~~~~~~~~
-
-The A&AI Inventory Provider Plugin requiries two-way SSL/TLS. After
-provisioning a certificate per A&AI guidelines, it will be necessary to
-securely install the certificate, key, and certificate authority bundle.
-
-When running conductor services as ``conductor:conductor``
-(recommended), consider co-locating all of these files under the
-configuration directory. For example, when using ``/etc/conductor``:
-
-.. code:: bash
-
- $ # Certificate files (crt extension, 644 permissions)
- $ sudo mkdir /etc/conductor/ssl/certs
- $ # Private Certificate Key files (key extension, 640 permissions)
- $ sudo mkdir /etc/conductor/ssl/private
- $ # Certificate Authority (CA) Bundles (crt extension, 644 permissions)
- $ sudo mkdir /etc/conductor/ssl/ca-certificates
- $ # Add files to newly created directories, then set ownership
- $ sudo chmod -R conductor:conductor /etc/conductor/ssl
-
-For a hypothetical domain name ``imacculate.client.research.att.com``,
-example filenames could be as follows:
-
-.. code:: bash
-
- $ find ssl -type f -printf '%M %u:%g %f\n'
- -rw-r----- conductor:conductor imacculate.client.research.att.com.key
- -rw-r--r-- conductor:conductor Symantec_Class_3_Secure_Server_CA.crt
- -rw-r--r-- conductor:conductor imacculate.client.research.att.com.crt
-
-When running conductor services as ``root``, consider these existing
-Ubuntu filesystem locations for SSL/TLS files:
-
-**Certificate** files (``crt`` extension) are typically stored in
-``/etc/ssl/certs`` with ``root:root`` ownership and 644 permissions.
-
-**Private Certificate Key** files (``key`` extension) are typically
-stored in ``/etc/ssl/private`` with ``root:root`` ownership and 640
-permissions.
-
-**Certificate Authority (CA) Bundles** (``crt`` extension) are typically
-stored in ``/usr/share/ca-certificates/conductor`` with ``root:root``
-ownership, and 644 permissions. These Bundle files are then symlinked
-within ``/etc/ssl/certs`` using equivalent filenames, a ``pem``
-extension, and ``root:root`` ownership.
-
-**NOTE**: LUKS (Linux Unified Key Setup) is not supported by Conductor
-at this time.
-
-Configuration
-~~~~~~~~~~~~~
-
-Configuration files are located in ``etc/conductor`` relative to the
-python environment Conductor is installed in.
-
-To generate a sample configuration file, change to the directory just
-above where ``etc/conductor`` is located (e.g., ``/`` for the default
-environment, or the virtual environment root directory). Then:
-
-.. code:: bash
-
- $ oslo-config-generator --config-file=etc/conductor/conductor-config-generator.conf
-
-This will generate ``etc/conductor/conductor.conf.sample``.
-
-Because the configuration directory and files will include credentials,
-consider removing world permissions:
-
-.. code:: bash
-
- $ find etc/conductor -type f -exec chmod 640 {} +
- $ find etc/conductor -type d -exec chmod 750 {} +
-
-The sample config may then be copied and edited. Be sure to backup any
-previous ``conductor.conf`` if necessary.
-
-.. code:: bash
-
- $ cd etc/conductor
- $ cp -p conductor.conf.sample conductor.conf
-
-``conductor.conf`` is fully annotated with descriptions of all options.
-Defaults are included, with all options commented out. Conductor will
-use defaults even if an option is not present in the file. To change an
-option, simply uncomment it and edit its value.
-
-With the exception of the ``DEFAULT`` section, it's best to restart the
-Conductor services after making any config changes. In some cases, only
-one particular service actually needs to be restarted. When in doubt,
-however, it's best to restart all of them.
-
-A few options in particular warrant special attention:
-
-::
-
- [DEFAULT]
-
- # If set to true, the logging level will be set to DEBUG instead of the default
- # INFO level. (boolean value)
- # Note: This option can be changed without restarting.
- #debug = false
-
-For more verbose logging across all Conductor services, set ``debug`` to
-true.
-
-::
-
- [aai]
-
- # Base URL for A&AI, up to and not including the version, and without a
- # trailing slash. (string value)
- #server_url = https://controller:8443/aai
-
- # SSL/TLS certificate file in pem format. This certificate must be registered
- # with the A&AI endpoint. (string value)
- #certificate_file = certificate.pem
-
- # Private Certificate Key file in pem format. (string value)
- #certificate_key_file = certificate_key.pem
-
- # Certificate Authority Bundle file in pem format. Must contain the appropriate
- # trust chain for the Certificate file. (string value)
- #certificate_authority_bundle_file = certificate_authority_bundle.pem
-
-Set ``server_url`` to the A&AI server URL, to but not including the
-version, omitting any trailing slash. Conductor supports A&AI API v9 at
-a minimum.
-
-Set the ``certificate`` prefixed keys to the appropriate SSL/TLS-related
-files.
-
-**IMPORTANT**: The A&AI server may have a mismatched host/domain name
-and SSL/TLS certificate. In such cases, certificate verification will
-fail. To mitigate this, ``certificate_authority_bundle_file`` may be set
-to an empty value. While Conductor normally requires a CA Bundle
-(otherwise why bother using SSL/TLS), this requirement has been
-temporarily relaxed so that development and testing may continue.
-
-::
-
- [messaging_server]
-
- # Log debug messages. Default value is False. (boolean value)
- #debug = false
-
-When the ``DEFAULT`` section's ``debug`` option is ``true``, set this
-section's ``debug`` option to ``true`` to enable detailed Conductor-side
-RPC-over-Music debug messages.
-
-Be aware, it is voluminous. "You have been warned." ``:)``
-
-::
-
- [music_api]
-
- # List of hostnames (round-robin access) (list value)
- #hostnames = localhost
-
- # Log debug messages. Default value is False. (boolean value)
- #debug = false
-
-Set ``hostnames`` to match wherever the Music REST API is being hosted
-(wherever Apache Tomcat and ``MUSIC.war`` are located).
-
-When the ``DEFAULT`` section's ``debug`` option is ``true``, set this
-section's ``debug`` option to ``true`` to enable detailed Conductor-side
-MUSIC API debug messages.
-
-The previous comment around the volume of log lines applies even more so
-here. (Srsly. We're not kidding.)
-
-**IMPORTANT**: Conductor does not presently use Music's atomic
-consistency features due to concern around lock creation/acquisition.
-Instead, Conductor uses eventual consistency. For this reason,
-consistency issues may occur when using Music in a multi-server, High
-Availability configuration.
-
-::
-
- [sdnc]
-
- # Base URL for SDN-C. (string value)
- #server_url = https://controller:8443/restconf
-
- # Basic Authentication Username (string value)
- #username = <None>
-
- # Basic Authentication Password (string value)
- #password = <None>
-
-Set ``server_url`` to the SDN-C server URL, omitting any trailing slash.
-
-Set ``username`` and ``password`` to the appropriate values as directed
-by SDN-C.
-
Running for the First Time
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -506,51 +154,6 @@ separate terminal session and in the following order:
conductor-reservation --config-file=/etc/conductor/conductor.conf
conductor-api --port=8091 -- --config-file=/etc/conductor/conductor.conf
-Optionally, use an application like
-`screen <http://packages.ubuntu.com/trusty/screen>`__ to nest all five
-terminal sessions within one detachable session. (This is also the same
-package used by
-`DevStack <https://docs.openstack.org/developer/devstack/>`__.)
-
-To verify that ``conductor-api`` can be reached, browse to
-``http://HOST:8091/``, where HOST is the hostname ``conductor-api`` is
-running on. No AuthN/AuthZ is required at this time. Depending on
-network considerations, it may be necessary to use a command like
-``wget`` instead of a desktop browser.
-
-The response should look similar to:
-
-.. code:: json
-
- {
- "versions": {
- "values": [
- {
- "status": "development",
- "updated": "2016-11-01T00:00:00Z",
- "media-types": [
- {
- "base": "application/json",
- "type": "application/vnd.ecomp.homing-v1+json"
- }
- ],
- "id": "v1",
- "links": [
- {
- "href": "http://127.0.0.1:8091/v1",
- "rel": "self"
- },
- {
- "href": "http://conductor.research.att.com/",
- "type": "text/html",
- "rel": "describedby"
- }
- ]
- }
- ]
- }
- }
-
Sample API Calls and Homing Templates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -558,192 +161,5 @@ A `Postman <http://getpostman.com/>`__ collection illustrating sample
requests is available upon request. The collection will also be added in
a future revision.
-`Sample homing templates </doc/examples/README.md>`__ are also
+`Sample homing templates <example.html>`__ are also
available.
-
-Ubuntu Service Scripts
-~~~~~~~~~~~~~~~~~~~~~~
-
-Ubuntu sysvinit (init.d) and upstart (init) scripts are typically
-installed at the Ubuntu package level. Since there is no such packaging
-at this time, example scripts have been provided in the repository.
-
-To install, place all Conductor `sysvinit
-scripts </examples/distribution/ubuntu/init.d>`__ in ``/etc/init.d``,
-and all `upstart scripts </examples/distribution/ubuntu/init>`__ in
-``/etc/init``.
-
-Set file permissions:
-
-.. code:: bash
-
- $ sudo chmod 644 /etc/init/conductor*
- $ sudo chmod 755 /etc/init.d/conductor*
-
-If a python virtual environment is being used, edit each
-``/etc/init/conductor*`` and ``/etc/init.d/conductor*`` prefixed file so
-that ``PYTHON_HOME`` is set to the python virtual environment root
-directory.
-
-Next, enable the scripts:
-
-.. code:: bash
-
- $ sudo update-rc.d conductor-api defaults
- $ sudo update-rc.d conductor-controller defaults
- $ sudo update-rc.d conductor-data defaults
- $ sudo update-rc.d conductor-solver defaults
- $ sudo update-rc.d conductor-reservation defaults
- $ sudo initctl reload-configuration
-
-Conductor components may now be started/stopped like any other Ubuntu
-service, for example:
-
-.. code:: bash
-
- $ sudo service conductor-api start
- $ sudo service conductor-api status
- $ sudo service conductor-api restart
- $ sudo service conductor-api stop
-
-Conductor service scripts automatically create directories for ``log``,
-``lock``, ``run``, ``lib``, and ``log`` files, e.g.,
-``/var/log/conductor`` and so on.
-
-Log File Rotation
-~~~~~~~~~~~~~~~~~
-
-Sample ``logrotate.d`` configuration files have been provided in the
-repository.
-
-To install, place all Conductor `logrotate
-files </examples/distribution/ubuntu/logrotate.d>`__ in
-``/etc/logrotate.d``.
-
-Set file ownership and permissions:
-
-.. code:: bash
-
- $ sudo chown root:root /etc/logrotate.d/conductor*
- $ sudo chmod 644 /etc/logrotate.d/conductor*
-
-``logrotate.d`` automatically recognizes new files at the next log
-rotation opportunity and does not require restarting.
-
-Running conductor-api Under apache2 httpd and mod\_wsgi
--------------------------------------------------------
-
-Sample configuration files have been provided in the repository.
-
-These instructions presume a ``conductor`` user exists. See the
-**Service Scripts** section for details.
-
-First, set up a few directories:
-
-.. code:: bash
-
- $ sudo mkdir -p /var/www/conductor
- $ sudo mkdir /var/log/apache2/conductor
-
-To install, place the Conductor `WSGI application
-file </conductor/api/app.wsgi>`__ in ``/var/www/conductor``.
-
-Set the owner/group of both directories/files to ``conductor``:
-
-.. code:: bash
-
- $ sudo chown -R conductor:conductor /var/log/apache2/conductor /var/www/conductor
-
-Next, place the Conductor `apache2 httpd site config
-file </examples/apache2/conductor.conf>`__ in
-``/etc/apache2/sites-available``.
-
-Set the owner/group to ``root``:
-
-.. code:: bash
-
- $ sudo chown -R root:root /etc/apache2/sites-available/conductor.conf
-
-If Conductor was installed in a python virtual environment, append
-``python-home=VENV`` to ``WSGIDaemonProcess``, where ``VENV`` is the
-python virtual environment root directory.
-
-**IMPORTANT**: Before proceeding, disable the ``conductor-api`` sysvinit
-and upstart services, as the REST API will now be handled by apache2
-httpd. Otherwise there will be a port conflict, and you will be sad.
-
-Enable the Conductor site, ensure the configuration syntax is valid, and
-gracefully restart apache2 httpd.
-
-.. code:: bash
-
- $ sudo a2ensite conductor
- $ sudo apachectl -t
- Syntax OK
- $ sudo apachectl graceful
-
-To disable the Conductor site, run ``sudo a2dissite conductor``, then
-gracefully restart once again. Optionally, re-enable the
-``conductor-api`` sysvinit and upstart services.
-
-Running conductor-api Under nginx and uWSGI
--------------------------------------------
-
-Sample configuration files have been provided in the repository.
-
-These instructions presume a ``conductor`` user exists. See the
-**Service Scripts** section for details.
-
-To install, place the Conductor `nginx config
-files </examples/nginx/>`__ and `WSGI application
-file </conductor/api/app.wsgi>`__ in ``/etc/nginx`` (taking care to
-backup any prior configuration files). It may be desirable to
-incorporate Conductor's ``nginx.conf`` into the existing config.
-
-Rename ``app.wsgi`` to ``conductor.wsgi``:
-
-.. code:: bash
-
- $ cd /etc/nginx
- $ sudo mv app.wsgi conductor.wsgi
-
-In ``nginx.conf``, set ``CONDUCTOR_API_FQDN`` to the server name.
-
-**IMPORTANT**: Before proceeding, disable the ``conductor-api`` sysvinit
-and upstart services, as the REST API will now be handled by nginx.
-Otherwise there will be a port conflict, and you will be sad.
-
-Restart nginx:
-
-.. code:: bash
-
- $ sudo service nginx restart
-
-Then, run ``conductor-api`` under nginx using uWSGI:
-
-.. code:: bash
-
- $ sudo uwsgi -s /tmp/uwsgi.sock --chmod-socket=777 --wsgi-file /etc/nginx/conductor.wsgi --callable application --set port=8091
-
-To use a python virtual environment, add ``--venv VENV`` to the
-``uwsgi`` command, where ``VENV`` is the python virtual environment root
-directory.
-
-Uninstallation
---------------
-
-Activate a virtual environment (venv) first, if necessary, then
-uninstall with:
-
-.. code:: bash
-
- $ pip uninstall ecomp-conductor
-
-Remove any previously made configuration file changes, user accounts,
-Ubuntu/SWM packages, and other settings as needed.
-
-Bug Reporting and Feedback
---------------------------
-
-... is encouraged. Please raise an issue at:
-https://jira.onap.org/projects/OPTFRA/summary
diff --git a/docs/sections/logging.rst b/docs/sections/logging.rst
index 86b63f9..2078ade 100644
--- a/docs/sections/logging.rst
+++ b/docs/sections/logging.rst
@@ -13,4 +13,24 @@ Timestamp|RequestId|ServiceInstanceId|ThreadId|Virtual Server Name|ServiceName|I
The logger util module can be found at:
-<>/has/conductor/conductor/common/utils/conductor_logging_util.py \ No newline at end of file
+<>/has/conductor/conductor/common/utils/conductor_logging_util.py
+
+Log File Rotation
+-----------------
+
+Sample ``logrotate.d`` configuration files have been provided in the
+repository.
+
+To install, place all Conductor `logrotate
+files </examples/distribution/ubuntu/logrotate.d>`__ in
+``/etc/logrotate.d``.
+
+Set file ownership and permissions:
+
+.. code:: bash
+
+ $ sudo chown root:root /etc/logrotate.d/conductor*
+ $ sudo chmod 644 /etc/logrotate.d/conductor*
+
+``logrotate.d`` automatically recognizes new files at the next log
+rotation opportunity and does not require restarting.