diff options
| -rw-r--r-- | docs/db-migrator/policy-db-migrator.rst | 297 | ||||
| -rw-r--r-- | docs/installation/oom.rst | 46 |
2 files changed, 326 insertions, 17 deletions
diff --git a/docs/db-migrator/policy-db-migrator.rst b/docs/db-migrator/policy-db-migrator.rst new file mode 100644 index 00000000..64e7c6d8 --- /dev/null +++ b/docs/db-migrator/policy-db-migrator.rst @@ -0,0 +1,297 @@ +.. This work is licensed under a Creative Commons Attribution +.. 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +Using Policy DB Migrator +######################## + +Policy DB Migrator is a set of shell scripts used to +install the database tables required to run ONAP Policy Framework. + +.. note:: + Currently the Istanbul versions of the PAP and API components require + ``db-migrator`` to run prior to initialization. + +Package contents +================ + +Policy DB Migrator is run as a docker container and consists of the following scripts: + +.. code:: + :number-lines: + + prepare_upgrade.sh + prepare_downgrade.sh + db-migrator + + +``prepare_upgrade.sh`` is included as part of the docker image and is used +to copy the upgrade sql files to the run directory. +This script takes one parameter: <SCHEMA NAME>. + +``prepare_downgrade.sh`` is included as part of the docker image and is used +to copy the downgrade sql files to the run directory. +This script takes one parameter: <SCHEMA NAME>. + +``db-migrator`` is included as part of the docker image and is used +to run either the upgrade or downgrade operation depending on user requirements. +This script can take up to four parameters: + +.. list-table:: + :widths: 20 20 20 + :header-rows: 1 + + * - Parameter Name + - Parameter flag + - Value (example) + * - operation + - -o + - upgrade/downgrade/report + * - schema + - -s + - policyadmin + * - to + - -t + - 0800/0900 + * - from + - -f + - 0800/0900 + +The container also consists of several sql files which are used to upgrade/downgrade +the policy database. + +The following environment variables need to be set to enable ``db-migrator`` +to run and connect to the database. + +.. list-table:: + :widths: 20 20 + :header-rows: 1 + + * - Name + - Value (example) + * - SQL_HOST + - mariadb + * - SQL_DB + - policyadmin + * - SQL_USER + - policy_user + * - SQL_PASSWORD + - policy_user + * - POLICY_HOME + - /opt/app/policy + +Prepare Upgrade +=============== + +Prior to upgrading the following script is run: + +.. code:: + + /opt/app/policy/bin/prepare_upgrade.sh <SCHEMA NAME> + +This will copy the upgrade files from ``/home/policy/sql`` to ``$POLICY_HOME/etc/db/migration/<SCHEMA NAME>/sql/`` + +Each individual sql file that makes up that release will be run as part of the upgrade. + + +Prepare Downgrade +================= + +Prior to downgrading the following script is run: +.. code:: + + /opt/app/policy/bin/prepare_downgrade.sh <SCHEMA NAME> + +This will copy the downgrade files from ``/home/policy/sql`` to ``$POLICY_HOME/etc/db/migration/<SCHEMA NAME>/sql/`` + +Each individual sql file that makes up that release will be run as part of the downgrade. + +Upgrade +======= + +.. code:: + + /opt/app/policy/bin/db-migrator -s <SCHEMA NAME> -o upgrade -f 0800 -t 0900 + +If the ``-f`` and ``-t`` flags are not specified, the script will attempt to run all available +sql files greater than the current version. + +The script will return either 1 or 0 depending on successful completion. + +Downgrade +========= + +.. code:: + + /opt/app/policy/bin/db-migrator -s <SCHEMA NAME> -o downgrade -f 0900 -t 0800 + +If the ``-f`` and ``-t`` flags are not specified, the script will attempt to run all available +sql files less than the current version. + +The script will return either 1 or 0 depending on successful completion. + +Logging +======= + + .. container:: paragraph + +After every upgrade/downgrade ``db-migrator`` runs the report operation to show the +contents of the db-migrator log table. + +.. code:: + + /opt/app/policy/bin/db-migrator -s <SCHEMA NAME> -o report + +Console output will also show the sql script command as in the example below: + +.. code:: + + upgrade 0100-jpapdpgroup_properties.sql + -------------- + CREATE TABLE IF NOT EXISTS jpapdpgroup_properties (name VARCHAR(120) NULL, version VARCHAR(20) NULL, + PROPERTIES VARCHAR(255) NULL, PROPERTIES_KEY VARCHAR(255) NULL) + + +migration schema +================ + +The migration schema contains two tables which belong to ``db-migrator``. + +* schema_versions - table to store the schema version currently installed by ``db-migrator`` + +.. list-table:: + :widths: 20 20 + :header-rows: 1 + + * - name + - version + * - policyadmin + - 0900 + +* policyadmin_schema_changelog - table which stores a record of each sql file that has been run + +.. list-table:: + :widths: 10 40 10 10 10 20 10 20 + :header-rows: 1 + + * - ID + - script + - operation + - from_version + - to_version + - tag + - success + - atTime + * - 1 + - 0100-jpapdpgroup_properties.sql + - upgrade + - 0 + - 0800 + - 1309210909250800u + - 1 + - 2021-09-13 09:09:26 + +* ID: Sequence number of the operation +* script: name of the sql script which was run +* operation: operation type - upgrade/downgrade +* from_version: starting version +* to_version: target version +* tag: tag to identify operation batch +* success: 1 if script succeeded and 0 if it failed +* atTime: time script was run + + +Partial Upgrade/Downgrade +========================= + +If an upgrade or downgrade ends with a failure status (success=0) the next time an upgrade +or downgrade is run it will start from the point of failure rather than re-run scripts +that succeeded. This allows the user to perform a partial upgrade or downgrade depending +on their requirements. + +Running db-migrator +=================== + +The script that runs ``db-migrator`` is part of the database configuration and is in the following directory: + +.. code:: + + oom/kubernetes/policy/resources/config/db_migrator_policy_init.sh + +This script is mounted from the host file system to the policy-db-migrator container. +It is setup to run an upgrade by default. + +.. code:: + + /opt/app/policy/bin/prepare_upgrade.sh ${SQL_DB} + /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o upgrade + rc=$? + /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o report + exit $rc + +The following table describes what each line does: + +.. list-table:: + :widths: 30 30 + :header-rows: 1 + + * - code + - description + * - /opt/app/policy/bin/prepare_upgrade.sh ${SQL_DB} + - prepare the upgrade scripts for the <SQL_DB> schema + * - /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o upgrade + - run the upgrade + * - rc=$? + - assign the return code from db-migrator to a variable + * - /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o report + - run the db-migrator report for the <SQL_DB> schema + * - exit $rc + - exit with the return code from db-migrator + +To alter how ``db-migrator`` is run the first two lines need to be modified. +The first line can be changed to call either ``prepare_upgrade.sh`` or ``prepare_downgrade.sh``. +The second line can be changed to use different input parameters for ``db-migrator`` : + +.. list-table:: + :widths: 10 20 10 + :header-rows: 1 + + * - flag + - value + - required + * - ``-o`` + - upgrade/downgrade + - ``Y`` + * - ``-s`` + - ${SQL_DB} + - ``Y`` + * - ``-f`` + - current version (e.g. 0800) + - ``N`` + * - ``-t`` + - target version (e.g. 0900) + - ``N`` + +This is an example of how a downgrade from version 0900 to version 0800 could be run: + +.. code:: + + /opt/app/policy/bin/prepare_downgrade.sh ${SQL_DB} + /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o downgrade -f 0900 -t 0800 + rc=$? + /opt/app/policy/bin/db-migrator -s ${SQL_DB} -o report + exit $rc + +Additional Information +====================== +If the target version of your upgrade or downgrade is the same as the current version, +no sql files are run. + +If an upgrade is run on a database where tables already exist in the policy schema, the +current schema version is set to 0800 and only sql scripts from later versions are run. + +.. note:: + It is advisable to take a backup of your database prior to running this utility. + Please refer to the mariadb documentation on how to do this. + +End of Document diff --git a/docs/installation/oom.rst b/docs/installation/oom.rst index 8485bb74..8e7c585d 100644 --- a/docs/installation/oom.rst +++ b/docs/installation/oom.rst @@ -50,6 +50,12 @@ The assumption is you have cloned the charts from the OOM repository into a loca From your local copy, edit any of the values.yaml files in the policy tree to make desired changes. +The policy schema will be installed automatically as part of the database configuration using ``db-migrator``. +By default the policy schema is upgraded to the latest version. +For more information on how to change the ``db-migrator`` setup please see: `Using Policy DB Migrator`_. + +.. _Using Policy DB Migrator: ../db-migrator/policy-db-migrator.html + **Step 2** Build the charts .. code-block:: bash @@ -65,39 +71,45 @@ After undeploying policy, loop on monitoring the policy pods until they go away. .. code-block:: bash - helm del --purge dev-policy + helm undeploy dev-policy kubectl get pods -n onap | grep dev-policy -**Step 4** Delete NFS persisted data for Policy + +**Step 4** Re-Deploy Policy pods + +After deploying policy, loop on monitoring the policy pods until they come up. .. code-block:: bash - rm -fr /dockerdata-nfs/dev/policy + helm deploy dev-policy local/onap --namespace onap + kubectl get pods -n onap | grep dev-policy -**Step 5** Make sure there is no orphan policy database persistent volume or claim. +.. note:: + If you want to purge the existing data and start with a clean install, + please follow these steps after undeploying: -First, find if there is an orphan database PV or PVC with the following commands: + **Step 1** Delete NFS persisted data for Policy -.. code-block:: bash + .. code-block:: bash - kubectl get pvc -n onap | grep policy - kubectl get pv -n onap | grep policy + rm -fr /dockerdata-nfs/dev/policy -If there are any orphan resources, delete them with + **Step 2** Make sure there is no orphan policy database persistent volume or claim. -.. code-block:: bash + First, find if there is an orphan database PV or PVC with the following commands: - kubectl delete pvc <orphan-policy-mariadb-resource> - kubectl delete pv <orphan-policy-mariadb-resource> + .. code-block:: bash -**Step 6** Re-Deploy Policy pods + kubectl get pvc -n onap | grep policy + kubectl get pv -n onap | grep policy -After deploying policy, loop on monitoring the policy pods until they come up. + If there are any orphan resources, delete them with -.. code-block:: bash + .. code-block:: bash + + kubectl delete pvc <orphan-policy-mariadb-resource> + kubectl delete pv <orphan-policy-mariadb-resource> - helm deploy dev-policy local/onap --namespace onap - kubectl get pods -n onap | grep dev-policy Restarting a faulty component ***************************** |