5.1.0 policy-handler - policy-updates from new PDP5.1.0

DCAEGEN2-1851:
- policy-handler now supports the policy-update notification
  from the new policy-engine thru DMaaP MR
  = no policy-filters - only policy-id values
- see README for discoverable config settings of dmaap_mr client
  = DMaaP MR client has the same flexibility as policy_engine
  = set the query.timeout to high value like 15000 (default)
- requests to DMaaP MR go through a single blocking connection
- first catch-up only after draining the policy-updates from DMaaP MR
  on the first loop
- safe parsing of messages from DMaaP MR
- policy-engine changed the data type for policy-version field
  from int to string that is expected to have the semver value
- related change to deployment-handler (DCAEGEN2-2085) has to be
  deployed to handle the non-numeric policyVersion
- on new PDP API: http /policy_latest and policy-updates
  return the new data from the new PDP API with the following fields
  added/renamed by the policy-handler to keep other policy related parts
  intact in R4-R6 (see pdp_api/policy_utils.py)
  * policyName = policy_id + "." + policyVersion.replace(".","-")
                                 + ".xml"
  * policyVersion = str(metadata["policy-version"])
  * "config" - is the renamed "properties" from the new PDP API response
- enabled the /catch_up and the periodic auto-catch-up for the new PDP
  API
- enabled GET /policies_latest - returns the latest policies for the
  deployed components
- POST /policies_latest - still disabled since no support for the
  policy-filters is provided for the new PDP API
- fixed hiding the Authorization value on comparing the configs
- logging of secrets is now sha256 to see whether they changed
- added X-ONAP-RequestID to headers the same way as X-ECOMP-RequestID
- on policy-update process the removal first, then addition
- changed the pool_connections=1 (number of pools) on PDP and DH sides
  == only a single destination is expected for each
- log the exception as fatal into error.log
- other minor fixes and refactoring
- unit-test coverage 74%
- integration testing is requested

DCAEGEN2-1976:
- policy-handler is enhanced to get user/password from env vars
  for PDP and DMaaP MR clients and overwriting the Authorization field
  in https headers received from the discoverable config
  = to override the Authorization value on policy_engine,
    set the environment vars $PDP_USER and $PDP_PWD in policy-handler
    container
  = to override the Authorization value on dmaap_mr,
    if using https and user-password authentication,
    set the environment vars $DMAAP_MR_USER and $DMAAP_MR_PWD in
    policy-handler container

Change-Id: Iad8eab9e20e615a0e0d2822f4735dc64c50aa55c
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-1851
Issue-ID: DCAEGEN2-1976

1 files changed, 0 insertions, 96 deletions

diff --git a/tests/main/mock_policy_engine.py b/tests/main/mock_policy_engine.py
deleted file mode 100644
index 1bac045..0000000
--- a/tests/main/mock_policy_engine.py
+++ /dev/null
@@ -1,96 +0,0 @@
-# ============LICENSE_START=======================================================
-# Copyright (c) 2018-2019 AT&T Intellectual Property. All rights reserved.
-# ================================================================================
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#      http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-# ============LICENSE_END=========================================================
-#
-"""mocking for the policy-engine - shared by many tests"""
-
-import copy
-import json
-
-from policyhandler.pdp_api.pdp_consts import (PDP_METADATA,
-                                              PDP_POLICY_ID,
-                                              PDP_POLICY_VERSION,
-                                              PDP_POLICIES, PDP_PROPERTIES)
-from policyhandler.pdp_api.policy_utils import PolicyUtils
-from policyhandler.utils import Utils
-
-_LOGGER = Utils.get_logger(__file__)
-
-
-class MockPolicyEngine(object):
-    """pretend this is the policy-engine"""
-    scope_prefix = "test_scope_prefix.pdp_desition_"
-    LOREM_IPSUM = """Lorem ipsum dolor sit amet consectetur ametist""".split()
-    LONG_TEXT = "0123456789" * 100
-    _policies = {}
-
-    _inited = False
-
-    @staticmethod
-    def init():
-        """init collection of policies: policy_version = policy_index + 1"""
-        if MockPolicyEngine._inited:
-            return
-        MockPolicyEngine._inited = True
-
-        MockPolicyEngine._policies = dict(
-            (policy_id, MockPolicyEngine._create_policy_body(policy_id, policy_version))
-            for policy_id, policy_version in
-            [(MockPolicyEngine.get_policy_id(policy_index), policy_index + 1)
-             for policy_index in range(1 + len(MockPolicyEngine.LOREM_IPSUM))]
-        )
-        _LOGGER.info("_policies: %s", json.dumps(MockPolicyEngine._policies))
-
-    @staticmethod
-    def get_policy_id(policy_index):
-        """get the policy_id by policy_index"""
-        return (MockPolicyEngine.scope_prefix
-                + MockPolicyEngine.LOREM_IPSUM[
-                    policy_index % len(MockPolicyEngine.LOREM_IPSUM)])
-
-    @staticmethod
-    def get_policy(policy_id):
-        """find policy the way the policy-engine finds"""
-        if policy_id not in MockPolicyEngine._policies:
-            return {}
-        return {PDP_POLICIES: {policy_id: copy.deepcopy(MockPolicyEngine._policies[policy_id])}}
-
-    @staticmethod
-    def gen_policy_latest(policy_index, version_offset=0):
-        """generate the policy response from policy-handler by policy_index = version - 1"""
-        policy_id = MockPolicyEngine.get_policy_id(policy_index)
-        policy = PolicyUtils.convert_to_policy(
-            MockPolicyEngine._create_policy_body(policy_id, policy_index + 1 - version_offset)
-        )
-        return policy_id, policy
-
-    @staticmethod
-    def _create_policy_body(policy_id, policy_version=1):
-        """returns a fake policy-body"""
-        return {
-            "type": "unit.test.type.policies",
-            "version": "1.0.0",
-            PDP_METADATA: {
-                PDP_POLICY_ID: policy_id,
-                PDP_POLICY_VERSION: policy_version,
-                "description": "description for {}".format(policy_id)
-            },
-            PDP_PROPERTIES: {
-                "policy_updated_from_ver": (policy_version - 1),
-                "policy_updated_to_ver": policy_version,
-                "policy_hello": "world!",
-                "updated_policy_id": policy_id
-            }
-        }