Age | Commit message (Collapse) | Author | Files | Lines |
|
- in R4 Dublin the policy-engine introduced a totally new API
- policy-handler now has a startup option to either use the new PDP API
or the old PDP API that was created-updated before the end of 2018
- see README.md and README_pdp_api_v0.md for instructions on how to
setup the policy-handler running either with the new PDP API
or the old (pdp_api_v0) PDP API
- this is a massive refactoring that changed almost all the source files,
but kept the old logic when using the old (pdp_api_v0) PDP API
- all the code related to PDP API version is split into two subfolders
= pdp_api/ contains the new PDP API source code
= pdp_api_v0/ contains the old (2018) PDP API source code
= pdp_client.py imports from either pdp_api or pdp_api_v0
= the rest of the code is only affected when it needs to branch
the logic
- logging to policy_handler.log now shows the path of the source file to
allow tracing which PDP API is actually used
- when the new PDP API is used, the policy-update flow is disabled
= passive mode of operation
= no web-socket
= no periodic catch_up
= no policy-filters
= reduced web-API - only a single /policy_latest endpoint is available
/policies_latest returns 404
/catch_up request is accepted, but ignored
- on new PDP API: http /policy_latest returns the new data from the
new PDP API with the following fields added by the policy-handler
to keep other policy related parts intact in R4
(see pdp_api/policy_utils.py)
= "policyName" = policy_id + "." + "policyVersion" + ".xml"
= "policyVersion" = str("metadata"."policy-version")
= "config" - is the renamed "properties" from the new PDP API response
- unit tests are split into two subfolders as well
= main/ for the new PDP API testing
= pdp_api_v0/ for the old (2018) PDP API
- removed the following line from the license text of changed files
ECOMP is a trademark and service mark of AT&T Intellectual Property.
- the new PDP API is expected to be extended and redesigned in R5 El Alto
- on retiring the old PDP API - the intention is to be able to remove
the pdp_api_v0/ subfolder and minimal related cleanup of the code
that imports that as well as the cleanup of the config.py, etc.
Change-Id: Ief9a2ae4541300308caaf97377f4ed051535dbe4
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-1128
|
|
- reconfigure == periodically retrieve the policy-handler config
from consul-kv and compare to previous config and subconfigs.
If changed, reconfigure the subunits
- selectively change one or any settings for the following
= catch_up timer interval
= reconfigure timer interval
= deployment-handler url and params (thread-safe)
= policy-engine url and params (thread-safe)
= web-socket url to policy-engine (through a callback)
- each subunit has its own Settings that keep track of changes
- try-catch and metrics around discovery - consul API
- hidden the secrets from logs
- froze the web-socket version to 0.49.0 because 0.50.0
and 0.51.0 are broken - looking around for stable alternatives
- fixed-adapted the callbacks passed to the web-socket lib
that changed its API in 0.49.0 and later
- log the stack on the exception occurring in the web-socket lib
- unit test refactoring
Change-Id: Id53bad59660a197f59d9aeb7c05ab761d1060cd0
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-470
|
|
- fixed removing the policy that no longer match the filter
= sending an empty policy_filter_matches for the policy_id
- cleaner workaround on getting the updated policy
when policy-engine sent policy-update notification
before finishing the update on there side
= using the collection of expected_versions instead
of min_version_expected
- some minor refactoring on policy_matcher and audit
Change-Id: Ica3cb810378e61d6991c616f88265ff170d32a64
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-492
|
|
- changed API and functionality - new dataflow
- new dataflow between policy-handler and deployment-handler
on policy-update and catchup
= GETting policy_ids+versions and policy-filters from
deployment-handler
= PUTting policy-update and catchup in the new message format
= data segmenting the policy-update/catchup messages to
deployment-handler to avoid 413 on deployment-handler side
= matching policies from policy-engine to policies
and policy-filters from deployment-handler
= coarsening the policyName filter received from deployment-handler
to reduce the number messages passed to policy-engine on catchup
= consolidating sequential policy-updates into a single request
when the policy-update is busy
- removed policy scope-prefixes from config and logic -
it is not needed anymore because
= the policy matching happens directly to policies and
policy-filters received from deployment-handler
= on catchup - the policy scope-prefix equivalents are calculated
based on the data received from deployment-handler
- API - GET /policies_latest now returns the info on deployed
policy_ids+versions and policy-filters, rather than policies
of the scope-prefixes previously found in config (obsolete)
- not sending an empty catch_up message to deployment-handler
when nothing changed
- send policy-removed to deployment-handler when getting
404-not found from PDP on removal of policy
- config change: removed catch_up.max_skips - obsolete
- brought the latest CommonLogger.py
- minor refactoring - improved naming of variables
Change-Id: I36b3412eefd439088cb693703a6e5f18f4238b00
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-492
|
|
- migrated from python 2.7 to 3.6
- brought up the latest versions of dependencies
-- Cherrypy 15.0.0, requests 2.18.4, websocket-client 0.48.0
- fixed migration errors
-- renamed the standard package Queue to queue
-- dict.items() instead of dict.iteritems()
-- dict.keys() instead of dict.viewkeys()
-- range() instead of xrange()
-- subprocess.check_output(..., universal_newlines=True) to
get str instead of byte-stream from stdout
- cleaned up migration warnings
-- super() instead of super(A, self)
-- logger.warning() instead of .warn()
- moved main() from policy_handler.py to __main__.py
- getting the policy_handler version directly from setup.py
instead of the env var on init of the audit
Change-Id: I0fc4ddc51c08a64f3cfdc5d2f010b1c6a1ae92f0
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-515
|
|
- in search of the memory leak that is falsely reported
by docker stats, the following runtime logging was added
= process_memory - rss and other memory of the current process
= virtual_memory - the memory info of the whole system
= thread_stacks - the active threads with the full stack on each
Change-Id: I5f5ab3a477bfba3aecc5963547aa82da6269670b
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-514
|
|
- periodically catchup - interval is configurable
= max_skips defines the number of times the catch_up
message that is identical to previous one can be skipped
- do not catchup more often than the interval
even between the manual catchup and auto catchup
- do not send the same catchup message twice in a row
to the deployment-handler but not exceed a hard limit
on catchup max_skips
- catchup if the deployment-handler instance is changed
Change-Id: I9a3fcc941e8a9e553abb3952dd882c37e0f5fdfe
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-389
|
|
- removed #org.onap.dcae from license text
Change-Id: I07f11e60c4677109ccb826c4e969b47acb4c498a
Signed-off-by: Alex Shatov <alexs@att.com>
Issue-ID: DCAEGEN2-347
|
|
Change-Id: I5626e0bbd3abaf96d8ab6a9b864329917b728c12
Signed-off-by: Lusheng Ji <lji@research.att.com>
Issue-ID: DCAEGEN2-325
|
|
Change-Id: I2a3628cb67d15ab2828f6818764d111df13e795a
Issue-ID: DCAEGEN2-249
Signed-off-by: Alex Shatov <alexs@att.com>
|
|
* new feature variable collection of policies per component in DCAE
* massive refactoring
* dissolved the external PolicyEngine.py into policy_receiver.py
- kept only the web-socket communication to PolicyEngine
* new /healthcheck - shows some stats of service running
* Unit Test coverage 75%
Change-Id: I816b7d5713ae0dd88fa73d3656f272b4f3e7946e
Issue-ID: DCAEGEN2-249
Signed-off-by: Alex Shatov <alexs@att.com>
|