aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2018-06-213.0.1 policy-handler - cleaning sonar smellsAlex Shatov16-260/+297
- no change of functionality or API - removed the unused enum34>=1.1.6 from requirements.txt and setup.py - refactored run_policy.sh to redirect the stdout+stderr only once - refactoring to remove smells+vulnerability reported by sonar -- renamed Config.config to Config.settings -- removed the commented out code in customizer.py -- renamed StepTimer.NEXT to StepTimer.STATE_NEXT to avoid the naming confusion with the method StepTimer.next. Also renamed the related StepTimer.STATE_* constants -- refactored several functions by extracting methods to eliminate 4 out of 5 "brain-overload" smells reported by sonar -- moved the literal string for the socket_host "0.0.0.0" to a constant on the web-server to avoid the reported vulnerability Change-Id: I4c7d47d41c6ecd7cb28f6704f5dad2053c1ca7d6 Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-515
2018-06-143.0.0 policy-handler - migrated to python 3.6Alex Shatov23-95/+106
- migrated from python 2.7 to 3.6 - brought up the latest versions of dependencies -- Cherrypy 15.0.0, requests 2.18.4, websocket-client 0.48.0 - fixed migration errors -- renamed the standard package Queue to queue -- dict.items() instead of dict.iteritems() -- dict.keys() instead of dict.viewkeys() -- range() instead of xrange() -- subprocess.check_output(..., universal_newlines=True) to get str instead of byte-stream from stdout - cleaned up migration warnings -- super() instead of super(A, self) -- logger.warning() instead of .warn() - moved main() from policy_handler.py to __main__.py - getting the policy_handler version directly from setup.py instead of the env var on init of the audit Change-Id: I0fc4ddc51c08a64f3cfdc5d2f010b1c6a1ae92f0 Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-515
2018-05-242.4.5 policy-handler - fix of unit tests2.0.0-ONAPbeijing2.0.0-ONAPAlex Shatov5-5/+8
- turned off test_gc unit-test on policy-handler to avoid get /gc/stats after shutdown of the web-server - made rougher comparison between execution time and timer interval Change-Id: Idcf6caae6f2a934dc2dc2d5a0fddd06543abd48a Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-532
2018-05-182.4.4 policy-handler - log process memoryAlex Shatov14-205/+421
- in search of the memory leak that is falsely reported by docker stats, the following runtime logging was added = process_memory - rss and other memory of the current process = virtual_memory - the memory info of the whole system = thread_stacks - the active threads with the full stack on each Change-Id: I5f5ab3a477bfba3aecc5963547aa82da6269670b Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-514
2018-05-102.4.3 policy-handler - try-catch top ExceptionsAlex Shatov16-601/+1358
- added try-except for top level Exception into all threads of policy-handler to avoid losing the thread and tracking the unexpected crashes - rediscover the deployment-handler if not found before and after each catchup - refactored audit - separated metrics from audit - added more stats and runtime info to healthcheck = gc counts and garbage info if any detected = memory usage - to detect the potential memory leaks = request_id to all stats = stats of active requests - avoid reallocating the whole Queue of policy-updates after catchup = clear of the internal queue under proper lock Change-Id: I3fabcaac70419a68bd070ff7d591a75942f37663 Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-483
2018-04-272.4.2 policy-handler - fixed race on step-timerAlex Shatov12-94/+162
- fixed the bug of unpredictably stopping of the periodic catch-up step-timer due to thread race condition in policy-handler = added critical sections under the reentrant lock on every group of local var change in step-timer - added more stats for healthcheck to track each type of job-operation separately = that helps narrowing down identifying the potential problems - unit test coverage 76% Change-Id: I92ddf6c92a3d225d9b87427e3edfb7f80669501a Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-472
2018-04-17policy-handler to accept CLAMP policy-updatesAlex Shatov1-1/+1
- change application_config of policy-handler to have CLAMP as a valid scope_prefix scope_prefixes : ["DCAE.Config_", "CLAMP"] Change-Id: Id7bc00d5e5272d22c3d588024dc44a0200599db6 Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-449
2018-04-02Fix docker tag based in CIMAN-132Lusheng Ji1-2/+10
The new mvn-phase-lib.sh contains the fix for pushing docker tags according to CIMAN-132. This change makes sure that we are using the new mvn-phase-lib.sh for R2. Issue-ID: DCAEGEN2-206 Change-Id: Id36ae5ebecd936d188dee1af8c5fd5eb93a097b3 Signed-off-by: Lusheng Ji <lji@research.att.com>
2018-03-222.4.1 better step-timer + audit + unit-testsAlex Shatov8-49/+410
- improved step-timer due to unit tests -- fixed events -- better logging - audit - collect list of package thru subprocess pip freeze - unit tests coverage 76% Change-Id: Ib1cb5f687612ecf18aa7414b1ff7dbf5774345b4 Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-389
2018-03-202.4.0 dns based discovery of deployment-handlerAlex Shatov7-24/+44
- policy-handler uses dns based discovery of deployment-handler - driven by config - new data structure for deploy_handler section of config -- changed from string "deployment_handler" in 2.3.1 to structure in 2.4.0 deploy_handler : # name of deployment-handler service # used by policy-handler for logging target_entity : "deployment_handler" # url of the deployment-handler service # for policy-handler to direct the policy-updates to # - expecting dns to resolve the name # deployment_handler to ip address url : "http://deployment_handler:8188" - logic is backwards compatible with 2.3.1 format - removed import pip from audit -- import pip broken in pip 9.0.2 (2018-03-19) -- import pip conflicts with requests -- pip API is not officially supported -- see links for more https://github.com/pypa/pip/issues/5079 https://github.com/pypa/pip/issues/5081 Change-Id: Ifcaba6cfd714f3099ab7a25fe979a3696a6460fc Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-404
2018-03-162.3.1 policy-handler - catch_up more oftenAlex Shatov6-28/+57
- enforce(do not skip) next catchup if failed to send to deployment-handler - more audit info and logging - cleaner queue handling in the policy-updater thread = queue reset and task_done on the to level in the run Change-Id: If3080d08f0f6560e3f0bd509fde94a7f8191b228 Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-389
2018-03-132.3.0 policy-handler - periodically catch_upAlex Shatov10-76/+255
- periodically catchup - interval is configurable = max_skips defines the number of times the catch_up message that is identical to previous one can be skipped - do not catchup more often than the interval even between the manual catchup and auto catchup - do not send the same catchup message twice in a row to the deployment-handler but not exceed a hard limit on catchup max_skips - catchup if the deployment-handler instance is changed Change-Id: I9a3fcc941e8a9e553abb3952dd882c37e0f5fdfe Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-389
2018-03-082.2.0 policy-handler - customization per companyAlex Shatov14-54/+326
- added etc_customize/ folder and customize.sh script = customize.sh script is expected to be overridden by company to customize Docker image build = the whole etc_customize/ folder is copied into docker image = it is up to the company what to put into that folder - any files - added customize/ folder with CustomizeBase and Customize classes = CustomizeBase defines the interface and the default=ONAP behavior = CustomizeBase is owned by ONAP and should not be changed by the company = Customize inherits CustomizeBase = policy-handler instantiates Customize to get the customized behavior = Customize is owned by the company and should be changed by the company = ONAP is not going to change Customize = the methods of Customize are expected to be overridden by the company to change the behavior of the policy-handler = sample Customize class can be found in README.md = Company is allowed to add more files to customize/ folder if that is required for better structuring of their code as soon as it is invoked by the methods of Customize Change-Id: I46f8170afaaa48e1005e4398a768a781db0a0e6c Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-379
2018-02-27up the version to 2.1.0 for policy-handlerAlex Shatov22-32/+9
- removed #org.onap.dcae from license text Change-Id: I07f11e60c4677109ccb826c4e969b47acb4c498a Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-347
2018-02-23fixed a bug in unit testsAlex Shatov1-1/+1
- minor list comparison bug - not affecting much Change-Id: I2f3a51ce2064601f0a08547d7c250eee551f6721 Signed-off-by: Alex Shatov <alexs@att.com> Issue-ID: DCAEGEN2-347
2018-02-21Exclude tests/ from SONAR reportLusheng Ji1-1/+2
Change-Id: I8d89fdf950d5e221b8c1eedccc1eb570d4648249 Signed-off-by: Lusheng Ji <lji@research.att.com> Issue-ID: DCAEGEN2-355
2018-02-20Fix for SONAR reportingLusheng Ji5-350/+45
Change-Id: I853af6b231b4b2f4eff7492d4770ce6a3a7fd786 Signed-off-by: Lusheng Ji <lji@research.att.com> Issue-ID: DCAEGEN2-355
2018-02-12Update license textLusheng Ji13-15/+15
Change-Id: I5626e0bbd3abaf96d8ab6a9b864329917b728c12 Signed-off-by: Lusheng Ji <lji@research.att.com> Issue-ID: DCAEGEN2-325
2018-02-05fix copyright from 2018 to 2017-2018Alex Shatov7-7/+7
Change-Id: I685c63758b7ce22766885d399f06e9ba14ca59f2 Issue-ID: DCAEGEN2-249 Signed-off-by: Alex Shatov <alexs@att.com>
2018-02-01improved message to deployment-handler and on APIAlex Shatov7-92/+115
* added errored_scopes and scope_prefixes to the message to deployment-handler - to prevent erroneous removal of policies * hardcoded condition for scope not found 404 at policy-engine to separate it from error on the scope retrieval 400 * adjusting the web API message in sync with notification to deployment-handler * unit test coverage 74% Change-Id: Ie736a1b7aee0631b6785669c6b765bd240dd77b8 Issue-ID: DCAEGEN2-249 Signed-off-by: Alex Shatov <alexs@att.com>
2018-01-25on not finding the policy return 404 and jsonAlex Shatov2-15/+40
* json of what is returned from policy-engine instead of 404 html Change-Id: I44fb7e113e23f68cac44e6ad21c1bc9b37cf3d58 Issue-ID: DCAEGEN2-249 Signed-off-by: Alex Shatov <alexs@att.com>
2018-01-11moved module docstring below the license textAlex Shatov15-54/+58
Change-Id: I2a3628cb67d15ab2828f6818764d111df13e795a Issue-ID: DCAEGEN2-249 Signed-off-by: Alex Shatov <alexs@att.com>
2018-01-10variable collection of policies per componentAlex Shatov24-1818/+1452
* new feature variable collection of policies per component in DCAE * massive refactoring * dissolved the external PolicyEngine.py into policy_receiver.py - kept only the web-socket communication to PolicyEngine * new /healthcheck - shows some stats of service running * Unit Test coverage 75% Change-Id: I816b7d5713ae0dd88fa73d3656f272b4f3e7946e Issue-ID: DCAEGEN2-249 Signed-off-by: Alex Shatov <alexs@att.com>
2017-11-25update versioningvagrant1-1/+1
Issue-ID: DCAEGEN2-211 Change-Id: I3b3cbb5ad62f9bdf8f7c99fbd60b9146df8c3136 Signed-off-by: vagrant <vv770d@att.com>
2017-11-20Bump minor versionJessica Wagantall2-3/+3
Bump minor version in preparation for Amsterdam branching. Change-Id: I284f133a2e0911eb9d2dd2178bc00eda7819c4a6 Issue-ID: CIMAN-120 Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
2017-10-31flexible Basic versus non-basic ClientAuth to PDPv1.1.01.0.0-ONAP1.0.0-AmsterdamAlex Shatov3-135/+145
PDP client (PolicyEngine.py) now can handle two formats of ClientAuth to match what we have in config * Basic <auth> * <auth> Change-Id: I4010d430a6675e3f259c1fc53b0b3373bd225352 Issue-Id: DCAEGEN2-128 Signed-off-by: Alex Shatov <alexs@att.com>
2017-10-27fixed ClientAuth and Authorization parsingAlex Shatov3-5/+7
two formats of ClientAuth and Authorization * Basic <auth> * <auth> Change-Id: I177a86caef6b2a2406277413d5de1972bcf19cfe Issue-Id: DCAEGEN2-128 Signed-off-by: Alex Shatov <alexs@att.com>
2017-09-26omit external libs from coverage on UTAlex Shatov1-1/+5
* omit the client libs of outside components * omit = # omit client libs policyhandler/PolicyEngine.py policyhandler/onap/CommonLogger.py Change-Id: If28158913b025a9f2217c61acd17a7d5e5cee668 Issue-Id: DCAEGEN2-62 Signed-off-by: Alex Shatov <alexs@att.com>
2017-09-22unit test and coverage with toxAlex Shatov5-0/+242
usage on local run: tox -c tox-local.ini usage on ONAP run: tox Change-Id: Ic455f0f44f5b3bee92b60ea282851e72c3a12b7e Issue-Id: DCAEGEN2-62 Signed-off-by: Alex Shatov <alexs@att.com>
2017-09-15version and proper deployment_handler in configAlex Shatov5-101/+3
Change-Id: I4cde9c4a2e1305519458658479a1c19c36c80a5b Issue-Id: DCAEGEN2-62 Signed-off-by: Alex Shatov <alexs@att.com>
2017-09-14Change version to SNAPSHOTLusheng Ji1-1/+1
Issue-Id: DCAEGEN2-60 Change-Id: I95b5c4b3f6f792d18774a73200362f56f1743a3a Signed-off-by: Lusheng Ji <lji@research.att.com>
2017-09-13setting the pool-size in requests sessionAlex Shatov3-4/+38
* to cache the connections - performance improvement Change-Id: I5a9b3ee30d40dc8d1b8154d5345a00b716e07964 Issue-Id: DCAEGEN2-62 Signed-off-by: Alex Shatov <alexs@att.com>
2017-09-121.0.0 and policy API to deployment-handlerAlex Shatov11-226/+48
* policy API to deployment-handler /policy * removed pycrypto of config - the same way as other apps * simple upload of config to consul - curl * preparation for policy-handler blueprint Change-Id: I424a1ded0795562ea36b5409304cbb8b5a7e8a24 Issue-Id: DCAEGEN2-62 Signed-off-by: Alex Shatov <alexs@att.com>
2017-09-12Add mvn script for docker buildLusheng Ji3-153/+467
Issue-Id: DCAEGEN2-60 Change-Id: Ic05eeb71ee94e29fcb2c0f2ca5a0ca4d36d9ca94 Signed-off-by: Lusheng Ji <lji@research.att.com>
2017-09-08Cleanup project's name in SonarJessica Wagantall1-1/+1
The name parameter in the root pom.xml should match the project name in gerrit to reflect consistency in Sonar. Change-Id: If1c433147c83b47eb790c11316ddc8fa5af7b0f4 Issue-id: CIMAN-65 Signed-off-by: Jessica Wagantall <jwagantall@linuxfoundation.org>
2017-08-29Update LICENSE.txt, pom.xml, version, mvn scriptLusheng Ji4-29/+431
Issue-Id: DCAEGEN2-86 Change-Id: I1352bf9e59aa64efd19b653b9d9d70d4db4d2c70 Signed-off-by: Lusheng Ji <lji@research.att.com>
2017-08-24policy-handler seed code - fix per Tommy's commentAlex Shatov1-2/+1
Change-Id: Iaed21de902e6875730ffc8ddc5d52c19ef702628 Issue-Id: DCAEGEN2-46 Signed-off-by: Alex Shatov <alexs@att.com>
2017-08-23policy-handler seed codealex_sh31-0/+4308
Change-Id: I35cd80b6e082f4b84740bab752774e8abc40ca35 Issue-Id: DCAEGEN2-46 Signed-off-by: Alex Shatov <alexs@att.com>
2017-06-28Initial empty repositoryJessica Wagantall0-0/+0