From 8823b04bb54266a0a1bb1d693d7d5f729339d6ce Mon Sep 17 00:00:00 2001 From: ToineSiebelink Date: Thu, 23 Jan 2025 11:30:14 +0000 Subject: RTD Updates for Policy Executor Integration - Updated page describing Policy Executor feature - Updated deployment page, added config parameters for Policy Executor - Corrected release note: Jira for Policy Executor should not have been included before - Clean up release note: removed redundant version mentioning - Clean up tables on deployment page: removed redundant empty lines Issue-ID: CPS-2443 Change-Id: I5750868b6c29e9de29cd08521b8efffff4f37eaf Signed-off-by: ToineSiebelink --- docs/deployment.rst | 149 ++++++++++++++++++++++++----------------------- docs/policy-executor.rst | 16 +++-- docs/release-notes.rst | 81 +------------------------- 3 files changed, 89 insertions(+), 157 deletions(-) diff --git a/docs/deployment.rst b/docs/deployment.rst index e17392a224..2af0dd0cd5 100644 --- a/docs/deployment.rst +++ b/docs/deployment.rst @@ -191,21 +191,16 @@ Any spring supported property can be configured by providing in ``config.additio | Property | Description | Default Value | +===========================================+=========================================================================================================+===============================+ | config.appUserName | User name used by cps-core service to configure the authentication for REST API it exposes. | ``cpsuser`` | -| | | | | | This is the user name to be used by cps-core REST clients to authenticate themselves. | | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | config.appUserPassword | Password used by cps-core service to configure the authentication for REST API it exposes. | Not defined | -| | | | | | If not defined, the password is generated when deploying the application. | | -| | | | | | See also :ref:`cps_common_credentials_retrieval`. | | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | postgres.config.pgUserName | Internal user name used by cps-core to connect to its own database. | ``cps`` | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | postgres.config.pgUserPassword | Internal password used by cps-core to connect to its own database. | Not defined | -| | | | | | If not defined, the password is generated when deploying the application. | | -| | | | | | See also :ref:`cps_common_credentials_retrieval`. | | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | postgres.config.pgDatabase | Database name used by cps-core | ``cpsdb`` | @@ -225,28 +220,24 @@ Any spring supported property can be configured by providing in ``config.additio +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | config.eventPublisher. | Kafka security protocol. | ``SASL_PLAINTEXT`` | | spring.kafka.security.protocol | Some possible values are: | | -| | | | | | * ``PLAINTEXT`` | | | | * ``SASL_PLAINTEXT``, for authentication | | | | * ``SASL_SSL``, for authentication and encryption | | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | config.eventPublisher. | Kafka security SASL mechanism. Required for SASL_PLAINTEXT and SASL_SSL protocols. | Not defined | | spring.kafka.properties. | Some possible values are: | | -| sasl.mechanism | | | -| | * ``PLAIN``, for PLAINTEXT | | +| sasl.mechanism | * ``PLAIN``, for PLAINTEXT | | | | * ``SCRAM-SHA-512``, for SSL | | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | config.eventPublisher. | Kafka security SASL JAAS configuration. Required for SASL_PLAINTEXT and SASL_SSL protocols. | Not defined | | spring.kafka.properties. | Some possible values are: | | -| sasl.jaas.config | | | -| | * ``org.apache.kafka.common.security.plain.PlainLoginModule required username="..." password="...";``, | | -| | for PLAINTEXT | | +| sasl.jaas.config | * ``org.apache.kafka.common.security.plain.PlainLoginModule required username="..." password="...";``, | | +| | for PLAINTEXT | | | | * ``org.apache.kafka.common.security.scram.ScramLoginModule required username="..." password="...";``, | | -| | for SSL | | +| | for SSL | | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | config.eventPublisher. | Kafka security SASL SSL store type. Required for SASL_SSL protocol. | Not defined | | spring.kafka.ssl.trust-store-type | Some possible values are: | | -| | | | | | * ``JKS`` | | +-------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------------------+ | config.eventPublisher. | Kafka security SASL SSL store file location. Required for SASL_SSL protocol. | Not defined | @@ -294,67 +285,77 @@ Any spring supported property can be configured by providing in ``config.additio Additional CPS-NCMP Customizations ================================== -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| Property | Description | Default Value | -+=================================================+=========================================================================================================+===============+ -| config.dmiPluginUserName | User name used by cps-core to authenticate themselves for using ncmp-dmi-plugin service. | ``dmiuser`` | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.dmiPluginUserPassword | Internal password used by cps-core to connect to ncmp-dmi-plugin service. | Not defined | -| | | | -| | If not defined, the password is generated when deploying the application. | | -| | | | -| | See also :ref:`cps_common_credentials_retrieval`. | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.ncmp.timers | Specifies the delay in milliseconds in which the module sync watch dog will wake again after finishing. | ``5000`` | -| .advised-modules-sync.sleep-time-ms | | | -| | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.ncmp.timers | Specifies the delay in milliseconds in which the data sync watch dog will wake again after finishing. | ``30000`` | -| .cm-handle-data-sync.sleep-time-ms | | | -| | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.additional.ncmp | Maximum size (in MB) of the in-memory buffer for HTTP response data. | ``16`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .maximumInMemorySizeInMegabytes | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.additional.ncmp | Maximum number of simultaneous connections allowed in the connection pool. | ``100`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .maximumConnectionsTotal | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.additional.ncmp | Maximum number of pending requests when the connection pool is full. | ``50`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .pendingAcquireMaxCount | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.additional.ncmp | Specifies the maximum time in seconds, to wait for establishing a connection for the HTTP Client. | ``30`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .connectionTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.additional.ncmp | Timeout (in seconds) for reading data from the server after the connection is established. | ``30`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .readTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.additional.ncmp | Timeout (in seconds) for writing data to the server. | ``30`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .writeTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ -| config.additional.ncmp | Total timeout (in seconds) for receiving a complete response, including all processing stages. | ``60`` | -| .[app] | | | -| .httpclient | | | -| .[services] | | | -| .responseTimeoutInSeconds | | | -+-------------------------------------------------+---------------------------------------------------------------------------------------------------------+---------------+ ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| Property | Description | Default Value | ++=================================================+=======================================================================================+=================================+ +| config.dmiPluginUserName | User name used by cps-core to authenticate themselves for using ncmp-dmi-plugin | ``dmiuser`` | +| | service. | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.dmiPluginUserPassword | Internal password used by cps-core to connect to ncmp-dmi-plugin service. | Not defined | +| | If not defined, the password is generated when deploying the application. | | +| | See also :ref:`cps_common_credentials_retrieval`. | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.ncmp.timers | Specifies the delay in milliseconds in which the module sync watch dog will wake again| ``5000`` | +| .advised-modules-sync.sleep-time-ms | after finishing. | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.ncmp.timers | Specifies the delay in milliseconds in which the data sync watch dog will wake again | ``30000`` | +| .cm-handle-data-sync.sleep-time-ms | after finishing. | | +| | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp | Maximum size (in MB) of the in-memory buffer for HTTP response data. | ``16`` | +| .[app] | | | +| .httpclient | | | +| .[services] | | | +| .maximumInMemorySizeInMegabytes | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp | Maximum number of simultaneous connections allowed in the connection pool. | ``100`` | +| .[app] | | | +| .httpclient | | | +| .[services] | | | +| .maximumConnectionsTotal | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp | Maximum number of pending requests when the connection pool is full. | ``50`` | +| .[app] | | | +| .httpclient | | | +| .[services] | | | +| .pendingAcquireMaxCount | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp | Specifies the maximum time in seconds, to wait for establishing a connection for the | ``30`` | +| .[app] | HTTP Client. | | +| .httpclient | | | +| .[services] | | | +| .connectionTimeoutInSeconds | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp | Timeout (in seconds) for reading data from the server after the connection is | ``30`` | +| .[app] | established. | | +| .httpclient | | | +| .[services] | | | +| .readTimeoutInSeconds | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp | Timeout (in seconds) for writing data to the server. | ``30`` | +| .[app] | | | +| .httpclient | | | +| .[services] | | | +| .writeTimeoutInSeconds | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp | Total timeout (in seconds) for receiving a complete response, including all processing| ``60`` | +| .[app] | stages. | | +| .httpclient | | | +| .[services] | | | +| .responseTimeoutInSeconds | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp.policy-executor | Enables or disables the policy-executor feature. | ``false`` | +| .enabled | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp.policy-executor | The default (fallback) decision in case a problem with the external service occurs. | ``allow`` | +| .defaultDecision | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp.policy-executor | The server address for the external policy executor service. | ``http://policy-executor-stub`` | +| .server.address | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ +| config.additional.ncmp.policy-executor | The port used for the external policy executor service. | ``8093`` | +| .server.port | | | ++-------------------------------------------------+---------------------------------------------------------------------------------------+---------------------------------+ .. note:: diff --git a/docs/policy-executor.rst b/docs/policy-executor.rst index b934a579b1..712b4fcf38 100644 --- a/docs/policy-executor.rst +++ b/docs/policy-executor.rst @@ -1,11 +1,10 @@ .. This work is licensed under a Creative Commons Attribution 4.0 International License. .. http://creativecommons.org/licenses/by/4.0 -.. Copyright (C) 2024 Nordix Foundation +.. Copyright (C) 2024-2025 Nordix Foundation -.. DO NOT CHANGE THIS LABEL FOR RELEASE NOTES - EVEN THOUGH IT GIVES A WARNING +.. DO NOT CHANGE THIS LABEL - EVEN THOUGH IT GIVES A WARNING .. _policy_executor: - Policy Executor ############### @@ -15,7 +14,16 @@ Policy Executor Introduction ============ -Work In Progress: This feature is not yet completed and does not affect current NCMP functionality. +The Policy Executor feature can be used to connect an external system to make decisions on CM write operation. +When the feature is enabled, NCMP will first call the configured external system and depending on the response, return an error or continue. +The details of the interface can be found in the ':ref:`policy_executor_consumed_apis`' section. + +This feature is available on 'legacy data interface' for operation on a single cm handle: "/v1/ch/{cm-handle}/data/ds/{datastore-name}" and only applies to "ncmp-datastore:passthrough-running". + +By default, the feature is not enabled. This is controlled by 'config.additional.ncmp.policy-executor.enabled' and other deployment parameters in the same group to enable it. See :ref:`additional-cps-ncmp-customizations` + +.. DO NOT CHANGE THIS LABEL - EVEN THOUGH IT GIVES A WARNING +.. _policy_executor_consumed_apis: Consumed APIs ------------- diff --git a/docs/release-notes.rst b/docs/release-notes.rst index 76d75cdec5..1f37612570 100644 --- a/docs/release-notes.rst +++ b/docs/release-notes.rst @@ -40,6 +40,7 @@ Bug Fixes Features -------- + - `CPS-2249 `_ NCMP to support Conflict Handling. .. ==================== @@ -68,14 +69,12 @@ Release Data Bug Fixes --------- -3.5.5 - `CPS-2509 `_ Fix module endpoints using alternate identifier. - `CPS-2517 `_ Make Content-Type header default to JSON for CPS APIs. - `CPS-2530 `_ NCMP Modules API giving empty response on READY CM Handles if two sub systems discovered in parallel. Features -------- -3.5.5 - `CPS-2009 `_ Update legacy NCMP APIs interfaces to support alternate id. - `CPS-2082 `_ Support XML content type to data node APIs in cps-core. - `CPS-2433 `_ Remove traces of unmaintained CPS-TBDMT repository. @@ -111,12 +110,10 @@ Release Data Bug Fixes --------- -3.5.4 - - `CPS-2403 `_ Improve lock handling and queue management during CM Handle Module Sync. + - `CPS-2403 `_ Improve lock handling and queue management during CM-handle Module Sync. Features -------- -3.5.4 - `CPS-2408 `_ One Hazelcast instance per JVM to manage the distributed data structures. Version: 3.5.3 @@ -141,7 +138,6 @@ Release Data Bug Fixes --------- -3.5.3 - `CPS-2353 `_ Slow cmHandle registration when we use moduleSetTag, alternateId and dataProducerIdentifier - `CPS-2395 `_ Retry mechanism (with back off algorithm) is removed with more frequent watchdog poll - `CPS-2409 `_ Return NONE for get effective trust level api if the trust level caches empty (restart case) @@ -150,9 +146,6 @@ Bug Fixes Features -------- -3.5.3 - - `CPS-2247 `_ Policy Executor: Invoke Policy Executor and handle 'deny' response - - `CPS-2412 `_ Policy Executor: handle errors - `CPS-2417 `_ Remove Hazelcast cache for prefix resolver @@ -178,14 +171,12 @@ Release Data Bug Fixes --------- -3.5.2 - `CPS-2306 `_ Update response message for data validation failure and make it consistent across APIs - `CPS-2319 `_ Fix "Create a node" and "Add List Elements" APIs response code - `CPS-2372 `_ Blank alternate ID overwrites existing one Features -------- -3.5.2 - `CPS-1812 `_ CM Data Subscriptions ( Create, Delete and Merging ) with positive scenarios - `CPS-2326 `_ Uplift liquibase-core dependency to 4.28.0 - `CPS-2353 `_ Improve registration performance with moduleSetTag @@ -213,12 +204,10 @@ Release Data Bug Fixes --------- -3.5.1 - `CPS-2302 `_ Fix handling of special characters in moduleSetTag. Features -------- -3.5.1 - `CPS-2121 `_ Enabled http client prometheus metrics and manage high cardinality using URL template. - `CPS-2289 `_ Support for CPS Path Query in NCMP Inventory CM Handle Search. @@ -242,13 +231,8 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.5.0 - Features -------- -3.5.0 - `CPS-989 `_ Replace RestTemplate with WebClient. - `CPS-2172 `_ Support for OpenTelemetry Tracing. @@ -278,12 +262,10 @@ Release Data Bug Fixes --------- -3.4.9 - `CPS-2211 `_ Toggle switch to disable CPS Core change events if not used by application. Set CPS_CHANGE_EVENT_NOTIFICATIONS_ENABLED environment variable for the same. Features -------- -3.4.9 - `CPS-1836 `_ Delta between anchor and JSON payload. Version: 3.4.8 @@ -308,15 +290,11 @@ Release Data Bug Fixes --------- -3.4.8 - `CPS-2186 `_ Report async task failures to client topic during data operations request - `CPS-2190 `_ Improve performance of NCMP module searches - `CPS-2194 `_ Added defaults for CPS and DMI username and password - `CPS-2204 `_ Added error handling for yang module upgrade operation -Features --------- - Version: 3.4.7 ============== @@ -339,12 +317,10 @@ Release Data Bug Fixes --------- -3.4.7 - `CPS-2150 `_ Fix for Async task execution failed by TimeoutException. Features -------- -3.4.7 - `CPS-2061 `_ Liquibase Steps Condensing and Cleanup. - `CPS-2101 `_ Uplift Spring Boot to 3.2.4 version. @@ -370,7 +346,6 @@ Release Data Bug Fixes --------- -3.4.6 - `CPS-2126 `_ Passing HTTP Authorization Bearer Token to DMI Plugins. @@ -406,10 +381,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.4.5 - Features -------- @@ -438,7 +409,6 @@ Release Data Bug Fixes --------- -3.4.4 - `CPS-2027 `_ Upgrade Yang modules using module set tag functionalities fix Features @@ -469,7 +439,6 @@ Release Data Bug Fixes --------- -3.4.3 - `CPS-2000 `_ Fix for Schema object cache not being distributed. - `CPS-2027 `_ Fixes for upgrade yang modules using module set tag. - `CPS-2070 `_ Add retry interval for Kafka consumer. @@ -506,11 +475,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.4.2 - - Features -------- - `CPS-1638 `_ Introduce trust level for CM handle. @@ -556,7 +520,6 @@ Release Data Bug Fixes --------- -3.4.1 - `CPS-1979 `_ Bug fix for Invalid topic name suffix. Features @@ -594,7 +557,6 @@ Release Data Bug Fixes --------- -3.4.0 - `CPS-1956 `_ Bug fix for No yang resources stored during cmhandle discovery. .. ======================== @@ -623,13 +585,9 @@ Release Data Bug Fixes --------- -3.3.9 - `CPS-1923 `_ CPS and NCMP changed management endpoint and port from /manage to /actuator and port same as cps application port. - `CPS-1933 `_ Setting up the class loader explicitly in hazelcast config. -Features --------- - Version: 3.3.8 ============== @@ -650,10 +608,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.3.8 - Features -------- - `CPS-1888 `_ Uplift Spring Boot to 3.1.2. @@ -680,7 +634,6 @@ Release Data Bug Fixes --------- -3.3.7 - `CPS-1866 `_ Fix ClassDefNotFoundError in opendaylight Yang parser Features @@ -713,7 +666,6 @@ Release Data Bug Fixes --------- -3.3.6 - `CPS-1841 `_ Update of top-level data node fails with exception - `CPS-1842 `_ Replace event-id with correlation-id for data read operation cloud event @@ -743,10 +695,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.3.5 - Features -------- - `CPS-1760 `_ Improve handling of special characters in Cps Paths @@ -771,10 +719,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.3.4 - Features -------- - `CPS-1767 `_ Upgrade CPS to java 17 @@ -799,10 +743,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.3.3 - Features -------- - `CPS-1515 `_ Support Multiple CM Handles for NCMP Get Operation @@ -831,7 +771,6 @@ Release Data Bug Fixes --------- -3.3.2 - `CPS-1716 `_ NCMP: Java Heap OutOfMemory errors and slow registration in case of 20k cmhandles Features @@ -862,11 +801,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.3.1 - - None - Features -------- - `CPS-1272 `_ Add Contains operation to CPS Path @@ -895,11 +829,6 @@ Release Data | | | +--------------------------------------+--------------------------------------------------------+ -Bug Fixes ---------- -3.3.0 - - None - Features -------- - `CPS-1215 `_ Add OR operation for CPS Path @@ -931,7 +860,6 @@ Release Data Bug Fixes --------- -3.2.6 - `CPS-1526 `_ Fix response message for PATCH operation - `CPS-1563 `_ Fix 500 response error on id-searches with empty parameters @@ -961,13 +889,8 @@ Release Data Bug Fixes --------- -3.2.5 - `CPS-1537 `_ Introduce control switch for model loader functionality. -Features --------- - - None - Version: 3.2.4 ============== -- cgit