diff options
51 files changed, 1310 insertions, 688 deletions
diff --git a/docs/automated-usecases.csv b/docs/automated-usecases.csv deleted file mode 100644 index af8c34b82..000000000 --- a/docs/automated-usecases.csv +++ /dev/null @@ -1,7 +0,0 @@ -Use Case;Description;Test Framework -basic_vm;Onboard, distribute and instantiate a simple Ubuntu VM based on its heat template.;`onap-tests <https://gitlab.com/Orange-OpenSource/lfn/onap/onap-tests/-/blob/master/onap_tests/xtesting/basic_vm.py>`__ -clearwater_ims;Onboard, distribute and instantiate a clearwater vIMS (6 VMs).;`onap-tests <https://gitlab.com/Orange-OpenSource/lfn/onap/onap-tests/-/blob/master/onap_tests/xtesting/clearwater_ims.py>`__ -freeradius_nbi;Onboard, distribute and instantiate a simple VM including freeradius, the instantiation is realized through the external API module (not direct call to SO);`onap-tests <https://gitlab.com/Orange-OpenSource/lfn/onap/onap-tests/-/blob/master/onap_tests/xtesting/freeradius_nbi.py>`__ -hv-ves;Automation of 5G Realtime PM and High Volume Stream Data Collection;`robot <https://git.onap.org/testsuite/tree/robot/testsuites/hvves-ci.robot>`__ -5GBulkPm;Automation of 5G Bulk PM;`robot <https://git.onap.org/testsuite/tree/robot/testsuites/usecases/5gbulkpm.robot>`__ -pnf-registrate;Automation of 5G - PNF PnP;`robot <https://git.onap.org/testsuite/tree/robot/testsuites/pnf-registration.robot>`__ diff --git a/docs/conf.py b/docs/conf.py index 68ef33b1a..adf15cc4a 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -13,7 +13,8 @@ linkcheck_ignore = [ 'http://so-monitoring:30224', r'http://SINK_IP_ADDRESS:667.*', r'http.*K8S_HOST:30227.*', - r'http.*K8S_NODE_IP.*' + r'http.*K8S_NODE_IP.*', + 'http://team.onap.eu' ] intersphinx_mapping = {} diff --git a/docs/docs_5G_oof_pci.rst b/docs/docs_5G_oof_pci.rst index 15a0e1c0c..8edabf40c 100644 --- a/docs/docs_5G_oof_pci.rst +++ b/docs/docs_5G_oof_pci.rst @@ -4,26 +4,33 @@ .. contents:: :depth: 3 .. + .. _docs_5G_oof_pci: OOF-PCI -------- + Description ~~~~~~~~~~~ The 5G OOF-PCI use case is an implementation of a SON (Self-Organizing Networks) algorithm + for Physical Cell ID (PCI) optimization and the centralized Automatic Neighbor Relations + (ANR) function (blacklisting a neighbor for handovers) in a 4G/5G network using the ONAP + Optimization Framework (OOF). This use case began with the implementation of PCI -optimization in Casablanca. In Dublin release, the SON-Handler MS was onboarded as a + +optimization in Casablanca. In Dublin release, the SON-Handler MS was onboarded asa + micro-service in DCAE. Enhancements were made to Policy and SDN-C components. Further + details of Dublin release scope and impacts for this use case are described in: + https://docs.onap.org/en/dublin/submodules/integration.git/docs/docs_5G_oof_pci.html#docs-5g-oof-pci -.. image:: https://wiki.onap.org/download/attachments/81401601/image2020-3-26_5-40-10.png?version=1&modificationDate=1585219211000&api=v2 - :alt: OOF-PCI In Frankfurt release, the following are the main enhancements: @@ -37,62 +44,75 @@ In Frankfurt release, the following are the main enhancements: https://wiki.onap.org/display/DW/CM+Notification+Support+in+ONAP -In Guilin release, the main enhancement was related to taking the first steps for enabling Machine-Learning (ML) based -SON functionality. Training is assumed to be done offline, and the ML model is then onboarded to ONAP as part -of OOF. The ML model provides additional inputs (based on the handover PM data) to the optimizer about cells -whose PCI values should not be modified during the PCI optimization process. To be able to do this, the ML model -that is part of OOF fetches historical PM data from a data base/data lake using the DES micro-service (in DCAE) -APIs. This is a new capability introduced in OOF which can be extended further for other scenarios. To be backward -compatible, the functionality in OOF to invoke the ML model for additional inputs before PCI optimization is -performed is configurable. + The end-to-end setup for the use case requires a Config DB which stores the cell related details of the RAN. + + This is updated by SDN-C (R), and is accessed by SON-Handler MS and OOF for fetching, e.g., neighbor list, PNF id, etc. + + + The Config DB implementation is available at: + + https://github.com/onap-oof-pci-poc/sdnc/tree/master/ConfigDB/Dublin. + -The end-to-end setup for the use case requires a Config DB which stores the cell related details of the RAN. This is -updated by SDN-C (R), and is accessed by SON-Handler MS and OOF for fetching, e.g., neighbor list, PNF id, etc. -The Config DB implementation is available at: -https://github.com/onap-oof-pci-poc/sdnc/tree/master/ConfigDB/Dublin + Swagger JSON API documentation can be found at: -Swagger JSON API documentation can be found at: -https://github.com/onap-oof-pci-poc/sdnc/blob/master/ConfigDB/Dublin/SDNC_ConfigDB_API_v3.0.0.json + https://github.com/onap-oof-pci-poc/sdnc/blob/master/ConfigDB/Dublin/SDNC_ConfigDB_API_v3.0.0.json. -As part of this use case work, a RAN Simulator providing a simulated Radio Access Network (RAN) with a number of -netconf servers simulating PNF elements has been implemented. The functionality of the RAN Simulator includes: + +As part of this use case work, a RAN Simulator providing a simulated Radio Access Network +(RAN) with a number of netconf servers simulating PNF elements has been implemented. The +functionality of the RAN Simulator includes: - Generation of neighbor-list-update messages - Generation of alarms for PCI collision/confusion and - Generation of handover metrics for different neighbor pairs (for the ANR use case). All above functionality are enabled using a simple UI. -All details regarding the use case for Guilin can be found here: -https://wiki.onap.org/display/DW/R7+OOF+SON+Use+Case All details regarding the use case for Frankfurt can be found here: + https://wiki.onap.org/display/DW/OOF+%28SON%29+in+R5+El+Alto%2C+OOF+%28SON%29+in+R6+Frankfurt The main use case page is: + https://wiki.onap.org/display/DW/5G+-+OOF+%28ONAP+Optimization+Framework%29+and+PCI+%28Physical+Cell+ID%29+Optimization + How to Use ~~~~~~~~~~ -The OOF-PCI use case is set up in the Rutgers University (Winlab) ONAP Wireless Lab (OWL). For details, please see: -https://wiki.onap.org/pages/viewpage.action?pageId=45298557. +The OOF-PCI use case is implemented in the Rutgers University (Winlab) ONAP Wireless Lab (OWL). + +For details, please see: https://wiki.onap.org/pages/viewpage.action?pageId=45298557. This page includes instructions for access to the lab. Setup and testing is done manually up to now. For all instructions about installing the components, please see: -- Frankfurt Installation: https://wiki.onap.org/display/DW/Demo+setup+steps+for+Frankfurt -- Guilin Installation: https://wiki.onap.org/display/DW/SON+use+case+demo+setup+steps+for+Guilin -- Son-Handler installation: https://docs.onap.org/projects/onap-dcaegen2/en/frankfurt/sections/services/son-handler/installation.html?highlight=dcaegen2 +Installation: https://wiki.onap.org/display/DW/Demo+setup+steps+for+Frankfurt + + +Son-Handler installation: + +https://docs.onap.org/projects/onap-dcaegen2/en/frankfurt/sections/services/son-handler/installation.html?highlight=dcaegen2 + Test Status and Plans ~~~~~~~~~~~~~~~~~~~~~ -For Guilin release, the enhancements described above were implemented. OOF was enhanced with invoking the onboarded -ML model for additional inputs on cells with fixed PCI values during the optimization. +For Frankfurt release, the enhancements described above were implemented. OOF was enhanced + +with handling cells with fixed PCI values during the optimization, SON-Handler MS was + +functionally enhanced for adaptive SON functionality, SDN-C (R) was enhanced to include + +handling of DMaaP message for config changes in the RAN, and Policy was also enhanced with + +Control Loop Co-ordination function. + +To see information about test plans, please see https://wiki.onap.org/display/DW/Testing. -To see information about test plans, please see https://wiki.onap.org/display/DW/Integration+Test+for+Guilin. Known Issues and Resolutions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -100,7 +120,10 @@ Known Issues and Resolutions (a) It is intended to have the RAN Simulator support sufficient Honeycomb netconf server instances to simulate 2000 cells. However, this number may be lower if there are hardware limitatons. (b) For Control Loop Co-ordination, the denial of a second Control Loop based on Target Lock (i.e., when a second Control - Loop tries to operate on the same target (in this case, a PNF) is successfully tested. - The CLC is also applied at Control Loop level only. -(c) There are some limitations in the DES APIs with respect to supporting generic queries. These will be addressed in - Honolulu release and beyond. + Loop tries to operate on the same target (in this case, a PNF) is successfully tested. The CLC is also applied at Control + Loop level only. However, some code updates are required in Policy to properly update the Operations History DB entry, and + to check the existence of active Control Loops by Policy. This will be addressed in Guilin release, and tracked via + https://jira.onap.org/browse/POLICY-2581 and https://jira.onap.org/browse/POLICY-2583. +(c) For Adaptive SON, the functionality in SON-Handler and OOF is implemented, however the OOF functionality is not + fully tested (this was anyhow a stretch goal). Further, the DCAE-CL-RSP message is not sent by Policy in Frankfurt release. + This is tracked via https://jira.onap.org/browse/POLICY-2580 and shall be part of Guilin release. diff --git a/docs/docs_E2E_network_slicing.rst b/docs/docs_E2E_network_slicing.rst index b27536440..e177f8d8b 100644 --- a/docs/docs_E2E_network_slicing.rst +++ b/docs/docs_E2E_network_slicing.rst @@ -12,6 +12,7 @@ E2E Network Slicing Use Case Overall Blueprint ----------------- + The objective of this use case is to realize End-to-End 5G Network Slicing using ONAP. An End-to-End Network Slice consists of RAN (Radio Access Network), Transport Network (TN) and Core Network (CN) slice @@ -82,190 +83,246 @@ Abbreviations +---------------+--------------------------------------------+ -Recap of Frankfurt functionality --------------------------------- -In Frankfurt release, CSMF and NSMF within ONAP was implemented, while connecting to an external Core NSSMF. -From the NSI Life Cycle perspective, the scope for Frankfurt included NSI design and pre-provision, NSI instantiation -and configuration, and NSI activation and deactivation. In particular: - -- CSMF: Functions of slice service creation, slice service activation and deactivation were implemented. - -- NSMF: Functions of NSI instantiation, NSI activation and deactivation were implemented. In addition, manual - intervention is also provided in NSMF slice task management portal to ensure the selected NSI/NSSI as well as - Service Profile and Slice Profile are OK or need adjustment. - -- Design of CST, NST and onboarding NSST that are required to support the run-time orchestration functions +Scope for Frankfurt +------------------- -- To connect to the external (core) NSSMF, an adaptor was implemented to provide interface between ONAP and 3rd party - core NSSMF. +To realize the three layers of the slice management function, we need to decide whether to implement CSMF, NSMF or NSMF within ONAP, or use the external CSMF, NSMF or NSSMF. This implies that for ONAP-based network slice management, we have different choices from an architectural perspective. For Frankfurt release, our scope is to implement CSMF and NSMF within ONAP, while connecting to an external Core NSSMF. -To support the above functions, code impacts in U-UI, SO, OOF and ExtAPI components, and schema change in A&AI -were implemented. For further details, see https://wiki.onap.org/display/DW/Proposed+Functions+for+R6+and+Impacted+Modules - -As part of Frankfurt release work, we supported the minimum-scope installation of ONAP to reduce the resource requirements. -From the module perspective, 5G E2E Slicing use case involves SDC, SO, A&AI, UUI, EXT-API, OOF and Policy modules of ONAP. -So we will configure these required modules along with the mandatory common modules such as DMaaP. Further, for each module, -the use case also does not use all of the charts，so we removed the not needed Charts under those modules to optimize the -resources required for setting up the use case. This approach will help to install a minimum-scope version ONAP for the -E2E Slicing use case. - -Further details of the installation steps are available at: -https://wiki.onap.org/display/DW/Install+Minimum+Scope+ONAP+for+5G+Network+Slicing +From the NSI Life Cycle perspective, the scope for Frankfurt includes NSI design and pre-provision, NSI instantiation and configuration, and NSI activation and deactivation. In particular: -Scope of Guilin release ------------------------ -From the architecture point of view, in Guilin release, besides the continuation of NSMF which was implemented in -Frankfurt release, the RAN NSSMF, TN NSSMF, CORE NSSMF have been implemented within ONAP, apart from interacting with -ezternal RAN NSSMF and external CORE NSSMF. +- CSMF: Functions of slice service creation, slice service activation and deactivation are implemented. -The following provides an overview of the enhancements done in Guilin release: +- NSMF: Functions of NSI instantiation, NSI activation and deactivation are + implemented. In addition, manual intervention is also provided in NSMF slice task + management portal to ensure the selected NSI/NSSI as well as ServiceProfile and + SliceProfile are fine or need adjustment. -- **Enhancements in NSMF**: Service Profile decomposition into Slice Profiles for 3 domains, NSI selection enhancement, - E2E slice instance creation including RAN, TN and CN slice sub-net instance creation/reuse, activation/deactivation - of E2E slice, and deciding whether to terminate E2E slice or not. +- Design of CST, NST and onboarding NSST that are required to support the run-time orchestration functions is also provided. -- **RAN NSSMF, TN NSSMF, CN NSSMF within ONAP**: Basic logic for all 3 NSSMFs to support NSSI allocation, activation, - deactivation, deletion and modification (in case of reuse of NSSI). +- To connect to the external (core) NSSMF, an adaptor is implemented to provide + interface between ONAP and 3rd party core NSSMF. -- **Enable NSMF interaction with RAN NSSMF, TN NSSMF, CN NSSMF**: Implement generic NSSMF adaptor for three domain NSSMFs, - alignment with standard intefaces (3GPP, IETF), enable the connection to external RAN NSSMF. +To support the above functions, code impacts in U-UI, SO, OOF and ExtAPI components, and schema change in A&AI are implemented. -- **Design of RAN NSST, TN NSST, CN NSST and Slice Profiles, TN information models**: Basic E2E Slicing model was provided - all the related templates designed from SDC, TN related information models. - -- **TMF 641 support**: Extension of the TMF 641 based interface from NB of ExtAPI to support service activation, - deactivation and termination. - -- **RAN and CN NFs set up and initial configurations**: CN NF simulators was developed: AMF, SMF, UPF and configure the - S-NSSAI on CN NFs; RAN NF Simulator was enhanced for PM data reporting, CU and Near-RT RIC configuration. - -- **KPI monitoring**: Implementation to request details of a KPI via UUI to ONAP DCAE. Providing the requested data to UUI - by DCAE using a new microservice (Data Exposure Service - DES). Enhancements in PM-Mapper to do KPI computation is - in progress, and will be completed in Honolulu release. - -- **Closed Loop**: First steps to realizing a simple Closed Loop in the RAN using PM data collected from the RAN was - implemented - update the allowed throughput for a S-NSSAI per Near-RT RIC coverage area based on DL/UL PRB for data - traffic that was reported from the RAN. The analysis of the PM data was done using a new Slice Analysis MS in DCAE, - and the Policy-based Control Loop framework was applied to trigger the updates in the RAN. - -- **Intelligent Slicing**: First steps to realizing a simple ML-based Closed Loop action in the RAN using PM data collected - from the RAN was implemented - update the maxNumberofConns for a S-NSSAI in each cell based on PDU session related - PM data that was reported from the RAN (PDU sessions requested, successfully setup and failed to be set up). The - training was done offline, and the ML model is onboarded as a micro-service to ONAP for demo purpose alone (it is - not part of ONAP code/repos). The ML model provides updates to the Slice Analysis MS, which then uses the - Policy-based Control Loop framework to trigger the updates in the RAN. +Further details can be obtained from: +https://wiki.onap.org/display/DW/Proposed+Functions+for+R6+and+Impacted+Modules + + +Impacted Modules for Frankfurt +------------------------------ + +SO +~~ + +CSMF and NSMF are implemented using SO BPMN workflows to support 5G +network slicing use case. CSMF workflow will process the user input +(service request) that comes from CSMF portal (UUI) and save the order +information into a communication service instance in AAI. Then CSMF will +send network slice request to NSMF workflow, and NSMF will then create +service profile, NSI and NSSI. Service profile is a logical concept +which exists only in AAI - it contains two AAI instances, one is a +profile instance that will hold the slice parameters, and the other is a +service instance which will be used to organize the NSI. NSI is also a +service instance in AAI which will be used to organize NSSI. NSSI is the +actual entity which will be created by NSSMF and an AAI service instance +will also be created to represent NSSI in ONAP context. NSI and NSSI can +both be shared. + +SO queries OOF for slice template selection and then slice instance +selection. In response to slice instance selection query, OOF may return +an existing slice instance or may recommend SO to create a new slice +instance. A new process called Orchestration Task is created to manage +recalibration of NSI&NSSI selection with manual intervention from the +portal. A new SO adapter is created to be the adapter of NSSMF which +will interact with external NSSMF for NSSI management. -- **Modeling enhancements**: Necessary modeling enhancements to support all the above functionalities. +Further details can be obtained from: +https://wiki.onap.org/display/DW/SO%3A+Impacts+and+Interfaces + +U-UI +~~~~ + +Usecase-UI (UUI) has added CSMF and NSMF portal components to ONAP to +support this use case. + +CSMF component includes the functions of creating network slicing, as +well as displaying and processing all the created network slices. The +customers need to fill the create communication service form to create a +network slice and then they can see the created network slice in the +list and execute operations of activating, deactivating or terminating +the network slice. + +NSMF component mainly includes two modules: slicing task management and +slice resource management which provides the functions of displaying and +processing all the slicing tasks and slice resources. In slicing task +management module, network operators can find all the slicing tasks +created by customers in CSMF component and executing proper operations +according to different task status. In slice resource management module, +there are three sub-modules which provide the functions of displaying +and processing the existing NS, NSI and NSSI. In addition, the NSMF +component provides the monitoring function so that users can check the +statistics of network slices. In this page, the statistics of slice +usage (traffic), online users and total bandwidth can be monitored and +displayed in the form of pi-charts and lines. -The base use case page for Guilin release is https://wiki.onap.org/display/DW/E2E+Network+Slicing+Use+Case+in+R7+Guilin. +Further details can be obtained from: +https://wiki.onap.org/display/DW/UUI%3A+Impacts + +OOF +~~~ + +For this use case OOF introduced two APIs which are used by SO, one for +slice template selection, and another for NSI/NSSI selection. Within +OOF, both the OSDF and HAS sub-components were enhanced for this use +case. OSDF maps the new API request contents to the appropriate format +for HAS to perform the optimization. After the optimization is done by +HAS, OSDF maps the response in the API response format as expected by +SO. Further, HAS always returns NSSI info (when existing NSSIs can be +reused) and OSDF then determines whether it refers to reuse of an +existing NSI or creation of a new NSI, and then prepares sends the +response to SO. + +HAS sub-component of OOF has been enhanced to use a couple of new policy +types, the AAI plug-in within HAS was enhanced to fetch the slice and +slice sub-net related details from AAI. Two new plug-ins were developed +in HAS – one for fetching slice templates and another for generating +slice profile candidates. Existing policies were reused and suitably +adapted for constraints and optimal selection of slice template and +slice instance. In case of new NSSI creation, HAS returns appropriate +slice profile for the sub-net for which a new NSSI has to be created. -Important Remarks -~~~~~~~~~~~~~~~~~ -(a) 2 deployment scenarios for RAN NSSI are supported. In the first scenario, the RAN NSSI comprises also of - TN Fronthaul (FH) and TN Midhaul (FH) NSSIs, and RAN NSSMF shall trigger TN NSSMF for TN FH and MH NSSI - related actions. In the second scenario, the RAN NSSI comprises only of RAN NFs. TN NSSMF shall be triggered by - NSMF for TN FH and MH NSSI related actions. This part is not yet implemented in NSMF, and will be realized in - Honolulu release. +Further details can be obtained from: +https://wiki.onap.org/display/DW/OOF%3A+Impacts+and+Interfaces + +EXT-API +~~~~~~~ + +The EXT-API has undergone some minimal enhancements for this use case in +Frankfurt release. A new value “CST” for the serviceType attribute in +the Service Order API has been introduced. + +The CSMF Portal in UUI captures the values for the requested +serviceCharacteristics that are required as inputs to CST Service model. +The relatedParty attribute in the Service Order is set according to the +Customer, where relatedParty.id will map to the AAI "global-customer-id“ +in the “customer” object. The serviceSpecification.id is to be set to +the UUID of the CST from SDC (i.e., this is the template for the Service +we are ordering from CSMF). The action field will be set to “add” to +indicate creation of a new service instance. CSMF Portal in UUI then +sends POST with the JSON body to /{api_url}/nbi/api/v4/serviceOrder/. +ExtAPI will generate a Service Order ID and send it in the response – +this ID can be used to track the order. ExtAPI will then invoke SO’s API +for creating the service. + +As can be seen from above explanation, the existing constructs of ExtAPI +has been reused with minor enhancements. -(b) The Guilin release scope focused on enhancing NSMF and implementing basic functionality of all 3 NSSMFs within - ONAP with the objective of realizing a simple E2E network slice using standard interfaces. So, further enhancements - and refinement will continue in Honolulu release and beyond. +Further details can be obtained from: +https://wiki.onap.org/display/DW/ExtAPI%3A+Impacts+and+Interfaces -(c) Details of the assumptions in Guilin release are documented at: - https://wiki.onap.org/display/DW/Assumptions+for+Guilin+release +A&AI +~~~~ -(d) Sequence/flow diagrams related to this use case are available at: - https://wiki.onap.org/display/DW/Use+case+flows +To support this use case，A&AI module has added 3 new nodes +(Communication-service-profile, Service-profile and +Slice-profile)，modified service-instance nodes, added 3 new nodes as +new attributes of service-instance node. To map to SDC templates +(Communication Service Template/Service Profile +Template/NST/NSST)，run-time instances of this use case have +Communication Service Instance/Service Profile Instance/NSI/NSSI. To +align with ONAP’s model-driven approach, this use case reuses +"service-instance" for all run-time instances. The relationship between +service-instances use the existing attribute "relationship-list" or +"allotted-resources". Communication-service-profile means the original +requirement of Communication-service-instance, such as latency, +data-rate, mobility-level and so on. Service-profile means the slice +parameter info of Service-profile-instance. Slice-profile holds the +slice sub-net parameter info of different network domain NSSIs, such as +(Radio) Access Network (AN), Transport Network (TN) and Core Network +(CN) NSSI. -(e) Details of the Modeling aspects are available at: - https://wiki.onap.org/display/DW/Modeling+enhancements?src=contextnavpagetreemode +A&AI provides query APIs to CSMF and NSMF, such as: +- Query + Communication-service-instances/Service-profile-instances/NSI/NSSI -Impacted Modules for Guilin ---------------------------- -The code-impacted modules of E2E Network Slicing in Guilin release are: +- Query Service-profile-instance by specified + Communication-service-instance -- **SO**: Enhancements in NSMF explained above, NSSMF adaptor, workflows for RAN, Core and TN NSSMFs, Closed Loop - trigger handling. +- Query NSI by specified Service-profile-instance, query NSSI by + specified NSSI. -- **OOF**: Enhancements for NSI selection, NSSI selection, Slice Profile generation, and NSI/NSSI termination - determination +A&AI also supply creation APIs to SO, such as: -- **DCAE**: 2 new MS were introduced: - (a) **Data Exposure Service (DES)** for querying database/datalake for PM/KPI data by any entity - (b) **Slice Analysis MS** for analyzing slice related PM/KPI data and determining necessary closed loop actions +- Create Communication-service-profile/Service-profile/Slice-profile, + and -- **AAI**: Updates for Slice Profile, NSST, Service Profile, TN slice models +- Create relationship between service-instances. -- **SDN-C**: TN NSSMF related functionality (allocation, activation/deactivation and termination of TN NSSI/S-NSSAI) +Further details can be obtained from: +https://wiki.onap.org/pages/viewpage.action?pageId=76875989 -- **CCSDK/SDN-R**: RAN NSSMF related functionality related to allocation, activation/deactivation and termination of RAN - NSSI/S-NSSAI, mainly the configuration of CUs, DUs and Near-RT RICs over O1 interface to the RAN NFs. In addition, - configuration updates related to Closed Loop and Intelligent Slicing shall also be sent over O1 interface. - -- **U-UI**: Enhancements related to configuration/selection of end-points. -- **ExtAPI**: Enhancements to support service activation/deactivation and termination using TMF 641 APIs +Functional Test Cases +--------------------- -- **Policy**: Minor updates to support the new Control Loop for Network Slicing. +The functional testing of this use case shall cover creation and +activation of a service with an E2E Network Slice Instance which +contains a Core Slice Sub-net instance. It also addresses the +termination of an E2E Network Slice Instance. It covers the following +aspects: -Apart from the above, the following modules had test-only impact: +- Creation of a new customer service via CSMF portal in UUI resulting + in creation of a new NSI -- **SDC**: Test of the supported functionality through creation of NST, NSST for all 3 domains, templates for Service - Profile and Slice Profile. +- Creation of a new customer service via CSMF portal in UUI resulting + in re-use of an existing NSI -- **CDS**: Support for configuration of S-NSSAI in the Core NFs. +- Activation of a customer service via CSMF portal in UUI -In addition: +- Creation of a new customer service via postman request to EXT-API + resulting in creation of a new NSI -- **Config DB** is enhanced to support storing and retrieval of RAN-related configuration data. This is not an official - ONAP component, and its functionality is expected to be performed by the Configuration Persistence Service in - Honolulu release. +- Creation of a new customer service via via postman request to ExtAPI + resulting in re-use of an existing NSI -- **Core NF simulators** have been developed for instantiating as part of Core NSSI creation/configuration, and also - to report PM data. +- Manual intervention via NSMF portal during NSI selection (NSI + selection adjustment) -- **RAN-Sim** has been enhanced to include CU and Near-RT RIC functionality, apart from enhancements to DU functionality. +- Termination of a NSI and associated NSSI -Details of the impacts/APIs of some of the modules listed above are available in the child pages of: -https://wiki.onap.org/display/DW/Impacted+Modules--Design+Details +- Interaction between ONAP and external NSSMF for new core NSSI + creation +- Checking inventory updates in AAI for NSIs, service and slice + profiles and NSSIs. -Functional Test Cases ---------------------- -The functional testing of this use case shall cover creation and activation of a service with an E2E Network Slice -Instance which contains RAN, Transport and Core Slice Sub-net instances. It also addresses the termination of an -E2E Network Slice Instance. We classify the test cases into 7 tracks: CSMF, NSMF, Common NSMMF/RAN NSSMF/TN NSSMF/ -Core NSSMF/KPI Monitoring/Close loop/Intelligent Slicing. Details of the test cases can be found at: -https://wiki.onap.org/display/DW/Track-wise+test+cases +Further details can be obtained from: +https://wiki.onap.org/display/DW/Functional+Test+Cases Operation Guidance ------------------ -The Guilin release setup details for the E2E Network Slicing use case will be available at the following page and its -sub-pages: -https://wiki.onap.org/display/DW/Demo+setup+steps+for+Guilin - - -Known Issues and Resolutions -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -(a) A limited testing of the supported scenarios is being carried out in Guilin. For example, for NSI allocation, only - a sub-set of combinations of NSI reuse, creation using existing or new NSSIs (RAN, Core, TN) is being tested. However, - since the remaining functionality will be activated only when triggered with the appropriate inputs, this limitation - shall not affect the stability of the components or other existing functionality. - -(b) The configuration updates for Closed Loop and Intelligent Slicing to the RAN is sent over O1 interface to Near-RT RIC. - It should be sent over A1 interface - this will be taken up in Honolulu release. -(c) Coverage area to TA list is not performed. Based on the coverage area provided as input, a set of cells are assumed to - be impacted. +How to install 5G E2E Slicing Minimum Scope +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +For 5G E2E Slicing use case, we support the minimum-scope installation +of ONAP to reduce the resource requirements. From the module +perspective, 5G E2E Slicing use case involves SDC, SO, A&AI, UUI, +EXT-API, OOF and Policy modules of ONAP. So we will configure these +required modules along with the mandatory common modules such as DMaaP. +Further, for each module, the use case also does not use all of the +charts，so we removed the not needed Charts under those modules to +optimize the resources required for setting up the use case. This +approach will help to install a minimum-scope version ONAP for 5G E2E +Slicing use case. -(d) NST selection requires a file to be stored corresponding to the templates created in SDC. In Honolulu release, OOF - functionality will be enhanced to fetch the NSTs from SDC. - -(e) RAN/Core NSSI endpoint determination, configuration and use is limited in Guilin release. +Further details of the installation steps are available at: +https://wiki.onap.org/display/DW/Install+Minimum+Scope+ONAP+for+5G+Network+Slicing -(f) Only creation of new TN NSSI is supported, reuse of existing TN NSSI is not yet supported. -(g) KPI computation functionality enhancements in PM-Mapper in DCAE is not part of Guilin release, though the functionality - is implemented and available. +Configuration aspects +~~~~~~~~~~~~~~~~~~~~~ +The template design, UI configuration, as well as manual configurations for some +of the components are all described in the following wiki page and its sub-pages: +https://wiki.onap.org/display/DW/Operation+Guidance+for+5G+Network+Slicing+Use+Case diff --git a/docs/docs_robot.rst b/docs/docs_robot.rst index c80281abf..96b4b7c65 100644 --- a/docs/docs_robot.rst +++ b/docs/docs_robot.rst @@ -61,15 +61,12 @@ At the end you shall see that logs and reporting are automatically generated by robot and stored on the pod under /share/logs/. You can retrieve the logs directly from the pods or through the web interface integrated in the robot pod by typing -http://portal.api.simpledemo.onap.org:30209/logs/ - -.. note:: - Login credentials: (test/test) +http://portal.api.simpledemo.onap.org:30209/logs/ (test/test) .. figure:: files/robot/robot_logs.png :align: center -In every test directory, You shall see 3 files: +You shall see 3 files: * log.html * output.xml @@ -160,8 +157,8 @@ Onboard and Instantiate VNF/PNF with Robot ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Robot scripts have been also created to provision components and perform end to end tests. -All the robot scripts are hosted on the tesuite repository (https://git.onap.org/testsuite/tree/robot/testsuites) -and deal with various test cases and/or components. +All the robot scripts are hosted on the tesuite repository and deal with various +test cases and/or components. :: @@ -194,47 +191,6 @@ the SDC by running the command: :: $ ete-k8s.sh onap healthdist - ++ export NAMESPACE=onap - ++ NAMESPACE=onap - +++ kubectl --namespace onap get pods - +++ sed 's/ .*//' - +++ grep robot - ++ POD=dev-robot-robot-6d444f4cdd-n7vg5 - ++ TAGS='-i healthdist' - +++ dirname ./ete-k8s.sh - ++ DIR=. - ++ SCRIPTDIR=scripts/etescript - ++ ETEHOME=/var/opt/ONAP - ++ [[ healthdist == \e\x\e\c\s\c\r\i\p\t ]] - +++ kubectl --namespace onap exec dev-robot-robot-6d444f4cdd-n7vg5 -- bash -c 'ls -1q /share/logs/ | wc -l' - ++ export GLOBAL_BUILD_NUMBER=5 - ++ GLOBAL_BUILD_NUMBER=5 - +++ printf %04d 5 - ++ OUTPUT_FOLDER=0005_ete_healthdist - ++ DISPLAY_NUM=95 - ++ VARIABLEFILES='-V /share/config/robot_properties.py' - ++ VARIABLES='-v GLOBAL_BUILD_NUMBER:32515' - ++ kubectl --namespace onap exec dev-robot-robot-6d444f4cdd-n7vg5 -- /var/opt/ONAP/runTags.sh -V /share/config/robot_properties.py -v GLOBAL_BUILD_NUMBER:32515 -d /share/logs/0005_ete_healthdist -i healthdist --display 95 - Starting Xvfb on display :95 with res 1280x1024x24 - Executing robot tests at log level TRACE - ============================================================================== - Testsuites - ============================================================================== - Testsuites.Health-Check :: Test that ONAP components are available via basi... - ============================================================================== - Health Distribution Test | PASS | - ------------------------------------------------------------------------------ - Testsuites.Health-Check :: Test that ONAP components are available... | PASS | - 1 critical test, 1 passed, 0 failed - 1 test total, 1 passed, 0 failed - ============================================================================== - Testsuites | PASS | - 1 critical test, 1 passed, 0 failed - 1 test total, 1 passed, 0 failed - ============================================================================== - Output: /share/logs/0005_ete_healthdist/output.xml - Log: /share/logs/0005_ete_healthdist/log.html - Report: /share/logs/0005_ete_healthdist/report.html If you consider the ete-k8s.sh script, the following testsuites are referenced: @@ -248,13 +204,11 @@ If you consider the ete-k8s.sh script, the following testsuites are referenced: * oof-cmso.robot, oof-homing.robot, oof-has.robot: oof related tests * pnf-registration.robot: ete, pnf_registrate * post-install-tests.robot dmaapacl, postinstall -* sdc-dcae-d.robot: sdc-dcae-d -* security.robot: security * update_onap_page.robot: UpdateWebPage * vnf-orchestration-direct-so.robot: instantiateVFWdirectso * vnf-orchestration.robot: instantiate, instantiateNoDelete, stability72hr -If you use the demo-k8s.sh script, you may see the following options: +If you use the demo-k8s.s script, you may see the following options: * init_robot * init @@ -264,7 +218,6 @@ If you use the demo-k8s.sh script, you may see the following options: * appc * instantiateVFW * instantiateVFWdirectso -* instantiateVLB_CDS * deleteVNF * heatbridge * cds @@ -273,53 +226,6 @@ If you use the demo-k8s.sh script, you may see the following options: * instantiateDemoVFWDT * vfwclosedloop -Below is the sample output for init execution with demo-k8s.sh script - -:: - - $ ./demo-k8s.sh onap init - Number of parameters: 2 - KEY: init - ++ kubectl --namespace onap get pods - ++ sed 's/ .*//' - ++ grep robot - + POD=dev-robot-robot-6d444f4cdd-6t77j - ++ dirname ./demo-k8s.sh - + DIR=. - + SCRIPTDIR=scripts/demoscript - + ETEHOME=/var/opt/ONAP - + '[' ']' - ++ kubectl --namespace onap exec dev-robot-robot-6d444f4cdd-6t77j -- bash -c 'ls -1q /share/logs/ | wc -l' - + export GLOBAL_BUILD_NUMBER=4 - + GLOBAL_BUILD_NUMBER=4 - ++ printf %04d 4 - + OUTPUT_FOLDER=0004_demo_init - + DISPLAY_NUM=94 - + VARIABLEFILES='-V /share/config/robot_properties.py' - + kubectl --namespace onap exec dev-robot-robot-6d444f4cdd-6t77j -- /var/opt/ONAP/runTags.sh -V /share/config/robot_properties.py -d /share/logs/0004_demo_init -i InitDemo --display 94 - Starting Xvfb on display :94 with res 1280x1024x24 - Executing robot tests at log level TRACE - ============================================================================== - Testsuites - ============================================================================== - Testsuites.Demo :: Executes the VNF Orchestration Test cases including setu... - ============================================================================== - Initialize Customer And Models | PASS | - ------------------------------------------------------------------------------ - Initialize SO Openstack Identity For V3 | PASS | - ------------------------------------------------------------------------------ - Testsuites.Demo :: Executes the VNF Orchestration Test cases inclu... | PASS | - 2 critical tests, 2 passed, 0 failed - 2 tests total, 2 passed, 0 failed - ============================================================================== - Testsuites | PASS | - 2 critical tests, 2 passed, 0 failed - 2 tests total, 2 passed, 0 failed - ============================================================================== - Output: /share/logs/0004_demo_init/output.xml - Log: /share/logs/0004_demo_init/log.html - Report: /share/logs/0004_demo_init/report.html - See :ref:`Verified Use Cases and Functional Requirements <docs_usecases>` to see how to use these scripts. Some of them may need specific prerequisites on the environment and may support specific versions and/or configurations. diff --git a/docs/docs_usecases.rst b/docs/docs_usecases.rst index ff5e18bff..95a0c67a0 100644 --- a/docs/docs_usecases.rst +++ b/docs/docs_usecases.rst @@ -4,10 +4,10 @@ .. _docs_usecases: Verified Use Cases and Functional Requirements ----------------------------------------------- +============================================== Description -~~~~~~~~~~~ +----------- This session includes use cases and functional requirements which have been officially verified in Frankfurt release by the ONAP community. @@ -21,66 +21,60 @@ This documentation deals with links to download the related assets and resources 3. Known issues and workarounds -The final testing status can be found at `Frankfurt Release Integration Testing -Status <https://wiki.onap.org/display/DW/2%3A+Frankfurt+Release+Integration+Testing+Status>`_ - -31 use cases/functional requirements have been considered for the Frankfurt release. - Use cases ~~~~~~~~~ .. csv-table:: use case table - :file: usecases.csv - :widths: 60,20,20 + :file: ./files/csv/usecases.csv + :widths: 10,40,20,30 + :delim: ; :header-rows: 1 Functional Requirements ~~~~~~~~~~~~~~~~~~~~~~~ .. csv-table:: functional requirements table - :file: functional-requirements.csv - :widths: 60,20,20 - :header-rows: 1 - -.. csv-table:: 5G functional requirements table - :file: functional-requirements-5g.csv - :widths: 60,20,20 + :file: ./files/csv/usecases-functional-requirements.csv + :widths: 10,40,10,10,30 + :delim: ; :header-rows: 1 Automated Use Cases ~~~~~~~~~~~~~~~~~~~ -Most of the use cases include some automation through robot or bash scripts. -These scripts are detailed in the documentation. - -Some use cases have been integrated in ONAP gates. It means the tests are run on -each daily or gating CI chain. The goal is to detect any regression as soon as -possible and demonstrate the ability to automate the use cases. +.. csv-table:: Infrastructure Healthcheck Tests + :file: ./files/csv/tests-infrastructure-healthcheck.csv + :widths: 20,40,20,20 + :delim: ; + :header-rows: 1 -.. csv-table:: automated use cases table - :file: automated-usecases.csv - :widths: 10,80,10 +.. csv-table:: Healthcheck Tests + :file: ./files/csv/tests-healthcheck.csv + :widths: 20,40,20,20 :delim: ; :header-rows: 1 -The robot scripts can be found in ONAP testsuite repository, an execution -run-time is provided through the robot pod. +.. csv-table:: Smoke Tests + :file: ./files/csv/tests-smoke.csv + :widths: 20,40,20,20 + :delim: ; + :header-rows: 1 -The python onap_tests framework is hosted on -https://gitlab.com/Orange-OpenSource/lfn/onap/onap-tests. Please not that this -framework is valid up to Frankfurk and will be deprecated in Guilin. It will -be replaced by scenarios leveraging python-onapsdk -https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk. +.. csv-table:: Security Tests + :file: ./files/csv/tests-security.csv + :widths: 20,40,20,20 + :delim: ; + :header-rows: 1 Deprecated Use Cases ~~~~~~~~~~~~~~~~~~~~ -The following use cases were included in El Alto or previous release but have +The following use cases were included in Frankfurt or previous release but have not been tested in Frankfurt, usually due to a lack of resources. The resources are still available in previous branches, some adaptations may however be needed for Frankfurt. .. csv-table:: deprecated use case table - :file: usecases-deprecated.csv + :file: ./files/csv/usecases-deprecated.csv :widths: 50,20,10,20 :header-rows: 1 diff --git a/docs/files/csv/issues-integration.csv b/docs/files/csv/issues-integration.csv new file mode 100644 index 000000000..70ead70c8 --- /dev/null +++ b/docs/files/csv/issues-integration.csv @@ -0,0 +1,11 @@ +Issue key;Issue id;Summary;Priority;Status;Created +INT-1770;56498;PNF registarte test case very offen fails at general step Submit SDC Software Product;Medium;Open;05/Nov/20 7:32 PM +INT-1755;56342;[PMSH] Wrong AAI api version in CSITs;Medium;Open;28/Oct/20 11:20 AM +INT-1738;55962;SO CSIT failing due to invalid syntax;Medium;Open;09/Oct/20 2:54 PM +INT-1726;55683;SDC intitlement pool creation is failing;Medium;Open;24/Sep/20 2:44 PM +INT-1720;55314;Robot certificates is too long and has bad owner;Medium;Open;14/Sep/20 6:57 AM +INT-1717;55218;Robot has python 2.7 pods;High;Open;10/Sep/20 7:31 AM +INT-1716;55175;robot pod is launched as root;High;Open;09/Sep/20 4:32 PM +INT-1660;53648;Fix 5g bulkpm CSITs for pm-mapper java 11;Medium;Open;16/Jul/20 5:28 AM +INT-1659;53599;MDONS Extnesion: Integration Test;Medium;In Progress;15/Jul/20 2:58 AM +INT-1655;53455;SDNC CSIT failing for test plan sdnc_netconf_tls_post_deploy.;Medium;Delivered;08/Jul/20 5:24 PM diff --git a/docs/files/csv/issues-testsuite.csv b/docs/files/csv/issues-testsuite.csv new file mode 100644 index 000000000..2cbe8f9b6 --- /dev/null +++ b/docs/files/csv/issues-testsuite.csv @@ -0,0 +1,2 @@ +Issue key;Issue id;Summary;Priority;Status;Created +TEST-277;56680;Issues with basic_clamp test;Medium;Open;12/Nov/20 9:24 AM diff --git a/docs/files/csv/release-demo-features.csv b/docs/files/csv/release-demo-features.csv new file mode 100644 index 000000000..b7d99bf6e --- /dev/null +++ b/docs/files/csv/release-demo-features.csv @@ -0,0 +1,8 @@ +Issue-ID;Description +NT-1658;vFW CNF CDS scripts update for native Helm support +INT-1240;ROBOT e2e test cases improvements for csar/zip tosca package validation in VNFSDK and SDC Onbarding +INT-1658;vFW CDS CNF CBA with native k8s profile upload +INT-1724;Update INFO.yaml +INT-1600;VNF Software Version parameter added to CDS Heat templates +INT-1625;Update pom files in master to 1.7.0-SNAPSHOT +SO-2983;Fix for missing VNFC rel information diff --git a/docs/files/csv/release-integration-features.csv b/docs/files/csv/release-integration-features.csv new file mode 100644 index 000000000..9dbba01c8 --- /dev/null +++ b/docs/files/csv/release-integration-features.csv @@ -0,0 +1,112 @@ +Issue-ID;Description +NT-1736;Refactor Integration official documentation +INT-1777;Updated docs_E2E_network_slicing.rst +INT-1776;[DOC] Adding few notes and sample outputs to the robot framework documentation +INT-1772;Update OOF PCI document +INT-1740;PNF Software Upgrade documentation +INT-1759;Minor corrections to doc index +INT-1764;Display waiver content for the security tests +INT-1764;Display waiver content for the security tests +INT-1763;Update rke config for windriver deployment +INT-1690;Update for MDONS Extersion in Guilin +INT-1616;optimize size and time using "--no-cache-dir" +INT-1723;Setup basic functional test of NSSMF RAN simulator +INT-1601;Set up and tear down test environment properly +INT-1724;Update INFO.yaml +INT-1723;Add missing dependency for the werkzeug module in setooptools as well +INT-1723;Add missing dependency for the werkzeug module +INT-1707;Add an External RAN NSSMF simulator +INT-1570;Add timeout on SSL internal cert retrieval +INT-1710;Remove old SECCOM waiver lists source +INT-1570;check_certificates: create result dir if needed +TEST-239;update vcpe case +TEST-259;Fixed CBA sdnc_model_version into vFW_CDS_CNF documentation +POLICY-2794;update policy staging images +INT-1601;Add support for detecting Java 8 +INT-1704;AAI-Simulator: Pulling from PNF-SW-UP CSIT to Integration repo +DCAEGEN2-2267;Update tests for Datafile Collector (DCAE SDK update) +INT-1570;Add MANIFEST.in to include j2 file in the python package +POLICY-2794;update policy staging images +INT-1695;Reset security xfail list for Guilin +INT-1696;Update release note for Frankfurt maintenance release +INT-1672;Drop using symlinks for documentation markup rendering +INT-1601;Replace deprecated "with_items" with loops +INT-1601;Add versions of tools used during verification +INT-1677;Manual revert of kubernetes version in rke scripts +INT-1677;Update Kubernetes rke version +INT-1601;Use external volumes for OpenStack virtual machines +INT-1601;Add hosts configuration for Service Mesh ONAP +INT-1601;Make Vagrant-based environment safe defaults +INT-1601;Add separate NICs to access OpenStack virtual machines +INT-1601;Add Ansible roles for OpenStack security groups +INT-1601;Add Ansible roles for OpenStack hosts +INT-1601;Add Ansible roles for OpenStack network +INT-1601;Add Vagrant-based development environment +INT-1601;Introduce ONAP CI Lab infrastructure setup guide +INT-1601;Introduce ONAP CI Lab setup guide +DCAEGEN2-2367;Adjust DFC test files to use FTPES instead of FTPS +INT-1571;Import upstream component version inspection tool +INT-1662;Fix vsftpd config setup +INT-1663;Add README file +INT-1662;Add HOT parameters for the simulator +INT-1661;Setup build env on instance +INT-1661;Setup runner script for the simulator in HOT +DCAEGEN2-2367;Update tls-init-container +INT-1657;[PMSHSIM] Refactoring pmsh-pnf-sim +DCAEGEN2-2219;Add sftp strict host key checking to DFC consul configs +INT-1652;Update release notes for Frankfurt Maintenance release +INT-1570;Fix check certification python pakaging +INT-1649;Add basic cloud-init setup for the simulator instance +INT-1641;Create basic Heat template +INT-1570;Initiate check certificate validity test +INT-1571;Update Operation box in Vagrant environment +INT-1639;netconf-pnp-simulator: fix supervisor patching failure +INT-1642;Create runtime setup for the script +INT-1643;Fix --conf option validation +OOM-2437;Add compare NodePort and Ingress scan +OOM-2434;Fix ingress default http and https ports +INT-1630;Start/stop the 3GPP measurements file generator at simulator start/stop +INT-1629;Pass options to each lifecycle method separately +INT-1629;Validate the subcommands on each instance method invocation +INT-1629;Remove the existing_sim_instances instance attribute +INT-1627;Support setting custom username/password for the file server service +INT-1623;Drop shell wrapper for docker compose operations +INT-1623;Run the 3GPP measurements file generator in bootstrap phase +INT-1621;Generate vsftpd config file in bootstrap phase +INT-1621;Make the config generation private method more generic +INT-1625;Update release note to include demo artifacts +INT-1626;OUpdate OOF_SON use case documentation +ONAPARC-551;Allow using separate storage pool for Vagrant-based minimal ONAP deployment +ONAPARC-551;Adjust default storage device argument on Vagrant-based minimal ONAP deployment +ONAPARC-551;Increase Helm timeout on Vagrant-based minimal ONAP deployment +ONAPARC-551;Set masterPassword on Vagrant-based minimal ONAP deployment +ONAPARC-551;Disable Helm charts linting on Vagrant-based minimal ONAP deployment +ONAPARC-551;Make Helm timeout adjustable on Vagrant-based minimal ONAP deployment +INT-1551;Update stability test documentation +INT-1619;Add a helper method to generate docker compose file +INT-1619;Simplify network name in Compose template +INT-1618;Add helper method to generate simulator config +INT-1617;Update Mass PNF simulator documentation +INT-1617;Drop action methods decorator +INT-1617;Setup and start docker-compose simulator app directly in Python module +INT-1617;Chown vsftpd config file to root while bootstrapping +INT-1617;Add internal routine for logs archiving +INT-1617;Remove useless RUNNING_COMPOSE_CONFIG assignment +INT-1617;Drop shell debug mode in command wrapper method +OOM-2331;Update helm and kubectl versions +INT-1615;Correct ReadTheDoc theme +INT-1457;[vFW_CNF_CDS] Fix typo in Postman collection request +INT-1613;Support stopping sending PNF registration messages in Python module +INT-1613;Add test case for invalid '--count' option value +INT-1611;Drop shell wrapper for 'trigger_custom' action as well +INT-1611;Support triggering simulator app directly from MassPnfSim module +INT-1610;Drop shell wrapper for stopping simulator instances +INT-1531;[vFW_CNF_CDS] UAT testing doc update +INT-1551;Update doc to reference 1.6.3 RC2 frankfurt image +INT-1468;Update E2E_network_slicing doc +INT-1610;Add helper routine for getting iter range in lifecycle methods +INT-1606;Drop shell wrapper for getting simulators status +INT-1606;Add helper method to get running docker containers names list +INT-1606;Add helper method to get simulator instance data +INT-1466;Update OOF_SON use case documentation +INT-1608;Remove postman subdirectory diff --git a/docs/files/csv/release-integration-ref.csv b/docs/files/csv/release-integration-ref.csv new file mode 100644 index 000000000..cf1399f46 --- /dev/null +++ b/docs/files/csv/release-integration-ref.csv @@ -0,0 +1,22 @@ +Repository;Revision +demo; +integration; +integration/csit;e50b1f4517d353e31626d06fa3a5bea079a07c44 +integration/docker/onap-java11;1508c9d6e2afa05808e50bb835c48669b1cf9b03 +integration/docker/onap-python;b2f19c061037bbf899330f5394972fb6a2ab1ed1 +integration/seccom;a8a3e4197944e10d162173cea355c567823fe723 +integration/simulators/5G-core-nf-simulator; +integration/simulators/dc-simulator; +integration/simulators/ran-simulator; +integration/terraform; +integration/terragrunt; +integration/usecases/BBS; +integration/usecases/mdons; +integration/xtesting; +oparent;a03c1fa9ec368a4352c1e2c3853747133be3aa29 +testsuite; +testsuite/cds; +testsuite/heatbridge; +testsuite/oom; f3afcb0e60f54840d386084ee3ee2e364b2650f1 +testsuite/python-testing-utilestestsuite/pythonsdk-tests; +testsuite/robot-utils; diff --git a/docs/files/csv/release-oparent-features.csv b/docs/files/csv/release-oparent-features.csv new file mode 100644 index 000000000..d97c5fd2e --- /dev/null +++ b/docs/files/csv/release-oparent-features.csv @@ -0,0 +1,14 @@ +Issue-ID;Description +INT-1756;Set default to guilin +INT-1486;Bump oparent to version 3.1.1-SNAPSHOT +INT-1486;Release oparent version 3.1.0 +INT-1489;Add CVE Profile +POLICY-2387;Updating jetty and including jetty-security +INT-1488;Add formatter plugin +INT-1271;Upgrade checkstyle to version 8.32 Google style +INT-1486;Upgrade latest dependencies - part 2 +INT-1575;Upgrade to latest dependencies +INT-1464;Add jetty-servlets as a dependency +INT-1574;Do not use alpha version of logback +INT-1540;Bump oparent master to 3.1.0 +INT-1539;Update INFO.yaml diff --git a/docs/files/csv/release-testsuite-features.csv b/docs/files/csv/release-testsuite-features.csv new file mode 100644 index 000000000..3d3cd0d6a --- /dev/null +++ b/docs/files/csv/release-testsuite-features.csv @@ -0,0 +1,49 @@ +Issue-ID;Description +INT-1770;PNF registarte - prevent test case fail at general step Submit SDC Software Product +INT-1768;Create testsuite:1.7.2 for RC1 +INT-1240;ROBOT e2e test cases improvements for csar/zip tosca package validation in VNFSDK and SDC Onbarding +INT-1258;Adapt suite setup to wait until topic is created +INT-1761;Bump Guilin 1.7.1 testcase docker +INT-1258;Adapt events to have the newest 3GPP path in schemaReference +INT-1754;5gbulkpm k8splugin version update for sftp component +INT-1731;Add man in the middle CMPv2 E2E test case in robot test suite certificate SANs verification +INT-1258;Fix resource paths for xtesting purposes +INT-1258;Add Suite Setup to Ves Test Suite +INT-1258;Add E2E integration tests to cover new stndDefined VES events +INT-1716;Fix Docker file +SO-3295;Revert "Adding NSSMF adapter healthcheck in SO" +INT-1731;Add CMPv2 E2E test case in robot test suite +CCSDK-2768;Improved CDS Health Check +SO-3295;Adding NSSMF adapter healthcheck in SO +SDC-3287;Enhanced SDC Healthcheck +TEST-266;Self release 1.7.0 +INT-1727;Fix docker staging build +INT-1637;Synch csar download path for vCPE +INT-1726;Add entitlement pool type to entitlement_pool.jinja +INT-1636;Add customer to Load vCPE Models +INT-1724;Update INFO.yaml +SDNC-1349;SDNC new Healthcheck TC +OPTFRA-851;Modify healthcheck for OOF-OSDF +INT-1668;fix Get Service Id issue in pnf registartion test case in xtesting gating and master +AAI-3123;A&AI health check improvement: +INT-1668;Remove from CI PNF registration test that uses deprecated vnf api +POLICY-2817;fix delete policy test in new policy healthcheck +POLICY-2817;Missing Variable Added +POLICY-2817;Improved Policy Health Check +INT-1668;Remove from CI PNF registration test that uses deprecated vnf api +POLICY-2003;Remove deprecated Policy Component Tests +INT-1702;Add tcagen2 blueprint +INT-1702;Update dcaegen2 tca policy type +INT-1675;hv-ves bad certificate +INT-1678;5gBulkPm sftp bp fix +INT-1668;Fix randomly failing test case for pnf refgstarte +INT-1668;Fix randomly failing test case for pnf refgstarte in xtetsting daily +VFC-1712;Remove health-check related to components which are not in Guilin Release +TEST-248;run cds bootstrap before vlb_cds instantiation +INT-1652;Self release testsuite 1.6.4 +VFC-1689;[VFC]Remove ems driver health check +TEST-247;Some corrections for VLB CDS +TEST-234;change owning-entity-id from hard coded to variable +INT-1553;Release testsuite 1.6.3 for Frankfurt RC2 +INT-1521;Fix 5g_bulk_PM and pnf_registrate test case errors +INT-1604;Fix problem with SSL in HV-Ves automated test case diff --git a/docs/files/csv/repo-demo.csv b/docs/files/csv/repo-demo.csv new file mode 100644 index 000000000..3d5c872fc --- /dev/null +++ b/docs/files/csv/repo-demo.csv @@ -0,0 +1,2 @@ +Repository;Description;Link +demo;Historical repository to host use case artifacts (heat templates, json files,..);`link <https://gerrit.onap.org/r/admin/repos/demo>`__ diff --git a/docs/files/csv/repo-integration-external.csv b/docs/files/csv/repo-integration-external.csv new file mode 100644 index 000000000..40843dba5 --- /dev/null +++ b/docs/files/csv/repo-integration-external.csv @@ -0,0 +1,4 @@ +Repository;Description;Link +pythononapsdk;Python SDK for ONAP, used to create use cases;`link <https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk/-/tree/develop/src/onapsdk/cds>`__ +xtesting-onap;Repository in charge of the test section of the CD chains;`link <https://gitlab.com/Orange-OpenSource/lfn/onap/xtesting-onap/>`__ +integration-view;Repository integration hosting the itegration portal including the hosting of the web site;`link <https://gitlab.com/Orange-OpenSource/lfn/onap/integration-view>`__ diff --git a/docs/integration-repositories.csv b/docs/files/csv/repo-integration.csv index bb585afd6..e96ab325d 100644 --- a/docs/integration-repositories.csv +++ b/docs/files/csv/repo-integration.csv @@ -3,14 +3,13 @@ integration;Historical main repository including documentation, simulators (e.g. integration/csit;Repository hosting some tooling to start component functional tests in Jenkins (To be deprecated in Guilin as such tests must be reinsourced by the projects);`link <https://gerrit.onap.org/r/admin/repos/integration/csit>`__ integration/docker/onap-java11;Java11 baseline image conformed to SECCOM recommendations;`link <https://gerrit.onap.org/r/admin/repos/integration/docker/onap-java11>`__ integration/docker/onap-python;Python baseline image conformed to SECCOM recommendations;`link <https://gerrit.onap.org/r/admin/repos/integration/docker/onap-python>`__ +integration/seccom;Repory hosting seccom recommended versions and security test waivers;`link <https://gerrit.onap.org/r/admin/repos/integration/seccom>`__ +**integration/simulators/5G-core-nf-simulator**;**5G core nf simulator**;`link <https://gerrit.onap.org/r/admin/repos/integration/simulators/5G-core-nf-simulator>`__ integration/simulators/dc-simulator;Data Center simulator;`link <https://gerrit.onap.org/r/admin/repos/integration/simulators/dc-simulator>`__ integration/simulators/pnf-simulator;PNF Simulator;`link <https://gerrit.onap.org/r/admin/repos/integration/simulators/pnf-simulator>`__ integration/simulators/ran-simulator;RAN simulator;`link <https://gerrit.onap.org/r/admin/repos/integration/simulators/ran-simulator>`__ +**integration/terraform**;**Terraform based alternative infrastructure installation**;`link <https://gerrit.onap.org/r/admin/repos/integration/terraform>`__ +**integration/terragrunt**;**Compagnon repository of terraform**;`link <https://gerrit.onap.org/r/admin/repos/integration/terragrunt>`__ integration/usecases/bbs;BBS use case introduced in Dublin and extracted from global repository in frankfurt;`link <https://gerrit.onap.org/r/admin/repos/integration/usecases/bbs>`__ integration/usecases/mdons;MDONS use case introduced in Frankfurt;`link <https://gerrit.onap.org/r/admin/repos/integration/usecases/mdons>`__ -testsuite;repository hosting the robot test suites;`link <https://gerrit.onap.org/r/admin/repos/testsuite>`__ -testsuite/heatbridge;python utils to manage the heatbridge function to enrich cloud information to AAI (deprecated);`link <https://gerrit.onap.org/r/admin/repos/testsuite/heatbridge>`__ -testsuite/oom;Helm chart for robot pod (to be deprecated in Guilin and moved back to OOM);`link <https://gerrit.onap.org/r/admin/repos/testsuite/oom>`__ -testsuite/python-testing-utils;Python and robot util libraries used for robot tests;`link <https://gerrit.onap.org/r/admin/repos/testsuite/python-testing-utils>`__ -demo;Historical repository to host use case artifacts (heat templates, json files,..);`link <https://gerrit.onap.org/r/admin/repos/demo>`__ -oparent;Java dependencies for JAVA projects;`link <https://gerrit.onap.org/r/admin/repos/oparent>`__ +integration/xtesting;Repository in charge to build th xtesting dockers used in CI/CD chains;`link <https://gerrit.onap.org/r/admin/repos/integration/xtesting>`__ diff --git a/docs/files/csv/repo-oparent.csv b/docs/files/csv/repo-oparent.csv new file mode 100644 index 000000000..40e4a3074 --- /dev/null +++ b/docs/files/csv/repo-oparent.csv @@ -0,0 +1,2 @@ +Repository;Description;Link +oparent;Java dependencies for JAVA projects;`link <https://gerrit.onap.org/r/admin/repos/oparent>`__ diff --git a/docs/files/csv/repo-testsuite.csv b/docs/files/csv/repo-testsuite.csv new file mode 100644 index 000000000..770428b0b --- /dev/null +++ b/docs/files/csv/repo-testsuite.csv @@ -0,0 +1,8 @@ +Repository;Description;Link +testsuite;repository hosting the robot test suites;`link <https://gerrit.onap.org/r/admin/repos/testsuite>`__ +**testsuite/cds**;**Repository hosting (standalone) CDS test suites shared by Bell Canada team, not yet integrated in CI/CD**;`link <https://gerrit.onap.org/r/admin/repos/testsuite/cds>`__ +testsuite/heatbridge;python utils to manage the heatbridge function to enrich cloud information to AAI (deprecated);`link <https://gerrit.onap.org/r/admin/repos/testsuite/heatbridge>`__ +testsuite/oom;Helm chart for robot pod (to be deprecated in Honolulu and moved back to OOM);`link <https://gerrit.onap.org/r/admin/repos/testsuite/oom>`__ +testsuite/python-testing-utils;Python and robot util libraries used for robot tests;`link <https://gerrit.onap.org/r/admin/repos/testsuite/python-testing-utils>`__ +**testsuite/pythonsdk-tests**;**Repository hosting the test scenarios leveraging python-onapsdk for end to end smoke tests**;`link <https://gerrit.onap.org/r/admin/repos/testsuite/pythonsdk-tests>`__ +**testsuite/robot-utils**;**Repository aiming to provide a robot wrapper for python-onapsdk**;`link <https://gerrit.onap.org/r/admin/repos/testsuite/robot-utils>`__ diff --git a/docs/files/csv/tests-healthcheck.csv b/docs/files/csv/tests-healthcheck.csv new file mode 100644 index 000000000..0df88a94b --- /dev/null +++ b/docs/files/csv/tests-healthcheck.csv @@ -0,0 +1,5 @@ +Tests;Description;Code;Comments +core;Robot healthcheck tests of the core components (AA&I, DMAAP, Portal, SDC, SDNC, SO);`robot tests <https://git.onap.org/testsuite/tree/robot/testsuites/health-check.robot>`__;`robotframework <https://robotframework.org/>`__ +full;Robot healthcheck tests for all the components;`robot tests <https://git.onap.org/testsuite/tree/robot/testsuites/health-check.robot>`__;`robotframework <https://robotframework.org/>`__ +healthdist;Check the onboarding and distribution of the vFW;`robot tests <https://git.onap.org/testsuite/tree/robot/testsuites/health-check.robot>`__;`robotframework <https://robotframework.org/>`__ +postinstall;Check dmaap and AA&I Design model DB tests;`robot tests <https://git.onap.org/testsuite/tree/robot/testsuites/post-install-tests.robot>`__;`robotframework <https://robotframework.org/>`__ diff --git a/docs/files/csv/tests-infrastructure-healthcheck.csv b/docs/files/csv/tests-infrastructure-healthcheck.csv new file mode 100644 index 000000000..43e4b139e --- /dev/null +++ b/docs/files/csv/tests-infrastructure-healthcheck.csv @@ -0,0 +1,4 @@ +Tests;Description;Code;Comments +onap-helm;Verify Helm chart status;`python module <https://gitlab.com/Orange-OpenSource/lfn/tools/helm-onap-status>`__; +onap-k8s;Check common resources of the ONAP Kubernetes namespace ;`python module <https://gitlab.com/Orange-OpenSource/lfn/tools/kubernetes-status>`__;kubernetes python library +nodeport_check_certs;This test list the nodeports and tries to get SSL information to evaluate the validity of the certificates (expiration and issuer) used on the nodeports;`python module <https://git.onap.org/integration/tree/test/security/check_certificates>`__;pyopenssl, kubernetes python libraries diff --git a/docs/files/csv/tests-security.csv b/docs/files/csv/tests-security.csv new file mode 100644 index 000000000..07e05d0ba --- /dev/null +++ b/docs/files/csv/tests-security.csv @@ -0,0 +1,9 @@ +Tests;Description;Code;Comments +root_pods;check that pods are nor using root user or started as root; `bash script <https://git.onap.org/integration/xtesting/tree/security/scripts/check_security_root.sh>`__; kubectl +unlimitted_pods;check that limits are set for pods;`bash script <https://git.onap.org/integration/xtesting/tree/security/scripts/check_unlimitted_pods.sh>`__; kubectl +cis_kubernetes;perform the k8s cis test suite (upstream src aquasecurity);`bash script <https://git.onap.org/integration/xtesting/tree/security/scripts/check_cis_kubernetes.sh>`__;`kube-bench <https://github.com/aquasecurity/kube-bench>`__ +nonssl_endpoints;check that all public HTTP endpoints exposed in ONAP cluster use SSL tunnels;`Go script <https://git.onap.org/integration/plain/test/security/sslendpoints/main.go>`__;kubetl, nmap +http_public_endpoints;check that there is no public http endpoints exposed in ONAP cluster;`bash script <https://git.onap.org/integration/plain/test/security/check_for_nonssl_endpoints.sh>`__;kubectl,nmap +jdpw_ports;check that there are no internal java ports;`bash script <https://git.onap.org/integration/plain/test/security/check_for_jdwp.sh>`__;kubectl, procfs +kube_hunter;security suite to search k8s vulnerabilities (upstream src aquasecurity);`kube-Hunter <https://github.com/aquasecurity/kube-hunter>`__; `kube-Hunter <https://github.com/aquasecurity/kube-hunter>`__ +versions;check that Java and Python are available only in versions recommended by SECCOM. This test is long and run only in Weekly CI chains;`python module <https://git.onap.org/integration/tree/test/security/check_versions>`__;cerberus, kubernetes python lib, diff --git a/docs/files/csv/tests-smoke.csv b/docs/files/csv/tests-smoke.csv new file mode 100644 index 000000000..bff73f2b3 --- /dev/null +++ b/docs/files/csv/tests-smoke.csv @@ -0,0 +1,9 @@ +Tests;Description;Code;Comments +basic_vm;Onboard, distribute and instantiate an Openstack VM using à la carte BPMN, replaced the former basic_vm test;`code <https://git.onap.org/testsuite/pythonsdk-tests/tree/src/onaptests/scenario/basic_vm.py>`__;`onap_pythonsdk <https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk>`__, `pythonsdk-tests <https://git.onap.org/testsuite/pythonsdk-tests/tree/>`__ +basic_network (new);Onboard, distribute and instantiate a Neutron network;`code <https://git.onap.org/testsuite/pythonsdk-tests/tree/src/onaptests/scenario/basic_network.py>`__;`onap_pythonsdk <https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk>`__, `pythonsdk-tests <https://git.onap.org/testsuite/pythonsdk-tests/tree/>`__ +basic_cnf;Onboard (new), distribute and instantiate a Kubernetes pods;`code <https://git.onap.org/testsuite/pythonsdk-tests/tree/src/onaptests/scenario/basic_cnf.py>`__;`onap_pythonsdk <https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk>`__, `pythonsdk-tests <https://git.onap.org/testsuite/pythonsdk-tests/tree/>`__ +cmpv2 (new);CMPv2 Usecase functionality;`code <https://git.onap.org/testsuite/tree/robot/testsuites/cmpv2.robot>`__;`robotframework <https://robotframework.org/>`__ +ves-collector (new);Suite for checking handling events by VES Collector;`code <https://git.onap.org/testsuite/tree/robot/testsuites/ves.robot>`__;`robotframework <https://robotframework.org/>`__ +hv-ves;HV-VES 'Sunny Scenario' Robot Framework test - message is sent to the collector and Kafka topic is checked if the message has been published. Content is decoded and checked.;`code <https://git.onap.org/testsuite/tree/robot/testsuites/hvves-ci.robot>`__;`robotframework <https://robotframework.org/>`__ +5gbulkpm;5G Bulk PM Usecase functionality;`code <https://git.onap.org/testsuite/tree/robot/testsuites/usecases/5gbulkpm.robot>`__;`robotframework <https://robotframework.org/>`__ +pnf-registrate;Executes the PNF registration test cases including setup and teardown;`code <https://git.onap.org/testsuite/tree/robot/testsuites/pnf-registration.robot>`__;`robotframework <https://robotframework.org/>`__ diff --git a/docs/files/csv/usecases-deprecated.csv b/docs/files/csv/usecases-deprecated.csv new file mode 100644 index 000000000..5484b47a2 --- /dev/null +++ b/docs/files/csv/usecases-deprecated.csv @@ -0,0 +1,15 @@ +Use Case,Link,Last Valid Version, Comments +vCPE Use Case,:ref:`link <docs_vcpe>` ,El Alto, No resources to test on Frankfurt +vIPsec with HPA Use Case ,:ref:`link<docs_vipsec_hpa>` ,El Alto, No resources to test on Frankfurt +Change Management Schedule Optimization,:ref:`link<docs_CM_schedule_optimizer>` ,El Alto, No resources to test on Frankfurt +Change Management Flexible Designer and Orchestrator ,:ref:`link<docs_CM_flexible_designer_orchestrator>` ,El Alto, No resources to test on Frankfurt +vFirewall with closed loop,:ref:`official doc <docs_vfw>` , Frankfurt, No resources to test on Guilin but it shall still work +vFirewall/vDNS with HPA ,:ref:`official doc <docs_vfw_hpa>`,Frankfurt, No resources to test on Guilin +Scale Out,:ref:`official doc <docs_scaleout>`,Frankfurt, No resources to test on Guilin +BBS (Broadband Service),:ref:`official doc <docs_bbs>`,Frankfurt, No resources to test on Guilin +vFirewall CNF with multicloud k8s plugin,:ref:`official doc <docs_vfw_edgex_multicloud_k8s>`, Frankfurt,No resources to test on Guilin +EdgeXFoundry CNF with multicloud k8s plugin,:ref:`official doc <docs_vfw_edgex_multicloud_k8s>`,Frankfurt, No resources to test on Guilin +vCPE with Tosca,:ref:`official doc <docs_vcpe_tosca_local>`, Frankfurt, No resources to test on Guilin +E2E Automation vLB with CDS,`wiki page <https://wiki.onap.org/pages/viewpage.action?pageId=71838891>`__, Frankfurt, No resources to test on Guilin +vFirewall In-Place Software Upgrade with Traffic Distribution,:ref:`official doc <docs_vfw_traffic>`, Frankfurt, No resources to test on Guilin +vFirewall CNF With CDS,:ref:`official doc <docs_vFW_CNF_CDS>`, Frankfurt, No resources to test on Guilin diff --git a/docs/functional-requirements-5g.csv b/docs/files/csv/usecases-functional-requirements-5g.csv.old index 286fd3aba..286fd3aba 100644 --- a/docs/functional-requirements-5g.csv +++ b/docs/files/csv/usecases-functional-requirements-5g.csv.old diff --git a/docs/files/csv/usecases-functional-requirements.csv b/docs/files/csv/usecases-functional-requirements.csv new file mode 100644 index 000000000..9679d404e --- /dev/null +++ b/docs/files/csv/usecases-functional-requirements.csv @@ -0,0 +1,21 @@ +Issue key;Summary;Priority;Status;Created +REQ-324;Support xNF Software Upgrade in association to schema updates;High;In Review;08/May/20 8:46 PM +REQ-335;Support for Test Topology Auto Design （NFV Testing Automatic Platform）;High;To Do;27/May/20 3:28 AM +REQ-336;Support for Test Environment Auto Deploy（NFV Testing Automatic Platform）;High;To Do;27/May/20 3:37 AM +REQ-337;Support for Test Task Auto Execution （NFV Testing Automatic Platform）;High;To Do;27/May/20 3:37 AM +REQ-338;Support for Test Result Auto Analysis & Certification （NFV Testing Automatic Platform）;High;To Do;27/May/20 3:38 AM +REQ-352;Extend ORAN A1 Adapter and add A1 Policy Management;High;In Review;27/May/20 2:21 PM +REQ-339;Container Network Function Test Platform (CNTP): OVP 2.0 support and Enable Network Service testing;High;To Do;27/May/20 4:06 AM +REQ-388;Interactive documentation aligned with a deployment;High;To Do;02/Jun/20 7:05 AM +REQ-329;Guilin-R7 - Support for Intent-based Network;High;To Do;20/May/20 2:29 AM +REQ-334;ETSI-Alignment for Guilin and Honolulu;Medium;In Review;27/May/20 2:08 AM +REQ-384;Define and prove the concept of model driven in the context of CPS;Medium;To Do;28/May/20 1:35 PM +REQ-353;"xNF License Management, Guilin R7";Medium;To Do;27/May/20 2:38 PM +REQ-320;5G Service Modeling in R7;Medium;To Do;07/May/20 6:01 PM +REQ-327;ONAP/3GPP & O-RAN Alignment-Standards Defined Notifications over VES (Guilin);Medium;In Progress;19/May/20 8:43 PM +REQ-322;Configuration & Persistency Service in R7;Medium;To Do;07/May/20 6:15 PM +REQ-321;CMPv2 Enhancements for R7;Medium;In Progress;07/May/20 6:06 PM +REQ-318;PNF Plug & Play in R7;Medium;To Do;07/May/20 5:36 PM +REQ-319;PNF PreOnboarding in R7;Medium;To Do;07/May/20 5:50 PM +REQ-417;Epic task for all logging fix tasks;Medium;To Do;10/Aug/20 12:59 PM +REQ-326;Self Serve Control Loops;Medium;To Do;11/May/20 4:18 PM diff --git a/docs/functional-requirements.csv b/docs/files/csv/usecases-functional-requirements.csv.old index e06df187b..cf5117e8e 100644 --- a/docs/functional-requirements.csv +++ b/docs/files/csv/usecases-functional-requirements.csv.old @@ -7,6 +7,6 @@ VSP Compliance and Validation Check within SDC,`wiki page <https://wiki.onap.org Enable PNF software version at onboarding,`wiki page <https://jira.onap.org/browse/REQ-88?src=confmacro>`__,A.Schmid xNF communication security enhancements, `wiki page <https://wiki.onap.org/display/DW/xNF+communication+security+enhancements+-+Tests+Description+and+Status>`__,M.Przybysz ETSI Alignment SO plugin to support SOL003 to connect to an external VNFM,`wiki page <https://wiki.onap.org/display/DW/ETSI+Alignment+Support>`__,F.Oliveira Byung-Woo Jun -Integration of CDS as an Actor, `wiki page <https://docs.onap.org/projects/onap-ccsdk-cds/en/latest/ui/designer.html>`__, B.Sakoto R.K.Verma Y.Malakov +Integration of CDS as an Actor, `official doc <https://docs.onap.org/projects/onap-ccsdk-cds/en/latest/ui/designer.html>`__, B.Sakoto R.K.Verma Y.Malakov 3rd Party Operational Domain Manager, `wiki page <https://wiki.onap.org/display/DW/Third-party+Operational+Domain+Manager>`__, D.Patel Configuration & persistency, `wiki page <https://wiki.onap.org/pages/viewpage.action?pageId=64003184>`__,Reshmasree c Swaminathan S diff --git a/docs/files/csv/usecases-non-functional-requirements.csv b/docs/files/csv/usecases-non-functional-requirements.csv new file mode 100644 index 000000000..58574194d --- /dev/null +++ b/docs/files/csv/usecases-non-functional-requirements.csv @@ -0,0 +1,28 @@ +Issue key;Issue id;Summary;Priority;Status;Created +REQ-402;53055;PoC - TOSCA Defined Control Loop ;High;To Do;01/Jul/20 9:46 AM +REQ-350;52019;Each ONAP project shall improve its CII Badging score by improving input validation and documenting it in their CII Badging site. ;High;To Do;27/May/20 1:56 PM +REQ-358;52034;No root (superuser) access to database from application container;High;To Do;27/May/20 4:09 PM +REQ-376;52056;Flow management must be activated for ONAP.;High;To Do;27/May/20 5:16 PM +REQ-379;52059;ONAP projects must use only approved and verified base images for their containers;High;To Do;27/May/20 6:41 PM +REQ-380;52060;ONAP container repository (nexus) must not contain upstream docker images;High;To Do;27/May/20 6:44 PM +REQ-360;52036;Application config should be fully prepared before starting the application container;High;To Do;27/May/20 4:11 PM +REQ-361;52037;Continue hardcoded passwords removal;High;To Do;27/May/20 4:12 PM +REQ-362;52038;All containers must run as non-root user;High;To Do;27/May/20 4:12 PM +REQ-365;52041;Containers must have no more than one main process;High;To Do;27/May/20 4:14 PM +REQ-366;52042;Containers must crash properly when a failure occurs;High;To Do;27/May/20 4:14 PM +REQ-369;52045;Replace nfs share with storage class as a default deployment option;High;To Do;27/May/20 4:15 PM +REQ-373;52053;ONAP must complete update of the Python language (from 2.7 -> 3.8);High;In Progress;27/May/20 5:07 PM +REQ-374;52054;ONAP shall use STDOUT for logs collection;High;To Do;27/May/20 5:09 PM +REQ-382;52069;"Support Pylog repository used by VF-C, Modeling, MultiCloud and OOF";High;To Do;27/May/20 11:22 PM +REQ-385;52108;IPv4/IPv6 dual stack support in ONAP (Guilin);High;In Progress;28/May/20 2:49 PM +REQ-351;52020;ONAP must complete update of the java language (from v8 -> v11) ;High;In Progress;27/May/20 1:59 PM +REQ-359;52035;Container rootfs must be mounted readOnly;High;To Do;27/May/20 4:11 PM +REQ-364;52040;Replace NodePorts with ingress controller as a default deployment option;High;To Do;27/May/20 4:13 PM +REQ-377;52057;ONAP must implement IAM solutions.;High;In Progress;27/May/20 5:18 PM +REQ-363;52039;ONAP components should be able to run without AAF and MSB;High;To Do;27/May/20 4:13 PM +REQ-370;52046;Components may use HTTP as server and client;High;To Do;27/May/20 4:16 PM +REQ-348;52017;Requirements about ONAP installation result for Guilin release;High;To Do;27/May/20 1:48 PM +REQ-368;52044;Service mesh PoC;High;To Do;27/May/20 4:15 PM +REQ-340;51957;ONAP to support Multi - tenancy ;Medium;To Do;27/May/20 8:08 AM +REQ-386;52109;Apply common Swagger style and documentation generation tools to create robust ONAP API documentation;Medium;To Do;28/May/20 3:01 PM +REQ-416;54263;Harbor POC with LNF Team;Medium;To Do;31/Jul/20 7:57 AM diff --git a/docs/files/csv/usecases.csv b/docs/files/csv/usecases.csv new file mode 100644 index 000000000..3f4f2adff --- /dev/null +++ b/docs/files/csv/usecases.csv @@ -0,0 +1,5 @@ +Ref;Summary;Link;Contacts +REQ-342;E2E Network Slicing;:ref:`official doc <docs_E2E_network_slicing>`;Swaminathan S +REQ-345;5G OOF SON;:ref:`official doc <docs_5G_oof_pci>`;Reshmasree c +REQ-347;CCVPN-Transport Slicing;:ref:`official doc <docs_ccvpn>`;X.Miao,G.Agrawal,Shashikanth VH +REQ-325;MDONS;:ref:`official doc <docs_CCVPN>`;X.Miao diff --git a/docs/files/integration-project.png b/docs/files/integration-project.png Binary files differnew file mode 100644 index 000000000..d590c842f --- /dev/null +++ b/docs/files/integration-project.png diff --git a/docs/files/tests/test-basic-cnf.png b/docs/files/tests/test-basic-cnf.png Binary files differnew file mode 100644 index 000000000..87701587e --- /dev/null +++ b/docs/files/tests/test-basic-cnf.png diff --git a/docs/files/tests/test-certif.png b/docs/files/tests/test-certif.png Binary files differnew file mode 100644 index 000000000..69d6264c2 --- /dev/null +++ b/docs/files/tests/test-certif.png diff --git a/docs/files/tests/test-dashboard.png b/docs/files/tests/test-dashboard.png Binary files differnew file mode 100644 index 000000000..afd4eee49 --- /dev/null +++ b/docs/files/tests/test-dashboard.png diff --git a/docs/files/tests/test-onap-helm.png b/docs/files/tests/test-onap-helm.png Binary files differnew file mode 100644 index 000000000..e5f5b5366 --- /dev/null +++ b/docs/files/tests/test-onap-helm.png diff --git a/docs/files/tests/test-onap-k8s.png b/docs/files/tests/test-onap-k8s.png Binary files differnew file mode 100644 index 000000000..69693f7f5 --- /dev/null +++ b/docs/files/tests/test-onap-k8s.png diff --git a/docs/files/windriver/windriver_CPU.png b/docs/files/windriver/windriver_CPU.png Binary files differnew file mode 100644 index 000000000..abf334b81 --- /dev/null +++ b/docs/files/windriver/windriver_CPU.png diff --git a/docs/files/windriver/windriver_RAM.png b/docs/files/windriver/windriver_RAM.png Binary files differnew file mode 100644 index 000000000..1333f01cb --- /dev/null +++ b/docs/files/windriver/windriver_RAM.png diff --git a/docs/files/windriver/windriver_disks.png b/docs/files/windriver/windriver_disks.png Binary files differnew file mode 100644 index 000000000..1f7fc5265 --- /dev/null +++ b/docs/files/windriver/windriver_disks.png diff --git a/docs/files/windriver/windriver_servers.png b/docs/files/windriver/windriver_servers.png Binary files differnew file mode 100644 index 000000000..39671782f --- /dev/null +++ b/docs/files/windriver/windriver_servers.png diff --git a/docs/files/windriver/windrivers_servers2.png b/docs/files/windriver/windrivers_servers2.png Binary files differnew file mode 100644 index 000000000..90d63c35a --- /dev/null +++ b/docs/files/windriver/windrivers_servers2.png diff --git a/docs/index.rst b/docs/index.rst index 0096406d6..395213ef5 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,62 +1,16 @@ -.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. SPDX-License-Identifier: CC-BY-4.0 .. _master_index: INTEGRATION =========== -The Integration project is in charge of: +.. toctree:: + :maxdepth: 1 -- Providing testing environment and support for the release use cases -- Executing project-specific Continuous System Integration Testing (CSIT) -- Managing full ONAP CI chains (daily master, staging, stable) to ensure the - stability of the integration -- Developing and performing tests within full ONAP context (healthcheck, End to - End, performance, robustness...) -- Validating the ONAP release - -For each release, the integration team provides the following artifacts: - -- Test suites and tools to check the various ONAP components -- Use-case documentation and artifacts -- a testsuite docker included in ONAP cluster to execute the tests -- baseline JAVA and Python images, as well as a docker managing Java libraries -- Configuration files and Heat templates to simplify the creation of the virtual - resources needed for the ONAP deployment and the use cases. - -The integration team manages several official ONAP repositories: - -.. csv-table:: Integration repositories table - :file: integration-repositories.csv - :widths: 30,50,20 - :delim: ; - :header-rows: 1 - -Since Frankfurt, we tried to create smaller repositories for the use cases and -the simulators. It shall help us to maintain the use cases and the simulators. -It shall also help to leverage and adopt existing simulators rather than -systematically re-inventing the wheel. - -The main wiki page of the Integration team can be found in -https://wiki.onap.org/display/DW/Integration+Project, you will find different -menus, Q&As, and release pages. - -Environment Installation ------------------------- - -In addition of the official OOM scripts, Integration provides some guidelines to -install your OpenStack configuration thanks to a heat template. -See :ref:`Integration heat guideline <integration-installation>` for details. - -Integration CI --------------- - -Integration project is responsible of the Continuous Integration Chains. -A guide has been created to setup your own CI chain. -See :ref:`Integration CI guideline <integration-ci>` for details. - -Stability Testing ------------------ - -Ensuring the stability of ONAP is one of the missions of the Integration team. -CI chains and stability tests are performed to help stabilising the release. -See :ref:`Integration stability tests <integration-s3p>` for details. + integration-missions.rst + release-notes.rst + integration-repositories.rst + integration-resources.rst + integration-labs.rst + integration-CICD.rst + integration-tests.rst diff --git a/docs/integration-CICD.rst b/docs/integration-CICD.rst new file mode 100644 index 000000000..298b292f4 --- /dev/null +++ b/docs/integration-CICD.rst @@ -0,0 +1,81 @@ +.. This work is licensed under a + Creative Commons Attribution 4.0 International License. +.. integration-CICD: + +.. integration_main-doc: + +CI/CD +===== + +.. important:: + Integration team deals with 2 different CI/CD systems. + + - jenkins CI/CD, CI managed by LF IT and CD by Integration team + - gitlab-ci managed by Integration and OOM team + +Continuous Integration +---------------------- + +The CI part provides the following features: + +- Repository Verification (format of the INFO.yaml) +- Patchset verification thank to json/yaml/python/go/rst/md linters. These jenkins + verification jobs are hosted in the ci-management repository. They can vote + +1/-1 on patchset submission. Integration team systematically enables linters + on any new repository +- Docker build: Integration team builds testsuite dockers and xtesting dockers. + These dockers are built then pushed to Nexus through a jjb also hosted in the + ci-management repository. + +The different verification chains are defined in https://jenkins.onap.org/: + +- CSIT: https://jenkins.onap.org/view/CSIT/ +- testsuite: https://jenkins.onap.org/view/testsuite/ +- integration: https://jenkins.onap.org/view/integration/ +- integration-terragrunt: https://jenkins.onap.org/view/integration-terragrunt/ +- testsuite-robot-utils: https://jenkins.onap.org/view/testsuite-robot-utils/ + +The jenkins jobs (jjb) are hosted in https://git.onap.org/ci-management/. + +Continuous Deployment +--------------------- + +There are 2 Continuous Deployment architectures. + +Jenkins CD on Windriver/Intel lab +.................................. + +The CD part on windriver/Intel is based on jenkins. + +It is based on a standalone VM hosting a jenkins server. +The credentials of this VM as well as the jenkins server have been provided to +integration committers. + +Several jobs can be triggered from this jenkins interface. +Historically several chains were run daily (staging/release) but due to +performance issues, they have all been stopped. +Only SB-00 has been kept for use case support. +The jenkins interface was however used to launch the installation of SB-00. + +This jenkins script is leveraging resources available in OOM and integration +repositories. + +It was planned to replaced this CD by a gitlab runner based CD to unify the CD +management. But due to performance issue in the DC it was not possible to +finalize the operation in Guilin. + +Gitlab CD +......... + +This CD is leveraging public gitlab-ci mechanism and used to deploy several ONAP +labs: + +- Daily Master: daily run using OOM Master +- Daily Frankfurt: daily run using the last stable version during Guilin Release + processing +- Daily Guilin: daily run setup at RC0 (candidate dockers available for integration) +- Weekly Master: run once a week with longer tests +- Gating: run on OOM, clamp or SO patchset submission. It means a full ONAP + deployment on demand based on new patchset declared in gerrit. + +See :ref:`Integration CI guideline <integration-ci>` for details. diff --git a/docs/integration-labs.rst b/docs/integration-labs.rst new file mode 100644 index 000000000..f00a5324c --- /dev/null +++ b/docs/integration-labs.rst @@ -0,0 +1,76 @@ +.. This work is licensed under a + Creative Commons Attribution 4.0 International License. +.. integration-labs: + +.. integration_main-doc: + +ONAP Integration Labs +===================== + +.. important:: + The Integration team deals with several community labs: + + - The Windriver/Intel lab + - The Azure staging lab + - The Continuous Deployment (CD) Labs + - The Orange openlab + +Additionnaly integration contributors may deal with their own lab pushing results +in the integration portal (See DT http://testresults.opnfv.org/onap-integration/dt/dt.html) + +Windriver/Intel lab +------------------- + +The Historical Community Lab +............................ + +This lab is the historical lab of ONAP integration team based on Openstack Ocata. +During Guilin, it has been audited and performance issues have been reported +(RAM and CPU). +A reinstallation has to be planned. + +The figure hereafter shows all the ONAP project consuming Windriver/Intel lab +resources (April 2020). + +.. figure:: files/windriver/windriver_servers.png + :align: center + +In order to avoid disturbing the projects, the reinstallation has been postponed +after Guilin. +A huge cleanup has been done in order to save the resources. +The historical CI/CD chains based on a stand alone jenkins VM hosted in Windriver +have been stopped.For guilin only SB-00 has been kept and re-installed for the +use case support. + +If you want to use this lab, you need a VPN access. The procedure is described in +the `wiki <https://wiki.onap.org/pages/viewpage.action?pageId=29787070>`__. + +Environment Installation Scripts +................................ + +In addition of the official OOM scripts, Integration used to provide some +exwtra scripts/guidelines to install your OpenStack infrastructure thanks to a +heat template. See :ref:`Integration heat guideline <integration-installation>` +for details. This scripts were used mainly in windriver labs but are not actively +maintained. + +.. caution: + The official reference for installation is the OOM documentation. + +Azure staging lab +----------------- + +An additional Azure staging lab has been created for Guilin. It is installed as +any daily/weekly/gating labs (see CI/CD sections). +Contact the Integration team to get an access. + +Orange Openlab +-------------- + +This lab is a community use lab. It is always provided the last stable version, +so the frankfurt release during Guilin release time. +Please note that such labs do not provide admin rights and is shared with all +the users. It can be used to discover ONAP. + +See `Orange Openlab access procedure <https://wiki.onap.org/display/DW/Orange+OpenLab>`__ +for details. diff --git a/docs/integration-missions.rst b/docs/integration-missions.rst new file mode 100644 index 000000000..331c92109 --- /dev/null +++ b/docs/integration-missions.rst @@ -0,0 +1,38 @@ +.. This work is licensed under a + Creative Commons Attribution 4.0 International License. +.. _integration-missions: + +Integration missions +==================== + +.. important:: + The Integration project is in charge of: + + - Providing testing environment + - Supporting the use case teams + - Managing ONAP CI/CD chains + - Developing tests + - Providing baseline images + - Validating the ONAP releases + +The different activities may be summarized as follows (proportions are indicative): + +.. figure:: files/integration-project.png + + +For each release, the integration team provides the following artifacts: + +- A daily CI chain corresponding to the release +- Staging labs to perform the pairwise testing (when not automated) and support + the use case teams +- Baseline JAVA and Python images +- oparent library to manage Java dependencies +- Test suites and tools to check the various ONAP components +- Use-case documentation and artifacts +- A testsuite docker included in the ONAP cluster to execute the robot based tests +- Configuration files (scripts, Heat templates, csa files) to help installing + and testing ONAP +- Wiki release follow-up tables (blocking points, docker versions,...) + +Please see the `integration wiki page <https://wiki.onap.org/display/DW/Integration+Project>`__ +for details. diff --git a/docs/integration-repositories.rst b/docs/integration-repositories.rst new file mode 100644 index 000000000..17b59710b --- /dev/null +++ b/docs/integration-repositories.rst @@ -0,0 +1,142 @@ +.. This work is licensed under a + Creative Commons Attribution 4.0 International License. +.. integration-repositories: + +Integration repositories +======================== + +.. important:: + The Integration project deals with lots of code repositories. + + +Most of the repositories are internal ONAP repositories. + +.. code-block:: bash + + ├── csit + ├── demo + ├── integration + │ ├── docker + │ │ ├── onap-java11 + │ │ └── onap-python + │ ├── integration + │ ├── seccom + │ ├── simulators + │ │ ├── 5G-core-nf-simulator + │ │ ├── dc-simulator + │ │ ├── pnf-simulator + │ │ └── ran-simulator + │ ├── terraform + │ ├── terragrunt + │ ├── usecases + │ │ ├── bbs + │ │ └── mdons + │ └── xtesting + ├── oparent + └── testsuite + ├── cds + ├── heatbridge + ├── oom + ├── pythonsdk-tests + ├── python-testing-utils + ├── robot-utils + └── testsuite + +Please note that integration and teststuite are repositories AND groups hosting +several sub-repositories. + +Integration +----------- + +The integration repository is the historical repository. +As a consequence it includes several elements in the same reppository: + +- Deployment scripts (deployment directory) +- Tests: the first non robot tests (security, vCPE,..) +- Simulators/emulators (test/mocks) +- Integration and use cases documentation (docs) +- Tools (bootstrap, S3Ptools) + +Since Frankfurt version, we created more smaller repositories especially for the use +cases and the simulators. +It shall help improving the maintenance of the different elements. +It shall also help identifying, leveraging and adopting existing simulators +rather than systematically re-inventing the wheel. + +.. attention:: + The **new integration sub-repositories** introduced for Guilin are: + + - 5G-core-nf-simulator + - terraform + - terragrunt + +.. csv-table:: Integration Repositories + :file: ./files/csv/repo-integration.csv + :widths: 30,50,20 + :delim: ; + :header-rows: 1 + +Testsuite +--------- + +The testsuite repository and it sub repositories deal excelusively with tests. + +The testsuite repository includes all the robotframework scripts. +The robot pod that can be installed as part of the ONAP cluster is build from +this repository. + +Several tooling repositories are associated with the robot tests (heatbridge, +robot-python-testing-utils). + +.. attention:: + For Guilin, 3 new sub repositories have been created: + + - testsuite/cds + - pythonsdk-test + - robot-utils + +.. csv-table:: Testsuite Repositories + :file: ./files/csv/repo-testsuite.csv + :widths: 30,50,20 + :delim: ; + :header-rows: 1 + +Demo +---- + +In this repository you will find any artifacts needed for demo, PoC and use cases +if they do not have their own repository (mainly old use cases). + +.. csv-table:: Demo Repository + :file: ./files/csv/repo-demo.csv + :widths: 30,50,20 + :delim: ; + :header-rows: 1 + +Oparent +------- + +.. csv-table:: Oparent Repository + :file: ./files/csv/repo-oparent.csv + :widths: 30,50,20 + :delim: ; + :header-rows: 1 + + +External repositories +--------------------- + +Additionnaly, the Integration team also deals with external gitlab.com +repositories. + +.. csv-table:: Integration external repositories table + :file: ./files/csv/repo-integration-external.csv + :widths: 30,50,20 + :delim: ; + :header-rows: 1 + +The python-onapsdk has been developped out of ONAP as gitlab provided more +enhanced built-in features for this kind of developments. + +The xtesting-onap repository is also hosted in gitlab.com as the CD part of +Integration work is based on public gitlab-ci chains. diff --git a/docs/integration-resources.rst b/docs/integration-resources.rst new file mode 100644 index 000000000..0ae3e7013 --- /dev/null +++ b/docs/integration-resources.rst @@ -0,0 +1,79 @@ +.. This work is licensed under a + Creative Commons Attribution 4.0 International License. +.. _integration-resources: + +.. integration_main-doc: + +Integration resources +===================== + +.. important:: + The Integration project maintains several community resources: + + - Public Portal: http://testresults.opnfv.org/onap-integration/ + - Test Database & API: http://testresults.opnfv.org/onap/api/v1/projects + - CI/CD logs & artifacts: https://logs.onap.org/onap-integration + - VNF demo artifacts:https://nexus.onap.org/content/repositories/releases/org/onap/demo/vnf/ + - Simulators + - Test frameworks (e.g. python-onapsdk) + +Integration portal +------------------ + +A portal is built to report the status of the different labs collaborating in +Integration, see http://testresults.opnfv.org/onap-integration/ + +.. figure:: files/CI/ONAP_CI_3.png + :align: center + :width: 6.5in + +The code of this web site is shared on a public gitlab project. + +Integration Test database +------------------------- + +The integration team shares a Test Result Database with the OPNFV project. All +the test results of the CD are automatically pushed in this database. +It is possible to retrieve the results through the Test API associated with this +test Database. + +The following information are available: + +- List of pods allowed to push results: http://testresults.opnfv.org/onap/api/v1/pods +- List of projects that declared test cases for CI/CD: http://testresults.opnfv.org/onap/api/v1/projects +- List of test cases per projects: + http://testresults.opnfv.org/onap/api/v1/projects/integration/cases + http://testresults.opnfv.org/onap/api/v1/projects/security/cases +- Results with lots of possible filter combinations: http://testresults.opnfv.org/onap/api/v1/results?last=3 + +It is possible to get results according to several criteria (version, case name, +lab, period, last, CI id,..) +See the `OPNFV test API documentation <https://wiki.opnfv.org/pages/viewpage.action?pageId=2926452>`__. + +Any company running ONAP Integration tests can be referenced to push their results +in this database. +This Database is hosted on a LF OPNFV server. Results are backuped daily. +Integration committers can have access to this server. + +VNF demo Artifacts +------------------ + +They are hosted in the demo repositories and published in +https://nexus.onap.org/content/repositories/releases/org/onap/demo/vnf/. + +Simulators +---------- + +There is a list of `available simulators <https://wiki.onap.org/pages/viewpage.action?pageId=81399210>`_. +Please consider any of them (they are open source and can be modified, +updated...) before creating a new one. + +Communication channels +---------------------- + +The main communication channel for real time support is the rocket chat channel +http://team.onap.eu. +You can also send a mail to onap-discuss AT lists.onap.org +with [ONAP] [Integration] prefix in the title. + +All the different links are reachable from the Integration portal. diff --git a/docs/integration-s3p.rst b/docs/integration-s3p.rst index 49c67850f..2e0f6bfce 100644 --- a/docs/integration-s3p.rst +++ b/docs/integration-s3p.rst @@ -17,188 +17,27 @@ Guilin, several requirements have been created to improve the S3P testing domain Stability ========= -ONAP stability was tested through a 72 hour test. -The intent of the 72 hour stability test is not to exhaustively test all -functions but to run a steady load against the system and look for issues like -memory leaks that cannot be found in the short duration install and functional -testing during the development cycle. - -Integration Stability Testing verifies that the ONAP platform remains fully -functional after running for an extended amounts of time. -This is done by repeated running tests against an ONAP instance for a period of -72 hours. - -:: - - **The 72 hour stability run result was PASS** - -The onboard and instantiate tests ran for over **115 hours** before environment -issues stopped the test. There were errors due to both tooling and environment -errors. - -The overall memory utilization only grew about **2%** on the work nodes despite -the environment issues. Interestingly the kubernetes ochestration node memory -grew more which could mean we are over driving the API's in some fashion. - -We did not limit other tenant activities in Windriver during this test run and -we saw the impact from things like the re-installation of SB00 in the tenant -and general network latency impacts that caused openstack to be slower to -instantiate. -For future stability runs we should go back to the process of shutting down -non-critical tenants in the test environment to free up host resources for -the test run (or other ways to prevent other testing from affecting the stability -run). - -The control loop tests were **100% successful** and the cycle time for the loop was -fairly consistent despite the environment issues. Future control loop stability -tests should consider doing more policy edit type activites and running more -control loop if host resources are available. The 10 second VES telemetry event -is quite aggressive so we are sending more load into the VES collector and TCA -engine during onset events than would be typical so adding additional loops -should factor that in. The jenkins jobs ran fairly well although the instantiate -Demo vFWCL took longer than usual and should be factored into future test planning. +TODO Methodology ~~~~~~~~~~~ -The Stability Test has two main components: - -- Running "ete stability72hr" Robot suite periodically. This test suite - verifies that ONAP can instantiate vDNS, vFWCL, and VVG. -- Set up vFW Closed Loop to remain running, then check periodically that the - closed loop functionality is still working. - -The integration-longevity tenant in Intel/Windriver environment was used for the -72 hour tests. - -The onap-ci job for "Project windriver-longevity-release-manual" was used for -the deployment with the OOM set to frankfurt and Integration branches set to -master. Integration master was used so we could catch the latest updates to -integration scripts and vnf heat templates. - -The jenkins job needs a couple of updates for each release: - -- Set the integration branch to 'origin/master' -- Modify the parameters to deploy.sh to specify "-i master" and "-o frankfurt" - to get integration master an oom frankfurt clones onto the nfs server. - -The path for robot logs on dockerdata-nfs changed in Frankfurt so the -/dev-robot/ becomes /dev/robot - -.. note:: - For Frankfurt release, the stability test has been executed on an - kubernetes infrastructure based on El Alto recommendations. The kubernetes - version was 1.15.3 (frankfurt 1.15.11) and the helm version was 2.14.2 - (frankfurt 2.16.6). However the ONAP dockers were updated to Frankfurt RC2 - candidate versions. The results are informative and can be compared with - previous campaigns. The stability tests used robot container image - **1.6.1-STAGING-20200519T201214Z**. Robot container was patched to use GRA_API - since VNF_API has been deprecated. - -Shakedown consists of creating some temporary tags for stability72hrvLB, -stability72hrvVG,stability72hrVFWCL to make sure each sub test ran successfully -(including cleanup) in the environment before the jenkins job started with the -higher level testsuite tag stability72hr that covers all three test types. - -Clean out the old buid jobs using a jenkins console script (manage jenkins) - -:: - - def jobName = "windriver-longevity-stability72hr"= - def job = Jenkins.instance.getItem(jobName) - job.getBuilds().each { it.delete() } - job.nextBuildNumber = 1 - job.save() - - -appc.properties updated to apply the fix for DMaaP message processing to call -http://localhost:8181 for the streams update. - -Results: 100% PASS -~~~~~~~~~~~~~~~~~~ -=================== ======== ========== ======== ========= ========= -Test Case Attempts Env Issues Failures Successes Pass Rate -=================== ======== ========== ======== ========= ========= -Stability 72 hours 77 19 0 58 100% -vFW Closed Loop 60 0 0 100 100% -**Total** 137 19 0 158 **100%** -=================== ======== ========== ======== ========= ========= - -Detailed results can be found at https://wiki.onap.org/display/DW/Frankfurt+Stability+Run+Notes - -.. note:: - - Overall results were good. All of the test failures were due to - issues with the unstable environment and tooling framework. - - JIRAs were created for readiness/liveness probe issues found while - testing under the unstable environment. Patches applied to oom and - testsuite during the testing helped reduce test failures due to - environment and tooling framework issues. - - The vFW Closed Loop test was very stable and self recovered from - environment issues. - -Resources overview -~~~~~~~~~~~~~~~~~~ -============ ====================== =========== ========== ========== -Date #1 CPU #1 RAM CPU* RAM** -============ ====================== =========== ========== ========== -May 20 18:45 dcae-tca-anaytics:511m appc:2901Mi 1649 36092 -May 21 12:33 dcae-tca-anaytics:664m appc:2901Mi 1605 38221 -May 22 09:35 dcae-tca-anaytics:425m appc:2837Mi 1459 38488 -May 23 11:01 cassandra-1:371m appc:2849Mi 1829 39431 -============ ====================== =========== ========== ========== - -.. note:: - - Results are given from the command "kubectl -n onap top pods | sort -rn -k 3 - | head -20" - - * sum of the top 20 CPU consumption - - ** sum of the top 20 RAM consumption CI results ========== -A daily Frankfurt CI chain has been created after RC0. +A daily Guilin CI chain has been created after RC0. The evolution of the full healthcheck test suite can be described as follows: |image1| -Full healthcheck testsuite verifies the status of each component. It is -composed of 47 tests. The success rate from the 9th to the 28th was never under -95%. -4 test categories were defined: - -- infrastructure healthcheck: test of ONAP kubernetes cluster and help chart status -- healthcheck tests: verification of the components in the target deployment - environment -- smoke tests: basic VM tests (including onboarding/distribution/instantiation), - and automated use cases (pnf-registrate, hvves, 5gbulkpm) -- security tests - -The security target (66% for Frankfurt) was reached after the RC1. A regression -due to the automation of the hvves use case (triggering the exposition of a -public port in HTTP) was fixed on the 28th of May. - -|image2| - -Orange Openlab -============== - -The Orange Openlab is a community lab targeting ONAP end user. It provides an -ONAP and cloud resources to discover ONAP. -A Frankfurt pre-RC0 version was installed beginning of May. The usual gating -testing suite was run daily in addition of the traffic generated by the lab -users. The VM instantiation has been working well without any reinstallation -over the **27** last days. Resilience ========== -The resilience test executed in El Alto was not realized in Frankfurt. .. |image1| image:: files/s3p/daily_frankfurt1.png :width: 6.5in - -.. |image2| image:: files/s3p/daily_frankfurt2.png - :width: 6.5in diff --git a/docs/integration-tests.rst b/docs/integration-tests.rst new file mode 100644 index 000000000..be3a04c7b --- /dev/null +++ b/docs/integration-tests.rst @@ -0,0 +1,149 @@ +.. This work is licensed under a + Creative Commons Attribution 4.0 International License. +.. integration-tests: + +Tests +===== + +.. important:: + Integration is in charged of several types of tests: + + - Use Cases: developped by use case teams, usually complex, demonstrating high value capabilities ofr ONAP. They may be partially automated and even + integrated in CD. + - CSIT Tests: functional tests created by the projects, partially hosted in CSIT repository + - Automatic Test Cases: these use cases are usually more simple and aims to validate that ONAP is working properly. + These tests have been developed to validate ONAP as a software solution. + In theory all the main function shall be covered by such tests in order to have more robust CI/CD and then avoid regressions. + These tests are usually developed and maintained by the integration team. + +We may also indicate that when the development of the test framework python-onapsk +follows standard developement quality rules and imposes the creation of +unit/functional/integration tests. +As an example python-onapsdk requires a unit test coverage of 98% before merging +a new feature, which is far above the project criteria in SonarCLoud today. + +Use Cases +--------- + +The use cases are described in :ref:`Verified Use cases <docs_usecases>`. + +CSIT Tests +---------- + +The CSIT tests are functional tests executed by the projects on mocked +environement to validate their components. +Historically it was hosted in a CSIT repository. + +Integration team invited the projects to bring back such tests bacdk to home +repository for 2 main reasons: + +- integration cannot be a bottleneck: +2/merge from integration needed for each + project +- most of the tests are abandonned and not maintained when hosted in a third party + repository leading to CI/CD time waste and misleading test reporting + +In Guilin a PoC to help the project to re-insource their functional tests have +been initiated. +See `CSIT wiki page <https://wiki.onap.org/display/DW/Maximizing+Benefits+of+CSIT+in+ONAP+Development>`__ +for details. + +Automatic Tests +--------------- + +These tests are run daily/weekly on each new gate (new patchset in OOM, clamp +or SO). They can be in any language (bash, go, python,...), leveraging any test +framework (robotframework, MTS, python-onapsdk). +They are all embedded in `xtesting <https://pypi.org/project/xtesting/>`__ dockers. + +.. hint:: + Automatic tests are currently divided in 4 different categories: + + - infrastructure-healthcheck: tests from OOM checking the ONAP namespace, certificates,.. + - healthcheck: basic tests on components + - smoke tests: end to end tests + - security tests + +A dashboard summarizing the status and providing the links to the test result +page or the logs is automatically created at the end of the execution of the +tests. + +.. figure:: files/tests/test-dashboard.png + +All the pages and artifacts are pushed to LF backend: + +- Daily chaines: https://logs.onap.org/onap-integration/daily +- Weekly chains: https://logs.onap.org/onap-integration/weekly +- Gating chains: https://logs.onap.org/onap-integration/gating + + +Infrastructure Healthcheck Tests +................................ + +.. csv-table:: Infrastructure Healthcheck Tests + :file: ./files/csv/tests-infrastructure-healthcheck.csv + :widths: 20,40,20,20 + :delim: ; + :header-rows: 1 + +See `Infrastructure Healthcheck README <https://git.onap.org/integration/xtesting/tree/infra-healthcheck/README.md>`__ +to adapt then run infrastructure healthcheck tests on your own system. + +Please note that the onap-k8s is run 2 times in CD chains. It is run just after +the installation (onap-k8s) and at the end of the test execution (onap-k8s-teardown) +in order to collect the logs of the different components during the test execution. + +.. figure:: files/tests/test-onap-k8s.png + +Healhcheck Tests +................ + +.. csv-table:: Healthcheck Tests + :file: ./files/csv/tests-healthcheck.csv + :widths: 20,40,20,20 + :delim: ; + :header-rows: 1 + +See `Healthcheck README <https://git.onap.org/integration/xtesting/tree/healthcheck/README.md>`__ +to adapt then run healthcheck tests on your own system. + +Smoke Tests +........... + +.. csv-table:: Smoke Tests + :file: ./files/csv/tests-smoke.csv + :widths: 20,40,20,20 + :delim: ; + :header-rows: 1 + +See `Python smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-robot/README.md>`__ +to adapt and run robot based smoke tests. +An html page is generated by the pythonsdk-test tests. + +.. figure:: files/tests/test-basic-cnf.png + + +See `Robot smoke test README <https://git.onap.org/integration/xtesting/tree/smoke-usecases-pythonsdk/README.md>`__ +to adapt and run pythonsdk based smoke tests. +Standard Robot html pages are generated. See :ref:`Robot page <docs_robot>`. + +Security Tests +............... + +.. csv-table:: Security Tests + :file: ./files/csv/tests-security.csv + :widths: 20,40,20,20 + :delim: ; + :header-rows: 1 + +See `Security test README <https://git.onap.org/integration/xtesting/tree/security/README.md>`__ +to adapt then run the security tests on your own system. + +Note for security tests, integration team follows `SECCOM recommendations and +apply waivers granted by SECCOM if needed through xfail lists <https://git.onap.org/integration/seccom/tree/>`__. + +Stability Testing +----------------- + +Ensuring the stability of ONAP is one of the missions of the Integration team. +CI chains and stability tests are performed to help stabilising the release. +See :ref:`Integration stability tests <integration-s3p>` for details. diff --git a/docs/release-notes.rst b/docs/release-notes.rst index 67de6643d..93de5cebc 100644 --- a/docs/release-notes.rst +++ b/docs/release-notes.rst @@ -6,115 +6,134 @@ Integration Release Notes ========================= -Integration Repo ----------------- +.. csv-table:: Integration Releases + :file: ./files/csv/release-integration-ref.csv + :widths: 50,50 + :delim: ; + :header-rows: 1 -Version: 6.0.1 -.............. +.. highlight:: rst -:Release Date: 2020-07-07 + - New use cases (cmpv2, ves-collector) + - New smoke tests based on pythonsdk-tests (replace onap_tests) + - Bug fixes + - New Daily Guilin CI chain + Quick Links: -**New Features** + - `Guilin Integration page <https://wiki.onap.org/display/DW/Integration+G+Release>`_ + - `Guilin Integration JIRA follow-up <https://wiki.onap.org/display/DW/Guilin+Docker+version+follow-up>`_ + - `Guilin use case testing status page <https://wiki.onap.org/display/DW/Guilin+Integration+blocking+points>` + - `Guilin Integration weather Board <https://wiki.onap.org/display/DW/0%3A+Integration+Weather+Board+for+Guilin+Release>`_ -- Update of CDS robot files (new testsuite 1.6.4) -Version: 6.0.0 -.............. +Code changes +------------ -:Release Date: 2020-06-15 +Integration Repo +................. -**New Features** +Version: 7.0.0 (aka Guilin) +^^^^^^^^^^^^^^^^^^^^^^^^^^^ -- provide a new testsuite docker version (1.6.3) including several test updates - for the different tests -- Creation of new repositories to host the use cases: bbs, mdons -- Creation of new repositories to host the simulators: dc-simulator, ran-simulator, - pnf-simulator -- Creation of new repositories to host baseline images: java11, python -- Update oparent library to fix security Vulnerabilities -- Support new use cases (mdons, vCPE_tosca,..) -- Creation of a web page to host CI daily results -- Creation and Integration in CI of new security tests (http public end points, - rooted pods, kubernetes CIS verification, jdpw ports) -- Update of the onap-k8s use case to report the full status of the cluster after - the installation and provide a dashboard -- Include healthdist and postinstall robot tests in healthcheck tests -- Add new smoke use cases in CI (pnf-registrate, 5gbulkpm,...) +:Release Date: 2020-11-xx -Quick Links: +.. csv-table:: Integration Changes + :file: ./files/csv/release-integration-features.csv + :widths: 30,70 + :delim: ; + :header-rows: 1 - - `Integration project page <https://wiki.onap.org/display/DW/Integration+Project>`_ - - ` Frankfurt use testing status page <https://wiki.onap.org/display/DW/2%3A+Frankfurt+Release+Integration+Testing+Status>` -ONAP Maturity Testing Notes ---------------------------- +Robot (Testsuite) +................. -A Frankfurt daily CI chain has bee put in place after the RC0 milestone. This CI -chain is used to track the stability of the release from the RC0. it will be -maintained as the latest stable CI branch and replaces the El Alto branch. -The daily results can be found in <https://gating-results.onap.eu/results/>. -A 72 stability test has been executed after RC1. +Version: 1.7.2 +^^^^^^^^^^^^^^ -See :ref:`S3P page<integration-s3p>` for further details. +:Release Date: 2020-11-xx -Verified Use Cases and Functional Requirements ----------------------------------------------- +.. csv-table:: Testsuite Changes + :file: ./files/csv/release-testsuite-features.csv + :widths: 30,70 + :delim: ; + :header-rows: 1 -The Integration team verified 31 use cases. -The details can be found at -:ref:`Verified Use Cases and Functional Requirements <docs_usecases>` session. O-Parent --------- +........ Version: 3.0.2 -.............. - -:Release Date: 2020-05-08 +^^^^^^^^^^^^^^ -**New Features** +:Release Date: 2020-11-xx -- Update tomcat 9.0.30 -- Update latest security dependencies -- Update settings.xml to use https maven -- Update sonar configuration -- Update checkstyle rules to exclude methods -- Upgrade oparent to checkstyle 8.27 -- Revert "update version of checkstyle for latest google style and jdk 8" -- update version of checkstyle for latest google style and jdk 8 -- Add compiler-plugin example -- Uprev to 3.0.0 for Java 11 -- qos logback to 1.2.3 +.. csv-table:: Oparent Changes + :file: ./files/csv/release-oparent-features.csv + :widths: 30,70 + :delim: ; + :header-rows: 1 Demo Artifacts (HEAT Templates) -------------------------------- +............................... + +Version: 1.7.0 +^^^^^^^^^^^^^^ + +:Release Date: 2020-11-** + +.. csv-table:: Demo Changes + :file: ./files/csv/release-demo-features.csv + :widths: 30,70 + :delim: ; + :header-rows: 1 -Version: 1.6.0 -.............. +The demo artifacts are pushed to https://nexus.onap.org/content/repositories/releases/org/onap/demo/vnf -:Release Date: 2020-06-15 +Other Repositories +.................. -https://nexus.onap.org/content/repositories/releases/org/onap/demo/vnf/ +New Guilin repositories: + +- 5G-core-nf-simulator +- terraform +- terragrunt +- testsuite/cds +- pythonsdk-test +- robot-utils + + +Verified Use Cases and Functional Requirements +---------------------------------------------- + +:ref:`Use Cases page <docs_usecases>` + +ONAP Maturity Testing Notes +--------------------------- -**New Features** +:ref:`Maturity testing page <integration-s3p>` -* Update POM and heat env to use 1.6.0 -* Helm chart for visualization operator -* bug fixes -* Robot enhancements for various use cases +Open JIRAs/Known issues +----------------------- -Robot Test Suites ------------------ +Integration +........... -Version: 1.6.4 -.............. +.. csv-table:: Integration Known Issues + :file: ./files/csv/issues-integration.csv + :widths: 10,10,40,10,10,20 + :delim: ; + :header-rows: 1 -:Release Date: 2020-07-07 -:sha1: f863e0060b9e0b13822074d0180cab11aed87ad5 +`Integration JIRA page <https://jira.onap.org/issues/?jql=project%20%3D%20Integration%20>`_ +Testsuite +......... -**New Features** +.. csv-table:: Testsuite Known Issues + :file: ./files/csv/issues-testsuite.csv + :widths: 10,10,40,10,10,20 + :delim: ; + :header-rows: 1 -- Some corrections for vLB CDS -- Change owning-entity-id from hard coded to variable +`Testsuite JIRA page <https://jira.onap.org/issues/?jql=project%20%3D%20Test>`_ diff --git a/docs/onap-integration-ci.rst b/docs/repo-onap-integration-ci.rst index 99a54323a..6f26c265a 100644 --- a/docs/onap-integration-ci.rst +++ b/docs/repo-onap-integration-ci.rst @@ -3,12 +3,13 @@ Integration Continuous Integration Guide ---------------------------------------- -Continuous Integration is key due to the complexity of the ONAP projects. -Several chains have been created: +.. important:: + Continuous Integration is key due to the complexity of the ONAP projects. + Several chains have been created: -- Daily stable chain -- Daily master chain -- Gating: On demand deployment of a full ONAP solution to validate patchsets + - Daily stable chain + - Daily master chain + - Gating: On demand deployment of a full ONAP solution to validate patchsets They are run on different environments (Orange labs, DT labs, Azure Cloud). @@ -77,9 +78,6 @@ can be used to visualize them. A web site has been created to centralize the links on http://testresults.opnfv.org/onap-integration/index.html -.. figure:: files/CI/ONAP_CI_3.png - :align: center - For Gating and gitlab.com based CI chains, the pipelines consist in pipelines of pipelines managed through the chaining of .gitlab-ci.yml file thanks to an Open Source deployment called chained-ci (https://gitlab.com/Orange-OpenSource/lfn/ci_cd/chained-ci). @@ -88,6 +86,7 @@ figure below: .. figure:: files/CI/ONAP_CI_1.png :align: center + :width: 10in If you click on any element of the chain, you will open a new window: @@ -99,12 +98,14 @@ created to summarize the tests and grant access to their associated logs: .. figure:: files/CI/ONAP_CI_8.png :align: center + :width: 6.5in Additionnaly, for the daily chain, another page displays the results as time series, allowing to see the evolution of the tests over time. .. figure:: files/CI/ONAP_CI_9.png :align: center + :width: 6.5in Setup Your Own CI Chains @@ -126,44 +127,3 @@ You should be able to chain your automation projects: If you want to report your results to the community, do not hesitate to contact the integration team. The Test database is public but the pods must be declared to be allowed to report results from third party labs. - - -ONAP Integration Testing Gate -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -5 categories have been defined for the ONAP integration testing gate: - -* infrastructure healthcheck: verify ONAP from a k8S perspective. It includes - 2 tests: onap-k8s (all the deployments, jobs, statefulste,..must be OK at - the end of an installation), onap-helm (all the helm chart must be - completed at the end of the installation) -* healthcheck: the traditionnal robot tests run from the cluster to perform - tests on the different components. -* smoke-usecases: End to end tests -* candidate-usecases: New end to end tests introduced in the automation chain - for the release -* security tests (security of kubernetes (CVE, CIS tests) and ONAP (exposed - ports, check the containers run as root,...)) -* benchmarking (robustness, stress tests): not yet available - -All these tests have been packaged thanks to the OPNFV Open Source tool xtesting. -Xtesting is a python package allowing to unify the way to declare, run tests. It -also ensures a consistent way to get the test results whatever the test framework -used (python, robotframework, bash, ...). It includes the mechanism to -automatically push the results to the test database using the test API. -It simplifies the integration in CI. - -The package can be found here https://pypi.org/project/xtesting/. - -The different ONAP xtesting dockers can be found on https://gitlab.com/Orange-OpenSource/lfn/onap/integration/xtesting/container_registry - -As an illustration, you can run the infrastructure healthcheck by typing the -following command:: - - docker run -v <the kube config>:/root/.kube/config -v <result directory>: - /var/lib/xtesting/results registry.gitlab.com/orange-opensource/lfn/onap/integration/xtesting/infra-healthcheck:latest - -All the xtesting tests are included in Daily and gating chains. -Please note that you can build your own onap-xtesting docker if you want to -include your tests. See https://wiki.onap.org/pages/viewpage.action?pageId=79202765 -for details. diff --git a/docs/usecases-deprecated.csv b/docs/usecases-deprecated.csv deleted file mode 100644 index e7a5250e7..000000000 --- a/docs/usecases-deprecated.csv +++ /dev/null @@ -1,5 +0,0 @@ -Use Case,Link,Last Valid Version, Comments -vCPE Use Case,:ref:`link <docs_vcpe>` ,El Alto, No resources to test on Frankfurt -vIPsec with HPA Use Case ,:ref:`link<docs_vipsec_hpa>` ,El Alto, No resources to test on Frankfurt -Change Management Schedule Optimization,:ref:`link<docs_CM_schedule_optimizer>` ,El Alto, No resources to test on Frankfurt -Change Management Flexible Designer and Orchestrator ,:ref:`link<docs_CM_flexible_designer_orchestrator>` ,El Alto, No resources to test on Frankfurt diff --git a/docs/usecases.csv b/docs/usecases.csv deleted file mode 100644 index 920c1251e..000000000 --- a/docs/usecases.csv +++ /dev/null @@ -1,13 +0,0 @@ -Use Case,Link,Contacts -vFirewall with closed loop,:ref:`official doc <docs_vfw>` ,M.Platania B.Freeman -vFirewall/vDNS with HPA ,:ref:`official doc <docs_vfw_hpa>`,E.Multanen -vFirewall In-Place Software Upgrade with Traffic Distribution,:ref:`official doc <docs_vfw_traffic>`,L.Rajewski -vFirewall CNF With CDS,:ref:`official doc <docs_vFW_CNF_CDS>`,K.Bańka L.Rajewski -Scale Out,:ref:`official doc <docs_scaleout>`,M.Platania S.Blandford -CCVPN-E LINE over OTN NNI,:ref:`official doc <docs_ccvpn>`,G.Agrawal -CCVPN - MDONS ,:ref:`official doc <docs_CCVPN>`,X.Miao -BBS (Broadband Service),:ref:`official doc <docs_bbs>`,D.Perez Caparros D.Balsiger -vFirewall CNF with multicloud k8s plugin,:ref:`official doc <docs_vfw_edgex_multicloud_k8s>`,E.Multanen R.Sood -EdgeXFoundry CNF with multicloud k8s plugin,:ref:`official doc <docs_vfw_edgex_multicloud_k8s>`,E.Multanen -vCPE with Tosca,:ref:`official doc <docs_vcpe_tosca_local>`, H.Haibin L.Zhao -E2E Automation vLB with CDS,`wiki page <https://wiki.onap.org/pages/viewpage.action?pageId=71838891>`__,A.Seaudi Y.Malakov |
