diff options
Diffstat (limited to 'docs/Chapter7')
| -rwxr-xr-x | docs/Chapter7/Monitoring-And-Management.rst | 836 |
1 files changed, 497 insertions, 339 deletions
diff --git a/docs/Chapter7/Monitoring-And-Management.rst b/docs/Chapter7/Monitoring-And-Management.rst index c40eea2..f4b5fb2 100755 --- a/docs/Chapter7/Monitoring-And-Management.rst +++ b/docs/Chapter7/Monitoring-And-Management.rst @@ -16,439 +16,602 @@ Monitoring & Management ----------------------- -This section addresses data collection and event processing -functionality that is directly dependent on the interfaces -provided by the VNFs' APIs. These can be in the form of asynchronous -interfaces for event, fault notifications, and autonomous data streams. -They can also be synchronous interfaces for on-demand requests to -retrieve various performance, usage, and other event information. - -The target direction for VNF interfaces is to employ APIs that are -implemented utilizing standardized messaging and modeling protocols -over standardized transports. Migrating to a virtualized environment -presents a tremendous opportunity to eliminate the need for proprietary -interfaces for VNF provider equipment while removing the traditional -boundaries between Network Management Systems and Element Management -Systems. Additionally, VNFs provide the ability to instrument the -networking applications by creating event records to test and monitor -end-to-end data flow through the network, similar to what physical or -virtual probes provide without the need to insert probes at various -points in the network. The VNF providers must be able to provide the -aforementioned set of required data directly to the ONAP collection -layer using standardized interfaces. +This section addresses data collection and event processing functionality that +is directly dependent on the interfaces provided by the xNFs’ APIs. These can +be in the form of asynchronous interfaces for event, fault notifications, and +autonomous data streams. They can also be synchronous interfaces for on-demand +requests to retrieve various performance, usage, and other event information. +It should be understood that events are well structured packages of information, +identified by an eventName, which are communicated to subscribers who are +interested in the eventName. Events are simply a way of communicating +well-structured packages of information to one or more instances of an Event +Listener service. + +The target direction for xNF interfaces is to employ APIs that are implemented +utilizing standardized messaging and modeling protocols over standardized +transports. Virtualized environments present a tremendous opportunity to +eliminate the need for proprietary interfaces for xNF provider equipment while +removing the traditional boundaries between Network Management Systems and +Element Management Systems. Additionally, virtualized NFs provide the ability +to instrument networking applications by creating event records to test and +monitor end-to-end data flow through the network, similar to what physical or +virtual probes provide without the need to insert probes at various points in +the network. The xNF providers must be able to provide the aforementioned set +of required data directly to the ONAP collection layer using standardized +interfaces. Data Model for Event Records ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -This section describes the data model for the collection of telemetry -data from VNFs by Service Providers (SPs) to manage VNF health and -runtime lifecycle. This data model is referred to as the VNF Event -Streaming (VES) specifications. While this document is focused on -specifying some of the records from the ONAP perspective, there may -be other external bodies using the same framework to specify additional -records. For example, OPNFV has a VES project that is looking to specify -records for OpenStack's internal telemetry to manage Application (VNFs), -physical and virtual infrastructure (compute, storage, network devices), -and virtual infrastructure managers (cloud controllers, SDN controllers). -Note that any configurable parameters for these data records (e.g., -frequency, granularity, policy-based configuration) will be managed -using the “Configuration” framework described in the prior sections -of this document. +This section describes the data model for the collection of telemetry data from +xNFs by Service Providers (SPs) to manage xNF health and run-time life cycle. +This data model is referred to as the VES Data Model. The VES acronym originally +stood for Virtual-function Event Streaming, but VES has been generalized to +support network-function event streaming, whether virtualized or not. + +The VES Data Model describes a vendor-agnostic common vocabulary of event +payloads. Vendor-specific, product-specific or service-specific data is +supported by the inclusion of a flexible additional information field structure. +The VES Data Models' common vocabulary is used to drive standard and automated +data analytics (policy-driven analytics) within the ONAP DCAE Framework. + +While this document is focused on specifying some of the records from the ONAP +perspective, there may be other external bodies using the same framework to +specify additional records. For example, OPNFV has a VES project that is +looking to specify records for OpenStack’s internal telemetry to manage +application (xNFs), physical and virtual infrastructure (compute, storage, +network devices, etc.) and virtual infrastructure managers (cloud controllers, +SDN controllers). It uses ONAP’s VES Agent to generate VES events from the xNF +and Intel’s collectD agent to generate infrastructure VES events. Note that any +configurable parameters for these data records (e.g., frequency, granularity, +policy-based configuration) will be managed using the “Configuration” framework +described in the prior sections of this document. The infrastructure metrics have +been funneled via the ONAP Multi-VIM Project and are now included in current +specifications. The Data Model consists of: -- Common Header Record: This data structure precedes each of the - Technology Independent and Technology Specific records sections of - the data model. - -- Technology Independent Records: This version of the document - specifies the model for Fault, Heartbeat, State Change, Syslog, - Threshold Crossing Alerts, and VNF Scaling* (short for - measurementForVfScalingFields – actual name used in JSON - specification) records. In the future, these may be extended to - support other types of technology independent records. Each of - these records allows additional fields (name/ value pairs) for - extensibility. The VNF provider can use these VNF Provider-specific - additional fields to provide additional information that may be - relevant to the managing systems. - -- Technology Specific Records: This version of the document specifies - the model for Mobile Flow records, Signaling and Voice Quality records. - In the future, these may be extended to support other types of records - (e.g. Network Fabric, Security records, etc.). Each of these records - allows additional fields (name/value pairs) for extensibility. The VNF - providers can use these VNF-specific additional fields to provide - additional information that may be relevant to the managing systems. - A placeholder for additional technology specific areas of interest to - be defined in the future documents has been depicted. + * Common Header Record: This data structure precedes each of the Technology + Independent and Technology Specific records sections of the data model. + + * Technology Independent Records: This version of the document specifies the + model for Fault, Heartbeat, Measurements, Notification, pnfRegistration, + State Change, Syslog, and Threshold Crossing Alerts records. In the future, + these may be extended to support other types of technology independent + records (work is currently progressing to define a new Performance Domain + that would be able to support already defined 3GPP Metrics for xNF, e.g. + 5G RAN device Use Case in Casablanca). Each of these records allows + additional fields (name/ value pairs) for extensibility. The xNF provider + may use these xNF provider-specific additional fields to provide additional + information that may be relevant to the managing systems. + + * Technology Specific Records: This version of the document specifies the + model for Mobile Flow records, Signaling and Voice Quality records. In the + future, these may be extended to support other types of records (e.g. + Network Fabric, Security records, etc.). Each of these records allows + additional fields (name/value pairs) for extensibility. The xNF provider + can use these xNF-specific additional fields to provide additional + information that may be relevant to the managing systems. A placeholder for + additional technology specific areas of interest to be defined in the + future documents has been depicted. |image0| Figure 1. Data Model for Event Records Event Records - Data Structure Description -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The data structure for event records consists of: -- a Common Event Header block; +- a Common Event Header block; and -- zero or more technology independent domain blocks; and +- zero (Heartbeat) or more technology independent domain blocks; or - - e.g., Fault domain, State Change domain, Syslog domain, etc. + - e.g. Fault, Measurements, Notification, PNF Registration, State Change, + Syslog, TCA, Other Fields etc. -- zero or more technology specific domain blocks. +- technology specific domain blocks. - - e.g., Mobile Flow domain, Signaling domain, Voice Quality domain, - etc. + - e.g. Mobile Flow, Signaling, Voice Quality, etc. Common Event Header ~~~~~~~~~~~~~~~~~~~~~ The common header that precedes any of the domain-specific records contains -information identifying the type of record to follow, information about -the sender and other identifying characteristics related to timestamp, -sequence number, etc. +information identifying the type of record to follow, information about the +sender and other identifying characteristics related to the domain and event. +(e.g., name, priority, sequence number, source, timestamp, type, etc.). + +.. req:: + :id: R-528866 + :target: VNF + :introduced: casablanca + :validation_mode: in_service + :impacts: dcae + :keyword: MUST + + The VNF **MUST** produce VES events that include the following mandatory + fields in the common event header + + * ``domain`` - the event domain enumeration + * ``eventId`` - the event key unique to the event source + * ``eventName`` - the unique event name + * ``lastEpochMicrosec`` - the latest unix time (aka epoch time) associated + with the event + * ``priority`` - the processing priority enumeration + * ``reportingEntityName`` - name of the entity reporting the event or + detecting a problem in another xNF + * ``sequence`` - the ordering of events communicated by an event source + * ``sourceName`` - name of the entity experiencing the event issue, which + may be detected and reported by a separate reporting entity + * ``startEpochMicrosec`` - the earliest unix time (aka epoch time) + associated with the event + * ``version`` - the version of the event header + * ``vesEventListenerVersion`` - Version of the VES event listener API spec + that this event is compliant with Technology Independent Records – Fault Fields ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The Fault Record, describing a condition in the Fault domain, contains -information about the fault such as the entity under fault, the -severity, resulting status, etc. +The current version of the data model supports the following technology +independent event records: + + * ``Fault`` - the Fault Record, describing a condition in the Fault domain, + contains information about device failures. The fault event provides data + such as the entity experiencing a fault, the severity, resulting status, + etc. + + * ``Heartbeat`` - the Heartbeat Record provides an optional structure for + communicating information about device health. Heartbeat records would only + have the Common Event Header block. An optional heartbeat domain is + available to specify information such as heartbeat interval and recommended + action upon missing heartbeat interval. Heartbeat avoids the need to ping + a device. A communication failure can be determined via missing heartbeat + events being delivered to DCAE and appropriate action (e.g. restart VM, + rebuild xNF or create ticket) can be taken by DCAE CLAMP. + + * ``Measurements`` - the Measurements Record contains information about xNF + and xNF resource structure and its condition to help in the management of + the resources for purposes of capacity planning, elastic scaling, + performance management and service assurance. These are soft alarms + providing an opportunity for proactive maintenance. + + * ``Notification`` - the Notification Record provides a structure for + communicating notification information from the NF. It can contain + notification information related to the current operational state that is + reported by the NF. As an example, when cards or port name of the entity + have changed state. (e.g., offline -> online) Other use cases include + notification of file ready for collection using Bulk Data Transfer or + notification on configuration changes to a device. + + * ``Other`` - the Other Record defines fields for events that do not have a + defined domain but are needed to be collected and sent to DCAE. This + record provides a mechanism to convey a complex set of fields (possibly + nested or opaque) and is purely intended to address miscellaneous needs + such as addressing time-to-market considerations or other proof-of-concept + evaluations. Hence, use of this record type is discouraged and should be + minimized. (Note: the Other domain could be used to create and test new + domain ideas.) + + * ``pnfRegistration`` - the pnfRegistration Record provides a structure for + registration of a physical network function. The pnfRegistration Record can + contain information about attributes related to the physical network function + including serial number, software revision, unit type and vendor name. + + * ``State Change`` - the State Change Record provides a structure for + communicating information about data flow through the xNF. The State + Change Record can contain information about state change related to + physical device that is reported by the xNF. As an example, when cards or + port name of the entity that has changed state. Note: The Notification + Domain can also communicate similar information. + + * ``Syslog`` - the Syslog Record provides a structure for communicating any + type of information that may be logged by the xNF. It can contain + information about system internal events, status, errors, etc. It is + recommended that low volume control or session logs are communicated via a + push mechanism, while other large volume logs should be sent via file + transfer. + + * ``Threshold Crossing Alert`` - the Threshold Crossing Alert (TCA) Record + provides a structure for communicating information about threshold + crossing alerts. It uses data from the Measurement or a similar domain to + watch for a Key Performance Indicator (KPI) threshold that has been + crossed. TCA provides alert definitions and types, actions, events, + timestamps and physical or logical details. + + +Technology Specific Records +~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Technology Independent Records – Heartbeat Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The current version of the data model supports the following technology +specific event records: -The Heartbeat Record provides an optional structure for communicating -information about heartbeat or watchdog signaling events. It can -contain information about service intervals, status information etc. -as required by the heartbeat implementation. + * ``Mobile Flow`` - the Mobile Flow Record provides a structure for + communicating information about data flow through the NF. It can contain + information about connectivity and data flows between serving elements for + mobile service, such as between LTE reference points, etc. -Note: Heartbeat records would only have the Common Event Header block. -An optional heartbeat domain is available if required by the heartbeat -implementation. + * ``Signaling`` - the Signaling Record provides a structure for + communicating information about signaling messages, parameters and + signaling state. It can contain information about data flows for signaling + and controlling multimedia communication sessions such as voice and video + calls. -Technology Independent Records – State Change Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + * ``Voice Quality`` - the Voice Quality Record provides a structure for + communicating information about voice quality statistics including media + connection information, such as transmitted octet and packet counts, + packet loss, packet delay variation, round-trip delay, QoS parameters and + codec selection. -The State Change Record provides a structure for communicating information -about data flow through the VNF. It can contain information about state -change related to physical device that is reported by VNF. As an example, -when cards or port name of the entity that has changed state. + * ``Future Domains`` - the Future Domains Record is a placeholder for + additional technology specific areas of interest that will be defined and + described in the future documents. -Technology Independent Records – Syslog Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Miscellaneous +~~~~~~~~~~~~~ -The Syslog Record provides a structure for communicating any type of -information that may be logged by the VNF. It can contain information -about system internal events, status, errors, etc. +The event specification contains various extensible structures (e.g. hashMap) +that enable event publishers to send information that has not been explicitly +defined. -Technology Independent Records – Threshold Crossing Alert Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. req:: + :id: R-283988 + :target: VNF + :introduced: casablanca + :validation_mode: in_service + :impacts: dcae + :keyword: MUST NOT + + The VNF, when publishing events, **MUST NOT** send information through + extensible structures if the event specification has explicitly defined + fields for that information. -The Threshold Crossing Alert (TCA) Record provides a structure for -communicating information about threshold crossing alerts. It can -contain alert definitions and types, actions, events, timestamps -and physical or logical details. +.. req:: + :id: R-470963 + :target: VNF + :introduced: casablanca + :validation_mode: in_service + :impacts: dcae + :keyword: MUST + + The VNF, when publishing events, **MUST** leverage camel case to separate + words and acronyms used as keys that will be sent through extensible fields. + When an acronym is used as the key, then only the first letter shall be + capitalized. -Technology Independent Records - VNF Scaling Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. req:: + :id: R-408813 + :target: VNF + :keyword: MUST + :introduced: casablanca + :validation_mode: none + :impacts: dcae -The VNF Scaling\* (short for measurementForVfScalingFields – -actual name used in JSON specification) Record contains information -about VNF and VNF resource structure and its condition to help in -the management of the resources for purposes of elastic scaling. + The VNF, when publishing events, **MUST** pass all information it is + able to collect even if the information field is identified as optional. + However, if the data cannot be collected, then optional fields can be + omitted. -Technology Independent Records – otherFields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The otherFields Record defines fields for events belonging to the -otherFields domain of the Technology Independent domain enumeration. -This record provides a mechanism to convey a complex set of fields -(possibly nested or opaque) and is purely intended to address -miscellaneous needs such as addressing time-to-market considerations -or other proof-of-concept evaluations. Hence, use of this record -type is discouraged and should be minimized. +Data Structure Specification of the Event Record +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Technology Specific Records – Mobile Flow Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +.. req:: + :id: R-520802 + :target: XNF PROVIDER + :keyword: MUST + :introduced: casablanca + :validation_mode: static + :impacts: dcae + + The xNF provider **MUST** provide a YAML file formatted in adherence with + the :doc:`VES Event Registration specification<../../../../vnfsdk/module.git/files/VESEventRegistration_3_0>` + that defines the following information for each event produced by the VNF: + + * ``eventName`` + * Required fields + * Optional fields + * Any special handling to be performed for that event -The Mobile Flow Record provides a structure for communicating -information about data flow through the VNF. It can contain -information about connectivity and data flows between serving -elements for mobile service, such as between LTE reference points, etc. +.. req:: + :id: R-120182 + :target: XNF PROVIDER + :keyword: MUST + :introduced: casablanca + :validation_mode: static + :impacts: dcae -Technology Specific Records – Signaling Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + The xNF provider **MUST** indicate specific conditions that may arise, and + recommend actions that may be taken at specific thresholds, or if specific + conditions repeat within a specified time interval, using the semantics and + syntax described by the :doc:`VES Event Registration specification<../../../../vnfsdk/module.git/files/VESEventRegistration_3_0>`. -The Signaling Record provides a structure for communicating information -about signaling messages, parameters and signaling state. It can -contain information about data flows for signaling and controlling -multimedia communication sessions such as voice and video calls. +**NOTE:** The Service Provider may override xNF provider Event Registrations using +the ONAP SDC Design Studio to finalizes Service Provider engineering rules +for the processing of the xNF events. These changes may modify any of the +following: -Technology Specific Records – Voice Quality Fields -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The Voice Quality Record provides a structure for communicating information -about voice quality statistics including media connection information, -such as transmitted octet and packet counts, packet loss, packet delay -variation, round-trip delay, QoS parameters and codec selection. +* Threshold levels +* Specified actions related to conditions -Technology Specific Records – Future Domains -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The futureDomains Record is a placeholder for additional technology -specific areas of interest that will be defined and described -in the future documents. +.. req:: + :id: R-570134 + :target: XNF + :keyword: MUST + :introduced: casablanca + :validation_mode: in_service + :impacts: dcae + + The events produced by the xNF **MUST** must be compliant with the common + event format defined in the + :doc:`VES Event Listener<../../../../vnfsdk/model.git/docs/files/VESEventListener_7_0_1>` + specification. -Data Structure Specification of the Event Record -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +.. req:: + :id: R-123044 + :target: XNF PROVIDER + :keyword: MUST + :introduced: casablanca + :validation_mode: in_service + :impacts: dcae + + The xNF Provider **MAY** require that specific events, identified by their + ``eventName``, require that certain fields, which are optional in the common + event format, must be present when they are published. -For additional information on the event record formats of the data -structures mentioned above, please refer to `VES Event -Listener <https://onap.readthedocs.io/en/latest/submodules/vnfsdk/model.git/docs/files/VESEventListener.html>`__. Transports and Protocols Supporting Resource Interfaces ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -Delivery of data from VNFs to ONAP must use the common transport -mechanisms and protocols for all VNFs as defined in this document. -Transport mechanisms and protocols have been selected to enable both -high volume and moderate volume datasets, as well as asynchronous and -synchronous communications over secure connections. The specified -encoding provides self-documenting content, so data fields can be -changed as needs evolve, while minimizing changes to data delivery. - -The term 'Event Record' is used throughout this document to represent -various forms of telemetry or instrumentation made available by the -VNF including, faults, status events, various other types of VNF -measurements and logs. Headers received by themselves must be used -as heartbeat indicators. Common structures and delivery protocols for -other types of data will be given in future versions of this document -as we get more insight into data volumes and required processing. - -In the following sections, we provide options for encoding, serialization -and data delivery. Agreements between Service Providers and VNF providers -shall determine which encoding, serialization and delivery method to use -for particular data sets. The selected methods must be agreed to prior to -the on-boarding of the VNF into ONAP design studio. - -VNF Telemetry using VES/JSON Model +Transport mechanisms and protocols have been selected to enable both high +volume and moderate volume data sets, as well as asynchronous and synchronous +communications over secure connections. The specified encoding provides +self-documenting content, so data fields can be changed as needs evolve, while +minimizing changes to data delivery. + +.. req:: + :id: R-798933 + :target: XNF + :keyword: SHOULD + :impacts: dcae + :validation_mode: in_service + :introduced: casblanca + + The xNF **SHOULD** deliver event records that fall into the event domains + supported by VES. + +.. req:: + :id: R-821839 + :target: XNF + :keyword: MUST + :impacts: dcae + :validation_mode: in_service + :introduced: casblanca + + The xNF **MUST** deliver event records to ONAP using the common transport + mechanisms and protocols defined in this document. + +The term ‘Event Record’ is used throughout this document to represent various +forms of telemetry or instrumentation made available by the xNFs +including, faults, status events, various other types of xNF measurements +and logs. + +Common structures and delivery protocols for other types of data will be given +in future versions of this document as we gain more insight into data volumes +and required processing. + +In the following sections, we provide options for encoding, serialization and +data delivery. Agreements between Service Providers and xNF providers determine +which encoding, serialization and delivery method to use for particular +data sets. + +.. req:: + :id: R-932071 + :target: XNF + :keyword: MUST + :impacts: dcae + :validation_mode: none + :introduced: casblanca + + The xNF provider **MUST** reach agreement with the Service Provider on + the selected methods for encoding, serialization and data delivery + prior to the on-boarding of the xNF into ONAP SDC Design Studio. + + +xNF Telemetry using VES/JSON Model ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -The preferred model for data delivery from a VNF to ONAP DCAE is -the JSON driven model as depicted in Figure 2. +.. req:: + :id: R-659655 + :target: XNF + :keyword: SHOULD + :impacts: dcae + :validation_mode: in_service + :introduced: casablanca + + The xNF **SHOULD** leverage the JSON-driven model, as depicted in Figure 2, + for data delivery unless there are specific performance or operational + concerns agreed upon by the Service Provider that would warrant using an + alternate model. |image1| Figure 2. VES/JSON Driven Model -VNF providers will provide a YAML artifact to the Service Provider -that describes: - -* standard VES/JSON model information elements (key/values) that - the VNF provides -* any additional non-standard (custom) VES/JSON model information - elements (key/values) that the VNF provides +xNF Telemetry using Google Protocol Buffers +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Using the semantics and syntax supported by YAML, VNF providers -will indicate specific conditions that may arise, and recommend -actions that should be taken at specific thresholds, or if specific -conditions repeat within a specified time interval. +.. req:: + :id: R-697654 + :target: XNF + :keyword: MAY + :impacts: dcae + :validation_mode: in_service + :introduced: casablanca + + The xNF **MAY** leverage the Google Protocol Buffers (GPB) delivery model + depicted in Figure 3 to support real-time performance management (PM) data. + In this model the VES events are streamed as binary-encoded GBPs over via + TCP sockets -Based on the VNF provider's recommendations, the Service Provider may -create additional YAML artifacts (using ONAP design Studio), which -finalizes Service Provider engineering rules for the processing of -the VNF events. The Service Provider may alter the threshold levels -recommended by the VNF providor, and may modify and more clearly -specify actions that should be taken when specified conditions arise. -The Service Provider-created version of the YAML artifact will be -distributed to ONAP applications by the Design framework. +|image2| -VNF Telemetry using YANG Model -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Figure 3. xNF Telemetry using Google Protocol Buffers -In addition to the JSON driven model described above, a YANG -driven model can also be supported, as depicted in Figure 3. -|image2| +**NOTE:** For high-volume xNF telemetry, native (binary) Google Protocol Buffers +(GPB) is the preferred serialization method. While supporting the GPB +telemetry delivery approach described above, the default delivery method +is the VES/REST JSON based model in DCAE. The purpose of the diagram above +is to illustrate the GPB delivery concept only and not to imply a specific +implementation. -Figure 3. YANG Driven Model - -VNF providers will provide to the Service Provider the following -YANG model artifacts: - -* common IETF YANG modules that support the VNF -* native (VNF provider-supplied) YANG modules that support the VNF -* open (OpenConfig) YANG modules and the following - configuration-related information, including: - - * telemetry configuration and operational state data; such as: - - * sensor paths - * subscription bindings - * path destinations - * delivery frequency - * transport mechanisms - * data encodings - -* a YAML artifact that provides all necessary mapping relationships - between YANG model data types to VES/JSON information elements -* YANG helper or decoder functions that automate the conversion between - YANG model data types to VES/JSON information elements -* OPTIONAL: YANG Telemetry modules in JSON format per RFC 7951 - -Using the semantics and syntax supported by YANG, VNF providers -will indicate specific conditions that may arise, and recommend -actions that should be taken at specific thresholds, or if specific -conditions repeat within a specified time interval. - -Based on the VNF provider's recommendations, the Service Provider may -create additional YAML artifacts (using ONAP design Studio), which -finalizes Service Provider engineering rules for the processing of the -VNF events. The Service Provider may alter the threshold levels recommended -by the VNF provider, and may modify and more clearly specify actions that -should be taken when specified conditions arise. The Service -Provided-created version of the YAML will be distributed to ONAP -applications by the Design framework. - -Note: While supporting the YANG model described above, we are still -leveraging the VES JSON based model in DCAE. The purpose of the -diagram above is to illustrate the concept only and not to imply a -specific implementation. - -VNF Telemetry using Google Protocol Buffers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +For additional information and uses cases for Real Time Performance +Management and High Volume Stream Data Collection, please refer to the +`5G - Real Time PM and High Volume Stream Data Collection ONAP Development <https://wiki.onap.org/display/DW/5G+-+Real+Time+PM+and+High+Volume+Stream+Data+Collection>`__ +Wiki page. -In addition to the data delivery models described above, support for -delivery of VNF telemetry using Google Protocol Buffers (GPB) can -also be supported, as depicted in Figure 4. +Bulk Telemetry Transmission +~~~~~~~~~~~~~~~~~~~~~~~~~~~ -VNF providers will provide to the Service Provider the additional -following artifacts to support the delivery of VNF telemetry to DCAE -via the open-source gRPC mechanism using Google's Protocol Buffers: +.. req:: + :id: R-908291 + :target: XNF + :keyword: MAY + :introduced: casablanca + :impacts: dcae, dmaap + :validation_mode: in_service -* the YANG model artifacts described in support of the - "VNF Telemetry using YANG Model" -* valid definition file(s) for all GPB / KV-GPB encoded messages -* valid definition file(s) for all gRPC services -* gRPC method parameters and return types specified as Protocol - Buffers messages + The XNF **MAY** leverage bulk xNF telemetry transmission mechanism, as + depicted in Figure 4, in instances where other transmission methods are not + practical or advisable. |image3| -Figure 4. Protocol Buffers Driven Model +Figure 4. xNF Telemetry using Bulk Transmission -Note: if Google Protocol Buffers are employed for delivery of VNF -telemetry, Key-Value Google Protocol Buffers (KV-GPB) is the -preferred serialization method. Details of specifications and -versioning corresponding to a release can be found at: -`VES Event Listener <https://onap.readthedocs.io/en/latest/submodules/vnfsdk/model.git/docs/files/VESEventListener.html>`__. +**NOTE:** An optional VES mapper micro-service can be leveraged to to extract +measurements and publish them as VES events. -Note: While supporting the VNF telemetry delivery approach described above, -we are still leveraging the VES JSON based model in DCAE. The purpose of -the diagram above is to illustrate the concept only and not to imply a -specific implementation. +For additional information and use cases for the Bulk Telemetry Transmission +Mechanism, please refer to the `5G - Bulk PM ONAP Development <https://wiki.onap.org/display/DW/5G+-+Bulk+PM>`__ +Wiki page. Monitoring & Management Requirements -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ VNF telemetry via standardized interface -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -All VNF telemetry data (e.g. fault event records, syslog records, -performance records, etc.) need to be delivered to ONAP using the -standardized models and interfaces described in this section. +.. req:: + :id: R-821473 + :target: XNF + :keyword: MUST + :introduced: casablanca + :impacts: dcae + :validation_mode: in_service -Encoding and Serialization -~~~~~~~~~~~~~~~~~~~~~~~~~~~ + The xNF MUST produce heartbeat indicators consisting of events containing + the common event header only per the VES Listener Specification. -Content delivered from VNFs to ONAP is to be encoded and serialized using JSON: JSON ~~~~~~~~~~~~~~~~~~ - .. req:: :id: R-19624 :target: XNF :keyword: MUST - The xNF **MUST** encode and serialize content delivered to - ONAP using JSON (RFC 7159) plain text format. High-volume data - is to be encoded and serialized using `Avro <http://avro.apache.org/>`_, - where the Avro [#7.4.1]_ data format are described using JSON. - - Note: - - - JSON plain text format is preferred for moderate volume data sets - (option 1), as JSON has the advantage of having well-understood simple - processing and being human-readable without additional decoding. Examples - of moderate volume data sets include the fault alarms and performance - alerts, heartbeat messages, measurements used for xNF scaling and syslogs. - - Binary format using Avro is preferred for high volume data sets - (option 2) such as mobility flow measurements and other high-volume - streaming events (such as mobility signaling events or SIP signaling) - or bulk data, as this will significantly reduce the volume of data - to be transmitted. As of the date of this document, all events are - reported using plain text JSON and REST. - - Avro content is self-documented, using a JSON schema. The JSON schema is - delivered along with the data content - (http://avro.apache.org/docs/current/ ). This means the presence and - position of data fields can be recognized automatically, as well as the - data format, definition and other attributes. Avro content can be - serialized as JSON tagged text or as binary. In binary format, the - JSON schema is included as a separate data block, so the content is - not tagged, further compressing the volume. For streaming data, Avro - will read the schema when the stream is established and apply the - schema to the received content. + The xNF, when leveraging JSON for events, **MUST** encode and serialize + content delivered to ONAP using JSON (RFC 7159) plain text format. + High-volume data is to be encoded and serialized using + `Avro <http://avro.apache.org/>`_, where the Avro [#7.4.1]_ data + format are described using JSON. + +Note: + + - JSON plain text format is preferred for moderate volume data sets + (option 1), as JSON has the advantage of having well-understood simple + processing and being human-readable without additional decoding. Examples + of moderate volume data sets include the fault alarms and performance + alerts, heartbeat messages, measurements used for xNF scaling and syslogs. + - Binary format using Avro is preferred for high volume data sets + (option 2) such as mobility flow measurements and other high-volume + streaming events (such as mobility signaling events or SIP signaling) + or bulk data, as this will significantly reduce the volume of data + to be transmitted. As of the date of this document, all events are + reported using plain text JSON and REST. + - Avro content is self-documented, using a JSON schema. The JSON schema is + delivered along with the data content + (http://avro.apache.org/docs/current/ ). This means the presence and + position of data fields can be recognized automatically, as well as the + data format, definition and other attributes. Avro content can be + serialized as JSON tagged text or as binary. In binary format, the + JSON schema is included as a separate data block, so the content is + not tagged, further compressing the volume. For streaming data, Avro + will read the schema when the stream is established and apply the + schema to the received content. In addition to the preferred method (JSON), content can be delivered from xNFs to ONAP can be encoded and serialized using Google Protocol Buffers (GPB). -KV-GPB/GPB -~~~~~~~~~~~~~~~~~~ +Google Protocol Buffers (GPB) +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. req:: - :id: R-10623 - :target: XNF - :keyword: MUST - :introduced: casablanca + :id: R-257367 + :target: XNF + :keyword: MUST + :introduced: casablanca + :validation_mode: in_service + + The xNF, when leveraging Google Protocol Buffers for events, **MUST** + serialize the events using native Google Protocol Buffers (GPB) where: + + * keys are represented as integers pointing to the system resources for + the xNF being monitored + * values correspond to integers or strings that identify the operational + state of the VNF resource, such a statistics counters and the state of + an xNF resource. + * required Google Protocol Buffers (GPB) metadata is provided in the + form of .proto files. - Telemetry data delivered using Google Protocol Buffers v3 (proto3) - **MUST** be serialized in one of the following methods: - - * Key-value Google Protocol Buffers (KV-GPB) is also known as - self-describing GPB: - - * keys are strings that correspond to the path of the system - resources for the VNF being monitored. - * values correspond to integers or strings that identify the - operational state of the VNF resource, such a statistics counters - and the state of a VNF resource. - * VNF providers must supply valid KV-GPB definition file(s) to allow - for the decoding of all KV-GPB encoded telemetry messages. - - * Native Google Protocol Buffers (GPB) is also known as compact GPB: - - * keys are represented as integers pointing to the system resources for - the VNF being monitored. - * values correspond to integers or strings that identify the operational - state of the VNF resource, such a statistics counters and the state - of a VNF resource. - * Google Protocol Buffers (GPB) requires metadata in the form of .proto - files. VNF providers must supply the necessary GPB .proto files such that - GPB telemetry messages can be encoded and decoded. - -In the future, we may consider support for other types of -encoding & serialization methods based on industry demand. +.. req:: + :id: R-978752 + :target: XNF PROVIDER + :keyword: MUST + :introduced: casablanca + :validation_mode: static + + The xNF providers **MUST** provide to the Service Provider the additional + following artifacts to support the delivery of high-volume xNF telemetry to + DCAE via GPB over TLS/TCP: + + * a valid VES Event .proto definition file, to be used validate and + decode an event + * a valid high volume measurement .proto definition file, to be used for + processing high volume events + * a supporting PM content metadata file to be used by analytics + applications to process high volume measurement events Reporting Frequency ~~~~~~~~~~~~~~~~~~~~~ .. req:: + :id: R-146931 + :target: XNF + :keyword: MUST + :introduced: casablanca + :validation_mode: in_service + + The xNF **MUST** report exactly one Measurement event per period + per source name. + +.. req:: :id: R-98191 :target: XNF :keyword: MUST @@ -694,7 +857,7 @@ Security regular procedures for securing access and delivery. Bulk Performance Measurement -~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. req:: :id: R-841740 @@ -718,7 +881,7 @@ Bulk Performance Measurement when supporting the event-driven bulk transfer of monitoring data. .. req:: - :id: R-75943 + :id: R-75943 :target: XNF :keyword: SHOULD :introduced: casablanca @@ -727,22 +890,17 @@ Bulk Performance Measurement The xNF **SHOULD** support the data schema defined in 3GPP TS 32.435, when supporting the event-driven bulk transfer of monitoring data. -.. [#7.4.1] - This option is not currently supported in ONAP and it is currently - under consideration. .. |image0| image:: ../Data_Model_For_Event_Records.png - :width: 7in - :height: 8in .. |image1| image:: ../VES_JSON_Driven_Model.png :width: 5in :height: 3in -.. |image2| image:: ../YANG_Driven_Model.png - :width: 5in - :height: 3in +.. |image2| image:: ../Protocol_Buffers_Driven_Model.png + :width: 4.74in + :height: 3.3in -.. |image3| image:: ../Protocol_Buffers_Driven_Model.png +.. |image3| image:: ../Bulk_Data_Transfer_Mechv1.PNG :width: 4.74in :height: 3.3in |