diff options
Diffstat (limited to 'docs')
4 files changed, 157 insertions, 9 deletions
diff --git a/docs/sections/services/ves-hv/deployment.rst b/docs/sections/services/ves-hv/deployment.rst index 9fdbceb1..07d26b94 100644 --- a/docs/sections/services/ves-hv/deployment.rst +++ b/docs/sections/services/ves-hv/deployment.rst @@ -117,12 +117,3 @@ Result: .. code-block:: bash kubectl get pods --namespace ${ONAP_NAMESPACE} --selector app=dcae-hv-ves-collector - -Healthcheck -=========== - -Inside HV-VES docker container runs small http service for healthcheck - exact port for this service can be configured -at deployment using `--health-check-api-port` command line option. - -This service exposes single endpoint **GET /health/ready** which returns **HTTP 200 OK** in case HV-VES is healthy -and ready for connections. Otherwise it returns **HTTP 503 Service Unavailable** with short reason of unhealthiness. diff --git a/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst b/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst new file mode 100644 index 00000000..8437180d --- /dev/null +++ b/docs/sections/services/ves-hv/healthcheck-and-monitoring.rst @@ -0,0 +1,89 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _healthcheck_and_monitoring: + +Healthcheck and Monitoring +========================== + +Healthcheck +----------- +Inside HV-VES docker container runs a small HTTP service for healthcheck. Port for healthchecks can be configured +at deployment using ``--health-check-api-port`` command line option or via `VESHV_HEALTHCHECK_API_PORT` environment variable (for details see :ref:`deployment`). + +This service exposes endpoint **GET /health/ready** which returns a **HTTP 200 OK** when HV-VES is healthy +and ready for connections. Otherwise it returns a **HTTP 503 Service Unavailable** message with a short reason of unhealthiness. + + +Monitoring +---------- +HV-VES collector allows to collect metrics data at runtime. To serve this purpose HV-VES application exposes an endpoint **GET /monitoring/prometheus** +which returns a **HTTP 200 OK** message with a specific data in its body. Returned data is in a format readable by Prometheus service. +Prometheus endpoint shares a port with healthchecks. + +Metrics provided by HV-VES metrics: + ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| Name of metric | Unit | Description | ++===============================================+==============+==========================================================================================+ +| hvves_clients_rejected_cause_total | cause/piece | number of rejected clients grouped by cause | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_clients_rejected_total | piece | total number of rejected clients | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_connections_active | piece | number of currently active connections | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_connections_total | piece | total number of connections | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_data_received_bytes_total | bytes | total number of received bytes | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_disconnections_total | piece | total number of disconnections | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_messages_dropped_cause_total | cause/piece | number of dropped messages grouped by cause | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_messages_dropped_total | piece | total number of dropped messages | ++-----------------------------------------------+--------------+-----------------------------------+------------------------------------------------------+ +| hvves_messages_latency_seconds_count | piece | latency is a time between | counter for number of latency occurance | ++-----------------------------------------------+--------------+ message.header.lastEpochMicrosec +------------------------------------------------------+ +| hvves_messages_latency_seconds_max | seconds | and time when data has been sent | maximal observed latency | ++-----------------------------------------------+--------------+ from HV-VES to Kafka +------------------------------------------------------+ +| hvves_messages_latency_seconds_sum | seconds | | sum of latency parameter from each message | ++-----------------------------------------------+--------------+-----------------------------------+------------------------------------------------------+ +| hvves_messages_processing_time_seconds_count | piece | processing time is time meassured | counter for number of processing time occurance | ++-----------------------------------------------+--------------+ between decoding of WTP message +------------------------------------------------------+ +| hvves_messages_processing_time_seconds_max | seconds | and time when data has been sent | maximal processing time | ++-----------------------------------------------+--------------+ From HV-VES to Kafka +------------------------------------------------------+ +| hvves_messages_processing_time_seconds_sum | seconds | | sum of processing time from each message | ++-----------------------------------------------+--------------+-----------------------------------+------------------------------------------------------+ +| hvves_messages_received_payload_bytes_total | bytes | total number of received payload bytes | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_messages_received_total | piece | total number of received messages | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_messages_sent_topic_total | topic/piece | number of sent messages grouped by topic | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ +| hvves_messages_sent_total | piece | number of sent messages | ++-----------------------------------------------+--------------+------------------------------------------------------------------------------------------+ + +JVM metrics: + +- jvm_buffer_memory_used_bytes +- jvm_classes_unloaded_total +- jvm_gc_memory_promoted_bytes_total +- jvm_buffer_total_capacity_bytes +- jvm_threads_live +- jvm_classes_loaded +- jvm_gc_memory_allocated_bytes_total +- jvm_threads_daemon +- jvm_buffer_count +- jvm_gc_pause_seconds_count +- jvm_gc_pause_seconds_sum +- jvm_gc_pause_seconds_max +- jvm_gc_max_data_size_bytes +- jvm_memory_committed_bytes +- jvm_gc_live_data_size_bytes +- jvm_memory_max_bytes +- jvm_memory_used_bytes +- jvm_threads_peak + +Sample response for **GET monitoring/prometheus**: + +.. literalinclude:: metrics_sample_response.txt diff --git a/docs/sections/services/ves-hv/index.rst b/docs/sections/services/ves-hv/index.rst index 5cfaed3f..5bb83ddc 100644 --- a/docs/sections/services/ves-hv/index.rst +++ b/docs/sections/services/ves-hv/index.rst @@ -36,3 +36,4 @@ High Volume VES Collector overview and functions HV-VES Offered APIs <../../apis/ves-hv/index> authorization example-event + healthcheck-and-monitoring diff --git a/docs/sections/services/ves-hv/metrics_sample_response.txt b/docs/sections/services/ves-hv/metrics_sample_response.txt new file mode 100644 index 00000000..da54e3ea --- /dev/null +++ b/docs/sections/services/ves-hv/metrics_sample_response.txt @@ -0,0 +1,67 @@ +jvm_threads_live 26.0 +system_cpu_count 4.0 +jvm_gc_memory_promoted_bytes_total 6740576.0 +process_cpu_usage 5.485463521667581E-4 +jvm_buffer_count{id="mapped",} 0.0 +jvm_buffer_count{id="direct",} 14.0 +jvm_threads_peak 26.0 +jvm_classes_unloaded_total 0.0 +hvves_clients_rejected_cause_total{cause="too_big_payload",} 7.0 +hvves_messages_sent_topic_total{topic="HV_VES_PERF3GPP",} 20000.0 +jvm_threads_daemon 25.0 +jvm_buffer_memory_used_bytes{id="mapped",} 0.0 +jvm_buffer_memory_used_bytes{id="direct",} 1.34242446E8 +jvm_gc_max_data_size_bytes 4.175429632E9 +hvves_messages_received_total 32000.0 +hvves_messages_dropped_total 12000.0 +hvves_clients_rejected_total 7.0 +system_cpu_usage 0.36204059243006037 +jvm_memory_max_bytes{area="nonheap",id="CodeHeap 'non-nmethods'",} 5832704.0 +jvm_memory_max_bytes{area="nonheap",id="Metaspace",} -1.0 +jvm_memory_max_bytes{area="nonheap",id="CodeHeap 'profiled nmethods'",} 1.22912768E8 +jvm_memory_max_bytes{area="nonheap",id="Compressed Class Space",} 1.073741824E9 +jvm_memory_max_bytes{area="heap",id="G1 Eden Space",} -1.0 +jvm_memory_max_bytes{area="heap",id="G1 Old Gen",} 4.175429632E9 +jvm_memory_max_bytes{area="heap",id="G1 Survivor Space",} -1.0 +jvm_memory_max_bytes{area="nonheap",id="CodeHeap 'non-profiled nmethods'",} 1.22912768E8 +hvves_messages_dropped_cause_total{cause="invalid",} 12000.0 +hvves_messages_received_payload_bytes_total 1.0023702E7 +jvm_buffer_total_capacity_bytes{id="mapped",} 0.0 +jvm_buffer_total_capacity_bytes{id="direct",} 1.34242445E8 +system_load_average_1m 2.77 +hvves_data_received_bytes_total 1.052239E7 +jvm_gc_pause_seconds_count{action="end of minor GC",cause="Metadata GC Threshold",} 2.0 +jvm_gc_pause_seconds_sum{action="end of minor GC",cause="Metadata GC Threshold",} 0.087 +jvm_gc_pause_seconds_count{action="end of minor GC",cause="G1 Evacuation Pause",} 8.0 +jvm_gc_pause_seconds_sum{action="end of minor GC",cause="G1 Evacuation Pause",} 0.218 +jvm_gc_pause_seconds_max{action="end of minor GC",cause="Metadata GC Threshold",} 0.03 +jvm_gc_pause_seconds_max{action="end of minor GC",cause="G1 Evacuation Pause",} 0.031 +hvves_messages_processing_time_seconds_max 0.114395 +hvves_messages_processing_time_seconds_count 20000.0 +hvves_messages_processing_time_seconds_sum 280.282544 +hvves_disconnections_total 11.0 +jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'non-nmethods'",} 1312640.0 +jvm_memory_used_bytes{area="nonheap",id="Metaspace",} 3.624124E7 +jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'profiled nmethods'",} 1.1602304E7 +jvm_memory_used_bytes{area="nonheap",id="Compressed Class Space",} 4273752.0 +jvm_memory_used_bytes{area="heap",id="G1 Eden Space",} 1.38412032E8 +jvm_memory_used_bytes{area="heap",id="G1 Old Gen",} 7638112.0 +jvm_memory_used_bytes{area="heap",id="G1 Survivor Space",} 7340032.0 +jvm_memory_used_bytes{area="nonheap",id="CodeHeap 'non-profiled nmethods'",} 4083712.0 +jvm_memory_committed_bytes{area="nonheap",id="CodeHeap 'non-nmethods'",} 2555904.0 +jvm_memory_committed_bytes{area="nonheap",id="Metaspace",} 3.7486592E7 +jvm_memory_committed_bytes{area="nonheap",id="CodeHeap 'profiled nmethods'",} 1.1730944E7 +jvm_memory_committed_bytes{area="nonheap",id="Compressed Class Space",} 4587520.0 +jvm_memory_committed_bytes{area="heap",id="G1 Eden Space",} 1.58334976E8 +jvm_memory_committed_bytes{area="heap",id="G1 Old Gen",} 9.8566144E7 +jvm_memory_committed_bytes{area="heap",id="G1 Survivor Space",} 7340032.0 +jvm_memory_committed_bytes{area="nonheap",id="CodeHeap 'non-profiled nmethods'",} 4128768.0 +jvm_gc_memory_allocated_bytes_total 1.235222528E9 +hvves_connections_total 12.0 +jvm_classes_loaded 7120.0 +hvves_messages_sent_total 20000.0 +hvves_connections_active 1.0 +jvm_gc_live_data_size_bytes 7634496.0 +hvves_messages_latency_seconds_max 1.5459828692292638E9 +hvves_messages_latency_seconds_count 20000.0 +hvves_messages_latency_seconds_sum 2.91400110035487E9
\ No newline at end of file |