diff options
21 files changed, 752 insertions, 75 deletions
diff --git a/docs/sections/build.rst b/docs/sections/build.rst index b6fcdc81..5323c3a0 100644 --- a/docs/sections/build.rst +++ b/docs/sections/build.rst @@ -45,7 +45,12 @@ Below is a list of the repositories and their sub-modules, and the language they - dcaegen2.services.pm-mapper (Java) - dcaegen2.services.ves-mapper (Java) - dcaegen2.services.son-handler (Java) + - dcaegen2.services.kpi-ms (Java) + - dcaegen2.services.pmsh (Python) + - dcaegen2.services.datalake-handler (Java) +* dcaegen2.services.son-handler (Java) + * dcaegen2.deployments - scripts (bash, python) @@ -57,6 +62,7 @@ Below is a list of the repositories and their sub-modules, and the language they - multisite-init-container (python) - dcae-remote-site (helm chart) + * dcaegen2.platform * dcaegen2.platform.blueprints @@ -77,9 +83,7 @@ Below is a list of the repositories and their sub-modules, and the language they * dcaegen2.platform.plugins - - cdap (Python) - dcae-policy (Python) - - docker (Python) - relationships (Python) - k8splugin (Python) @@ -87,6 +91,8 @@ Below is a list of the repositories and their sub-modules, and the language they * dcaegen2.platform.servicechange-handler (Clojure) +* dcaegen2.platform.ves-openapi-manager (Java) + * dcaegen2.utils - onap-dcae-cbs-docker-client (Python) diff --git a/docs/sections/configuration.rst b/docs/sections/configuration.rst index 575dcdc1..bd2608f9 100644 --- a/docs/sections/configuration.rst +++ b/docs/sections/configuration.rst @@ -80,3 +80,12 @@ The ``componentImages.ves`` value is provided in the ``oom/kubernetes/dcaegen2/c componentImages: ves: onap/org.onap.dcaegen2.collectors.ves.vescollector:1.5.4 + +Config maps +----------- + +During installation of DCAEGEN2 module two config maps are installed by default: dcae-external-repo-configmap-schema-map and dcae-external-repo-configmap-sa88-rel16. + +Config maps are used by DCAEGEN VES and VES OPEN API components. + +Instruction how to generate the content of config maps is described in `README <https://gerrit.onap.org/r/gitweb?p=oom/utils.git;a=blob_plain;f=external-schema-repo-generator/README.md;hb=refs/heads/master>`_ file.
\ No newline at end of file diff --git a/docs/sections/design-components/DCAE-MOD/DCAE-MOD-User-Guide.rst b/docs/sections/design-components/DCAE-MOD/DCAE-MOD-User-Guide.rst index d5e640c2..fb51d84a 100644 --- a/docs/sections/design-components/DCAE-MOD/DCAE-MOD-User-Guide.rst +++ b/docs/sections/design-components/DCAE-MOD/DCAE-MOD-User-Guide.rst @@ -130,9 +130,9 @@ design tool's controller settings. +------------------------+--------------------------------------------+ With Guilin release, OOM/ingress template has been updated to enable virtual host by default. -All MOD API's and UI access via ingress should use dcaemod.api.simpledemo.onap.org. +All MOD API's and UI access via ingress should use dcaemod.simpledemo.onap.org. -In order to access Design UI from local, add an entry for dcaemod.api.simpledemo.onap.org in /etc/hosts with the correct IP (any K8S node IP can be specified). +In order to access Design UI from local, add an entry for dcaemod.simpledemo.onap.org in /etc/hosts with the correct IP (any K8S node IP can be specified). Using DCAE MOD without an Ingress Controller @@ -170,7 +170,7 @@ Distribution target URL will be -Now let’s access the Nifi (DCAE designer) UI - http://dcaemod.api.simpledemo.onap.org/nifi +Now let’s access the Nifi (DCAE designer) UI - http://dcaemod.simpledemo.onap.org/nifi IPAddress is the host address or the DNS FQDN, if there is one, for one of the Kubernetes nodes. @@ -181,9 +181,18 @@ IPAddress is the host address or the DNS FQDN, if there is one, for one of the K Let's fetch the artifacts/ spec files -**Sample Component DCAE-VES-Collector :** https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json +**Component Spec for DCAE-VES-Collector :** https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json + +**Component Spec for DCAE-TCAgen2 :** https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/spec/vescollector-componentspec.json + +**VES 5.28.4 Data Format :** https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/data-formats/VES-5.28.4-dataformat.json + +**VES 7.30.2.1 Data Format :** https://git.onap.org/dcaegen2/collectors/ves/tree/etc/CommonEventFormat_30.2.1_ONAP.jsonormat.json + +**VES Collector Response Data Format :** https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/data-formats/ves-response.json + +**TCA CL Data Format :** https://git.onap.org/dcaegen2/analytics/tca-gen2/tree/dcae-analytics/dpo/dcaeCLOutput.json -**Sample Data Format :** https://git.onap.org/dcaegen2/collectors/ves/tree/dpo/data-formats/VES-5.28.4-dataformat.json For the purpose of onboarding, a Sample Request body should be of the type -:: @@ -218,10 +227,9 @@ curl -X POST http://<onboardingapi host>/onboarding/components -H "Conte In our case, -curl -X POST http://dcaemod.api.simpledemo.onap.org/onboarding/dataformats -H "Content-Type: application/json" -d @<filepath to request> - -curl -X POST http://dcaemod.api.simpledemo.onap.org/onboarding/components -H "Content-Type: application/json" -d @<filepath to request> +curl -X POST http://dcaemod.simpledemo.onap.org/onboarding/dataformats -H "Content-Type: application/json" -d @<filepath to request> +curl -X POST http://dcaemod.simpledemo.onap.org/onboarding/components -H "Content-Type: application/json" -d @<filepath to request> @@ -233,7 +241,7 @@ curl -X GET http://<IPAddress>/onboarding/components **f. Verify the genprocessor (which polls onboarding periodically to convert component specs to nifi processor), converted the component** -Open http://dcaemod.api.simpledemo.onap.org/nifi-jars in a browser. +Open http://dcaemod.simpledemo.onap.org/nifi-jars in a browser. These jars should now be available for you to use in the nifi UI as processors diff --git a/docs/sections/images/R8_architecture_diagram.png b/docs/sections/images/R8_architecture_diagram.png Binary files differindex 94235253..c29228b1 100644 --- a/docs/sections/images/R8_architecture_diagram.png +++ b/docs/sections/images/R8_architecture_diagram.png diff --git a/docs/sections/images/architecture_diagram b/docs/sections/images/architecture_diagram index 4ab70d75..b79a4b7a 100644 --- a/docs/sections/images/architecture_diagram +++ b/docs/sections/images/architecture_diagram @@ -1 +1 @@ -<mxfile host="app.diagrams.net" modified="2021-02-16T01:21:18.194Z" agent="5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" etag="fBCrF5w--oVR_YudS1rJ" version="14.3.2" type="device"><diagram id="tDj8KXzGa5euc6s5cB2h" name="Page-1">7V1bc6O4Ev41qZp5SIqrgEdf4snWJpPUeHbP7NMWNthmB4MX40yyv/5IGGEQMsg2ICUkfogRCIO6++tWd6t1pY7WL18ie7N6CB3Xv1Ik5+VKHV8pimzIJvyHWl73LaZu7RuWkeekFx0apt5/btoopa07z3G3hQvjMPRjb1NsnIdB4M7jQpsdReGv4mWL0C/+6sZeuqWG6dz2y63/85x4lbbKwDqcuHO95Sr9aVMx9ifWNr44fZPtynbCX7km9fZKHUVhGO+/rV9Gro8GD4/Lvt/kyNnswSI3iFk6jH48L5Z/L/+RbHkdjX6Aob+2r9O7PNv+Ln3hKwX48H7DRQhvC586fk2HAvy7C/GJ621CqAG8QAWbl8NJ+G2J/o9Hg1t4chDNV14M6bKLXHj46S4MQn/n7z7jX4GPu/+hfbd0pLLfVGL3BbWv4rUPG2T4dRtH4U93FPphBFuCMHDRM3m+TzTZvrcM4OEcDo8L24fPbhR7kLCD9MTacxz0M8Nf6BGnG3uOfvMXZGPYFoW7wHHRyEnZY+WHGo8bvKf7kmtKh/6LG67dOHqFl/w6cI0upaywyjGMnrbZKaMus64HWsIvKTlPIK1yMWnhKxdIC4VsP8T5RnVhow+FBR4GgydWOh/GWz5CkaMsAB/BVGYqAPDMMrIdDxIpd04dWxb8UuQR1Ecbyya4kLgvRRqmGGekh3nS6xTSa22RXqWQ/uiAS6cOOEXm4HiODPRpZTxlrTygcjZ6hRFV2hpSreEhJTk1HdQSa0+Sv+wM1kFamQCWNpxMGiKASRAAlAkAaBytmm2Nv16HZnRsmqNPAcPqUU+pUGhTN3r25tAg6Y3+Ss9aBEeYFJE0aBzRFkOA8xgi+SuT9/tocL10A6VFfYVFV6HrqaEKkH1QEutbyZgk7SVoUG7RpxnyluhrUehLM19a02Fmo/Qd31/f2QGUl4hZdN8biXVVu9EFI7J1sY0qa5fYqHehv2YH8+Y5QpIAoHGEBo1WOkfIsjaQGsL0jNYpPygUNU/lh9bmLHjW3z5DHNcDPLlBnwwnFG4YmdZoQuMGydSNpEcTKkARjhsu905cxg13rh3FM9eO+8oTKhCNJ7Aqyg266yzdaXoYRvEqXIaB7d8eWglr+HDNfRhuUmL848bxa+p+tHdxWCSV++LFP1B3qEH3R3/lzoxf0jsnB6/4IIDvu++k6Pg464YODv2SI9xxG0OuGyDfJTL/fXu79ea4eeL5+JmOkncb7qK5WyVXqX8A3nDpxlUXprNeNMKV7BK5vh17z0WfafN4QPNrdIkH08ev2IzsKyJoZtGG5I8IqskXEYyzIMEoQIJUAwkXyLrGKOvYoSWIrKsWX6oqZ1FVMJxnpb3VNOmTrvDV7NfcBZvQC+Jt7s5PqOEALUAtQouqEgEt4nqZcBMT18Mv+yc4MGH2KhfoIJojuEsd9PTtrq+6B8iiWaOqXBp0btaoxApSp1mjjr1dZTx0ARbprDanJAQYKYRvRNWqwUjR9KrrWwKj2qhI22D0cP1gbzb9NYdLOkjnDUkKZ8OJHZIuABPACia6UEatXBu0allex3ZsQzlBuThQIHx3Hof9FV0C4TValkG31oTyVkT36ES2K2vCeGPWhHGiNWHxsCYMzuh09+f1n7fTvuIRGX8RAI9UvniknAVIXPDIZMQjIAQcqRad1Y7BEXl9HXzh853Cl6rx5VajfW7tgEFxzFUUi7k2DahlnZQopA9jWdWLBokAykl/g8ZyV9JuMUq7Ipi0d5YPdizA+/XhCfb5jkQ0/eFZhE+WQOBwqq+woJlkTqEAwAA+gOGovEtvExjwc3MDhm+30++jMFj0VdDJ0JsAYm58iHmd9HJI8DprPqqTkf+a+SUAHOaXmdDwnYr0O95Gpp8JEG/j7HaQjDwQ3WjVUAQPntzIg6+O1pTxgifm/NPG3WWXkZrzlLPoDz0cvg9aG0KoIlU3CwCDF24fjdQQkR3i+pZUEecpjlbgw5rsyDfGhqYYbGhWslXpek3SObAhZxMckID4rhix8aTcy3Qf5yx7sxheuXlXug+7N0Sh9eXrbMjSMfvpycvXCevEBT6mt9kiJtqu7A1qnPvhzmGdxOSmHw6YAR1QpzfEFAbOupT5vJmJikpE83VarjKthIbV1kRF54vWTO4SRckEm1V865bOFFZZZFVWDksssmvSVwDJHZzyugzYmFuVAY9yHYqsdVkugaYKYYGQPhZLrTRAgHrR5bh2yFH7BlReX7Rv6nuTQrZH6LQXIWcNGEpaLaDSHUEK+pxYvUalAe9g8Bsr8GZwu5u59Wg72+PD/SxrsOc/lwlqPO5i30PilrQ7dvTzEd7GixPBvJH0YqNyhacSZVkSrnACKRsUbLco0A70lqBdo8VNSaw/YNrMD+c/aVozG3S5Au9IwKwc01oQK5THKw8ZbjsN68qzdCJ8IcsEAOxRuQQA9TcC3SKJzlDardeUVq1SlRuSRKy0ptyKZJu2qU3Lfjrq4ZdYjeOKInKmhD40/Jwo6NMMfhrEqpks9TXPIzKFSTKrunnrmHfqydjd+OHr2k1+q+dFBgzC+8U/2mxVL/RM5xVvKrzMYWKkUIEF/lUbZrVTJozw9enXjXt3L6thyDutJS1nCbuNVnawdD+gxxJuVRgQI+/gceMGgyc4jZQe7MBe9pdDZEm4hTpA4cwiT6HvzV/L6PGRISvLmnDcwnudH0qk9Jaw19ALHC9A3zJF1E8uIQ1e/mlNFsWv0KLBi78zBfcygzdv7uasX/4GL828LbPhBQYv83pDsaLIgPdyrt+CZzjYYfLKiTXTT7whrVz+eANq/S8n0Rn33W7s4ByuwmbNNv4SoQLR0jhvwezv2nPDhjSD+fOQwXs6Pba3q1loR05fmYK0YxTaNhHd8kTt7LmbjUOeoHpdhNGalTPezcYhBllCmuLrz0I0newcYvDNnr064tjVDrkvreSxieLvlaSRdaG/12BNrNPFMn8N3nkgo/sBWuD8kQnSVCaIYRpEjFg3FNxSUHp6GeFaywcxxEz1U3uBbw3EswzWws1Yb4qCb3xrTx7jO4MpP7zAPzhxhizG/e4YqPF1/mclm8oSkQ+qp5WVj9b+lsh1V8UO7Sx4wcPKTX9Px6MP7d2c9raIuQldd3eayoknzIJBaF9UtyRdirysdc5xzJY38soKYb/W7bogWyTydrDU0Dhz/9fGkBdHuD/At7ndB0lG4o++VnUxh8bT/PJwW7ueH4NrhqZpNwvUwGtyVGnudhYNzZw/TUZDTVZ3kFjbmVnV5SSazymVgFJgOElSzuM4UBdq581xeb6iz/Wa1/xvlAtN3pvqjdC6Xm+B3rjnmYUWkSpG3d/I7DJ6Zp60AOVkKmjlkR6YAGg6nTqTiTUEVokKGEpWtpMARWLtILDAwv7TjeerqwYjWmqBTDIOKOXIpNHMltYiWiYto6/3ZMIL3rN1YCZvMtVu/8I4icnSKLc79KMjf7eNO4HNt8oJRxZt8uMEzps0nVer/rStXnilW6aGHJcJhilWwTqr7EzkzkR95SHAumWRaDxUvQmMGCsfi/NTQ6BM8LZTYXCGSz1bNV5W5jK2otlCJJ+1WcjgnJUHbbgN8HsLUj/BUKwbK/dXrIapGtoNAMV7spZTMIxT79xydQU88tw8H8MhWlh5+5ys+ScXzD1F4dzdbj92mzhwEFmEnn/GuSxRUKuzgAHrMilhC6YRsajRqGGby2K1uZqvC3BWMNbisaO9VbsclBHUGODSrMiQL+Pcw+OYH9QNVaDTgqO3kjGhB0eVW/RpBupkmUybp/gssNsvD3XZcuMWsK7rGihEyddL6p0TESyRZgitgyBlhvB1pd8N9HB0/y9YfX92R8Nv69U1qyHackJKqaJ6uuXD0YSU0ua6QL9qHzUZSu7hLBA72TSsHpaY/KElpgFz050tSkxDcdU6tmsuGqpvK5Pb1erAoGRv0GYqRmte1be4JWBvnao4AY6D5+LMTDly11ylLlOOLP1xaoe6fZBkyeCxT7jEUITyKF8I4udQCaUByJLArG4NsrZw6UYtezFkqZyUnroUpLEd2/DfwofCrwB7jRROMNtuaBa0GAtOO4sJaoTkANaVyq1VoMyWiFWaFE0kMQsqggY5tCeI4I2U+1Oqb9u6QJZd21l9rHlSZm9bnuvagXOF9plGlZF2ERzWMMgkt+eSSgCsIVMkVe1WUhnybRoJ/wgiqUCtErCzVSeUzFNu27rclpPd8kV5+y2FBjHdE0FfskzBm1hu+Sak8GztWS2FnWtPjO45qh6q1DoeJJ832+3V44dECqYXZVqxnXP0YuoGFV0iTVNvQy9axkm3bV0ilRJVH4OkzNW+ducnx90iMdmbq5/7LpcW6VnoUlNWOfY7CPDRt30rFuM7Kej3UWmPMRZIS19uJBZI5Sjea4aeHqZ3fWUHjdyCyGLElya4Ybpe3P21lv9Qr1c/nD+WX3/Xpo/XZ5bYa7V05/j+Oq1FTnG2fJq4roMWTEwGztoLPlemWL0TCCoxGIUNj/Kc0mF9angYhYiWByMGDtnqIXRcdMX/AQ==</diagram></mxfile>
\ No newline at end of file +<mxfile host="app.diagrams.net" modified="2021-03-19T19:54:37.159Z" agent="5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36" etag="3AP2DIiOsC0p_-szBS0Z" version="14.4.3" type="device"><diagram id="tDj8KXzGa5euc6s5cB2h" name="Page-1">7V1Zl5u4Ev41fU7y0H1YBTx6aadzp7cTZ3IzT/dgQ9tMMHgA9zK//koYYRAyYBuQHNx+aCMQYKnqq0VVpSt5tHr/Epjr5YNv2e6VJFjvV/L4SpJEWVfgP9TysW3RZHHbsAgcK7lo1zB1/rWTRiFp3TiWHeYujHzfjZx1vnHue549j3JtZhD4b/nLXnw3/9S1ubALDdO56RZb/+tY0TJpFYGxO3FnO4tl8mhd0rYnVia+OPkl4dK0/LdMk3x7JY8C34+231bvI9tFg4fHZdtvsuds+mKB7UV1Oox+vr4s/rf4WzDFVTD6CYbuyrxO7vJqupvkB19JwIX3G7748LbwraOPZCjAPxsfn7gO44kawAtksH7fnYTfFuj/eDS4hScHwXzpRHBeNoENDz/d+Z7vbtzNZ/wU+LrbB227JSOVPlOK7HfUvoxWLmwQ4dcwCvxf9sh3/QC2eL5no3dyXJdoMl1n4cHDORweG7YPX+0gcuDEDpITK8ey0GOGb+gVp2tzjp75BskYtgX+xrNsNHJC+lrZocbjBu9pv2eakqH/YvsrOwo+4CVvO6pRhYQUlhmCUZM2MyHURdp1N5fwSzKdB0ytdPLUwp+cm1rIZNshzjbKLyb6UEjgYTB4rjvPu/EW98zIXhKAr6BLMxkAeGYRmJYDJylzTh4bBvySpxHURxmLOjhxct/zc4gxLjnMTr1KmXqlramXKVO/d8CFQwecwnNwPEca+rQynqJSHFAxHb3ciEptDanS8JCSlJoMaoG0J/FfegbLIKU4AYYynEwamgCdmABQnABAo2hZb2v81So0o2PTHH1yGFaNelKJQJvawaszhwpJb+RXctYgKEKnsKRGo4i2CEI/jiDiv+L0/vH8FXYZ+av1JjIjx/fg0cO0RemFGVmiS62hDJC2UGDyW0GbxO0FoJBu0ach/JXV/GwblNmmKTOtSTRsDpygzYhKLW1mD4F8Hw2uF7YnsSMJQVAnw0mRJJSRbowmNJIQdFWLezRAEqkZk5CERBEJVJJoTb8VT7ddTiOJO9sMopltRn2lCRnwRhMYfzKDblvQnk8O/SBa+gvfM93bXSshK3fX3Pv+OpmMv+0o+kicE+Ym8vNTZb870U/U/UZNjv7KnBm/J3eODz7wgQd/77aTpOLjtBs62PWLj3DHMIJUN0CeDaQcuGYYOnPcPHFc/E57pzf0N8HcLuOrxHqAN1zYUdmFiU6MRriUXALbhTL1Ne9RaR4PaFZPl3gwfXq8vjM9qJQFfUUERc8rDuwRQdbZIoJ2FCRoOUgQKiDhBF5XavI6Nnc54XXZYDur0lGzyhnONw7fcVf4yuZH5oK173hRmLnzM2rYQQYgbA1ZJtzYxPUi4Rwirodftm9Qs7dIAM+WE5JeBJWmY3KCkKL5kboUUs/f7voqnIDIm7qK18J4UFeFuih2mLpqmeEypaETwEqtq5QKXKCaJBA4o5SjmqSoZdfnUa0xMKp0qrYNRg/XD+Z63V99uSCOVNaQJDHWrOpD0glgAuqCicqV1ovfmxm/js3IhHyClvIhQ7j2PPL7y7oEwiu0RcputQnpXFh3r6XblTahnZk2oR2oTRgstAmNMTrd/bj+cctwzY6vBRoO8Ehmi0fSEYAkZtEo47Fp3vOmnxcEyQadvPZBEHl9FWTh88deLyWe51YhTlbYUrR2lIg9SMJ2QdEiX1p1ZSRJy3IrFloXhVpW80oLBwJMPUOFuituN2pyu8QZtxuMuX36+PAM+3xHLJo8eBbgkwUQ2J3qKywounLDHTCACzDs5XfhPIEBvzczYPh2O/0+8r2XvjI6uTzHAZtrFzav4l4GUWJH2a9kSHOVfQlAqT3ajn2ZMg1bU6Tfa3JkDBsHa3KM3Q6ClgWiG6UciuDBsx048KejtBVW8FQ7iBXwpYUwNjnzPtPd4e8x1xoXokhW9RzA4NzQvas5xOoPcX1LooixiaPk6LAixPLMyFDngwz1UrIqXK8IKgMyZKyCAxIQfytCNPiSfYxD9fX88srNbyX7sHuDl7k+PVmHrE6xNU/eHyd1DRf4ms46REQULs01apy7/saqa8RkzA8LzIAKqOYNYcJAq0uaz5sxVGRixV+lxTPTsvSNtgwVlS1a13KXSFLK2HXZtyr/JpeqkRZy2OVppNckPwHEd7CKyR2wMZPaAY8yHfKkdVr8kyJzoYGQPhZDLlVAgHzS5bg8wV79BpReX5FYUuhNMlnLiSVKJaDSHUES+hxYIEOmAe9g8LUu8KZwu5nZ1Wg72+LD/SxtMOe/FjFqPG0i10HsFrdbZvDrCd7GiWLGvBHUfKN0hU2JIi9RvVCQPgWKF2psGPQSDKKoDIRSeX6AN5ygJwq2GxRoB2pL0K7Q1k1JrN9h2sz1579oUjMddLEE70jALB3TShDLVeAqDhluOwzrilY6sXwhkpllW1QuAED1jUC3SKLWqB7V65mWDXJFujBFdeeacquOExJVWvTTXg+/UFc5LqlTpQvoQ8PPiYQ+zeCnRmTWpOGxWRoRKUSSatXNa8esQ0/G9tr1P1Z2/KyeVyrQCO8X+9VmozwZNLErzmp5mYFhJFGBBf6VK2aVJhNG+EpHC2jcu3sSVeH3Zhfvtq2YB7uNlqa3sC/QY3CXOQb4iDt4WtveIC7E92B65qK/FCIK3CXzAIkxiTz7rjP/KKLHJUJWFBXuqIV1LiAKpHQWsNfQ8SzHQ99SQdRPKiEVXvZhTQbFr9Ciwou/11rcSxXerLpbnQ/YocJLU2+LZHiCwls3nQvvhMCLwss6neur9woH249/cqzN9BNvSC2XPd6ASv/LQfOM+4Zr0zuGqrBaE0ZfAlRQXBhnNZjtXXuu2JBqMHsa0lib02MzXM58M7D6ShSkHiPRKtF3SxOV1nM3exM8Q/H64gerupTx2+xNoJF1qCm+/nSJppPNCTS20bNXexy7yi72pZU4Nl78vYIwMk7092p1A+tUvtRfjXUcyOh+gBKcL5EgTUWCaLpGrBGrmoRbckJPLSJca/EgGp+hfnIv8K2B9SytbuVvLDd5wTe29Sn30Z1WKz48Rz84cIas6P3bEVDjef5HBZuKAhEPqibVl/cWGhfIvKt8h3YSXvCwMpPf0/HoIr2bk94GYZvQZXenoZzYYOYMQvsiugXhVOStWwsdr9myRl5RIvTXqi0eRINE3g5SDbUjt5hsDHnxCvcFfJsCX1EgCYk9+hrlxRwaD/PLwm1lPv+eMqhQilXAa3xUqu52thqaOn+aXA3V67qD+NoTzSgvJ9F8TKkApBzBCYJ0HMWBqqV21hSXpSu6rde85D9TKtRZ78w3Qnm9zgv6xT2PLDTIssm0nGG9y9Uz/aAElINnQSmO9EAHQFHpszOZGENgFGYBQ8nStGKgiLUdBBaY2X/Z0Xx51eCKlpybJhEvKGWmSaGpLe1tt02L6Ov9NOGE9zQPTGc9TZVbxNQ0YtIwynCDHjpyN2HUCWyeKyXsSdpkRwmMN3I6rlb9YdvBsAq3TBQ5JgaGzlfBOqPoTGRORH2lIVB3WyPeaKh8oxg+Mh/z9qnGUSR426EwOMKlmqwaLytzGlnRdCGSztosZHBM5kEbbgP8uzmpn6BJxo2R+ctXw5Q15QaA/D3rllPQtEPv3HJ1BTzyzDwfwyFKrLx9jXP+yYS558Cf22F42W1iR0FkEXr2EeeiQEGtzhYM6qZJcVswjViLGo0a1rmMujpX83UBjlqMNcgdh7tYWjUq00FrgloNuNRLIuSLOPfwNGYHdUMZqLTF0VtBm9AXR6Vb9GkG6kSRDJun+Cyw2y8LdWm6cQtY13UNFKLk6yn1zokVLJ4shNZBkGIhPC7Vu4Hqj+7/Acvvr/Zo+G21vK6riLYckFKoqJ7fPLK6gww62G0Sj2qZ/YKjQMx407BqWKrlDy0QDZjr9uylQDQUV61l2vpLQ/VtRXJLWxVolOgNmqWiteZVPcctAXvrVMUBcAw8F0dGypG77EpVkXJk6Y9DO1TtgyQKGou9xIUaRSj30gUnfg6ZEBqALAlc161B1hYu3KhlL4YoFIPSE5eCMDYjE/57cSHzS8BcIYHjzcI1TYPmI+G0szVBheAcUDdTubUKlGmKWKlK0UQQM6csqJFDewAL3giZP6n8tq0zZNG1ndbHmsdl9sKirWt61hXaZxpVRtoEcFh9L+XcnnMqAbCaSOFUuVtOrRFv08jyDyecCuQyBjtadELOPOS2rfNtMdgtW5S331yoEeYeD/KyjgneRLrlWXDh0dKznAs7l54Y3TOzuqtSazlw+pzZZiseLxzJmVwUacV2jpGLiRuUd47UdbUNuWhoB922dY6UCrP65MVlrra1Oz9ZdojYZKuufu47XxqkZ6FLSVnm2O9ggY++7Vu+GN9Bi36XSns11wJp4cuNrAVSKYp1ztDzw/Sur+SgkFsQGTXxpQlqmK5e7v5aiX/K18uf1p+Lxz+U6dP1kSX2Wi3dOb6/TmqRU5wtnya2baGEicnAWjne59IQqwsEZeo4soi8+o81Xime+Phk/VjdGxvr/U3ymCPQjr6QBjS+nX4+Y9ooEAKFXPaLJ6FaOnVLG5W1ItreVcXdWqsDz3Q/QgeVD34I+0oeJHS0Ka7gYeCjqdjZP3DIlg++ZaMr/g8=</diagram></mxfile>
\ No newline at end of file diff --git a/docs/sections/installation_oom.rst b/docs/sections/installation_oom.rst index 443b970a..c2701c88 100644 --- a/docs/sections/installation_oom.rst +++ b/docs/sections/installation_oom.rst @@ -21,11 +21,10 @@ At deployment time, with a single **helm deploy** command, Helm resolves all the and invokes Kubernetes deployment operations for all the resources. All ONAP Helm charts are organized under the **kubernetes** directory of the **OOM** project, where roughly each ONAP component occupies a subdirectory. -DCAE charts are placed under the **dcaegen2** directory. +DCAE platform components are deployed using Helm charts under the **dcaegen2** directory. +Four DCAE services (the HV VES collector, the VES collector, the PNF Registration Handler, and the TCA (Gen 2) analytics service) are deployed using Helm charts under the **dcaegen2-services** directory. -All DCAE platform components have corresponding Helm chart which will be used to trigger the deployment. -All DCAE Services are deployed through Cloudify Blueprint. The default ONAP DCAE deployment includes small subset of DCAE services deployed through Bootstrap pod to meet -ONAP Integration usecases. Optionally operators can deploy on-demand other MS required for their usecases as described in :doc:`On-demand MS Installation +Other DCAE Services are deployed on-demand, after ONAP/DCAE installation, through Cloudify Blueprints. Operators can deploy on-demand other MS required for their usecases as described in :doc:`On-demand MS Installation <./installation_MS_ondemand>`. @@ -51,7 +50,19 @@ The dcaegen2 chart has the following sub-charts: * ``dcae-policy-handler``: deploys the DCAE policy handler service. * ``dcae-redis``: deploys the DCAE Redis cluster. * ``dcae-dashboard``: deploys the DCAE Dashboard for managing DCAE microservices deployments -* ``dcae-servicechange-handler``: deploys the DCAE service change handler service. A subchart (``dcae-inventory-api``) deploys the DCAE inventory API service. +* ``dcae-servicechange-handler``: deploys the DCAE service change handler service. +* ``dcae-inventory-api``: deploys the DCAE inventory API service. +* ``dcae-ves-openapi-manager``: deploys the DCAE service validator of VES_EVENT type artifacts from distributed services. + +The dcaegen2-services chart has the following sub-charts: + +* ``dcae-hv-ves-collector``: deploys the DCAE High-Volume VES collector service. +* ``dcae-ms-healthcheck``: deploys a health check component that tests the health of the 4 DCAE services deployed via Helm. +* ``dcae-prh``: deploys the DCAE PNF Registration Handler service. +* ``dcae-tcagen2``: deploys the DCAE TCA analytics service. +* ``dcae-ves-collector``: deploys the DCAE VES collector service. + +The dcaegen2-services sub-charts depend on a set of common templates, found under the ``common`` subdirectory under ``dcaegen2-services``. DCAE Deployment --------------- @@ -69,10 +80,12 @@ These include: * Deployment handler * Policy handler * Service change handler - * Inventory API service (launched as a subchart of service change handler) + * Inventory API service * Inventory postgres database service (launched as a dependency of the inventory API service) * DCAE postgres database service (launched as a dependency of the bootstrap service) * DCAE Redis cluster + * DCAE Mongo database service (launched as a dependency of the bootstrap service) + * VES OpenAPI Manager Some of the DCAE subcharts include an initContainer that checks to see if other services that they need in order to run have become ready. The installation @@ -85,7 +98,6 @@ directory, not as part of the DCAE chart hierarchy. The dcae-bootstrap service has a number of prerequisites because the subsequently deployed DCAE components depends on a number of resources having entered their normal operation state. DCAE bootstrap job will not start before these resources are ready. They are: - * dcae-cloudify-manager * consul-server * msb-discovery @@ -94,32 +106,10 @@ The dcae-bootstrap service has a number of prerequisites because the subsequentl * dcae-db * dcae-mongodb * dcae-inventory-api - + Additionaly tls-init-container invoked during component deployment relies on AAF to generate the required certificate hence AAF -must be enabled under OOM deployment configuration. - -Once started, the DCAE bootstrap service will call Cloudify Manager to deploy -a series of blueprints which specify the additional DCAE microservice components. -These blueprints use the DCAE Kubernetes plugin (``k8splugin``) to deploy -Docker images into the ONAP Kubernetes cluster. For each component, the plugin -creates a Kubernetes deployment and other Kubernetes resources (services, volumes, logging sidecar, etc.) -as needed. - -The DCAE bootstrap service creates the following Kubernetes deployments: - -* deploy/dep-dcae-hv-ves-collector -* deploy/dep-dcae-prh -* deploy/dep-dcae-tca-analytics -* deploy/dep-dcae-tcagen2 -* deploy/dep-dcae-ves-collector -* deploy/dep-holmes-engine-mgmt -* deploy/dep-holmes-rule-mgmt - -After deploying all of the blueprints, the DCAE bootstrap service -continues to run. The bootstrap container can be useful for -troubleshooting or for launching additional components. The bootstrap -container logs (accessed using the ``kubectl logs`` command) show the -details of all of the component deployments. +must be enabled under OOM deployment configuration. + DCAE Configuration ------------------ @@ -147,6 +137,9 @@ In addition, for DCAE components deployed through Cloudify Manager blueprints, t Now we walk through an example, how to configure the Docker image for the DCAE VESCollector, which is deployed by Cloudify Manager. +(*Note: Beginning with the Honolulu release, VESCollector is no longer deployed using Cloudify Manager. However, the example is still +useful for understanding how to deploy other components using a Cloudify blueprint.*) + In the `k8s-ves.yaml <https://git.onap.org/dcaegen2/platform/blueprints/tree/blueprints/k8s-ves.yaml>`_ blueprint, the Docker image to use is defined as an input parameter with a default value: .. code-block:: yaml @@ -154,7 +147,7 @@ In the `k8s-ves.yaml <https://git.onap.org/dcaegen2/platform/blueprints/tree/bl tag_version: type: string default: "nexus3.onap.org:10001/onap/org.onap.dcaegen2.collectors.ves.vescollector:1.5.4" - + The corresponding input file, ``https://git.onap.org/oom/tree/kubernetes/dcaegen2/components/dcae-bootstrap/resources/inputs/k8s-ves-inputs-tls.yaml``, it is defined again as: @@ -162,7 +155,7 @@ it is defined again as: {{ if .Values.componentImages.ves }} tag_version: {{ include "common.repository" . }}/{{ .Values.componentImages.ves }} {{ end }} - + Thus, when ``common.repository`` and ``componentImages.ves`` are defined in the ``values.yaml`` files, their values will be plugged in here and the resulting ``tag_version`` value @@ -182,22 +175,23 @@ DCAE Service Endpoints ---------------------- Below is a table of default hostnames and ports for DCAE component service endpoints in Kubernetes deployment: - ================== ================================= ====================================================== - Component Cluster Internal (host:port) Cluster external (svc_name:port) - ================== ================================= ====================================================== - VES dcae-ves-collector:8443 xdcae-ves-collector.onap:30417 - HV-VES dcae-hv-ves-collector:6061 xdcae-hv-ves-collector.onap:30222 - TCA dcae-tca-analytics:11011 xdcae-tca-analytics.onap:32010 - TCA-Gen2 dcae-tcagen2:9091 NA - PRH dcae-prh:8100 NA - Policy Handler policy-handler:25577 NA - Deployment Handler deployment-handler:8443 NA - Inventory inventory:8080 NA - Config binding config-binding-service:10000/10001 config-binding-service:30415 - DCAE Healthcheck dcae-healthcheck:80 NA - Cloudify Manager dcae-cloudify-manager:80 NA - DCAE Dashboard dcae-dashboard:8443 xdcae-dashboard:30419 - ================== ================================= ====================================================== + =================== ================================== ======================================================= + Component Cluster Internal (host:port) Cluster external (svc_name:port) + =================== ================================== ======================================================= + VES dcae-ves-collector:8443 dcae-ves-collector.onap:30417 + HV-VES dcae-hv-ves-collector:6061 dcae-hv-ves-collector.onap:30222 + TCA-Gen2 dcae-tcagen2:9091 NA + PRH dcae-prh:8100 NA + Policy Handler policy-handler:25577 NA + Deployment Handler deployment-handler:8443 NA + Inventory inventory:8080 NA + Config binding config-binding-service:10000/10001 NA + DCAE Healthcheck dcae-healthcheck:80 NA + DCAE MS Healthcheck dcae-ms-healthcheck:8080 NA + Cloudify Manager dcae-cloudify-manager:80 NA + DCAE Dashboard dashboard:8443 dashboard:30418 + DCAE mongo dcae-mongo-read:27017 NA + =================== ================================== ======================================================= In addition, a number of ONAP service endpoints that are used by DCAE components are listed as follows for reference by DCAE developers and testers: @@ -205,10 +199,10 @@ for reference by DCAE developers and testers: ==================== ============================ ================================ Component Cluster Internal (host:port) Cluster external (svc_name:port) ==================== ============================ ================================ - Consul Server consul-server:8500 consul-server:30270 + Consul Server consul-server-ui:8500 NA Robot robot:88 robot:30209 TCP - Message router message-router:3904 message-router:30227 - Message router message-router:3905 message-router:30226 + Message router message-router:3904 NA + Message router message-router:3905 message-router-external:30226 Message router Kafka message-router-kafka:9092 NA MSB Discovery msb-discovery:10081 msb-discovery:30281 Logging log-kibana:5601 log-kibana:30253 @@ -250,4 +244,8 @@ DCAE components deployed by Cloudify Manager are uninstalled: The DCAE cleanup script uses Cloudify Manager and the DCAE Kubernetes plugin to instruct Kubernetes to delete the components deployed by Cloudify Manager. This includes the components deployed when the DCAE bootstrap -service ran and any components deployed after bootstrap.
\ No newline at end of file +service ran and any components deployed after bootstrap. + +To undeploy the DCAE services deployed via Helm (the hv-ves-collector, ves-collector, tcagen2, +and prh), use the ``helm undeploy`` command against the `top_level_release_name`-``dcaegen2-services`` +Helm sub-release.
\ No newline at end of file diff --git a/docs/sections/release-notes.rst b/docs/sections/release-notes.rst index 38012811..277367fe 100644 --- a/docs/sections/release-notes.rst +++ b/docs/sections/release-notes.rst @@ -14,6 +14,247 @@ DCAE Release Notes .. +.. ========================== +.. * * * HONOLULU * * * +.. ========================== + + +Version: 8.0.0 +============== + + +Abstract +-------- + +This document provides the release notes for Honolulu release. + +Summary +------- + +Following DCAE components are available with default ONAP/DCAE installation. + + - Platform components + + - Cloudify Manager (helm chart) + - Bootstrap container (helm chart) + - Configuration Binding Service (helm chart) + - Deployment Handler (helm chart) + - Policy Handler (helm chart + - Service Change Handler (helm chart) + - Inventory API (helm chart) + - Dashboard (helm chart) + - VES OpenAPI Manager (helm chart) + + - Service components + + - VES Collector (helm chart & cloudify blueprint) + - HV-VES Collector (helm chart & cloudify blueprint) + - PNF-Registration Handler (helm chart & cloudify blueprint) + - Docker based Threshold Crossing Analytics (TCA-Gen2) (helm chart & cloudify blueprint) + - Holmes Rule Management (helm chart & cloudify blueprint) + - Holmes Engine Management (helm chart & cloudify blueprint) + + - Additional resources that DCAE utilizes deployed using ONAP common charts: + + - Postgres Database + - Mongo Database + - Consul Cluster + +Below service components (mS) are available to be deployed on-demand (through Cloudify Blueprint) + + - SNMPTrap Collector + - RESTConf Collector + - DataFile Collector + - PM-Mapper + - BBS-EventProcessor + - VES Mapper + - Heartbeat mS + - SON-Handler + - PM-Subscription Handler + - DataLake Handler (Admin and Feeder) + - Slice Analysis mS + - DataLake Extraction Service + - KPI-Ms + + Notes: + + \* These components are delivered by the Holmes project. + + + +Under OOM (Kubernetes) all DCAE component containers are deployed as Kubernetes Pods/Deployments/Services into Kubernetes cluster. DCAE components are deployed using combination of Helm charts and Cloudify blueprint as noted above. DCAE provides a Cloudify Manager plugin (k8splugin) that is capable of expanding a Cloudify blueprint node specification for a service component to a full Kubernetes specification, with additional enhancements such as replica scaling, sidecar for logging to ONAP ELK stack, registering services to MSB, etc. + + +Release Data +------------ + ++--------------------------------------+--------------------------------------------------------+ +| **DCAE Project** | | +| | | ++--------------------------------------+--------------------------------------------------------+ +| **Docker images** | Refer :any:`Deliverable <honolulu_deliverable>` | +| | | ++--------------------------------------+--------------------------------------------------------+ +| **Release designation** | 8.0.0 Honolulu | +| | | ++--------------------------------------+--------------------------------------------------------+ +| **Release date** | 2021-04-01 | +| | | ++--------------------------------------+--------------------------------------------------------+ + + +New features +------------ + +- DCAE Enhancements + + +Functional Updates + + - (New) VES-Openapi-Manager component added to DCAE, allowing to notify of missing openAPI description, at xNF distribution phase + - Added VES 7.2.1 support in VESCollector + - DCAE MS deployment through helm (common template defined for easier migration of DCAE services) + - (New) KPI-Computation MS introduced for support for E2E Slicing Usecase + - K8S configMap support through onboarding/design/deployment via DCAE-MOD and DCAE-Platform + - BP-generation Enhancements (Native-kafka support/Config-map) + - CFY plugin enhancements (support IPV6 service exposure + Config-Map + Cert-Manager's CMPv2 issuer integration) + - DCAE SDK enhancement (Dmaap Client update for timeout/retry + CBS client update) + - DFC enhancement - support in HTTP/HTTPS/enroll certificate from CMPv2 server + +Non-Functional + - DCAE Cloudify py3 upgrade (including plugins/bootstrap cli) + - CII Badging improvements + - Policy-Handler Py3 upgrade + - Vulnerability updates for several DCAE MS (DataFile Collector, RESTConf, VESCollector, InventoryAPI, MOD/RuntimeAPI, VES-mapper, PM-Mapper, PRH, SON-Handler) + - Code Coverage improvement (DataFile, SDK, Blueprint-generator, Plugins, Acumos Adapter) + - Documentation/user-guide updates + +Bug Fixes + + - BPGenerator yaml Fixes are different for yaml file and string (DCAEGEN2-2489) + - Slice Analysis - Avoid removal of data when insufficient samples are present (DCAEGEN2-2509) + + +- Following new services are delivered this release + - VES OpenAPI Manager + - Analytics/RCA + - KPI MS + +.. _honolulu_deliverable: + +Deliverables +------------ + +Software Deliverables + +.. csv-table:: + :header: "Repository", "SubModules", "Version & Docker Image (if applicable)" + :widths: auto + + "dcaegen2/analytics/tca-gen2", "", "onap/org.onap.dcaegen2.analytics.tca-gen2.dcae-analytics-tca-web:1.2.1" + "dcaegen2/collectors/datafile", "", "onap/org.onap.dcaegen2.collectors.datafile.datafile-app-server:1.5.5" + "dcaegen2/collectors/hv-ves", "", "onap/org.onap.dcaegen2.collectors.hv-ves.hv-collector-main:1.5.0" + "dcaegen2/collectors/restconf", "", "onap/org.onap.dcaegen2.collectors.restconfcollector:1.2.4" + "dcaegen2/collectors/snmptrap", "", "onap/org.onap.dcaegen2.collectors.snmptrap:2.0.4" + "dcaegen2/collectors/ves", "", "onap/org.onap.dcaegen2.collectors.ves.vescollector:1.8.0" + "dcaegen2/deployments", "cm-container", "onap/org.onap.dcaegen2.deployments.cm-container:4.4.2" + "dcaegen2/deployments", "consul-loader-container", "onap/org.onap.dcaegen2.deployments.consul-loader-container:1.1.0" + "dcaegen2/deployments", "dcae-k8s-cleanup-container", "onap/org.onap.dcaegen2.deployments.dcae-k8s-cleanup-container:1.0.0" + "dcaegen2/deployments", "healthcheck-container", "onap/org.onap.dcaegen2.deployments.healthcheck-container:2.1.0" + "dcaegen2/deployments", "tls-init-container", "onap/org.onap.dcaegen2.deployments.tls-init-container:2.1.0" + "dcaegen2/deployments", "dcae-services-policy-sync", "onap/org.onap.dcaegen2.deployments.dcae-services-policy-sync:1.0.0" + "dcaegen2/platform", "mod/onboardingapi", "onap/org.onap.dcaegen2.platform.mod.onboardingapi:2.12.5" + "dcaegen2/platform", "mod/distributorapi", "onap/org.onap.dcaegen2.platform.mod.distributorapi:1.1.0" + "dcaegen2/platform", "mod/designtool", "onap/org.onap.dcaegen2.platform.mod.designtool-web:1.0.2" + "dcaegen2/platform", "mod/genprocessor", "onap/org.onap.dcaegen2.platform.mod.genprocessor-http:1.0.2" + "dcaegen2/platform", "mod/genprocessor", "onap/org.onap.dcaegen2.platform.mod.genprocessor-job:1.0.2" + "dcaegen2/platform", "mod/designtool/mod-registry", "onap/org.onap.dcaegen2.platform.mod.mod-registry:1.0.0" + "dcaegen2/platform", "mod/runtimeapi", "onap/org.onap.dcaegen2.platform.mod.runtime-web:1.2.3" + "dcaegen2/platform", "adapter/acumos", "onap/org.onap.dcaegen2.platform.adapter.acumos:1.0.4" + "dcaegen2/platform/blueprints", "", "onap/org.onap.dcaegen2.deployments.k8s-bootstrap-container:3.0.3" + "dcaegen2/platform/configbinding", "", "onap/org.onap.dcaegen2.platform.configbinding:2.5.3" + "dcaegen2/platform/deployment-handler", "", "onap/org.onap.dcaegen2.platform.deployment-handler:4.4.1" + "dcaegen2/platform/inventory-api", "", "onap/org.onap.dcaegen2.platform.inventory-api:3.5.2" + "dcaegen2/platform/policy-handler", "", "onap/org.onap.dcaegen2.platform.policy-handler:5.1.2" + "dcaegen2/platform/servicechange-handler", "", "onap/org.onap.dcaegen2.platform.servicechange-handler:1.4.0" + "dcaegen2/platform/ves-openapi-manager", "", "onap/org.onap.dcaegen2.platform.ves-openapi-manager:1.0.1" + "dcaegen2/services", "components/datalake-handler", "onap/org.onap.dcaegen2.services.datalakefeeder:1.1.0" + "dcaegen2/services", "components/datalake-handler", "onap/org.onap.dcaegen2.services.datalakeadminui:1.1.0" + "dcaegen2/services", "components/datalake-handler", "onap/org.onap.dcaegen2.services.datalake.exposure.service:1.1.0" + "dcaegen2/services", "components/pm-subscription-handler", "onap/org.onap.dcaegen2.services.pmsh:1.1.2" + "dcaegen2/services", "components/slice-analysis-ms", "onap/org.onap.dcaegen2.services.components.slice-analysis-ms:1.0.4" + "dcaegen2/services", "components/bbs-event-processor", "onap/org.onap.dcaegen2.services.components.bbs-event-processor:2.0.1" + "dcaegen2/services", "components/kpi-ms", "onap/org.onap.dcaegen2.services.components.kpi-ms:1.0.0" + "dcaegen2/services/heartbeat", "", "onap/org.onap.dcaegen2.services.heartbeat:2.1.1" + "dcaegen2/services/mapper", "", "onap/org.onap.dcaegen2.services.mapper.vesadapter.universalvesadaptor:1.2.0" + "dcaegen2/services/pm-mapper", "", "onap/org.onap.dcaegen2.services.pm-mapper:1.5.2" + "dcaegen2/services/prh", "", "onap/org.onap.dcaegen2.services.prh.prh-app-server:1.5.6" + "dcaegen2/services/son-handler", "", "onap/org.onap.dcaegen2.services.son-handler:2.1.3" + "dcaegen2/platform", "mod/bpgenerator", "Blueprint Generator 1.7.3 (jar)" + "dcaegen2/services/sdk", "", "DCAE SDK 1.7.0 (jar)" + "ccsdk/dashboard", "", "onap/org.onap.ccsdk.dashboard.ccsdk-app-os:1.4.0" + + +Known Limitations, Issues and Workarounds +----------------------------------------- + +None + + +*System Limitations* + +None + +*Known Vulnerabilities* + +None + +*Workarounds* + +Documented under corresponding jira if applicable. + +Security Notes +-------------- + +*Fixed Security Issues* + Listed above + +*Known Security Issues* + None + +*Known Vulnerabilities in Used Modules* + None + +DCAE code has been formally scanned during build time using NexusIQ and all Critical vulnerabilities have been addressed, items that remain open have been assessed for risk and determined to be false positive. The DCAE open Critical security vulnerabilities and their risk assessment have been documented as part of the `project <https://wiki.onap.org/pages/viewpage.action?pageId=51282478>`_. + +Test Results +------------ + + - `DCAE R8 Pairwise Test <https://wiki.onap.org/display/DW/DCAE+Pair+Wise+Testing+for+Honolulu+Release>`_ + - `DCAE R8 Functional Test <https://wiki.onap.org/display/DW/DCAE+R8+Testplan>`_ + + +References +---------- + +For more information on the ONAP Honolulu release, please see: + +#. `ONAP Home Page`_ +#. `ONAP Documentation`_ +#. `ONAP Release Downloads`_ +#. `ONAP Wiki Page`_ + + +.. _`ONAP Home Page`: https://www.onap.org +.. _`ONAP Wiki Page`: https://wiki.onap.org +.. _`ONAP Documentation`: https://docs.onap.org +.. _`ONAP Release Downloads`: https://git.onap.org + +Quick Links: + + - `DCAE project page <https://wiki.onap.org/display/DW/Data+Collection+Analytics+and+Events+Project>`_ + - `Passing Badge information for DCAE <https://bestpractices.coreinfrastructure.org/en/projects/1718>`_ + .. =================================== .. * * * GUILIN MAINTENANCE * * * diff --git a/docs/sections/services/serviceindex.rst b/docs/sections/services/serviceindex.rst index 10c2bd1b..3102b953 100644 --- a/docs/sections/services/serviceindex.rst +++ b/docs/sections/services/serviceindex.rst @@ -7,22 +7,46 @@ DCAE Service components .. Add or remove sections below as appropriate for the platform component.
+
+Collectors
+----------
+
.. toctree::
:maxdepth: 1
+ ./dfc/index.rst
+ ./restconf/index.rst
./snmptrap/index.rst
- ./ves-http/index.rst
./ves-hv/index.rst
+ ./ves-http/index.rst
+
+Event Processor
+---------------
+.. toctree::
+ :maxdepth: 1
+
+ ./bbs-event-processor/index.rst
+ ./datalake-handler/index.rst
./mapper/index.rst
- ./prh/index.rst
- ./dfc/index.rst
- ./heartbeat-ms/index.rst
./pm-mapper/index.rst
+
+Analytics
+---------
+
+.. toctree::
+ :maxdepth: 1
+
+ ./heartbeat-ms/index.rst
+ ./kpi-computation-ms/index.rst
./pm-subscription-handler/index.rst
- ./bbs-event-processor/index.rst
+ ./slice-analysis-ms/index.rst
./son-handler/index.rst
- ./restconf/index.rst
./tcagen2-docker/index.rst
- ./datalake-handler/index.rst
- ./slice-analysis-ms/index.rst
- ./kpi-computation-ms/index.rst
\ No newline at end of file +
+Miscellaneous Services
+----------------------
+
+.. toctree::
+ :maxdepth: 1
+
+ ./ves-openapi-manager/index.rst
diff --git a/docs/sections/services/ves-openapi-manager/architecture.rst b/docs/sections/services/ves-openapi-manager/architecture.rst new file mode 100644 index 00000000..2e4c4276 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/architecture.rst @@ -0,0 +1,30 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-architecture: + +VES OpenAPI Manager architecture +================================ +Functionalities of VES OpenAPI Manager require communication with other ONAP components. Because of that, SDC +Distribution Client has been used as a library to achieve such a communication. There are two components required by +application to work: SDC BE and Message Router. SDC Distribution Client provides communication with both of them when +it's properly configured (for application configuration instruction refer to: :ref:`ves-openapi-manager-deployment` ). + +.. image:: resources/architecture.png + + +.. _ves-openapi-manager-flow: + +VES OpenAPI Manager workflow +---------------------------- +VES OpenAPI Manager workflow can be split into phases: + +1) Listening for Service Model distribution events +2) Optional downloading of artifacts depending on Service Model contents. At least one Service Model resource must contain *VES_EVENT* type artifacts. +3) Optional validation of artifacts depending on content of downloaded artifacts. Artifact must contain stndDefined events declaration. + + +VES OpenAPI Manager workflow is presented on the diagram below. + +.. image:: resources/workflow.png + diff --git a/docs/sections/services/ves-openapi-manager/artifacts.rst b/docs/sections/services/ves-openapi-manager/artifacts.rst new file mode 100644 index 00000000..f42a470a --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/artifacts.rst @@ -0,0 +1,26 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-artifacts: + +VES OpenAPI Manager artifacts and delivery +========================================== +VES OpenAPI Manager is delivered as a docker container and published in ONAP Nexus repository following image naming +convention. + +Image +----- +Full image name is *onap/org.onap.dcaegen2.platform.ves-openapi-manager*. + +Versioning +---------- +VES OpenAPI Manager keeps its Changelog in the repository. It's available here: +`Changelog <https://github.com/onap/dcaegen2-platform-ves-openapi-manager/blob/master/Changelog.md>`_ + +Use *latest* image tag to get the most recent version of VES OpenAPI Manager. + +Repository +---------- +Repository with the code of VES OpenAPI Manager is available on ONAP Gerrit: +`Gerrit <https://gerrit.onap.org/r/admin/repos/dcaegen2/platform/ves-openapi-manager>`_ + diff --git a/docs/sections/services/ves-openapi-manager/deployment.rst b/docs/sections/services/ves-openapi-manager/deployment.rst new file mode 100644 index 00000000..fdec791a --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/deployment.rst @@ -0,0 +1,69 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-deployment: + +VES OpenAPI Manager deployment +============================== +VES OpenAPI Manager is a simple Java application which can be started by using only Java 11+, yet it has some +prerequisites to work correctly: + +1) File with OpenAPI schemas mappings. +2) Access to two ONAP services: SDC BE and Message Router. + +These prerequisites are met by default when using Helm charts created for VES OpenAPI Manager in OOM. It's described in +more detail in *Helm chart* section. + +There is also available a simple configuration via environment variables which are optional. It's described in more +detail in *Environment variables* section. + +File with OpenAPI schemas mappings +---------------------------------- +VES OpenAPI checks whether schemaReferences of distributed service align with stndDefined schemas from VES Collector. +To achieve that application should receive a file with mappings used by VES. Because there are few ways to run the +application, it contains its own default file which assure that application will work. Default file may be overwritten +or edited any time, even during application runtime. + +Helm charts which are used to deploy application in ONAP cluster are configured to overwrite default mapping file with +file from predefined ConfigMap (named *dcae-external-repo-configmap-schema-map*), which is also used by VES Collector. +Using ConfigMap ensures that both: VES OpenAPI Manager and VES Collector use the exact same file. + +.. warning:: + VES OpenAPI Manager does not check if the used mapping file is the same file that VES uses. Within ONAP, the working + assumption is, that both: VES openAPI Manager and VES Collector leverage the same Kubernetes ConfigMaps, which + contain the schema-mapping file and respective openAPI descriptions. + +VES OpenAPI Manager has a configurable property which contains path to the mapping file. It has to be set before the +application startup. It can be done by setting environment variable *SCHEMA_MAP_PATH*. Helm charts are preconfigured to +set this variable. + +Environment variables +--------------------- +There are environment variables which must be used for configuration. Helm chart contain predefined values which are +valid when running VES OpenAPI Manager from its released image in the ONAP cluster. + ++-----------------+---------------------------+----------------------+ +| Variable name | Description | Helm chart values | ++=================+===========================+======================+ +| SCHEMA_MAP_PATH | Path to the mapping file. | /app/schema-map.json | ++-----------------+---------------------------+----------------------+ +| ASDC_ADDRESS | URL to SDC BE. | sdc-be:8443 | ++-----------------+---------------------------+----------------------+ + + +Helm chart +---------- +By default VES OpenAPI Manger is deployed via Helm as the DCAE subcomponent in the ONAP cluster. Helm chart is +configured to deploy application with all prerequisites met. It achieves that by: + +1) Mounting ConfigMap with mapping file under */app/schema-map.json* path. +2) Proper setting environment variables to values described in section *Environment variables*. Mapping file path is set to point to mounted file and SDC BE URL is set to internal port available only from Kubernetes cluster. +3) Setting Readiness check. It waits for other ONAP components to start: SDC BE, Message Router. VES OpenAPI Manager Pod will not start until they are not ready. + +Local deployment +---------------- +It's possible to run VES OpenAPI Manager in local environment which connects to external lab with ONAP. This way +requires exposing ports of some services on lab, creating local port tunneling and running VES OpenAPI Manager (using +docker-compose or IDE e.g. IntelliJ). + +It's described in more detail in the README in project repository (`README <https://github.com/onap/dcaegen2-platform-ves-openapi-manager/blob/master/README.md>`_).
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/index.rst b/docs/sections/services/ves-openapi-manager/index.rst new file mode 100644 index 00000000..e6cc15c3 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/index.rst @@ -0,0 +1,27 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + + +VES OpenAPI Manager +=================== + +VES OpenAPI Manager has been created to validate the presence of OpenAPI schemas declared in *VES_EVENT* type artifacts, +within the DCAE run-time environment during Service Model distribution in SDC. When deployed, it automatically listens +to events of Service Models distributions by using SDC Distribution Client in order to read the declared OpenAPI +descriptions. Purpose of this component is to partially validate artifacts of type *VES_EVENT* from Resources of +distributed services. During validation phase it checks whether *stndDefined* events defined in VES_EVENT type artifact, +contain only *schemaReferences* that local copies are accessible by DCAE VES Collector. If any of schemaReference is +absent in local externalSchema repository, the VES OpenAPI Manager informs ONAP user which schemas need to be uploaded +to the DCAE run-time environment. + + +VES OpenAPI Manager overview and functions +------------------------------------------ + +.. toctree:: + :maxdepth: 1 + + ./architecture.rst + ./artifacts.rst + ./deployment.rst + ./use-cases.rst diff --git a/docs/sections/services/ves-openapi-manager/resources/architecture.png b/docs/sections/services/ves-openapi-manager/resources/architecture.png Binary files differnew file mode 100644 index 00000000..5df1198f --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/architecture.png diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml new file mode 100644 index 00000000..73c6919e --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-no-stndDefined.yaml @@ -0,0 +1,37 @@ +--- +event: + presence: required + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: notification} + eventName: {presence: required, value: Noti_MyPnf-Acme_FileReady} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.0.1} + vesEventListenerVersion: {presence: required, value: 7.0.1} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + notificationFields: + presence: required + structure: + changeIdentifier: {presence: required, value: PM_MEAS_FILES} + changeType: {presence: required, value: fileReady} + notificationFieldsVersion: {presence: required, value: 2.0} + arrayOfNamedHashMap: + presence: required + array: + - name: {presence: required} + hashMap: {presence: required, structure: { + keyValuePair: {presence: required, structure: {key: {presence: required, value: location}, value: {presence: required}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: compression}, value: {presence: required, value: gzip}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatType}, value: {presence: required, value: org.3GPP.32.435}}}, + keyValuePair: {presence: required, structure: {key: {presence: required, value: fileFormatVersion}, value: {presence: required, value: V10}}}} + } +...
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml new file mode 100644 index 00000000..6bba640e --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined-no-schemaReference.yaml @@ -0,0 +1,29 @@ +--- +event: + presence: required + comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification" + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: stndDefined} + eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.1} + vesEventListenerVersion: {presence: required, value: 7.2} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"} + stndDefinedFields: + presence: required + structure: + data: {presence: required} + stndDefinedFieldsVersion: {presence: required, value: "1.0"} + +...
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml new file mode 100644 index 00000000..ad3ca469 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/artifact-stndDefined.yaml @@ -0,0 +1,30 @@ +--- +event: + presence: required + comment: "stndDefined event to support 3GPP FaultSupervision NotifyNewAlarm notification" + structure: + commonEventHeader: + presence: required + structure: + domain: {presence: required, value: stndDefined} + eventName: {presence: required, value: stndDefined-gNB-Nokia-Notification} + priority: {presence: required, value: Normal} + eventId: {presence: required} + reportingEntityId: {presence: required} + reportingEntityName: {presence: required} + sequence: {presence: required, value: 0} + sourceId: {presence: required} + sourceName: {presence: required} + version: {presence: required, value: 4.1} + vesEventListenerVersion: {presence: required, value: 7.2} + startEpochMicrosec: {presence: required} + lastEpochMicrosec: {presence: required} + stndDefinedNamespace: {presence: required, value: "3GPP-FaultSupervision"} + stndDefinedFields: + presence: required + structure: + schemaReference: { presence: required, value: "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm" } + data: {presence: required} + stndDefinedFieldsVersion: {presence: required, value: "1.0"} + +...
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json new file mode 100644 index 00000000..e11851c0 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-example.json @@ -0,0 +1,14 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/comDefs.yaml" + }, + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/coslaNrm.yaml" + }, + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml" + } +]
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json new file mode 100644 index 00000000..fb34f5ae --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map-invalid.json @@ -0,0 +1,6 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/streamingDataMnS.yaml" + } +]
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/schema-map.json b/docs/sections/services/ves-openapi-manager/resources/schema-map.json new file mode 100644 index 00000000..97883fbc --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/schema-map.json @@ -0,0 +1,6 @@ +[ + { + "publicURL": "https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml", + "localURL": "3gpp/rep/sa5/MnS/tree/SA88-Rel16/OpenAPI/faultMnS.yaml" + } +]
\ No newline at end of file diff --git a/docs/sections/services/ves-openapi-manager/resources/workflow.png b/docs/sections/services/ves-openapi-manager/resources/workflow.png Binary files differnew file mode 100644 index 00000000..423792b6 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/resources/workflow.png diff --git a/docs/sections/services/ves-openapi-manager/use-cases.rst b/docs/sections/services/ves-openapi-manager/use-cases.rst new file mode 100644 index 00000000..aa3a4bf8 --- /dev/null +++ b/docs/sections/services/ves-openapi-manager/use-cases.rst @@ -0,0 +1,117 @@ +.. This work is licensed under a Creative Commons Attribution 4.0 International License. +.. http://creativecommons.org/licenses/by/4.0 + +.. _ves-openapi-manager-use-cases: + +VES OpenAPI Manager validation use-cases +======================================== +The main VES OpenAPI Manager use case is to verify if the schemaReferences declared in *VES_EVENT* type artifacts are +present in the local DCAE run-time externalSchemaRepo and show validation results to user in SDC UI. + +The general flow of VES OpenAPI Manager is available here :ref:`ves-openapi-manager-flow`. + +Based on the referenced flow, there are few possible behaviours of VES OpenAPI Manager. In this section two main flows: +successful and unsuccessful validation will be described step by step. + +Validation prerequisites +------------------------ +Validation phase takes place only when specific conditions are met. + +1) VES OpenAPI Manager is properly configured: client is connected to SDC and mapping file is present and pointed in configuration. Configuration is described in detail here: :ref:`ves-openapi-manager-deployment`. +2) Distribution of a Service Model takes place in SDC. +3) Service contains an *VES_EVENT* type artifact. +4) Artifact content is correctly downloaded. + +Validation description +---------------------- +When *schemaReference* field from artifact is being validated, only the part of the URI that indicates public openAPI +description file location is taken into consideration. + +For example when *schemaReference* with value +*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml#/components/schemas/NotifyNewAlarm* +is found in artifact, then only the part before # sign (public openAPI description file location URI part) is being +validated. This way part which would be validated is +*https://forge.3gpp.org/rep/sa5/MnS/blob/SA88-Rel16/OpenAPI/faultMnS.yaml*. + +Mapping file must have a predefined JSON format of list of objects (mappings) with publicURL and localURL fields. +Example with 3 mappings: + +.. literalinclude:: resources/schema-map-example.json + :language: json + +When *schemaReference* is split, it's compared to each publicURL from mapping file. If there is no publicURL in mapping +file which matches schemaReference, then schemaReference is marked as invalid. This process is executed for all +stndDefined events defined in *VES_EVENT* artifact, which declare a schemaReference. All invalid references are returned +to user via SDC UI when validation of a complete artifact ends. + +Based on returned information with invalid references user can take action and e.g. add mappings and schemas to DCAE +run-time environment by editing ConfigMaps which store them. + ++-----------------------------------------+-------------------------+ +| ConfigMap name | Content | ++=========================================+=========================+ +| dcae-external-repo-configmap-schema-map | Mapping file | ++-----------------------------------------+-------------------------+ +| dcae-external-repo-configmap-sa88-rel16 | OpenAPI schemas content,| +| | example stores 3GPP | +| | sa88-rel16 schemas | ++-----------------------------------------+-------------------------+ + + +Successful validation case +-------------------------- +There are few ways to get a successful validation status - *DEPLOY_OK*. + +1) When artifact *VES_EVENT* does not contain *stndDefined* events definitions. Only *stndDefined* event are validated. +2) When artifact *VES_EVENT* contains *stndDefined* events definitions but *schemaReference* fields are not present. +3) When artifact *VES_EVENT* contains *stndDefined* events definitions and each *schemaReference* of the event is present in the mapping file. + + +*VES_EVENT* artifact may contain more than one event definition. Examples of valid artifacts with single events are +below. + +Example of valid artifact without *stndDefined* event definition (case 1): + +.. literalinclude:: resources/artifact-no-stndDefined.yaml + :language: yaml + +Example of valid artifact with *stndDefined* event definition, but without schemaReference field (case 2): + +.. literalinclude:: resources/artifact-stndDefined-no-schemaReference.yaml + :language: yaml + +Example of artifact with *stndDefined* event definition (case 3): + +.. literalinclude:: resources/artifact-stndDefined.yaml + :language: yaml + +which is valid when mapping file contains a mapping of schemaReference field. +Example of mapping file content which makes example artifact valid: + +.. literalinclude:: resources/schema-map.json + :language: json + +Unsuccessful validation case +---------------------------- +Another case is an unsuccessful validation case which sends status *DEPLOY_ERROR* with error message containing listed +*schemaReference* that are missing from mapping file. Fail case might occur: + +1) When artifact *VES_EVENT* contains *stndDefined* events definitions and any of *schemaReference* is not present in mapping file. + +Example of artifact with *stndDefined* event definition: + +.. literalinclude:: resources/artifact-stndDefined.yaml + :language: yaml + +which is invalid when mapping file does not contain a mapping of schemaReference field. +Example of mapping file which makes example artifact invalid: + +.. literalinclude:: resources/schema-map-invalid.json + :language: json + +Validation results +------------------ +There are two ways to receive validation results. + +1) Via SDC UI. Results are available in *Service->Distributions* view. To see results in SDC UI user has to wait up to few minutes. +2) In VES OpenAPI Manager logs. They are printed right after validation.
\ No newline at end of file |
