aboutsummaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authormrichomme <morgan.richomme@orange.com>2020-10-29 15:36:09 +0100
committermrichomme <morgan.richomme@orange.com>2020-11-02 11:41:22 +0100
commit559771fe0fd39ef11b6fdd3c18af6a25f899ada0 (patch)
tree372722d382533cdcded26353cee1b8438014b3ed
parenta678715a0934ab44a710b82df351951f105d727d (diff)
Update xtesting READMEs
They can be used to run the xtesting dockers on any ONAP deployment Note a section in the official doc shall also be created Issue-ID: TEST-274 Signed-off-by: mrichomme <morgan.richomme@orange.com> Change-Id: If1d98e51c1dd476f5b1da81e51e4b4d99f513aa5 Signed-off-by: mrichomme <morgan.richomme@orange.com>
-rw-r--r--healthcheck/README.md94
-rw-r--r--infra-healthcheck/README.md16
-rw-r--r--security/README.md28
-rw-r--r--smoke-usecases-pythonsdk/README.md238
-rw-r--r--smoke-usecases-robot/README.md9
5 files changed, 325 insertions, 60 deletions
diff --git a/healthcheck/README.md b/healthcheck/README.md
index 455bcdf..b1826c5 100644
--- a/healthcheck/README.md
+++ b/healthcheck/README.md
@@ -6,15 +6,11 @@ This healthcheck docker includes the test suites checking ONAP components.
It includes 4 tests:
-* core: robot basic healthcheck of the components AAI, dmaap, portal, SDC, SDNC,
+- core: robot basic healthcheck of the components AAI, dmaap, portal, SDC, SDNC,
SO
-* small: robot basic healthcheck of the components AAI, dmaap, portal, SDC, SDNC,
- SO, AAF, APPC, CLI, LOG, MSB, Mulicloud, VID
-* medium: robot basic healthcheck of the components AAI, dmaap, portal, SDC, SDNC,
- SO, AAF, APPC, CLI, LOG, MSB, Mulicloud, VID, CDS, CLAMP, Dcaegen2, OOF,
- Policy, UUI
-* full: all the components
-* healthdist: test the onboarding and the distribution of the vFW
+- full: robot basic healthcheck covering all the components
+- healthdist: test the onboarding and the distribution of the vFW
+- postinstall: postinstallation tests including dmaap message router ACL update
## Usage
@@ -22,7 +18,7 @@ It includes 4 tests:
Mandatory:
-* The Robot configuration file: this file is setup at ONAP installation in the
+- The Robot configuration file: this file is setup at ONAP installation in the
configmap onap-robot-robot-eteshare-configmap. It includes lots of variables,
end points, urls, passwords, logins, needed by the robot scripts.
By default most of these data are set at ONAP installation.
@@ -34,7 +30,7 @@ Mandatory:
Optional:
-* The local result directory path: to store the results in your local
+- The local result directory path: to store the results in your local
environement. It shall corresponds to the internal result docker path
/var/lib/xtesting/results
@@ -45,27 +41,85 @@ the image as a jobs within onap namespace.
An example of job is provided in the job subdirectory. You need to setup the
following variables:
-* Mandatory:
- * run_type: the name of the use case: core, small, medium, full, healthdist
- * onap_namespace: onap name space (usually set to onap)
-* Optional:
- * deployment_name: parameter for the results pushed into the database (e.g.
+- Mandatory:
+ - run_type: the name of the use case: core, full, healthdist, postinstall
+ - onap_namespace: onap name space (usually set to onap)
+- Optional:
+ - deployment_name: parameter for the results pushed into the database (e.g.
daily-master)
- * deploy_scenario: parameter for the results pushed into the database (e.g.
+ - deploy_scenario: parameter for the results pushed into the database (e.g.
rke_istio_baremetal)
- * node_name: parameter for the results pushed into the database. This
+ - node_name: parameter for the results pushed into the database. This
parameter must match the pod declaration in the database otherwise the
results will not be accepted
- * test_result_url: the url of the test database
- * build_tag: parameter for the results pushed into the database. It is used to
+ - test_result_url: the url of the test database
+ - build_tag: parameter for the results pushed into the database. It is used to
identify the CI run. It can be used indirectly to retrieve the version
- * res_local_path: the path where you want to save the result logs on your
+ - res_local_path: the path where you want to save the result logs on your
local machine
By default (in scripts/cmd.sh), the -r option is enabled in the job template.
It means that xtesting will try to push the results to the DB.
If the parameters are not set, an error will be displayed (DB not reachable)
+### Example
+
+For the core test, the job description (test-core.yaml) shall look as follows:
+
+```
+apiVersion: batch/v1
+kind: Job
+metadata:
+ name: integration-onap-core
+ namespace: onap
+spec:
+ template:
+ spec:
+ containers:
+ - env:
+ - name: INSTALLER_TYPE
+ value: oom
+ - name: DEPLOY_SCENARIO
+ value: onap-nofeature-noha
+ - name: NODE_NAME
+ value: onap_daily_pod4_master-ONAP-oom
+ - name: TEST_DB_URL
+ value: http://testresults.opnfv.org/onap/api/v1/results
+ - name: BUILD_TAG
+ value: gitlab_ci-functest-kubespray-baremetal-daily-master-209039216-onap
+ - name: TAG
+ value: core
+ image: nexus3.onap.org:10003/onap/xtesting-healthcheck:master
+ imagePullPolicy: Always
+ name: functest-onap
+ volumeMounts:
+ - mountPath: /etc/localtime
+ name: localtime
+ readOnly: true
+ - mountPath: /share/config
+ name: robot-eteshare
+ - mountPath: /var/lib/xtesting/results/
+ name: robot-save-results
+ restartPolicy: Never
+ volumes:
+ - hostPath:
+ path: /etc/localtime
+ name: localtime
+ - configMap:
+ defaultMode: 493
+ name: onap-robot-eteshare-configmap
+ name: robot-eteshare
+ - hostPath:
+ path: /dockerdata-nfs/onap/integration/xtesting-healthcheck/core
+ name: robot-save-results
+```
+
+This job can be simply run using the command:
+
+```
+kubectl apply -f test-core.yaml
+```
+
### Output
```
diff --git a/infra-healthcheck/README.md b/infra-healthcheck/README.md
index 36628a1..4ff7f5d 100644
--- a/infra-healthcheck/README.md
+++ b/infra-healthcheck/README.md
@@ -9,12 +9,16 @@ It includes 2 tests:
- onap-k8s: list pods, deployments, events, cm, ... For any faulty pod, it
collects the logs and the describe. The success criteria is 100% of the pods
- are up&running
+ are up and running
- onap-helm: list the helm charts. The success criteria is all the helm charts
are completed.
-- nodeport_ingress: check that we have a 1:1 corresdpondance between nodeports
+- nodeport_ingress: check that we have a 1:1 correspondance between nodeports
and ingress (run only when the env variable DEPLOY_SCENARIO includes ingress)
+Please note that you will find another test (onap-k8s-teardown) in CI. It is exactly
+the same than onap-k8s (status of the onap cluster) executed at the end of the
+CI, after all the other tests. It allows to collect the logs of the components.
+
## Usage
### Configuration
@@ -39,7 +43,7 @@ You can run this docker by typing:
```
docker run -v <the kube config>:/root/.kube/config -v
<result directory>:/var/lib/xtesting/results
-nexus3.onap.org:10001/onap/xtesting-infra-healthcheck:latest
+nexus3.onap.org:10003/onap/xtesting-infra-healthcheck:latest
```
Options:
@@ -58,7 +62,7 @@ environment variables:
- INSTALLER_TYPE: precise how your ONAP has been installed (e.g. kubespray-oom,
rke-oom)
- BUILD_TAG: a unique tag of your CI system. It can be usefull to get all the
- tests of one CI run. It uses the regex (dai|week)ly-(.+?)-[0-9]\* to find the
+ tests of one CI run. It uses the regex (dai|week)ly-(.+?)-\[0-9]\* to find the
version (e.g. daily-elalto-123456789).
- DEPLOY_SCENARIO: your scenario deployment. ingress test run only if the
scenario includes 'ingress'
@@ -74,7 +78,7 @@ The command becomes:
docker run -v <the kube config>:/root/.kube/config -v
-v <the user home dir>/.helm/plugins/deploy/cache/onap/logs:/onap_helm_logs
<result directory>:/var/lib/xtesting/results
-nexus3.onap.org:10001/onap/xtesting-infra-healthcheck:latest:latest
+nexus3.onap.org:10003/onap/xtesting-infra-healthcheck:latest:latest
/bin/bash -c "run_tests -r -t all"
```
@@ -86,6 +90,6 @@ nexus3.onap.org:10001/onap/xtesting-infra-healthcheck:latest:latest
+------------------+-------------+-------------------+----------+--------+
| onap-k8s | integration | infra-healthcheck | 00:06 | PASS |
| onap-helm | integration | infra-healthcheck | 00:01 | PASS |
-| nodeport_ingress | security | security | 00:01 | FAIL |
+| nodeport_ingress | security | security | 00:01 | FAIL |
+------------------+-------------+-------------------+----------+--------+
```
diff --git a/security/README.md b/security/README.md
index 7dc7c0e..2a94f26 100644
--- a/security/README.md
+++ b/security/README.md
@@ -18,7 +18,8 @@ It includes 6 tests:
- kube_hunter: security suite to search k8s vulnerabilities (upstream src
aquasecurity)
- versions: check that Java and Python are available only in versions
- recommended by SECCOM
+ recommended by SECCOM. This test is long and run only in Weekly CI chains\
+ (https://logs.onap.org/onap-integration/weekly).
## Usage
@@ -44,7 +45,7 @@ You can run this docker by typing:
```
docker run -v <the kube config>:/root/.kube/config -v
<result directory>:/var/lib/xtesting/results
-nexus3.onap.org:10001/onap/xtesting-security:latest
+nexus3.onap.org:10003/onap/xtesting-security:latest
```
Options:
@@ -63,7 +64,7 @@ environment variables:
- INSTALLER_TYPE: precise how your ONAP has been installed (e.g. kubespray-oom,
rke-oom)
- BUILD_TAG: a unique tag of your CI system. It can be usefull to get all the
- tests of one CI run. It uses the regex (dai|week)ly-(.+?)-[0-9]\* to find the
+ tests of one CI run. It uses the regex (dai|week)ly-(.+?)-\[0-9]\* to find the
version (e.g. daily-elalto-123456789).
The command becomes:
@@ -71,20 +72,21 @@ The command becomes:
```
docker run -v <the kube config>:/root/.kube/config -v
<result directory>:/var/lib/xtesting/results
-nexus3.onap.org:10001/onap/xtesting-security:latest
+nexus3.onap.org:10003/onap/xtesting-security:latest
/bin/bash -c "run_tests -r -t all
```
### Output
```
-+-----------------------+------------+------------+------------+-----------+
-| TEST CASE | PROJECT | TIER | DURATION | RESULT |
-+-----------------------+------------+------------+------------+-----------+
-| root_pods | security | security | 03:48 | PASS |
-| unlimitted_pods | security | security | 00:37 | FAIL |
-| cis_kubernetes | security | security | 00:01 | PASS |
-| http_public_endpoints | security | security | 00:01 | PASS |
-| jdpw_ports | security | security | 05:39 | PASS |
-+-----------------------+------------+------------+------------+-----------+
++-----------------------+------------+------------+-----------+
+| TEST CASE | PROJECT | DURATION | RESULT |
++-----------------------+------------+------------+-----------+
+| root_pods | security | 03:48 | PASS |
+| unlimitted_pods | security | 00:37 | FAIL |
+| cis_kubernetes | security | 00:01 | PASS |
+| kube_hunter | security | 00:03 | PASS |
+| http_public_endpoints | security | 00:01 | PASS |
+| jdpw_ports | security | 05:39 | PASS |
++-----------------------+------------+------------+-----------+
```
diff --git a/smoke-usecases-pythonsdk/README.md b/smoke-usecases-pythonsdk/README.md
index a840ef1..d21f5d2 100644
--- a/smoke-usecases-pythonsdk/README.md
+++ b/smoke-usecases-pythonsdk/README.md
@@ -11,17 +11,17 @@ but the Dockerfile must be adapted accordingly.
The tests are:
-- basic_vm: it onboard/distribute/deploy a single Ubuntu VM in ONAP using
- VNF-API. The components used are SDC, SO, AA&I, SDNC.
-- freeradius_nbi: based on basic_vm, the instantiation part is done through the
- ONAP external API (NBI) module.
-- clearwater_ims: it consists in a full deployment of an clearwater vIMS in ONAP.
+- basic_vm: onboarding/distribution/deployment of a single Ubuntu VM in ONAP using
+ GR-API (A la Carte SO BPMN). The components used are SDC, SO, AAI, SDNC.
+- basic_cnf: onboarding/distribution/deployment of a single nginx pod in ONAP using
+ GR-API (A la Carte SO BPMN) and K8s module. The components used are SDC, SO,
+ AAI, SDNC, MultiCloud.
+- basic_network: onboarding/distribution/deployment of a Neutron network
## Usage
-Note this docker will be deprecated in Guilin. The existing 3 test cases and
-additional ones will be performed through test cases [2] consuming the Python
-ONAP SDK [1]
+Tests are performed through test cases [2] consuming the Python
+ONAP SDK [1].
### Configuration
@@ -29,35 +29,237 @@ Mandatory:
Assuming that these use cases were based on onap-tests, the service descriptions
must be overwritten with the right cloud informations as well as the onap-tests
-configuration. That is why 2 volumes are needed:
+configuration.
-- <custom onap-tests config>: The last section of the default configuration must
- be adapted to your environement [3]
-- <service description dir>: the customized service descriptions including the
- parameters to be overwritten (SDNC preload) [4]
+pythonsdk-tests configuration is given by a file whose path is specified in env
+variables. It is therefore needed to setup env files and precise the
+test configuration.
+
+There are optional and mandatory env variables
+
+Mandatory:
+
+- OS_TEST_CLOUD: it specifies the Openstack cloud you
+ are using (declared in the clouds.yaml)
+- ONAP_PYTHON_SDK_SETTINGS: the configuration to be considered for the tests.
+ See pythonsdk-tests README for details
Optional:
+- INSTALLER_TYPE: the name of the installer (used for results)
+- TEST_DB_URL: the url of the DB you want to push the results
+- NODE_NAME: the name of the lab or CI chain (used for results)
+- BUILD_TAG: an id used for CI to group tests in a CI run
+- DEBUG: by default xtesting logs are not enables
+
+An example of env file is given herefater:
+
+```
+INSTALLER_TYPE=oom
+TEST_DB_URL=http://testresults.opnfv.org/onap/api/v1/results
+NODE_NAME=onap_daily_pod4_master-ONAP-oom
+BUILD_TAG=gitlab_ci-functest-kubespray-baremetal-daily-master-209039216-onap
+DEBUG=True
+OS_TEST_CLOUD=onap-master-daily-vnfs-ci
+ONAP_PYTHON_SDK_SETTINGS=onaptests.configuration.ubuntu16_nomulticloud_settings
+```
+
+The generic configuration file settings.py can be also specified to overwritte
+default values.
+
+```
+"""Specific settings module.""" # pylint: disable=bad-whitespace
+
+######################
+# #
+# ONAP INPUTS DATAS #
+# #
+######################
+
+
+# Variables to set logger information
+# Possible values for logging levels in onapsdk: INFO, DEBUG , WARNING, ERROR
+LOG_CONFIG = {
+ "version": 1,
+ "disable_existing_loggers": False,
+ "formatters": {
+ "default": {
+ "class": "logging.Formatter",
+ "format": "%(asctime)s %(levelname)s %(lineno)d:%(filename)s(%(process)d) - %(message)s"
+ }
+ },
+ "handlers": {
+ "console": {
+ "level": "INFO",
+ "class": "logging.StreamHandler",
+ "formatter": "default"
+ },
+ "file": {
+ "level": "DEBUG",
+ "class": "logging.FileHandler",
+ "formatter": "default",
+ "filename": "/var/lib/xtesting/results/pythonsdk.debug.log",
+ "mode": "w"
+ }
+ },
+ "root": {
+ "level": "DEBUG",
+ "handlers": ["console", "file"]
+ }
+}
+
+# SOCK_HTTP = "socks5h://127.0.0.1:8080"
+REPORTING_FILE_PATH = "/var/lib/xtesting/results/reporting.html"
+K8S_REGION_TYPE = "k8s"
+```
+
+You can specify the log level, the log file, and the path for the reporting page.
+
+You may then indicate several volumes when launching the xtesting docker.
+Please note it can be run from anyplace assuming that the configuration and the
+connectivity are properly set. You then need to hav access to ONAP endpoints, you
+can either copy a /etc/hosts in the docker or add the hosts in the docker command.
+
+The different needed volumes are:
+
- The local result directory path: to store the results in your local
environement. It shall corresponds to the internal result docker path
/var/lib/xtesting/results
-- The cloud.yaml if you enable the Openstack verification option (Openstack
- client call to check that the stack is created in Openstack)
+- The kubernetes .kube/config configuration for use cases interacting with kubernetes
+ (basic_cnf)
+- The openstack cloud.yaml for use cases interacting with Openstack
+ infrastructures (basic_vm, basic_network)
+- The customization of your service. You can overwrite the model datas by the
+ values matching your environement. In this case you need to replace the default
+ service configuration file
+
+An example of clouds.yaml
+
+```
+clouds:
+ onap-master-daily-vnfs-ci:
+ auth:
+ auth_url: https://vim.pod4.opnfv.fr:5000
+ password: yGiJDEh82K9x0T69iIJJRPeUqVRPFnIR
+ project_id: 978ec1ae58ac492a936e0bcec22dfee9
+ project_name: onap-master-daily-vnfs
+ user_domain_name: Default
+ username: onap-master-daily-vnfs-ci
+ compute_api_version: '2.15'
+ identity_api_version: '3'
+ interface: public
+ project_domain_name: Default
+ region_name: RegionOne
+```
+
+A example of basic_vm customization is provided hereafter. You can customize
+this file according to your cloud environment (key, flavor name,..).
+The data will be preloaded to overwrite the default model values.
+
+```
+---
+ubuntu16test:
+ tosca_file_from_SDC: service-Ubuntu16tha-template
+ version: "1.0"
+ vnfs:
+ - vnf_name: ubuntu16test
+ heat_files_to_upload: onaptests/templates/heat-files/ubuntu16/ubuntu16.zip
+ vnf_parameters: [
+ {"name": "ubuntu16_image_name",
+ "value": "ubuntu-16.04-daily"
+ },
+ {"name": "ubuntu16_key_name",
+ "value": "cleouverte"
+ },
+ {"name": "ubuntu16_pub_key",
+ "value": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAA\
+BAQDY15cdBmIs2XOpe4EiFCsaY6bmUmK/GysMoLl4UG51JCfJwvwoWCoA+6mDIbymZxhxq9IGx\
+ilp/yTA6WQ9s/5pBag1cUMJmFuda9PjOkXl04jgqh5tR6I+GZ97AvCg93KAECis5ubSqw1xOCj4\
+utfEUtPoF1OuzqM/lE5mY4N6VKXn+fT7pCD6cifBEs6JHhVNvs5OLLp/tO8Pa3kKYQOdyS0xc3r\
+h+t2lrzvKUSWGZbX+dLiFiEpjsUL3tDqzkEMNUn4pdv69OJuzWHCxRWPfdrY9Wg0j3mJesP29EBh\
+t+w+EC9/kBKq+1VKdmsXUXAcjEvjovVL8l1BrX3BY0R8D imported-openssh-key"
+ },
+ {"name": "ubuntu16_flavor_name",
+ "value": "m1.small"
+ },
+ {"name": "VM_name",
+ "value": "ubuntu16test-VM-01"
+ },
+ {"name": "vnf_id",
+ "value": "ubuntu16test-VNF-instance"
+ },
+ {"name": "vf_module_id",
+ "value": "ubuntu16test-vfmodule-instance"
+ },
+ {"name": "vnf_name",
+ "value": "ubuntu16test-VNF"
+ },
+ {"name": "admin_plane_net_name",
+ "value": "admin"
+ }
+ ]
+```
### Command
+If you specify all the hosts
+
```
docker run
-v <your local env>:/var/lib/xtesting/conf/env_file
--v <custom onap-tests config>:/usr/lib/python3.8/site-packages/onap_tests/onap-conf/onap-testing.yaml
--v <service description dir>:/usr/lib/python3.8/site-packages/onap_tests/templates/vnf-services
-v <cloud.yaml file corresponding to your VNF tenant>/root/.config/openstack.yaml
+-v <kube config file corresponding to your k8s cluster>/root/.kube/config
+-v <service definition yaml matching your environment>:/usr/lib/python3.8/site-packages/onaptests/templates/vnf-services/ubuntu16test-service.yaml
-v <result directory>:/var/lib/xtesting/results
-nexus3.onap.org:10001/onap/xtesting-infra-healthcheck:latest
+--add-host="portal.api.simpledemo.onap.org:<your ONAP IP>"
+--add-host="vid.api.simpledemo.onap.org:<your ONAP IP>"
+--add-host="sdc.api.fe.simpledemo.onap.org:<your ONAP IP>"
+--add-host="sdc.api.be.simpledemo.onap.org:<your ONAP IP>"
+--add-host="aai.api.sparky.simpledemo.onap.org:<your ONAP IP>"
+--add-host="so.api.simpledemo.onap.org:<your ONAP IP>"
+--add-host="sdnc.api.simpledemo.onap.org:<your ONAP IP>"
+--add-host="sdc.workflow.plugin.simpledemo.onap.org:<your ONAP IP>"
+--add-host="sdc.dcae.plugin.simpledemo.onap.org:<your ONAP IP>"
+--add-host="msb.api.simpledemo.onap.org:<your ONAP IP>"
+nexus3.onap.org:10003/onap/xtesting-smoke-usecases-pythonsdk:latest /bin/sh -c "run_tests -t basic_vm"
```
+Unkike the other xtesting docker, 1 docker = 1 use case, the target -t all is
+not usable.
+
+Note you can run also the docker interactivly
+
+```
+docker run -it
+-v <your local env>:/var/lib/xtesting/conf/env_file
+-v <cloud.yaml file corresponding to your VNF tenant>/root/.config/openstack.yaml
+-v <kube config file corresponding to your k8s cluster>/root/.kube/config
+-v <service definition yaml matching your environment>:/usr/lib/python3.8/site-packages/onaptests/templates/vnf-services/ubuntu16test-service.yaml
+-v <result directory>:/var/lib/xtesting/results
+nexus3.onap.org:10003/onap/xtesting-smoke-usecases-pythonsdk:latest sh
+```
+
+Inside the docker you can edit the /etc/hosts and indicate the different hosts).
+Then you can run the test with the following command:
+
+```
+run_tests -t basic_vm
+```
+
+you can modify the env variable to run any of the use cases.
+
### Output
+```
++-------------------+---------------------+------------------+----------------+
+| TEST CASE | PROJECT | DURATION | RESULT |
++-------------------+---------------------+------------------+----------------+
+| basic_vm | integration | 09:52 | PASS |
++-------------------+---------------------+------------------+----------------+
+```
+
+### references
+
[1]: https://gitlab.com/Orange-OpenSource/lfn/onap/python-onapsdk "Python ONAP SDK repository"
[2]: https://git.onap.org/testsuite/pythonsdk-tests/tree/ "Scenario consuming Python SDK"
diff --git a/smoke-usecases-robot/README.md b/smoke-usecases-robot/README.md
index b2be51d..b253cbd 100644
--- a/smoke-usecases-robot/README.md
+++ b/smoke-usecases-robot/README.md
@@ -5,9 +5,10 @@
The tests are:
- pnf_registrate
-- vfw_cl
- 5gbulkpm
- hv-ves
+- vescollector
+- cmpv2
## Usage
@@ -15,8 +16,10 @@ The robot scripts have been planned to be launched from the cluster.
The easiest way to run the test consists in creating a kubernetes job.
You can run it as a sandalone dockers but the endpoints must be adapted
to be reachable.
+The robot code is under the ONAP testsuite repository
+(https://git.onap.org/testsuite/tree/).
-### Configuration
+### Example
An example of job test.yaml can be found hereafter:
@@ -43,7 +46,7 @@ spec:
value: gitlab_ci-functest-kubespray-baremetal-daily-master-559950989
- name: TAG
value: 5gbulkpm
- image: nexus3.onap.org:10001/onap/xtesting-smoke-usecases-robot:latest
+ image: nexus3.onap.org:10003/onap/xtesting-smoke-usecases-robot:latest
imagePullPolicy: Always
name: functest-onap
volumeMounts: