diff options
Diffstat (limited to 'test/mocks/datafilecollector-testharness/auto-test/README.md')
-rw-r--r-- | test/mocks/datafilecollector-testharness/auto-test/README.md | 217 |
1 files changed, 13 insertions, 204 deletions
diff --git a/test/mocks/datafilecollector-testharness/auto-test/README.md b/test/mocks/datafilecollector-testharness/auto-test/README.md index a94076f58..b73067dee 100644 --- a/test/mocks/datafilecollector-testharness/auto-test/README.md +++ b/test/mocks/datafilecollector-testharness/auto-test/README.md @@ -1,6 +1,8 @@ ## Running automated test case and test suites Test cases run a single test case and test suites run one or more test cases in a sequence. +The test cases and test suites are possible to run on both Ubuntu and Mac-OS. + ##Overall structure and setup Test cases and test suites are written as bash scripts which call predefined functions in two other bash scripts located in ../common dir. @@ -69,201 +71,8 @@ print_result ``` ----------------------------------------------------------- -The ../common/testcase_common.sh contains all functions needed for the test case file. - -The following is a list of the available functions in a test case file. Please see some of the defined test case for examples. - -**log_sim_settings**</br> -Print the env variables needed for the simulators and their setup - -**clean_containers**</br> -Stop and remove all containers including dfc apps and simulators - -**start_simulators**</br> -Start all simulators in the simulator group - -**start_dfc <dfc-instance-id>**</br> -Start the dfc application. The arg shall be an integer from 0 to 5 reprenting the dfc instance to start. DFC app will get a name like 'dfc_app0' to 'dfc_app4'. - -**kill_dfc <dfc-instance-id> **</br> -Stop and remove the dfc app container with the instance id. - -**consul_config_app <dfc-instance-id> <json-file-path>**</br> -Configure consul with json file with app config for a dfc instance using the dfc instance id and the json file. - -**consul_config_dmaap <dfc-instance-id> <json-file-path>**</br> -Configure consul with json file with dmaap config for a dfc instance using the dfc instance id and the json file. +The ../common/testcase_common.sh contains all functions needed for the test case file. See the README.md file in the ../common dir for a description of all available functions. -**kill_dr**</br> -Stop and remove the DR simulator container - -**kill_drr**</br> -Stop and remove the DR redir simulator container - -**kill_mr**</br> -Stop and remove the MR simulator container - -**kill_sftp <sftp-instance-id>**</br> -Stop and remove a SFTP container with the supplied instance id (0-5). - -**stop_sftp <sftp-instance-id>**</br> -Stop a SFTP container with the supplied instance id (0-5). - -**start_sftp <sftp-instance-id>**</br> -Start a previously stopped SFTP container with the supplied instance id (0-5). - -**kill_ftps <ftps-instance-id>**</br> -Stop and remove a FTPS container with the supplied instance id (0-5). - -**stop_ftps <ftps-instance-id>**</br> -Stop a FTPS container with the supplied instance id (0-5). - -**start_ftps <ftps-instance-id>**</br> -Start a previously stopped FTPS container with the supplied instance id (0-5). - -**mr_print <vaiable-name>**</br> -Print a variable value from the MR simulator. - -**dr_print <vaiable-name>**</br> -Print a varialle value from the DR simulator. - -**drr_print <vaiable-name>**</br> -Print a variable value from the DR redir simulator. - -**dfc_print <dfc-instance-id> <vaiable-name>**</br> -Print a variable value from an dfc instance with the supplied instance id (0-5). - -**mr_read <vaiable-name>**</br> -Read a variable value from MR sim and send to stdout - -**dr_read <vaiable-name>**</br> -Read a variable value from DR sim and send to stdout - -**drr_read <vaiable-name>**</br> -Read a variable value from DR redir sim and send to stdout - -**sleep_wait <sleep-time-in-sec>**</br> -Sleep for a number of seconds - -**sleep_heartbeat <sleep-time-in-sec>**</br> -Sleep for a number of seconds and prints dfc heartbeat output every 30 sec - -**mr_equal <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the MR simulator is equal to a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -equal to the targer or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value becomes equal to the target -value or not. - -**mr_greater <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the MR simulator is greater than a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -greater the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value is greater than the target -value or not. - -**mr_less <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the MR simulator is less than a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -less than the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value is less than the target -value or not. - -**mr_contain_str <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the MR simulator contains a substring target and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable contains -the target substring or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value contains the target -substring or not. - -**dr_equal <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR simulator is equal to a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -equal to the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value becomes equal to the target -value or not. - -**dr_greater <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR simulator is greater than a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -greater the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value is greater than the target -value or not. - -**dr_less <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR simulator is less than a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -less than the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value is less than the target -value or not. - -**dr_contain_str <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR simulator contains a substring target and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable contains -the target substring or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value contains the target -substring or not. - -**drr_equal <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR Redir simulator is equal to a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -equal to the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value becomes equal to the target -value or not. - -**drr_greater <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR Redir simulator is greater than a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -greater the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value is greater than the target -value or not. - -**drr_less <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR Redir simulator is less than a target value and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is -less than the target or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value is less than the target -value or not. - -**drr_contain_str <variable-name> <target-value> [<timeout-in-sec>]**</br> -Tests if a variable value in the DR Redir simulator contains a substring target and and optional timeout. -</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable contains -the target substring or not. -</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>`` - This test waits up to the timeout seconds -before setting pass or fail depending on if the variable value contains the target -substring or not. - -**dfc_contain_str <variable-name> <substring-in-quotes>**</br> -Test is a variable in the DFC contains a substring. - -**store_logs <log-prefix>**</br> -Store all dfc app and simulators log to the test case log dir. All logs gets a prefix to -separate logs stored at different steps in the test script. -If logs need to be stored in several locations, use different prefix to easily identify the location -when the logs where taken. - -**check_dfc_log**</br> -Check the dfc application log for WARN and ERR messages and print the count. - -**print_result**</br> -Print the test result. Only once at the very end of the script. - -**print_all**</br> -Print all variables from the simulators and the dfc heartbeat. - -In addition, comment in the file can be added using the normal comment sign in bash '#'. -Comments that shall be visible on the screen as well as in the test case log, use ``echo "<msg>"``. ##Test suite files## A test suite file contains one or more test cases to run in sequence. @@ -297,15 +106,15 @@ suite_complete ``` ----------------------------------------------------------- -The ../common/testsuite_common.sh contains all functions needed for a test suite file. - -The following is a list of the available functions in a test case file. Please see a existing test suite for examples. - -**suite_setup**</br> -Sets up the test suite and print out a heading. +The ../common/testsuite_common.sh contains all functions needed for a test suite file. See the README.md file in the ../common dir for a description of all available functions. -**run_tc <tc-script> <$1 from test suite script> <$2 from test suite script>**</br> -Execute a test case with arg from test suite script +##Known limitations## +When DFC has polled a new event from the MR simulator, DFC starts to check each file whether it has been already published or not. This check is done per file towards the DR simulator. +If the event contains a large amount of files, there is a risk that DFC will flood the DR simulator with requests for these checks. The timeout in DFC for the response is currently 4 sec and the DR simulator may not be able to answer all request within the timeout. +DR simulator is single threaded. This seem to be a problem only for the first polled event. For subsequent events these requests seem to be spread out in time by DFC so the DR simulator can respond in time. +The problem is visible in the DR simulator counters `ctr_publish_query` and`ctr_publish_query_not_published` in the auto-test scripts. They will have a count slightly less (1 to 5) than the actual number of files in the event. The application log in DFC also prints a timeout error for each failed request. +A number of the test script will report failure due to this limitation in the DR simulator. -**suite_complete**</br> -Print out the overall result of the executed test cases.
\ No newline at end of file +The FTP servers may deny connection when too many file download requests are made in a short time from DFC. +This is visible in the DFC application log as WARNINGs for failed downloads. However, DFC always retry the failed download a number of times to +minimize the risk of giving up download completely for these files.
\ No newline at end of file |