Updated README files and adaptation of consul/cbs config

Change-Id: I81494f56978a3d0ff06ec0d66968f33f08114103
Issue-ID: DCAEGEN2-1719
Signed-off-by: maximesson <maxime.bonneau@est.tech>

1 files changed, 13 insertions, 204 deletions

diff --git a/test/mocks/datafilecollector-testharness/auto-test/README.md b/test/mocks/datafilecollector-testharness/auto-test/README.md
index a94076f58..b73067dee 100644
--- a/test/mocks/datafilecollector-testharness/auto-test/README.md
+++ b/test/mocks/datafilecollector-testharness/auto-test/README.md
@@ -1,6 +1,8 @@
 ## Running automated test case and test suites
 Test cases run a single test case and test suites run one or more test cases in a sequence.
 
+The test cases and test suites are possible to run on both Ubuntu and Mac-OS. 
+
 ##Overall structure and setup
 Test cases and test suites are written as bash scripts which call predefined functions in two other bash scripts
 located in ../common dir.
@@ -69,201 +71,8 @@ print_result
 ```
 -----------------------------------------------------------
 
-The ../common/testcase_common.sh contains all functions needed for the test case file.
-
-The following is a list of the available functions in a test case file. Please see some of the defined test case for examples.
-
-**log_sim_settings**</br>
-Print the env variables needed for the simulators and their setup
-
-**clean_containers**</br>
-Stop and remove all containers including dfc apps and simulators
-
-**start_simulators**</br>
-Start all simulators in the simulator group
-
-**start_dfc <dfc-instance-id>**</br>
-Start the dfc application. The arg shall be an integer from 0 to 5 reprenting the dfc instance to start. DFC app will get a name like 'dfc_app0' to 'dfc_app4'.
-
-**kill_dfc  <dfc-instance-id> **</br>
-Stop and remove the dfc app container with the instance id.
-
-**consul_config_app <dfc-instance-id> <json-file-path>**</br>
-Configure consul with json file with app config for a dfc instance using the dfc instance id and the json file.
-
-**consul_config_dmaap <dfc-instance-id> <json-file-path>**</br>
-Configure consul with json file with dmaap config for a dfc instance using the dfc instance id and the json file.
+The ../common/testcase_common.sh contains all functions needed for the test case file. See the README.md file in the ../common dir for a description of all available functions.
 
-**kill_dr**</br>
-Stop and remove the DR simulator container
-
-**kill_drr**</br>
-Stop and remove the DR redir simulator container
-
-**kill_mr**</br>
-Stop and remove the MR simulator container
-
-**kill_sftp <sftp-instance-id>**</br>
-Stop and remove a SFTP container with the supplied instance id (0-5).
-
-**stop_sftp <sftp-instance-id>**</br>
-Stop a SFTP container with the supplied instance id (0-5).
-
-**start_sftp <sftp-instance-id>**</br>
-Start a previously stopped SFTP container with the supplied instance id (0-5).
-
-**kill_ftps <ftps-instance-id>**</br>
-Stop and remove a FTPS container with the supplied instance id (0-5).
-
-**stop_ftps <ftps-instance-id>**</br>
-Stop a FTPS container with the supplied instance id (0-5).
-
-**start_ftps <ftps-instance-id>**</br>
-Start a previously stopped FTPS container with the supplied instance id (0-5).
-
-**mr_print <vaiable-name>**</br>
-Print a variable value from the MR simulator.
-
-**dr_print <vaiable-name>**</br>
-Print a varialle value from the DR simulator.
-
-**drr_print <vaiable-name>**</br>
-Print a variable value from the DR redir simulator.
-
-**dfc_print <dfc-instance-id> <vaiable-name>**</br>
-Print a variable value from an dfc instance with the supplied instance id (0-5).
-
-**mr_read <vaiable-name>**</br>
-Read a variable value from MR sim and send to stdout
-
-**dr_read <vaiable-name>**</br>
-Read a variable value from DR sim and send to stdout
-
-**drr_read <vaiable-name>**</br>
-Read a variable value from DR redir sim and send to stdout
-
-**sleep_wait <sleep-time-in-sec>**</br>
-Sleep for a number of seconds
-
-**sleep_heartbeat <sleep-time-in-sec>**</br>
-Sleep for a number of seconds and prints dfc heartbeat output every 30 sec
-
-**mr_equal <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the MR simulator is equal to a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-equal to the targer or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value becomes equal to the target
-value or not.
-
-**mr_greater <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the MR simulator is greater than a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-greater the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value is greater than the target
-value or not.
-
-**mr_less <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the MR simulator is less than a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-less than the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value is less than the target
-value or not.
-
-**mr_contain_str <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the MR simulator contains a substring target and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable contains
-the target substring or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value contains the target
-substring or not.
-
-**dr_equal <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR simulator is equal to a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-equal to the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value becomes equal to the target
-value or not.
-
-**dr_greater <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR simulator is greater than a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-greater the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value is greater than the target
-value or not.
-
-**dr_less <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR simulator is less than a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-less than the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value is less than the target
-value or not.
-
-**dr_contain_str <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR simulator contains a substring target and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable contains
-the target substring or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value contains the target
-substring or not.
-
-**drr_equal <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR Redir simulator is equal to a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-equal to the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value becomes equal to the target
-value or not.
-
-**drr_greater <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR Redir simulator is greater than a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-greater the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value is greater than the target
-value or not.
-
-**drr_less <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR Redir simulator is less than a target value and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable is
-less than the target or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value is less than the target
-value or not.
-
-**drr_contain_str <variable-name> <target-value> [<timeout-in-sec>]**</br>
-Tests if a variable value in the DR Redir simulator contains a substring target and and optional timeout.
-</br>Arg: ``<variable-name> <target-value>`` - This test set pass or fail depending on if the variable contains
-the target substring or not.
-</br>Arg: ``<variable-name> <target-value> <timeout-in-sec>``  - This test waits up to the timeout seconds
-before setting pass or fail depending on if the variable value contains the target
-substring or not.
-
-**dfc_contain_str <variable-name> <substring-in-quotes>**</br>
-Test is a variable in the DFC contains a substring.
-
-**store_logs <log-prefix>**</br>
-Store all dfc app and simulators log to the test case log dir. All logs gets a prefix to
-separate logs stored at different steps in the test script.
-If logs need to be stored in several locations, use different prefix to easily identify the location
-when the logs where taken.
-
-**check_dfc_log**</br>
-Check the dfc application log for WARN and ERR messages and print the count.
-
-**print_result**</br>
-Print the test result. Only once at the very end of the script.
-
-**print_all**</br>
-Print all variables from the simulators and the dfc heartbeat.
-
-In addition, comment in the file can be added using the normal comment sign in bash '#'.
-Comments that shall be visible on the screen as well as in the test case log, use ``echo "<msg>"``.
 
 ##Test suite files##
 A test suite file contains one or more test cases to run in sequence.
@@ -297,15 +106,15 @@ suite_complete
 ```
 -----------------------------------------------------------
 
-The ../common/testsuite_common.sh contains all functions needed for a test suite file.
-
-The following is a list of the available functions in a test case file. Please see a existing test suite for examples.
-
-**suite_setup**</br>
-Sets up the test suite and print out a heading.
+The ../common/testsuite_common.sh contains all functions needed for a test suite file. See the README.md file in the ../common dir for a description of all available functions.
 
-**run_tc <tc-script> <$1 from test suite script> <$2 from test suite script>**</br>
-Execute a test case with arg from test suite script
+##Known limitations##
+When DFC has polled a new event from the MR simulator, DFC starts to check each file whether it has been already published or not. This check is done per file towards the DR simulator. 
+If the event contains a large amount of files, there is a risk that DFC will flood the DR simulator with requests for these checks. The timeout in DFC for the response is currently 4 sec and the DR simulator may not be able to answer all request within the timeout.
+DR simulator is single threaded. This seem to be a problem only for the first polled event. For subsequent events these requests seem to be spread out in time by DFC so the DR simulator can respond in time.
+The problem is visible in the DR simulator counters `ctr_publish_query` and`ctr_publish_query_not_published` in the auto-test scripts. They will have a count slightly less (1 to 5) than the actual number of files in the event. The application log in DFC also prints a timeout error for each failed request.
+A number of the test script will report failure due to this limitation in the DR simulator.
 
-**suite_complete**</br>
-Print out the overall result of the executed test cases.
\ No newline at end of file
+The FTP servers may deny connection when too many file download requests are made in a short time from DFC.
+This is visible in the DFC application log as WARNINGs for failed downloads. However, DFC always retry the failed download a number of times to 
+minimize the risk of giving up download completely for these files.
\ No newline at end of file