Refactor rancher1_api module

This rewrite enables to add easier more features supported by the rancher
API. The initial idea of a simple get and set through the JSON REST API
is not feasible. To achieve something with the API one may have to setup
more options on different URLs and in a particular order. For this reason
the module comes with the mechanism of "modes", which is a wrapper around
some feature in the rancher and which can require multiple steps to do.

Rancher1_api module could also support "raw" mode where the user will not
be limited by a few implemented modes in the module, but he can craft all
requests by hand - but due to the fact that such thing can be done easily
with just curl command and ansible shell module, there is no benefit in
doing so. Especially when rancher 1.6 is already obsoleted within the ONAP.

The useful value of this module is to give the user a simple means to set
something in the rancher and hide all the boilerplate from him via a mode.

- Original logic was rewritten to utilize the "mode" mechanism.
- New module structure is also easier to test outside of ansible.

Issue-ID: OOM-1681
Change-Id: I0e7932199df9ec1acd80af545060199867ad17fa
Signed-off-by: Petr Ospalý <p.ospaly@partner.samsung.com>

2 files changed, 224 insertions, 104 deletions

diff --git a/ansible/library/rancher1_api.py b/ansible/library/rancher1_api.py
index ce3df1b8..006de9ce 100644
--- a/ansible/library/rancher1_api.py
+++ b/ansible/library/rancher1_api.py
@@ -12,8 +12,16 @@ module: rancher1_api
 short_description: Client library for rancher API
 description:
   - This module modifies a rancher 1.6 using it's API (v1).
-  - WIP, as of now it can only change a current value to a new one.
-
+  - It supports some rancher features by the virtue of a 'mode'.
+  - 'modes' hide from you some necessary cruft and expose you to the only
+    important and interestig variables wich must be set. The mode mechanism
+    makes this module more easy to use and you don't have to create an
+    unnecessary boilerplate for the API.
+  - Only a few modes are/will be implemented so far - as they are/will be
+    needed. In the future the 'raw' mode can be added to enable you to craft
+    your own API requests, but that would be on the same level of a user
+    experience as running curl commands, and because the rancher 1.6 is already
+    obsoleted by the project, it would be a wasted effort.
 options:
   rancher:
     description:
@@ -23,29 +31,44 @@ options:
     required: true
     aliases:
       - server
-      - rancher_url
+      - rancher_server
       - rancher_api
       - api
-      - url
-  category:
+  account_key:
     description:
-      - The path in JSON API without the last element.
+      - The public and secret part of the API key-pair separated by colon.
+      - You can find all your keys in web UI.
+      - For example:
+        B1716C4133D3825051CB:3P2eb3QhokFKYUiXRNZLxvGNSRYgh6LHjuMicCHQ
     required: false
-    default: settings
-    aliases:
-      - rancher_category
-      - api_category
-  option:
+  mode:
     description:
-      - The name of the settings option.
+      - A recognized mode how to deal with some concrete configuration task
+        in rancher API to ease the usage.
+      - The implemented modes so far are:
+        'settings':
+            Many options under <api_server>/v1/settings API url and some can
+            be seen also under advanced options in the web UI.
+        'access_control':
+            It setups user and password for the account (defaults to 'admin')
+            and it enables the local authentication - so the web UI and API
+            will require username/password (UI) or apikey (API).
     required: true
     aliases:
-      - name
-      - key
+      - rancher_mode
+      - api_mode
+    choices:
       - settings
-  value:
+      - access_control
+  data:
     description:
-      - A new value to replace the current one.
+      - Dictionary with key/value pairs. The actual names and meaning of pairs
+        depends on the used mode.
+      - 'settings' mode:
+        option: Option/path in JSON API (url).
+        value: A new value to replace the current one.
+      - 'access_control' mode:
+        None - not yet implemented - placeholder only.
     required: true
   timeout:
     description:
@@ -54,6 +77,12 @@ options:
     default: 10.0
 """
 
+default_timeout = 10.0
+
+
+class ModeError(Exception):
+    pass
+
 
 def _decorate_rancher_api_request(request_method):
 
@@ -76,7 +105,7 @@ def _decorate_rancher_api_request(request_method):
 
 
 @_decorate_rancher_api_request
-def get_rancher_api_value(url, headers=None, timeout=10.0,
+def get_rancher_api_value(url, headers=None, timeout=default_timeout,
                           username=None, password=None):
 
     if username and password:
@@ -91,7 +120,7 @@ def get_rancher_api_value(url, headers=None, timeout=10.0,
 
 
 @_decorate_rancher_api_request
-def set_rancher_api_value(url, payload, headers=None, timeout=10.0,
+def set_rancher_api_value(url, payload, headers=None, timeout=default_timeout,
                           username=None, password=None):
 
     if username and password:
@@ -107,106 +136,114 @@ def set_rancher_api_value(url, payload, headers=None, timeout=10.0,
                             data=json.dumps(payload))
 
 
-def create_rancher_api_payload(json_data, new_value):
+def create_rancher_api_url(server, mode, option):
+    request_url = server.strip('/') + '/v1/'
 
-    payload = {}
+    if mode == 'raw':
+        request_url += option.strip('/')
+    elif mode == 'settings':
+        request_url += 'settings/' + option.strip('/')
+    elif mode == 'access_control':
+        request_url += option.strip('/')
 
-    try:
-        api_id = json_data['id']
-        api_activeValue = json_data['activeValue']
-        api_name = json_data['name']
-        api_source = json_data['source']
-    except Exception:
-        raise ValueError
-
-    payload.update({"activeValue": api_activeValue,
-                    "id": api_id,
-                    "name": api_name,
-                    "source": api_source,
-                    "value": new_value})
-
-    if api_activeValue != new_value:
-        differs = True
-    else:
-        differs = False
+    return request_url
 
-    return differs, payload
 
+def get_keypair(keypair):
+    if keypair:
+        keypair = keypair.split(':')
+        if len(keypair) == 2:
+            return keypair[0], keypair[1]
 
-def is_valid_rancher_api_option(json_data):
+    return None, None
 
-    try:
-        api_activeValue = json_data['activeValue']
-        api_source = json_data['source']
-    except Exception:
-        return False
 
-    if api_activeValue is None and api_source is None:
-        return False
+def mode_settings(api_url, data=None, headers=None, timeout=default_timeout,
+                  access_key=None, secret_key=None, dry_run=False):
 
-    return True
+    def is_valid_rancher_api_option(json_data):
 
+        try:
+            api_activeValue = json_data['activeValue']
+            api_source = json_data['source']
+        except Exception:
+            return False
 
-def main():
-    module = AnsibleModule(
-        argument_spec=dict(
-            rancher=dict(type='str', required=True,
-                         aliases=['server',
-                                  'rancher_api',
-                                  'rancher_url',
-                                  'api',
-                                  'url']),
-            category=dict(type='str', default='settings',
-                          aliases=['rancher_category', 'api_category']),
-            option=dict(type='str', required=True,
-                        aliases=['name', 'key', 'settings']),
-            value=dict(type='str', required=True),
-            timeout=dict(type='float', default=10.0),
-        ),
-        supports_check_mode=True
-    )
+        if api_activeValue is None and api_source is None:
+            return False
 
-    rancher_url = module.params['rancher'].strip('/')
-    rancher_option = module.params['option'].strip('/')
-    rancher_category = module.params['category']
-    rancher_value = module.params['value']
-    rancher_timeout = module.params['timeout']
-    # cattle_access_key = ''
-    # cattle_secret_key = ''
+        return True
 
-    # Assemble API url
-    request_url = rancher_url + '/v1/' + rancher_category + '/' \
-        + rancher_option
+    def create_rancher_api_payload(json_data, new_value):
 
-    http_headers = {'Content-Type': 'application/json',
-                    'Accept': 'application/json'}
+        payload = {}
+        differs = False
+
+        try:
+            api_id = json_data['id']
+            api_activeValue = json_data['activeValue']
+            api_name = json_data['name']
+            api_source = json_data['source']
+        except Exception:
+            raise ValueError
+
+        payload.update({"activeValue": api_activeValue,
+                        "id": api_id,
+                        "name": api_name,
+                        "source": api_source,
+                        "value": new_value})
+
+        if api_activeValue != new_value:
+            differs = True
+
+        return differs, payload
+
+    # check if data contains all required fields
+    try:
+        if not isinstance(data['option'], str) or data['option'] == '':
+            raise ModeError("ERROR: 'option' must contain a name of the \
+                            option")
+    except KeyError:
+        raise ModeError("ERROR: Mode 'settings' requires the field: 'option': \
+                        %s" % str(data))
+    try:
+        if not isinstance(data['value'], str) or data['value'] == '':
+            raise ModeError("ERROR: 'value' must contain a value")
+    except KeyError:
+        raise ModeError("ERROR: Mode 'settings' requires the field: 'value': \
+                        %s" % str(data))
+
+    # assemble request URL
+    request_url = api_url + 'settings/' + data['option'].strip('/')
 
     # API get current value
     try:
         json_response = get_rancher_api_value(request_url,
-                                              headers=http_headers,
-                                              timeout=rancher_timeout)
+                                              username=access_key,
+                                              password=secret_key,
+                                              headers=headers,
+                                              timeout=timeout)
     except requests.HTTPError as e:
-        module.fail_json(msg=str(e))
+        raise ModeError(str(e))
     except requests.Timeout as e:
-        module.fail_json(msg=str(e))
+        raise ModeError(str(e))
 
     if not json_response:
-        module.fail_json(msg='ERROR: BAD RESPONSE (GET) - no json value \
-                         in the response')
+        raise ModeError('ERROR: BAD RESPONSE (GET) - no json value in the \
+                        response')
 
     if is_valid_rancher_api_option(json_response):
         valid = True
         try:
             differs, payload = create_rancher_api_payload(json_response,
-                                                          rancher_value)
+                                                          data['value'])
         except ValueError:
-            module.fail_json(msg='ERROR: INVALID JSON - missing json values \
-                             in the response')
+            raise ModeError('ERROR: INVALID JSON - missing json values in \
+                            the response')
     else:
         valid = False
 
-    if valid and differs and module.check_mode:
+    if valid and differs and dry_run:
         # ansible dry-run mode
         changed = True
     elif valid and differs:
@@ -214,16 +251,18 @@ def main():
         try:
             json_response = set_rancher_api_value(request_url,
                                                   payload,
-                                                  headers=http_headers,
-                                                  timeout=rancher_timeout)
+                                                  username=access_key,
+                                                  password=secret_key,
+                                                  headers=headers,
+                                                  timeout=timeout)
         except requests.HTTPError as e:
-            module.fail_json(msg=str(e))
+            raise ModeError(str(e))
         except requests.Timeout as e:
-            module.fail_json(msg=str(e))
+            raise ModeError(str(e))
 
         if not json_response:
-            module.fail_json(msg='ERROR: BAD RESPONSE (PUT) - no json value \
-                             in the response')
+            raise ModeError('ERROR: BAD RESPONSE (PUT) - no json value in \
+                            the response')
         else:
             changed = True
     else:
@@ -231,9 +270,78 @@ def main():
 
     if changed:
         msg = "Option '%s' is now set to the new value: %s" \
-            % (rancher_option, rancher_value)
+            % (data['option'], data['value'])
     else:
-        msg = "Option '%s' is unchanged." % (rancher_option)
+        msg = "Option '%s' is unchanged." % (data['option'])
+
+    return changed, msg
+
+
+def mode_handler(server, rancher_mode, data=None, timeout=default_timeout,
+                 account_key=None, dry_run=False):
+
+    changed = False
+    msg = 'UNKNOWN: UNAPPLICABLE MODE'
+
+    # check API key-pair
+    if account_key:
+        access_key, secret_key = get_keypair(account_key)
+        if not (access_key and secret_key):
+            raise ModeError('ERROR: INVALID API KEY-PAIR')
+
+    # all requests share these headers
+    http_headers = {'Content-Type': 'application/json',
+                    'Accept': 'application/json'}
+
+    # assemble API url
+    api_url = server.strip('/') + '/v1/'
+
+    if rancher_mode == 'settings':
+        changed, msg = mode_settings(api_url, data=data,
+                                     headers=http_headers,
+                                     timeout=timeout,
+                                     access_key=access_key,
+                                     secret_key=secret_key,
+                                     dry_run=dry_run)
+    elif rancher_mode == 'access_control':
+        msg = "SKIP: 'access_control' Not yet implemented"
+
+    return changed, msg
+
+
+def main():
+    module = AnsibleModule(
+        argument_spec=dict(
+            rancher=dict(type='str', required=True,
+                         aliases=['server',
+                                  'rancher_api',
+                                  'rancher_server',
+                                  'api']),
+            account_key=dict(type='str', required=False),
+            mode=dict(required=True,
+                      choices=['settings', 'access_control'],
+                      aliases=['api_mode']),
+            data=dict(type='dict', required=True),
+            timeout=dict(type='float', default=default_timeout),
+        ),
+        supports_check_mode=True
+    )
+
+    rancher_server = module.params['rancher']
+    rancher_account_key = module.params['account_key']
+    rancher_mode = module.params['mode']
+    rancher_data = module.params['data']
+    rancher_timeout = module.params['timeout']
+
+    try:
+        changed, msg = mode_handler(rancher_server,
+                                    rancher_mode,
+                                    data=rancher_data,
+                                    account_key=rancher_account_key,
+                                    timeout=rancher_timeout,
+                                    dry_run=module.check_mode)
+    except ModeError as e:
+        module.fail_json(msg=str(e))
 
     module.exit_json(changed=changed, msg=msg)
 
diff --git a/ansible/roles/rancher/tasks/rancher_server.yml b/ansible/roles/rancher/tasks/rancher_server.yml
index add7b3c9..b71bf8d1 100644
--- a/ansible/roles/rancher/tasks/rancher_server.yml
+++ b/ansible/roles/rancher/tasks/rancher_server.yml
@@ -45,20 +45,32 @@
     - name: Main tables
       rancher1_api:
         server: "{{ rancher_server_url }}"
-        option: main_tables.purge.after.seconds
-        value: "{{ rancher.main_tables_purge_after_seconds }}"
+        account_key: "{{ key_public }}:{{ key_private }}"
+        mode: settings
+        data:
+          option: main_tables.purge.after.seconds
+          value: "{{ rancher.main_tables_purge_after_seconds }}"
     - name: Events
       rancher1_api:
         server: "{{ rancher_server_url }}"
-        option: events.purge.after.seconds
-        value: "{{ rancher.events_purge_after_seconds }}"
+        account_key: "{{ key_public }}:{{ key_private }}"
+        mode: settings
+        data:
+          option: events.purge.after.seconds
+          value: "{{ rancher.events_purge_after_seconds }}"
     - name: Service log
       rancher1_api:
         server: "{{ rancher_server_url }}"
-        option: service_log.purge.after.seconds
-        value: "{{ rancher.service_log_purge_after_seconds }}"
+        account_key: "{{ key_public }}:{{ key_private }}"
+        mode: settings
+        data:
+          option: service_log.purge.after.seconds
+          value: "{{ rancher.service_log_purge_after_seconds }}"
     - name: Audit log
       rancher1_api:
         server: "{{ rancher_server_url }}"
-        option: audit_log.purge.after.seconds
-        value: "{{ rancher.audit_log_purge_after_seconds }}"
+        account_key: "{{ key_public }}:{{ key_private }}"
+        mode: settings
+        data:
+          option: audit_log.purge.after.seconds
+          value: "{{ rancher.audit_log_purge_after_seconds }}"