CLAMP Dashboard
===============

ELK stack for CLAMP : Logstash is used to retrieve events and notifications from DMaaP and pushes them into Elasticsearch.
Kibana is then used to extract statistics.


Deployment instructions
-----------------------

Requirements: docker-compose

1. Update configuration in docker-compose file
2. `docker-compose up -d elasticsearch logstash kibana`

If you encounter connection problems with kibana, first deploy elasticsearch, wait for it to be available, then kibana.

Backup/restore
--------------

backup.py and restore.py scripts are available inside the kibana docker image for saving and restoring the configuration.
the default configuration is located in the kibana docker image under the directory "/saved-objects/default/"

### backup.py 
```
firts create the directory "/saved-objects/mybackup" if it doesn't exist
docker-compose exec kibana backup.py -C /saved-objects/mybackup
```
```
usage: backup.py [-h] [-v] [-C CONFIGURATION_PATH] [-f] [-H KIBANA_HOST]

Description of the script

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         Use verbose logging
  -C CONFIGURATION_PATH, --configuration_path CONFIGURATION_PATH
                        Path of the configuration to be backed up.
  -f, --force           If the save folder already exists, overwrite files
                        matching a configuration item that should be written.
                        Files already in the folder that do not match are left
                        as-is.
  -H KIBANA_HOST, --kibana-host KIBANA_HOST
                        Kibana endpoint.

```

### restore.py
```
to restore the configuration, you previously backed up, use the command below:
docker-compose exec kibana restore.py -C /saved-objects/mybackup -f
to restore the default confgiuration use the command below:
docker-compose exec kibana restore.py -C /saved-objects/default -f
```
```
usage: restore.py [-h] [-v] [-C CONFIGURATION_PATH] [-H KIBANA_HOST] [-f]

Restores the kibana configuration.

optional arguments:
  -h, --help            show this help message and exit
  -v, --verbose         Use verbose logging
  -C CONFIGURATION_PATH, --configuration_path CONFIGURATION_PATH
                        Path of the configuration to be restored.Should
                        contain at least one folder named index-
                        pattern,config,search,visualization or dashboard
  -H KIBANA_HOST, --kibana-host KIBANA_HOST
                        Kibana endpoint.
  -f, --force           Overwrite configuration if needed.
```

Tools
-----

The following tools are available in the 'tools/' folder.


### EsAutoQuery

Small script ease Elasticsearch /painless/ field development.
It reads a json file as a query for Elasticsearch, pushes it on the ES server, and display back the answer in a loop, each time the file is modified.


### DMaaP Service Mocker

Script that simulates control loop DMaaP services to provide sample data to logstash through DMaaP.

TODO
----
* Add a script that verifies that elasticsearch is available before starting loading the default configuration for kibana