diff --git a/Dockerfile b/Dockerfile index 5eebebbf..910842dd 100644 --- a/Dockerfile +++ b/Dockerfile @@ -14,7 +14,7 @@ ADD . /opt/grafana-backup-tool RUN chmod -R a+r /opt/grafana-backup-tool \ && find /opt/grafana-backup-tool -type d -print0 | xargs -0 chmod a+rx -RUN pip3 --no-cache-dir install . +RUN pip3 --no-cache-dir install --break-system-packages . RUN chown -R 1337:1337 /opt/grafana-backup-tool USER 1337 diff --git a/README.md b/README.md index a5e5063a..47d2a484 100644 --- a/README.md +++ b/README.md @@ -3,6 +3,7 @@ A Python-based application to backup Grafana settings using the [Grafana API](https://grafana.com/docs/grafana/latest/http_api/). The aim of this tool is to: + 1. Easily backup and restore Grafana. 2. Have versioned backups`(date and time in file name)` for restoring and saving to cloud storage providers. Currently support 1. `Amazon S3` @@ -10,6 +11,7 @@ The aim of this tool is to: 3. `GCP Cloud Storage` (Use service-account's credential file, [see here](https://cloud.google.com/storage/docs/reference/libraries#linux-or-macos)) ## Supported components + * Folder * Folder Permissions * Library Elements (doesn't work with Grafana 8.0.0 but 8.4.3) @@ -19,14 +21,14 @@ The aim of this tool is to: * Alert Rules (Supported in version 9.4.0 of grafana and up.) * Teams * Team Members (Needs Basic Authentication (username and password, see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#admin-organizations-api)) - * You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=YWRtaW46YWRtaW4=` - * Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization) + * You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=[Generated-Base64-Auth-Token]` + * Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization) * Organization (Needs Basic Authentication (username and password, see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#admin-organizations-api)) - * You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=YWRtaW46YWRtaW4=` - * Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization) + * You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=[Generated-Base64-Auth-Token]` + * Or Sets this ENV of the Grafana server `GF_USERS_ALLOW_ORG_CREATE=true`. see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#create-organization) * User (Needs Basic Authentication (username and password, see [grafana doc](https://grafana.com/docs/grafana/latest/http_api/org/#admin-organizations-api)) - * You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=YWRtaW46YWRtaW4=` - * Grafana's api doesn't provide user's password when backing up, so the `default_user_password` in `grafanaSettings.json`, or in ENV `DEFAULT_USER_PASSWORD`, E.g `export DEFAULT_USER_PASSWORD=supersecret` will be used when restoring. + * You need to set `Admin's account and password` in `grafanaSettings.json`, or set the base64 encoded `admin account and password` in ENV `GRAFANA_BASIC_AUTH`. E.g `export GRAFANA_BASIC_AUTH=[Generated-Base64-Auth-Token]` + * Grafana's api doesn't provide user's password when backing up, so the `default_user_password` in `grafanaSettings.json`, or in ENV `DEFAULT_USER_PASSWORD`, E.g `export DEFAULT_USER_PASSWORD=supersecret` will be used when restoring. * Snapshots * Dashboard Versions (only backup, no restore) * Annotations @@ -34,12 +36,14 @@ The aim of this tool is to: **NOTE** The only supported `orgId` right now is `1`, the default organization will be backed up only! ## Requirements + * Bash * Python 2.7 or Python 3.x * Access to a Grafana API server. * A `Token` of an `Admin` role (see `Configuration` section below for more info) ## Configuration + There are three ways to setup the configuration: 1. Use `environment variables` to define the variables for connecting to a Grafana server. @@ -47,6 +51,7 @@ There are three ways to setup the configuration: 3. Use `~/.grafana-backup.json` to define variables in json format. ### Example Config + * Copy [grafanaSettings.example.json](examples/grafanaSettings.example.json) and modify it for you to use, remove `azure`, `aws`, `gcp`, `influxdb` blocks (but keep the ones you used). * Check out the [examples](examples) folder for more configuration details. @@ -57,7 +62,7 @@ There are three ways to setup the configuration: ```bash ### Do not use a trailing slash on GRAFANA_URL export GRAFANA_URL=http://some.host.org:3000 -export GRAFANA_TOKEN=eyJrIjoidUhaU2ZQQndrWFN3RRVkUnVfrT56a1JoaG9KWFFObEgiLCJuIjoiYWRtaW4iLCJpZCI6MX0= +export GRAFANA_TOKEN=[Grafana-Access-Token] # GRAFANA_HEADERS is optional export GRAFANA_HEADERS=Host:some.host.org @@ -68,30 +73,39 @@ To create and obtain a `Token` for your Grafana server, please refer to the [off **NOTE** that you need to generate a `Token` with an `Admin` role for the backup to succeed, otherwise you will have potential permission issues. ## Installation + ### Virtual environment (optional but recommended) + Create a virtualenv, you could using something like `pyenv` if you'd prefer -``` + +```bash virtualenv -p $(which python3) venv source venv/bin/activate ``` ### Installation using pypi -``` + +```bash pip install grafana-backup ``` ### Installation using this repo + First clone this repo -``` + +```bash git clone https://github.com/ysde/grafana-backup-tool.git cd grafana-backup-tool ``` + Installation works best using `pip` -``` + +```bash pip install . ``` ## How to Use + * First perform the **Configuration** and **Installation** sections as described above. * Use the `grafana-backup save` command to backup all your folders, dashboards, datasources and alert channels to the `_OUTPUT_` subdirectory of the current directory. @@ -115,24 +129,28 @@ $ grafana-backup restore _OUTPUT_/202006272027.tar.gz ``` ## Docker + Replace variables below to use the Docker version of this tool + * `{YOUR_GRAFANA_TOKEN}`: Your Grafana site `Token`. * `{YOUR_GRAFANA_URL}`: Your Grafana site `URL`. * `{YOUR_BACKUP_FOLDER_ON_THE_HOST}`: The `backup folder` on the Grafana host machine. ## Kubernetes + Check out the CronJob in [examples](examples) for a simple example of how grafana-backup-tool can be ran within a Kubernetes environment ### Backup If you decide to use a volume (-v) then you'll need to create the volume first with 1337 uid/gid ownership first, example: -``` + +```bash mkdir /tmp/backup sudo chown 1337:1337 /tmp/backup ``` -``` +```bash docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ -e GRAFANA_TOKEN={YOUR_GRAFANA_TOKEN} \ -e GRAFANA_URL={YOUR_GRAFANA_URL} \ @@ -145,9 +163,9 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ ***Example:*** -``` +```bash docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ - -e GRAFANA_TOKEN="eyJrIjoiNGZqTDEyeXNaY0RsMXNhbkNTSnlKN2M3bE1VeHdqVTEiLCJuIjoiZ3JhZmFuYS1iYWNrdXAiLCJpZCI6MX0=" \ + -e GRAFANA_TOKEN="[Grafana-Access-Token]" \ -e GRAFANA_URL=http://192.168.0.79:3000 \ -e GRAFANA_ADMIN_ACCOUNT=admin \ -e GRAFANA_ADMIN_PASSWORD=adminpassword \ @@ -157,7 +175,8 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ ``` ***S3 Example:*** Set S3 configurations in `-e` or `grafanaSettings.json`([example](https://github.com/ysde/grafana-backup-tool/blob/master/examples/grafana-backup.example.json)) -``` + +```bash -e AWS_S3_BUCKET_NAME="my-backups-bucket" \ -e AWS_S3_BUCKET_KEY="grafana-backup-folder" \ -e AWS_DEFAULT_REGION="us-east-1" \ @@ -166,22 +185,23 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ ``` ***Azure Example:*** Set Azure configurations in `-e` or `grafanaSettings.json`([example](https://github.com/ysde/grafana-backup-tool/blob/master/examples/grafana-backup.example.json)) -``` + +```bash -e AZURE_STORAGE_CONTAINER_NAME="azure-storage-container-name" \ -e AZURE_STORAGE_CONNECTION_STRING="azure-storage-connection-string" ``` ***GCS Example:*** Set GCS configurations in `-e` or `grafanaSettings.json`([example](https://github.com/ysde/grafana-backup-tool/blob/master/examples/grafana-backup.example.json)) -``` + +```bash -e GCS_BUCKET_NAME="bucket-name" \ -e GCLOUD_PROJECT="gcp-project-name" \ -e GOOGLE_APPLICATION_CREDENTIALS="credential-file-path" ``` - ### Restore -``` +```bash docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ -e GRAFANA_TOKEN={YOUR_GRAFANA_TOKEN} \ -e GRAFANA_URL={YOUR_GRAFANA_URL} \ @@ -196,9 +216,9 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ ***Example:*** -``` +```bash docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ - -e GRAFANA_TOKEN="eyJrIjoiNGZqTDEyeXNaY0RsMXNhbkNTSnlKN2M3bE1VeHdqVTEiLCJuIjoiZ3JhZmFuYS1iYWNrdXAiLCJpZCI6MX0=" \ + -e GRAFANA_TOKEN="[Grafana-Access-Token]" \ -e GRAFANA_URL=http://192.168.0.79:3000 \ -e GRAFANA_ADMIN_ACCOUNT=admin \ -e GRAFANA_ADMIN_PASSWORD=adminpassword \ @@ -210,9 +230,11 @@ docker run --user $(id -u):$(id -g) --rm --name grafana-backup-tool \ ``` ### Building + You can build the docker image simply by executing `make` in the root of this repo. The image will get tagged as `ysde:grafana-backup` ### Monitoring + InfluxDB support has been added and Prometheus push gateway support will be added in the future. In order to monitor successful backups with InfluxDB simply configure `grafana-backup` InfluxDB settings using this [example](examples) configuration. diff --git a/grafana_backup/create_alert_rule.py b/grafana_backup/create_alert_rule.py index 9a3fcde5..7d87024b 100644 --- a/grafana_backup/create_alert_rule.py +++ b/grafana_backup/create_alert_rule.py @@ -32,8 +32,8 @@ def main(args, settings, file_path): get_response= get_alert_rule(uid, grafana_url, http_get_headers, verify_ssl, client_cert, debug) status_code=get_response[0] print("Got a code: {0}", status_code) + http_post_headers['x-disable-provenance']='*' if status_code == 404: - http_post_headers['x-disable-provenance']='*' result = create_alert_rule(json.dumps(alert_rule), grafana_url, http_post_headers, verify_ssl, client_cert, debug) else: result = update_alert_rule(alert_rule['uid'], json.dumps(alert_rule), grafana_url, http_post_headers, verify_ssl, client_cert, debug)