duplicity backup - Google Drive

This is one post in a series of tutorials on Duplicity. If you are just getting started with duplicity, I recommend to first read the first and second part to get a basic overview of duplicity and get GPG keys setup properly (this will not be covered in this post). You should also visit the duplicity overview post, which contains a list of all posts in this series. The basic configuration of this post will be used in this post to configure duplicity-backup.sh.

Google Drive

The duplicity manpage lists multiple ways to use duplicity with Google services:

• Google Docs
  • Gdata (Legacy backend)
  • Pydrive
• Google Cloud Storage
• pydrive

While Google Docs and pydrive backends save the content to Google Drive (using different methods), which comes for free with any Google account - the Google Cloud Storage backend is their paid service. There is a free tier, however this post will focus on Google drive using pydrive.

Pydrive itself works in 2 ways - by using the regular account (username + password) or with a service account. For now, we will focus on the regular account (it will only save the OAuth-token, not the password / username).

Getting started

The Google Drive backend requires pydrive (the python package) to be installed:

pip install pydrive

We will also need a Google account. Go to the Google developer console and create a new project. Insert a name you’d like (I’m using “duplicity-sync-xmatthias” for this demonstration).

Next, go to the API overview and enable the Google drive API.

After enabling the Drive API, you will be forwarded to the overview of this API.

Click on credentials on the left hand side and configure the OAuth consent screen to your liking. As only you are going to see this screen, it is important to insert something that is meaningful to, but does not need to be complete.

Next, we create a new OAuth Client ID, select other and insert the name for our client (in this case, i used cmd-client.

Now, we are presented with a client-id and a client secret.

(Don’t bother to try the above keys - by the time this post goes live I already deleted the test application)..

We will paste the client-id and client-secret into the pydrive settings file “pydrive.conf” as specified on the duplicity manpage.

client_config_backend: settings
client_config:
    client_id: <Client ID from developers’ console>
    client_secret: <Client secret from developers’ console>
save_credentials: True
save_credentials_backend: file
save_credentials_file: <filename to cache credentials>
get_refresh_token: True

Ours:

client_config_backend: settings
client_config:
    client_id: 76061484583-nfoocmj6e891ml1sbmrq7m4gob4b5qht.apps.googleusercontent.com
    client_secret: 8bXpRP9agTBBZiA7X11X6dKk
save_credentials: True
save_credentials_backend: file
save_credentials_file: /home/matt/pydrivecreds.json
get_refresh_token: True

Now, we need to export the environment variable so duplicity can find this configuration file

export GOOGLE_DRIVE_SETTINGS=/home/matt/pydrive.conf
duplicity full --encrypt-key 3E988E6866B39EE1 --sign-key E24E7891636093DB /home/matt pydrive+gdocs://developer.gserviceaccount.com/duplicity_backup_test/

During this first run, we will be asked to visit a Url. As we do so, we are asked to choose an account - and then allow the application to access the Drive folder.

After allowing, we are presented with a code to paste back into the command line.

Press enter and you should see “Authentication successful.”.

Duplicity will now backup and upload the data. After the process is complete, we can check what duplicity did by going to Google Drive and checking the new folder.

To revoke the application access, go to the Google account Permissions page and revoke access to the Application (It should show up under whatever name you gave it in the OAuth screen setup - in my case it is “duplicity-backup”).

duplicity-backup.sh

We can integrate this process fairly easily in the duplicity-backup.sh script this way:

Change DEST to the following:

DEST="pydrive+gdocs://developer.gserviceaccount.com/duplicity_backup_test/"

In addition to this, we also need to export GOOGLE_DRIVE_SETTINGS. We do this by adding the following line right below the DEST line.

export GOOGLE_DRIVE_SETTINGS=/home/matt/pydrive.conf

The remainder of the configuration remains identical to the configuration used in the duplicity-backup.sh post.

We now run the duplicity-backup.sh script.

./duplicity-backup.sh --config duplicity-backup.conf --backup

If everything goes well, Google Drive should show the incremental files as well.

If you still have the cronjob scheduled and used the same configuration file, the backup will automatically go to google Drive instead of the sftp server or a different backend.

Closing

This is yet another method to backup your files to a cloud service securely. Again, the drawback is that the server gains access to your whole Google Drive. Depending on your usage of Google Drive, this may or may not be a problem. As i don’t really use Google Drive for anything else and have plenty of free space available, I am fine with this (The OAuth-tokens saved on the server do only grant access to Google Drive, but to no other Service). Alternatively, you could also create a dedicated account for Duplicity to run the backups.