dupicity backup - Dropbox | xmatthias blog

This is one post in a series of tutorials on Duplicity. If you are just getting started with duplicity, I would recommend to first read the first and second part to get a basic overview of duplicity and get GPG keys setup properly (this will not be covered in this post, but is required for the tutorial to work). You should also have a look at the duplicity overview post, which contains a list of all posts in this series. The basic configuration from this post will be used as basis in this post to configure duplicity-backup.sh.

Dropbox

First of all, we obviously need a private Dropbox (Full disclosure: Referral link - non-referral link here) account. The free version currently offers 2 GB free storage, which however can be extended by inviting new people and completing some tasks. Mainly in the past, many were able to get ton of free storage from multiple different actions by Dropbox.

The dropbox-backend does require the dropbox python SDK, which we will install using the following command:

pip install dropbox==6.9.0

Note: version 6.9.0 or below is required, as version 7.1 introduced some breaking changes, which will result in error-messages saying something like TypeError: expected request_binary as binary type, got <type 'file'>.

Next, let’s log into Dropbox and go to the developer tools to create a new “Dropbox API” Application. We will use an “App folder” application here, which limits access to one folder of Dropbox instead of the whole Dropbox and name the application “duplicity_sync_xmatthias”.

Note: The Application name must be unique accross all Dropbox applications, because in theory the application could be published (which we will not do).

dropbox1_cut

Click on Create App to complete the application creation. In the next screen, we can see that the application is in “development” status. All this means is that it is not published (which don’t intend to do) and only this user (and invited test-users) are allowed to use this API key.

According to the duplicty documentation, we now have 2 options - generate the API key on this site - or supply duplicity with the variables DPBX_APP_KEY and DPBX_APP_SECRET and have duplicity run through the OAuth workflow. As everyone creates his own “Application” in Dropbox, there is no benefit to this approach (this would only be viable if you share the application with someone else) - therefore we generate the Access Token in this window. Generate the Access Token as shown in the picture below.

dropbox2_cut

Copy the string which will appear instead of the Button and use it as variable DPBX_ACCESS_TOKEN. (Mine is redacted as it would grant access to my Dropbox). Then, run the command we know from previous Posts to test connectivity with Dropbox.

export DPBX_ACCESS_TOKEN=teBo-PD8bsUAA<redacted>
duplicity --encrypt-key 3E988E6866B39EE1 --sign-key E24E7891636093DB  /tmp/backupTest/ "dpbx:///duplicitytest1"

When first running this command, you may encounter an error similar to mine:

duplicity --encrypt-key 3E988E6866B39EE1 --sign-key E24E7891636093DB  /home/matt/test/ dpbx:///duplictytest2
Exception [ApiError('87ed601063102a7aa75f9e400f9e718e', ListFolderError(u'path', LookupError(u'not_found', None)))]:
| Traceback (most recent call last):
|   File "/usr/lib/python2.7/dist-packages/duplicity/backends/dpbxbackend.py", line 76, in wrapper
|     return f(self, *args)
|   File "/usr/lib/python2.7/dist-packages/duplicity/backends/dpbxbackend.py", line 384, in _list
|     resp = self.api_client.files_list_folder(remote_dir)
|   File "/home/matt/.local/lib/python2.7/site-packages/dropbox/base.py", line 673, in files_list_folder
|     None,
|   File "/home/matt/.local/lib/python2.7/site-packages/dropbox/dropbox.py", line 272, in request
|     user_message_locale)
| ApiError: ApiError('87ed601064102f7da75f9e400f9e718e', ListFolderError(u'path', LookupError(u'not_found', None)))
Attempt 1 failed. BackendException: dpbx api error "ApiError('87ed601063102a7aa75f9e400f9e718e', ListFolderError(u'path', LookupError(u'not_found', None)))"

The duplicity documentation states that the folder needs to exist. Let’s suffice this requirement by going to our Dropbox home, navigate to “Apps/<your_app_name>” and create the folder. In my case, the url to this is https://www.dropbox.com/home/Apps/duplicity_sync_xmatthias - but your’s will differ depending on the selected name for your “application”. You can also use subfolders if multiple separate backups are required in the same account. Please note that Dropbox will sync the encrypted backup to all connected PC’s by default (unselect from “selective sync” if this is not desired).

dropbox3_cut

Create the folder you’d like to use (in my case, duplicitytest1) and rerun the above command.

export DPBX_ACCESS_TOKEN=teBo-PD8bsUAA<redacted>
duplicity --encrypt-key 3E988E6866B39EE1 --sign-key E24E7891636093DB  /tmp/backupTest/ "dpbx:///duplicitytest1"

Duplicity will now backup and upload the data.

duplicity-backup.sh

To add this to the duplicity-backup.sh script we used previously, change DEST to the following and add the access-token as DPBX_ACCESS_TOKEN to the script:

DPBX_ACCESS_TOKEN="teBo-PD8bsUAA<redacted>"
DEST="dpbx:///duplicitytest1"

The rest of the configuration remains identical to the configuration used in the duplicity-backup.sh post.

NOTE: should you receive the error BackendException: dpbx: DPBX_APP_KEY environment variable not set while running the above script, please update duplicity-backup.sh by issuing a git pull as the Dropbox-functionality required a pull-request to fully work. Also, make sure that you use the dev branch, as the PR has so far only been applied there. Previously, the access token has not been exported correctly.

We now run the duplicity-backup.sh script.

./duplicity-backup.sh --config duplicity-backup.conf --backup

A incremental backup should appear in your Dropbox folder.

dropbox4_cut

You may want to now schedule the backup using the cron job as shown in the post presenting duplicity-backup.sh.

Conclusion

Everyone prefers a different cloud-storage provider. All offer different possibilities at different prices and with different advantages / disadvantages. I don’t want to judge which one others should use, and have therefore shown the setup for multiple services. Dropbox presents a good choice from a security point of view by allowing to only expose a dedicated App-folder to duplicity (we’ve seen this differently from onedrive).

In the end, it does not matter which provider is used, as long as the backup is executed regularly, tested and implemented following the “3-2-1” rule:

3 total copies of the data - on 2 different mediums, at least 1 off site.

Full disclosure

If you register with Dropbox using the link above, I will get free space added through the referral program. If you want to avoid this, please use this link or type in https://www.dropbox.com into your browser manually.