The last years, I only used an external hard drive which is kept offline as my only backup. This approach has two big downsides:
- I'm lazy with backups and don't do them regularly.
- No redundancy in case of fire or accidentially deletion.
So I decided to use an easier solution instead. I came to the decision to use Amazon Web Services. Furthermore, I took some simplifications to keep the whole process as simple as possible:
- I only want to backup my private photo collections which are around 60 GB. It would not be too bad if I lose my MP3 or movie collection.
- Everything should be stored on the HDD in my laptop, on the external HDD and in the Cloud, i.e. I will be following the 3-2-1 rule.
- To make things easier, I only want to backup immutable folders (A photo collection of an event is immutable!). If a single file inside a folder changes, I'm willing to upload the whole folder again.
- Everything should be encrypted by strong cryptography.
After reviewing the cheaper, but more complicated Amazon Glacier, I decided to rather use Amazon Simple Storage Service instead. The total costs will still be below 1 USD/month, so the negligable savings when using Glacier are not worth the extra efforts (e.g. two stage retrieval) using it.
Installing the AWS Command Line Interface (aws-cli) was very easy as it was in the Arch Linux repository and done by a simple
pacman -Syu aws-cli.
After creating a AWS Account, I created a S3 bucket and an AWS IAM user which has full access to this bucket (following this tutorial).
The credentials for the IAM user had to be configured (this are not the real ones!):
$ cat ~/.aws/credentials [default] aws_access_key_id = A89ABXXZIVASSDDQ aws_secret_access_key = +jtJLidi3ld9vlsL9sl9dls9zoif/
Then, I set the cheapest region us-east-1 as the default region and enabled the dualstack support:
$ cat ~/.aws/config [default] region = us-east-1 output = json s3 = use_dualstack_endpoint = true
Now I was able to upload and download files from S3 with a similiar syntax as
An example command is
aws s3 cp s3://my-aws-bucket-name/remote-file.tar.gpg local-file.tar.gpg
For the actual backup, I decided to do it the following way:
- Whenever I get new pictures from my camera or from somebody else, I'll first copy them into a local folder somewhere.
- Then I run a simple script which takes the folder which should be backuped and the actual name as arguments.
- The script prompts me for an encryption passphrase which is identical for all archives. Symmetric encryption has the great advantage that I can't lose my private key file. It is secure enough, too, if only the PGP passphrase is strong enough.
- I need to make sure that the passphrase is not accidentially misspelled, because this would render the encrypted archive as unusable.
- The archive is created, encrypted and stored on the local harddisk in my laptop and also uploaded to the AWS Cloud immediately.
- Whenever I'm not lazy, I'll sync the backup directory of the local harddisk to the external hardddisk.
So whenever someone gives me a collection of photos, I'll run the following script. You can use it, too, all settings which must be changed are marked with "TODO".
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54
All it does is to do some validity checks, read in a passphrase, check the passphrase for typing errors by comparing it to the first few bytes of its SHA512 hash and afterwards copying the encrypted archive to the local disk as well as AWS S3.
The script in action looks like the following (backup-ing the local folder DCIM):
$ ~/bin/awsbackup.sh DCIM/ 2016-12-07_backup_demonstration enter passphrase: not-my-actual-passphrase-enter-your-own-long-one-here SHA512 of password is a044f02a4abc68f4378a86b3f4b32a198ba301845b0cd6e50106e874345700cc6663a86c1ea125dc5e92be17c98f9a0f85ca9d5f595db2012f7cc3571945c123 - Copying files... Creating and uploading encrypted archive "2016-12-07_backup_demonstration.tar.gpg" ... 2016-12-07_backup_demonstration/ 2016-12-07_backup_demonstration/IMG004.jpg 2016-12-07_backup_demonstration/IMG003.jpg 2016-12-07_backup_demonstration/IMG002.jpg 2016-12-07_backup_demonstration/IMG001.jpg Upload done! Removing symlink ...
From time to time, the local backup is manually copied to the external hard disk, too.
Finally, I wrote down the password for my AWS Account as well as the encryption password and stored it at two different locations. Furthermore, I told a relative about it and he was able to restore the backup using the AWS Console and my notes. This should be sufficient that I won't never loose any personal data.