Blog OS backup strategy using EC2 AMI tools

This article is linked to the previous article Your Data In the Cloud. This backup strategy is available for both instance root types. It was nevertheless originally designed to backup an instance whose root device was instance-store. If you intend to backup an EBS-backed AMI instance, please note that this process will create an S3-backed AMI that we will convert back to EBS afterwards. The main advantage of this solution is the ability to download your backup.

Creating an S3-backed AMI

This backup is a hot backup : it will create an AMI while your system is still running. To prevent any data inconsistency, at least stop all the main processes which could modify files during the backup process.

1. Upload the EC2 AMI Tools to the server you want to backup.
2. Make all the scripts available in bin executable :

chmod +x =/root/ec2-ami-tools-1.3-56066/bin/*

3. Export the required environment variables :

export PATH=/root/ec2-ami-tools-1.3-56066/bin:$PATH
export EC2_AMITOOL_HOME=/root/ec2-ami-tools-1.3-56066/
export EC2_HOME=/root/ec2-ami-tools-1.3-56066/

4. Bundle your volume to the non-persistent volume by default available on /mnt for all the AWS instances :

ec2-bundle-vol -d /mnt -k private_key.pem -c certificate.pem -u account_number -r i386
Copying / into the image file /mnt/image...
Excluding:
/sys/kernel/debug
/sys/kernel/security
/sys
/proc
/dev/pts
/dev
/dev
/media
/mnt
/proc
/sys
/mnt/image
/mnt/img-mnt
1+0 records in
1+0 records out
1048576 bytes (1.0 MB) copied, 0.0104679 s, 100 MB/s
mke2fs 1.41.11 (14-Mar-2010)
warning: Unable to get device geometry for /mnt/image
Splitting /mnt/image.tar.gz.enc...
Created image.part.00
...
Generating digests for each part...
Digests generated.
Unable to read instance meta-data for ancestor-ami-ids
Unable to read instance meta-data for ramdisk-id
Unable to read instance meta-data for product-codes
Creating bundle manifest...
ec2-bundle-vol complete.

5. Create a bucket (here nimbleapps) and a folder (here image) in your S3 Volume (AWS Management Console, Tab S3). Upload your image on your S3 storage :

ec2-upload-bundle -b nimbleapps/image -m /mnt/image.manifest.xml -a Access_Key_ID -s Secret_Access_Key
Uploading bundled image parts to the S3 bucket nimbleapps ...
Uploaded image.part.00
...
Uploading manifest ...
Uploaded manifest.
Bundle upload completed.

You should be able to see your image in your S3 bucket : you can now download your OS backup locally via any S3 tool such as S3 browser. You are no longer relying on Amazon to host your OS backup.

Launching an S3-backed instance from the AMI

1. Back in the AWS Management console, tab EC2, section AMIs, click the button Register New AMI.
2. Enter the link to your image manifest :

http://s3.amazonaws.com:80/bucket_name/folder/image.manifest.xml

3. Go to the Instances section and click Launch Instance.
4. Click on the MyAMIs tab. Select the AMI created earlier and follow the guidance. Your instance should startup automatically.

Launching an EBS backed instance from the AMI

The problem with the previous process is that it prevents you from having an EBS-backed AMI instance. You can choose to convert your S3-backed AMI to an EBS instance by doing the following :

1. Start any EBS-backed AMI instance.
2. Upload the ec2-ami-tools in your instance.
3. Create an extra EBS volume, attach it to your instance. (Eg. device /dev/sdg)
4. Format it. For example :

mkfs.ext3 /dev/sdg

5. Download your S3-backed AMI into the non persistent volume mount in /mnt :

ec2-download-bundle -b nimbleapps/image -a Access_Key_ID -s Secret_Access_Key -k private_key.pem -d /mnt
Downloading manifest image/image.manifest.xml from nimbleapps to /mnt/image.manifest.xml ...
Downloading part image/image.part.00 to /mnt/image.part.00 ...
...

6. Unbundle your image like this :

ec2-unbundle -k private_key.pem -m /mnt/image.manifest.xml -s /mnt/ -d /mnt/
Unbundle complete.

7. Dump the unbundled image on the device :

dd if=/mnt/image of=/dev/sdg

8. Mount the volume /dev/sdg (E.g in /media) and delete the following line in the fstab file (E.g /media/etc/fstab) :

/dev/sda2 /mnt auto defaults,comment=cloudconfig 0 0

9. Umount the volume, detach it in AWS Management Console.
10. In AWS Management Console, Volume section, right-click on your volume => Create Snapshot From Volume. You can then delete the previous volume.

To start the EBS-backed AMI instance from the snapshot :

1. Take an EBS-backed AMI instance with a Kernel ID compatible with the one you want to restore.
2. Stop this instance, detach the root volume.
3. Create a volume from the snapshot.
4. Attach this new volume as the root device of your instance.

Conclusion

The EC2 AMI tools allows you:

• to create a backup available for download : you no longer rely on Amazon only to host your backup.


but it also :

• makes a hot backup which could lead to some inconsistencies.