Amazon S3’s Reduced Redundancy Storage

Amazon has released a new feature that many of you will be taking advantage of.  It’s called Reduced Redundancy Storage.  Amazon typically stores your files on multiple devices at multiple data centers.  When you store a file with the reduced redundancy storage option, the file is still stored on multiple devices at multiple data centers but fewer times.  The benefit of this is a lower price to store your files.  It costs 33% less to store files with reduced redundancy.  The drawback is that Amazon may lose one of your files.  For this reason you should only use this option if the file you are storing can be regenerated or restored from another location.

A typical use for reduced redundancy would be for the storage of programmatically generated thumbnails of a picture.  Suppose a user uploads a picture to your site.  You generate a thumbnail of the picture and store the original and the thumbnail on Amazon S3.  You can choose to store the original with the standard storage class and the thumbnail with the reduced redundancy storage class.  If the thumbnail is ever lost you can regenerate it from the original.

Amazon calculates reduced redundancy storage is 99.99% durable as opposed to standard storage’s 99.999999999% durability.  Amazon also calculates reduced redundancy storage provides 400 times the durability of a typical disk drive.

You can choose to make a file reduced redundancy when you upload it. Setting a special request header in the upload tells Amazon you want to use reduced redundancy for that file. The code snippet below shows how to upload a file with the SprightlySoft S3 Component for .NET and make it reduced redundancy.

Dictionary<String, String> ExtraRequestHeaders = new Dictionary<String, String>();
ExtraRequestHeaders.Add("x-amz-storage-class", "REDUCED_REDUNDANCY");

SprightlySoftS3.Upload MyUpload = new SprightlySoftS3.Upload();
MyUpload.UploadFile("AWSAccessKeyId", "AWSSecretAccessKey", "BucketName", "KeyName", "FilePath", ExtraRequestHeaders);

You can tell which files are using reduced redundancy when you list a bucket.  The StorageClass value will be STANDARD or REDUCED_REDUNDANCY.

You can configure Amazon to send you a notification through Amazon’s Simple Notification Service in the event that Amazon loses one of your reduced redundancy files.  First you set up a Simple Notification Service topic that will load a web page, send an email or add an Amazon Simple Queue Service item when it is triggered.  Next you configure your bucket to trigger the topic when the ReducedRedundancyLostObject event occurs.  Say your application stores images and thumbnails.  Create a web page to regenerate the thumbnail from the original.  Set up a Simple Notification Service topic to call your web page what it runs.  Then configure your bucket to call the topic if it loses a file.It’s a bit more work but it can pay for itself it you store many files with the reduced redundancy storage option.

To see examples of working with reduced redundancy files in C# or VB.NET try the SimpleExamples project included with the SprightlySoft S3 Component for .NET.