SprightlySoft Releases S3 Delete

SprightlySoft is pleased to announce the release of S3 Delete. S3 Delete is a command line program that allows Windows users to delete files from Amazon S3.

Amazon S3 is a web service that allows you to store and retrieve files. You can upload as many files you’d like and they can be made available through the Internet. There is no upfront cost for using the service and you only pay for what you use.

SprightlySoft S3 Delete allows you to delete files and folders in your S3 account. The program uses parallel processing so multiple calls to delete files are made at the same time. This makes deleting many files faster. The program can be used to only delete files of a certain type or age.

S3 Delete is completely free to use and it is open source. You may make changes to the code or include it’s functionality in your own application. Download the application and .NET source code from http://sprightlysoft.com/S3Delete/.

Amazon Simple Notification Service Script

Amazon Simple Notification Service (SNS) is a web service that allows you to notify distributed applications by pushing a message to them.

Say you have an application that runs on many servers. In a traditional application the computers would continually poll a queue or database looking for a job to process. Many computers continually polling will reduce performance of the queue and result in higher usage fees. You can increase the time between polls to reduce load but this decreases the performance of your application.

One way to have your computers process a job immediately and eliminate the need to poll a queue is to use Amazon’s Simple Notification Service. First you set up a web server on each of the computers that run your application. Next, set up a SNS subscription in Amazon to send messages to your computers. Then create an application on your web servers to do something when they receive a SNS message. Finally, configure your application to call SNS when something needs to be done.

Here is a representation of the flow.
Your application –> Amazon SNS –> Your web servers –> Do work

To get familiar with Amazon SNS you can log into the AWS Console with your Amazon credentials. Start by creating a Topic and add a Subscription that will specify where the messages will go. You can send a message to an email address, web site URL, or an Amazon Simple Queue Service queue.

When you create a subscription you will need to confirm you have permission to send messages there. With an email subscription you will be sent an email and you will need to click a link in that email. When you create a web URL subscription Amazon sends a message to that URL. You will need to capture this message and respond to it by loading a special URL in the message or calling the API with a token that is included in the message. If you do not confirm the subscription, Amazon will not send messages to it.

The process of configuring a web URL to receive a SNS message is tricky. You need to configure the web server before you set up the subscription. The following describes how to set up a Windows web server to respond to Amazon SNS messages. The main points of the article are setting up a script to automatically confirm a SNS subscription and to verify that the SNS message came from Amazon.

Start by downloading the attached Visual Studio project. You will need Visual Studio 2010 to work with it. The project contains a file called AutoConfirm.aspx. This is the script you will point your SNS subscription to. At the beginning of the code you will see variables that hold your Amazon AccessKeyId and SecretAccessKey. Fill these variables with your values. Next you will see variables related to email settings. The script is configured to send an email whenever a message is received. Fill these values with your settings.

String AWSAccessKeyId = "xxxxxxxxxxxxxxxxxxxxx";
String AWSSecretAccessKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx";

String EmailFromAddress = "account@gmail.com";
String EmailToAddress = "me@company.com";
String EmailSmtpHost = "smtp.gmail.com";
int EmailSmtpPort = 587;
Boolean EmailSmtpEnableSsl = true;
String EmailSmtpUserName = "account@gmail.com";
String EmailSmtpPassword = "mypassowrd";

The script starts by gathering all information that was sent to it. An interesting fact of SNS messages is that you cannot access the data sent to you through the typical Request.Form or Request.QueryString methods. Information sent to your script will only be available through Request.InputStream. The code demonstrated how to convert Request.InputStream into a string.

//The LogMessage will contain information about what the script did. It will be sent through email at the end.
String LogMessage = "";
LogMessage += "DateTime = " + DateTime.Now + Environment.NewLine;
LogMessage += "SERVER_NAME = " + Request.ServerVariables["SERVER_NAME"] + Environment.NewLine;
LogMessage += "LOCAL_ADDR = " + Request.ServerVariables["LOCAL_ADDR"] + Environment.NewLine;
LogMessage += "REMOTE_ADDR = " + Request.ServerVariables["REMOTE_ADDR"] + Environment.NewLine;
LogMessage += "HTTP_USER_AGENT = " + Request.ServerVariables["HTTP_USER_AGENT"] + Environment.NewLine;
LogMessage += "CONTENT_TYPE = " + Request.ServerVariables["CONTENT_TYPE"] + Environment.NewLine;
LogMessage += "SCRIPT_NAME = " + Request.ServerVariables["SCRIPT_NAME"] + Environment.NewLine;
LogMessage += "REQUEST_METHOD = " + Request.ServerVariables["REQUEST_METHOD"] + Environment.NewLine;
LogMessage += "Request.QueryString = " + Request.QueryString.ToString() + Environment.NewLine;
LogMessage += "Request.Form = " + Request.Form.ToString() + Environment.NewLine;

//Convert the Request.InputStream into a string.
byte[] MyByteArray = new byte[Request.InputStream.Length];
Request.InputStream.Read(MyByteArray, 0, Convert.ToInt32(Request.InputStream.Length));

String InputStreamContents;
InputStreamContents = System.Text.Encoding.UTF8.GetString(MyByteArray);

At this point we have the data that was sent to us. Amazon sends the message in JSON format. The script converts the JSON message into a Dictionary using the System.Web.Script.Serialization.JavaScriptSerializer class. You can access the class by adding a reference to System.Web.Extensions.dll in your project. The class is only available in .NET Framework 4.0. If your web server does not support .NET 4.0 you will need to rewrite this part of the code to parse the variables from the message using another method.

//Convert the JSON data into a Dictionary.  Use of the System.Web.Script namespace requires a refrence to System.Web.Extensions.dll.
System.Web.Script.Serialization.JavaScriptSerializer MyJavaScriptSerializer = new System.Web.Script.Serialization.JavaScriptSerializer();

System.Collections.Generic.Dictionary<String, Object> MyObjectDictionary;
MyObjectDictionary = MyJavaScriptSerializer.DeserializeObject(InputStreamContents) as System.Collections.Generic.Dictionary<String, Object>;

Now that all the variables are in a dictionary it is simple to access them. For example use MyObjectDictionary[“TopicArn”].ToString() to give you the value of TopicArn in the message.

Next the code validates that the message really did come from Amazon. This is done by taking the values in the message, and then constructing a string with the values in a specific order. The construction of this string is described at http://sns-public-resources.s3.amazonaws.com/SNS_Message_Signing_Release_Note_Jan_25_2011.pdf. Amazon constructed the same string on their end and signed the string before sending the message to you. The signed string value is included in the Signature value of the message. You will use these values to confirm the signature is valid.

StringBuilder MyStringBuilder = new StringBuilder();

String GeneratedMessage;
GeneratedMessage = MyStringBuilder.ToString();

The VerifySignature function downloads Amazon’s public certificate key and uses it to create a hash of the variables in the message. If the hash matches the Signature value then the message did come from Amazon. Only Amazon has the private certificate key which is required to generate signatures.

private static Boolean VerifySignature(String GeneratedMessage, String SignatureFromAmazon, String SigningCertURL)
System.Uri MyUri = new System.Uri(SigningCertURL);

//Check if the domain name in the SigningCertURL is an Amazon URL.
if (MyUri.Host.EndsWith(".amazonaws.com") == true)
  byte[] SignatureBytes;
  SignatureBytes = Convert.FromBase64String(SignatureFromAmazon);

  //Check the cache for the Amazon signing cert.
  byte[] PEMFileBytes;
  PEMFileBytes = (byte[])System.Web.HttpContext.Current.Cache[SigningCertURL];

  if (PEMFileBytes == null)
    //Download the Amazon signing cert and save it to cache.
    System.Net.WebClient MyWebClient = new System.Net.WebClient();
    PEMFileBytes = MyWebClient.DownloadData(SigningCertURL);

    System.Web.HttpContext.Current.Cache[SigningCertURL] = PEMFileBytes;

  System.Security.Cryptography.X509Certificates.X509Certificate2 MyX509Certificate2 = new System.Security.Cryptography.X509Certificates.X509Certificate2(PEMFileBytes);

  System.Security.Cryptography.RSACryptoServiceProvider MyRSACryptoServiceProvider;
  MyRSACryptoServiceProvider = (System.Security.Cryptography.RSACryptoServiceProvider)MyX509Certificate2.PublicKey.Key;

  System.Security.Cryptography.SHA1Managed MySHA1Managed = new System.Security.Cryptography.SHA1Managed();
  byte[] HashBytes = MySHA1Managed.ComputeHash(Encoding.UTF8.GetBytes(GeneratedMessage));

  return MyRSACryptoServiceProvider.VerifyHash(HashBytes, System.Security.Cryptography.CryptoConfig.MapNameToOID("SHA1"), SignatureBytes);
  return false;

If the signature is valid the code automatically responds to ConfirmSubscription requests. You get a ConfirmSubscription request as soon as you create a subscription. The message contains a special token you will use to confirm the subscription. The code does this by calling the ConfirmSubscription API method. Information on this method can be found at http://docs.amazonwebservices.com/sns/latest/api/API_ConfirmSubscription.html. The code uses the SprightlySoft AWS Component to make the API request. This component is free and it makes it easy to send the required authorization parameters to the API. See http://sprightlysoft.com/AWSComponent/ for more information.

Boolean RetBool;
int ErrorNumber = 0;
String ErrorDescription = "";
String LogData = "";
Dictionary<String, String> RequestHeaders = new Dictionary<String, String>();
int ResponseStatusCode = 0;
String ResponseStatusDescription = "";
Dictionary<String, String> ResponseHeaders = new Dictionary<String, String>();
String ResponseString = "";

//This a SubscriptionConfirmation message. Get the token.
String Token;
Token = MyObjectDictionary["Token"].ToString();

//Get the endpoint from the SigningCertURL value.
String TopicEndpoint;
TopicEndpoint = MyObjectDictionary["SigningCertURL"].ToString();
TopicEndpoint = TopicEndpoint.Substring(0, TopicEndpoint.IndexOf(".amazonaws.com/") + 15);

//Amazon Simple Notification Service, ConfirmSubscription: <a href="http://docs.amazonwebservices.com/sns/latest/api/API_ConfirmSubscription.htmlString">http://docs.amazonwebservices.com/sns/latest/api/API_ConfirmSubscription.html
String</a> RequestURL;
RequestURL = TopicEndpoint + "?Action=ConfirmSubscription&Version=2010-03-31";
RequestURL += "&Token=" + System.Uri.EscapeDataString(Token);
RequestURL += "&TopicArn=" + System.Uri.EscapeDataString(TopicArn);

//Use the SprightlySoft AWS Component to call ConfirmSubscription on Amazon.
RetBool = MakeAmazonSignatureVersion2Request(AWSAccessKeyId, AWSSecretAccessKey, RequestURL, "GET", null, "", 3, ref ErrorNumber, ref ErrorDescription, ref LogData, ref RequestHeaders, ref ResponseStatusCode, ref ResponseStatusDescription, ref ResponseHeaders, ref ResponseString);

The code looks at the response from the API. A successful request will result in a XML document with the SubscriptionArn value.

if (RetBool == true)
  System.Xml.XmlDocument MyXmlDocument;
  System.Xml.XmlNamespaceManager MyXmlNamespaceManager;
  System.Xml.XmlNode MyXmlNode;

  MyXmlDocument = new System.Xml.XmlDocument();

  MyXmlNamespaceManager = new System.Xml.XmlNamespaceManager(MyXmlDocument.NameTable);
  MyXmlNamespaceManager.AddNamespace("amz", "http://sns.amazonaws.com/doc/2010-03-31/");

  MyXmlNode = MyXmlDocument.SelectSingleNode("amz:ConfirmSubscriptionResponse/amz:ConfirmSubscriptionResult/amz:SubscriptionArn", MyXmlNamespaceManager);

  LogMessage += "ConfirmSubscription was successful. SubscriptionArn = " + MyXmlNode.InnerText + Environment.NewLine;
  LogMessage += Environment.NewLine;
  LogMessage += "ConfirmSubscription failed." + Environment.NewLine;
  LogMessage += "Response Status Code: " + ResponseStatusCode + Environment.NewLine;
  LogMessage += "Response Status Description: " + ResponseStatusDescription + Environment.NewLine;
  LogMessage += "Response String: " + ResponseString + Environment.NewLine;
  LogMessage += Environment.NewLine;

The script finishes by sending an email with information on what took place. This email makes it easy to see when your script is called and what took place.

//Send an email with the log information.
System.Net.Mail.SmtpClient MySmtpClient = new System.Net.Mail.SmtpClient();
MySmtpClient.Host = EmailSmtpHost;
MySmtpClient.Port = EmailSmtpPort;
MySmtpClient.EnableSsl = EmailSmtpEnableSsl;
if (EmailSmtpUserName != "")
  System.Net.NetworkCredential MyNetworkCredential = new System.Net.NetworkCredential();
  MyNetworkCredential.UserName = EmailSmtpUserName;
  MyNetworkCredential.Password = EmailSmtpPassword;
  MySmtpClient.Credentials = MyNetworkCredential;
System.Net.Mail.MailAddress FromMailAddress = new System.Net.Mail.MailAddress(EmailFromAddress);
System.Net.Mail.MailAddress ToMailAddress = new System.Net.Mail.MailAddress(EmailToAddress);
System.Net.Mail.MailMessage MyMailMessage = new System.Net.Mail.MailMessage(FromMailAddress, ToMailAddress);

if (TopicArn == "")
  MyMailMessage.Subject = "Log Information For " + Request.ServerVariables["SERVER_NAME"] + Request.ServerVariables["SCRIPT_NAME"];
  MyMailMessage.Subject = "Log Information For " + TopicArn;

MyMailMessage.Body = LogMessage;


Now that you know how the script works it’s time to publish it to a web server. The script requires a Windows web server running the .NET framework 4.0. The web server must be accessible from the Internet in order for Amazon to send messages to it.

When you have published the script you can set up a SNS subscription in Amazon. Go to the AWS Console (http://aws.amazon.com/console/) and create a new topic. Once you have a topic, create a subscription under it. For protocol select HTTP. Under endpoint enter the URL to the script you published.

If everything went right your subscription will immediately be confirmed and you will receive an email from the script with details. You will then be able to publish messages to your subscription.

Download SprightlySoft_SNSAutoConfirm_v1 – 68.48 KB

SprightlySoft AWS Component Source Code Now Available

The source code for the SprightlySoft AWS Component is now available to download.  The AWS Component allows Microsoft developers to quickly build applications that interact with Amazon Web Services.  It supports all services offered by Amazon and includes sample code so you can see how to implement functions you need.

You may want a copy of the source code for the following reasons:

  • Add functionality that does not currently exist.
  • Review the code for corporate compliance.
  • Use functions in your project without the need to reference a dll file.
  • Have complete peace of mind.

Access to the source code is permitted with a paid support agreement.  For full details visit http://sprightlysoft.com/AWSComponent/SupportAgreement.aspx

SprightlySoft Releases S3 Upload

SprightlySoft is pleased to announce the release of S3 Upload.  S3 Upload is a command line program for Windows that allows you to upload a file to Amazon S3.

Amazon S3 is a web service that allows you to store and retrieve files.  There is no upfront cost for using the service and you only pay for what you use.  It’s reliable so you will never lose you data, and it is scalable so you can upload as much as you want.

SprightlySoft S3 Upload allows you to upload a file to Amazon S3.  It uses Amazon’s multipart upload feature so you can upload files up to 5 terabytes in size and you can resume an upload if it is interrupted. 

S3 upload is completely free to use and it is open source.  You may make changes to the code or include the functionality in your own application.  Download the application and .NET source code from http://sprightlysoft.com/S3Upload/.

Amazon Endorses the SprightlySoft AWS Component

SprightlySoft is pleased to announce that Amazon has recommended the SprightlySoft AWS Component for .NET to developers looking to interact with Amazon Route 53.  The endorsement is included in the Amazon Route 53 Getting Started Guide.  See http://docs.amazonwebservices.com/Route53/latest/GettingStartedGuide/WhereGoFromHere.html for details.

Amazon Route 53 is a scalable Domain Name System (DNS) web service.  You would use it to point your domain names to the Amazon services you use.  When using Amazon web services you are continually bringing computers online and taking them offline.  You need to update your domain names quickly to reflect these changes.  Amazon Route 53 allows you to programmatically make changes to your domain name configuration.  Route 53 is also super quick and can handle massive amounts of users using your services.

The SprightlySoft AWS Component allows Microsoft developers to quickly and easily create applications that interact with Amazon Route 53.  The component includes sample code in C# and VB.NET for all Route 53 functions.  Best of all it is completely free to use.

SprightlySoft hopes many developers will find the component useful and looks forward to building more great applications that use Amazon web services.

Amazon S3 Multipart Uploads

Sample code for Amazon S3 multipart uploads has been added to the SprightlySoft AWS Component for .NET.  Multipart uploads is a new feature in S3 that allows you to upload a file in parts.  When all the parts have been uploaded you tell S3 to reassemble all the parts into the original file.  Some of the benefits of multipart uploads are:

  • Ability to resume an upload.  Previously if an upload failed you needed to restart the upload from the beginning.  If you uploaded 800 MB of a 900 MB file and your Internet connection was interrupted, you need to re-upload the entire file again.  With multipart uploads you upload the file in 5 MB chunks.  When a chunk is complete it sits on S3 until you are ready to complete the upload.  If your Internet connection goes down while you are uploading only the current chunk you are working on is lost.  You resume uploading from the last complete chunk on S3.
  • Concurrent uploads.  With multipart uploads you can have multiple threads uploading different parts of the same file to S3 at the same time.  You may see increased transfer speed when uploading this ways and quicker speed mean a faster upload.
  • Large object support.  Amazon S3 recently increased the maximum size of a file on S3 from 5 GB to 5 TB.  5 GB may seem like a lot but there are cases where people need to store larger files.  For example a Blu-ray movie may be 50 GB and a 2-hour uncompressed 1080p HD video is around 1.5 terabytes.  Before large object support you would need to break large files into smaller parts before sending them to S3 and reassemble them after you download the parts from S3. 

You can find the multipart uploads examples in the S3AllOperations project with the SprightlySoft AWS Component for .NET.  There is sample code in C# and VB.NET.  The component is free so give it a try.

Amazon Route 53

SprightlySoft is pleased to announce the SprightlySoft AWS Component for .NET fully supports the new Amazon Route 53 service. 

Amazon Route 53 is a scalable Domain Name System (DNS) web service. It provides secure and reliable routing to your infrastructure that uses Amazon Web Services (AWS) products, such as Amazon Elastic Compute Cloud (Amazon EC2), Elastic Load Balancing, or Amazon Simple Storage Service (Amazon S3). You can also use Amazon Route 53 to route users to your infrastructure outside of AWS.

A function has been added to the SprightlySoft AWS Component for .NET that allows you easlity generate an authorization signature which is required when making requests to the service.  In addition sample code has been added in C# and VB.NET that shows you how to call each Amazon Route 53 function.

You can interact with the Amazon Route 53 API through the following web page: http://sprightlysoft.com/AmazonRoute53/.  Try listing your host zones or adding a new host zone.

Amazon S3 Sync

SprightlySoft has released an Amazon S3 synchronization application. This application allows you to take a folder on your computer and upload it to S3. You can make additions, deletions and changes to your local files and next time you run the application it will detect these changes and apply them to Amazon S3. This program allows you to create a mirror of a local folder on S3 and always keep it up to date.

Amazon Simple Storage Service (Amazon S3) is a service that allows you to store files in Amazon’s cloud computing environment. When your files are in Amazon’s system you can retrieve the files from anywhere on the web. You can also use Amazon’s CloudFront service in conjunction with S3 to distribute your files to millions of people. Amazon S3 is the same highly scalable, reliable, secure, fast, inexpensive infrastructure that Amazon uses to run its own global network of web sites. Best of all Amazon S3 is free for the first 5 GB of storage.

S3 Sync is open source and completely free to use. Visit http://sprightlysoft.com/S3Sync/ to download the application or get the source code.

The following gives you a rundown of how the source code works. This information is useful if you want to learn how the program works so you can make changes to it.

S3 Sync works by listing your files on Amazon S3, listing your files locally and comparing the differences between the two lists. Here is the logic of the program.

  1. The code starts by getting a list of all user settings. These include your Amazon AWS credentials, the S3 bucket you are syncing to, and the local folder you are syncing from.
  2. The next major call is the PopulateS3HashTable function. Here all the files in your Amazon S3 bucket is listed. The code calls the ListBucket function from the SprightlySoft AWS Component. This function returns an ArrayList of objects that represent each item in S3. Properties of each object include S3 key name, size, date modified and ETag. These objects are added to a HashTable so it can be used later on.
  3. The next call is the PopulateLocalArrayList. This function lists all local files and folders you want to synchronize. Each file and folder is added to an ArrayList so it can be used later on. The function has the option to only include certain files or exclude certain files. The function is recursive. That means it calls itself for each sub folder.
  4. Next the program creates a list of files that should be deleted on Amazom S3. This is done through the PopulateDeleteS3ArrayList function. This function goes through each item in the list of file on S3 and checks if they exist in the list of local items. If the S3 item does not exist locally, the item is added to the DeleteS3ArrayList.
  5. Next the program finds which local files do not exist on S3. This is done through the PopulateUploadDictionary function. This function goes through the local ArrayList and check if each items exists in the S3 HashTable. The item may exist on S3 and locally but the local file may have different content. To determine is a file is the same between S3 and locally the program has an option to compare files by ETag. An ETag is an identifier based on the content of a file. If the file changes the ETag changes. Amazon stores the Etag of each file you upload. When you list files on S3 the ETag for each file is returned. If you choose to compare by ETag the program will calculate the ETag of the local file and check if it matches the ETag returned by Amazon. Any files that don’t match are added to a Dictionary of items that need to be uploaded to S3.

Now we have a list of files that need to be deleted on S3 and a list of files that need to be uploaded to S3. The program has an option to list these changes or go ahead and apply these changes to S3.

If we are deleting files on S3 the DeleteExtraOnS3 function is called. This function goes through the DeleteS3ArrayList and deleted each file in it. This is done by calling the MakeS3Request function. This function uses the SprightlySoft AWS Component to send the appropriate command to S3. For more information about using the AWS Component see the documentation included with the source code and read the Amazon S3 API Reference documentation from Amazon. The function has the ability to retry the command if it fails.

Finally the program calls UploadMissingToS3 function to upload files from the UploadDictionary to S3. Here the program sets header information such as Content-MD5, Content-Type, and metadata to store the locate file’s timestamp in S3. It then calls the UploadFileToS3 function which is very similar to the MakeS3Request function. The difference is UploadFileToS3 function has parameters that are only relevant to uploading a file. The program uses a variable called MyUpload which is a SprightlySoft AWS Component object. This object raises an event whenever the progress on an upload changes. The program hooks into this progress event and shows the progress of the upload while it is taking place. This is done in the MyUpload_ProgressChangedEvent function.

When the program completes it has an option to send log information through an email. This is useful if you run the program as a scheduled task and you want to be notified if there is a failure.

SprightlySoft’s AWS Component for .NET vs. Amazon’s AWS SDK for .NET

SprightlySoft has released a new product that allows Microsoft developers to interact with all aspects of Amazon’s web services.  Amazon has many web service products such as EC2, S3, and Simple DB.  Amazon allows anyone to interact with these services through an API they provide.  Developers create programs that send requests to the API and process the result.  A typical use of the API would be uploading a file to Amazon S3 or listing all files in S3.

There are many different programming languages and it’s up to the developer to find a way to interact with Amazon’s API.  Amazon has recently released an SDK to help Microsoft .NET developers.  The problem with the SDK is that it does not support all Amazon services.  If you want to develop something for an unsupported service such as Amazon DevPay, you either need to create the missing functions or find another component.  Another problem is the SDK does not closely follow the API documentation.  When learning a service a developer needs to read Amazon’s API documentation then discover how that functionality has been exposed through the SDK.  It takes more work to develop a solution.

SprightlySoft has solved these problems with its AWS component for .NET.  The component is a simple consumer or Amazon’s REST interface.  It works with all services provided by Amazon and it closely follows the API documentation.  To work with an Amazon web service you connect to a URL and you send data.  The API documentation identifies what URL you should connect to and what parameters you should be sending.  It’s a simple process of applying information from the documentations to a developer’s code.

Another benefit of SprightlySoft’s component is the sample code provided.  There is at least one function call for every web service offered by Amazon.  It’s easy to modify any example to call any operation.  All sample code is provided in both C# and VB.NET.

SprightlySoft’s component really shines with the Amazon’s S3 service.  S3 is different than the other Amazon services.  Other services work by sending information to a URL and retrieving an XML response.  With S3 you work with files.  Sending files and retrieving files.  Working with files is a different skill than working with XML.  It takes much longer to transfer a large file, there is a need to see the progress of the transfer, there are error handling concerns, and there is a need to abort transfers in progress.  The SprightlySoft component addresses all these issues.  Some specialized features SprightlySoft offers are calculating the hash of a file with progress notification and encrypting a file with progress notification.

The best part of the SprightlySoft’s AWS Component for .NET is that it is free.  You can download it now and start using it in your project.  You will be able to leverage the power is Amazon’s cloud computing platform today.

Amazon S3’s Reduced Redundancy Storage

Amazon has released a new feature that many of you will be taking advantage of.  It’s called Reduced Redundancy Storage.  Amazon typically stores your files on multiple devices at multiple data centers.  When you store a file with the reduced redundancy storage option, the file is still stored on multiple devices at multiple data centers but fewer times.  The benefit of this is a lower price to store your files.  It costs 33% less to store files with reduced redundancy.  The drawback is that Amazon may lose one of your files.  For this reason you should only use this option if the file you are storing can be regenerated or restored from another location.

A typical use for reduced redundancy would be for the storage of programmatically generated thumbnails of a picture.  Suppose a user uploads a picture to your site.  You generate a thumbnail of the picture and store the original and the thumbnail on Amazon S3.  You can choose to store the original with the standard storage class and the thumbnail with the reduced redundancy storage class.  If the thumbnail is ever lost you can regenerate it from the original.

Amazon calculates reduced redundancy storage is 99.99% durable as opposed to standard storage’s 99.999999999% durability.  Amazon also calculates reduced redundancy storage provides 400 times the durability of a typical disk drive.

You can choose to make a file reduced redundancy when you upload it. Setting a special request header in the upload tells Amazon you want to use reduced redundancy for that file. The code snippet below shows how to upload a file with the SprightlySoft S3 Component for .NET and make it reduced redundancy.

Dictionary<String, String> ExtraRequestHeaders = new Dictionary<String, String>();
ExtraRequestHeaders.Add("x-amz-storage-class", "REDUCED_REDUNDANCY");

SprightlySoftS3.Upload MyUpload = new SprightlySoftS3.Upload();
MyUpload.UploadFile("AWSAccessKeyId", "AWSSecretAccessKey", "BucketName", "KeyName", "FilePath", ExtraRequestHeaders);

You can tell which files are using reduced redundancy when you list a bucket.  The StorageClass value will be STANDARD or REDUCED_REDUNDANCY.

You can configure Amazon to send you a notification through Amazon’s Simple Notification Service in the event that Amazon loses one of your reduced redundancy files.  First you set up a Simple Notification Service topic that will load a web page, send an email or add an Amazon Simple Queue Service item when it is triggered.  Next you configure your bucket to trigger the topic when the ReducedRedundancyLostObject event occurs.  Say your application stores images and thumbnails.  Create a web page to regenerate the thumbnail from the original.  Set up a Simple Notification Service topic to call your web page what it runs.  Then configure your bucket to call the topic if it loses a file.It’s a bit more work but it can pay for itself it you store many files with the reduced redundancy storage option.

To see examples of working with reduced redundancy files in C# or VB.NET try the SimpleExamples project included with the SprightlySoft S3 Component for .NET.  http://sprightlysoft.com/