S3 Sync and Server-Side Encryption

by Anton on March 30, 2015

S3 Sync now supports server-side encryption using Amazon KMS-Managed Keys and Customer-Provided Keys. Server-side encryption is the process where Amazon encrypts files after you upload them. If you provide the correct credentials when retrieving a file, Amazon decrypts the file and returns it to you. Server-side encryption makes storing files on Amazon more secure.

Server-Side Encryption using AWS KMS-Managed Keys (SSE-KMS)

Amazon allows you to generate encryption keys within Amazon and specify which encryption key to use when uploading a file. This method is called AWS KMS-Managed Keys. You specify the ID of the encryption key to use when uploading a file. When downloading a file you must specify the same ID of the encryption key. You generate and manage encryption keys through the Identity and Access Management service which is accessible through the AWS web management console. If someone gets access to your S3 bucket they will not be able to retrieve a file unless they specify the correct ID of the encryption key. When managing encryption keys through the Identity and Access Management service you can delegate encryption key administrators and  encryption key users. These are the users that will be able to encrypt data, decrypt data, and manage encryption keys. When using SSE-KMS, AWS CloudTrail is available. CloudTrail keeps and audit history of who used an encryption key and when.

To use SSE-KMS in S3 Sync you will need to use the UploadHeaders parameter. The following is an example of this parameter.

-UploadHeaders “x-amz-server-side-encryption:aws:kms|x-amz-server-side-encryption-aws-kms-key-id:c112fc98-2e5d-4eab-aebc-0f565aa7e6fc”

The x-amz-server-side-encryption section tells S3 you will be using SSE-KMS. The x-amz-server-side-encryption-aws-kms-key-id section tells S3 which encryption key to use for the operation. You can get the key ID for an encryption key you generated in the AWS web management console under the Identity and Access Management service.

Note: The UploadHeaders parameter applies to upload, download, and bidirectional syncs in S3 Sync.

Server-Side Encryption using Customer-Provided Keys (SSE-C)

Amazon allows you to generate your own encryption key and send that key to S3 when uploading a file. Amazon will encrypt the file with your encryption key after it receives it. Amazon will then discard the key. When requesting the file you must include the same encryption key that was sent when uploading the file. Amazon will decrypt the file and return it to you. The benefit here is that Amazon does not store the encryption key so there is no way for someone with access to your account to retrieve a file. The drawback is you need to securely store your encryption keys locally. If you loose your keys you cannot retrieve your files.

The following is an example of the S3 Sync UploadHeaders parameter for using SSE-C.

-UploadHeaders “x-amz-server-side-encryption-customer-algorithm:AES256|x-amz-server-side-encryption-customer-key:Qi1sHpQnppeJAo5WyM8w/BGJXJmTO/LBg6dcjThY6nM=|x-amz-server-side-encryption-customer-key-MD5:IcpiSupe46fQ0fb5AGQ2RQ==”

The x-amz-server-side-encryption-customer-algorithm section says you will be using the AES 256 encryption method on S3. The x-amz-server-side-encryption-customer-key section is the encryption key. The x-amz-server-side-encryption-customer-key-MD5 section is the MD5 hash of the encryption key. Amazon will store the x-amz-server-side-encryption-customer-key-MD5 value with the file but not the x-amz-server-side-encryption-customer-key value.

The following is code to generate an encryption key and it’s MD5 hash in C#.

System.Security.Cryptography.AesManaged MyAesManaged = new System.Security.Cryptography.AesManaged();
MyAesManaged.GenerateKey();
String CustomerKey = Convert.ToBase64String(MyAesManaged.Key);

System.Security.Cryptography.MD5 MyMD5 = System.Security.Cryptography.MD5.Create();
Byte[] MD5Bytes = MyMD5.ComputeHash(MyAesManaged.Key);
String CustomerKeyMD5 = Convert.ToBase64String(MD5Bytes);

To read more about server-side encryption see http://docs.aws.amazon.com/AmazonS3/latest/dev/serv-side-encryption.html

{ Comments on this entry are closed }

Microsoft Windows has a built in task scheduler. It allows you to automatically run a program at a certain time. Task scheduler is great for automating backups. You can configure a task to send files to Amazon S3 using SprightlySoft S3 Sync. The following article will take you step by step through the process of creating a scheduled task to automatically upload files Amazon S3.

First run the S3 Sync Command-Line Wizard to build a S3 Sync command line. Enter your parameters of which folder you want to upload and which S3 bucket you want to save files to. When all your parameters are entered you will arrive at the Command-Line tab in the wizard. This screen allows you to save the command line to a batch file. Click the Save Command-Line button.

Cmmand-Line Save
In the Save As dialogue box enter a name for your batch file.

Save Batch File
Now that your batch file is saved you can double click it to run the command-line. This makes it easier to run S3 Sync. However in some situations you may want to run the command-line often and you do not want to click the batch file each time. This is where Windows Task Scheduler comes in handy. You can create a task to run the batch file once an hour. The following shows how to create a scheduled task.

In Windows go to the Control Panel. Under System and Security you will see Administrative Tools. Click it to see the administrative tools.

Control Panel Administrative Tools
Under Administrative Tools click Task Scheduler.

Control Panel Task Scheduler
The Task Scheduler screen will open up. Right click in the main area and click Create New Task.

Create New Task
Enter a name for your scheduled task. You will need to select the user the task runs under. It can be your account or another account. It’s best to select “Run whether user is logged on or not” so the task will run even if the user is not logged in. Click OK when you are complete.

Create Task General
In the Create Task screen click on the Triggers tab and click the New button.

Create Task Triggers
On the New Trigger screen select how often the task will run. In the example below the trigger is set to run once an hour. You can choose different settings for your task if you’d like.

New Trigger
When you are finished with your trigger go to the Actions tab on the Create Task screen. Click the New button.

Create Task Actions
Under Program/script select the batch file you created earlier. Click the OK button when complete.

New Action
The settings for your new task are now entered. Click the OK button to create the task. You may be prompted to enter the password of the user the task runs under. Enter the password to continue.

Save Task
You will now see the task in the Task Scheduler. To confirm the task is configured correctly right click on it and click Run. The Task Schuler will execute the batch file which will run S3 Sync with your settings. View the S3 Sync log to make sure S3 Sync ran correctly.

Run Task
With the scheduled task configured you can relax and know your files will automatically be backed up to Amazon S3.

{ Comments on this entry are closed }

How to use Amazon Glacier in S3

February 24, 2014

Amazon Glacier is a service that allows you to archive your files on Amazon’s cloud infrastructure. It is designed for large files that are accessed infrequently such as video and picture backups. The main advantage of Glacier is the price. It costs $1 for 100 GB of storage space per month. This is 88% cheaper […]

Read the full article →

Restricting an IAM User to a Sub Folder in Amazon S3

September 13, 2013

Do you want to use multiple IAM users with a single S3 bucket but don’t want the users to access each other’s files? You can craft a S3 bucket policy to limit a user to a specific S3 sub folder. The following will show you how to create a bucket policy and use SprightlySoft S3 […]

Read the full article →

S3 Sync v4 Supports Restore from Glacier

January 24, 2013

S3 Sync version 4 has been released. The latest version supports downloading files that have been moved to the Glacier storage class. Amazon Glacier is a file archiving service. Storing a file in Glacier costs 1/10th the cost of storing it in S3. The drawback of Glacier is that if you want to retrieve a […]

Read the full article →

SprightlySoft Releases Glacier Sync

December 19, 2012

SprightlySoft is pleased to announce the release of Glacier Sync. Glacier Sync is a Windows application that allows you to synchronize a folder on your computer with a vault on Amazon Glacier. Amazon Glacier is an extremely inexpensive file archiving service. It is designed to store files for a long period of time where the […]

Read the full article →

Uploading Files to S3 with Plupload and ASP.NET

November 16, 2012

Plupload is a component that makes it easy to upload files through a web browser. When you add Plupload  to a web page, users can select multiple files to upload and users can see the progress while an upload is taking place. To read more about Plupload see http://www.plupload.com/. Plupload can be configured to upload files to […]

Read the full article →

S3 Sync Now Supports AWS GovCloud, GreenQloud, and Seeweb

May 11, 2012

SprightlySoft is pleased to announce S3 Sync now supports Amzon’s AWS GovCloud service. GovCloud provides compute and storage services for US government agencies. Data in this environment is segregated and complies with specific requirements for hosting government information.  To read more about GovCloud see http://aws.amazon.com/govcloud-us/. To use S3 Sync with GovCloud you must first have a […]

Read the full article →

SprightlySoft Releases S3 Delete

May 17, 2011

SprightlySoft is pleased to announce the release of S3 Delete. S3 Delete is a command line program that allows Windows users to delete files from Amazon S3. Amazon S3 is a web service that allows you to store and retrieve files. You can upload as many files you’d like and they can be made available […]

Read the full article →

Amazon Simple Notification Service Script

April 18, 2011

Amazon Simple Notification Service (SNS) is a web service that allows you to notify distributed applications by pushing a message to them. Say you have an application that runs on many servers. In a traditional application the computers would continually poll a queue or database looking for a job to process. Many computers continually polling […]

Read the full article →