Amazon has a service called AWS Security Token Service which allows you to generate temporary credentials to access your AWS resources. SprightlySoft S3 Sync can be configured to use temporary credentials. The following article shows how to generate temporary credentials and use them with S3 Sync.

AWS Security Token Service is typically used by large organizations that do not want to manage many permanent AWS Identity and Access Management (IAM) users. These organizations will generate a temporary IAM user through STS that exists for only a few hours. An organization user will be able to use this IAM account for their required task. No extra work is required to expire the temporary account. To read more about Security Token Service see http://docs.aws.amazon.com/STS/latest/UsingSTS/Welcome.html

The organization will call the GetFederationToken function of the STS service to generate temporary security credentials. When calling this function the organization will choose how long the credentials are valid for. To read more about the GetFederationToken function see http://docs.aws.amazon.com/STS/latest/APIReference/API_GetFederationToken.html

The GetFederationToken function will return an AccessKeyId, SecretAccessKey, and SessionToken. These values can be used with S3 Sync to preform a file synchronization. Enter the AccessKeyId and SecretAccessKey and you normally would in the application. The SessionToken value needs to go in the RequestHeaders parameter in the format of “x-amz-security-token:[STSVALUE]”. When S3 Sync makes calls to Amazon it will add a x-amz-security-token header in every request. Amazon will validate the token in conjunction with the AccessKeyId and SecretAccessKey.

The following is an example of a S3 Sync command line using temporary credentials and a session token.

C:\Program Files (x86)\SprightlySoft\S3 Sync\S3Sync.exe" -AWSAccessKeyId xxxxxxxxxxxx -AWSSecretAccessKey xxxxxxxxxxxx -BucketName mybucket -SyncDirection Upload -LocalFolderPath "C:\myfolder" -LicenseKey xxxxxxxxxxxx -RequestHeaders "x-amz-security-token:xxxxxxxxxxxx"

 

{ Comments on this entry are closed }

You can use a batch file to execute commands after S3 Sync completes. Say you want to upload files to S3 then delete the local files if the upload was successful. The following batch file is an example of this process.

Upload files to S3 then delete local files.

C:\Program Files (x86)\SprightlySoft\S3 Sync\S3Sync.exe" -AWSAccessKeyId xxxxxxxxxxxx -AWSSecretAccessKey xxxxxxxxxxxx -BucketName mybucket -S3FolderKeyName "myfolder/" -SyncDirection Upload -LocalFolderPath "C:\myfolder" -DeleteS3Items true -LogOnlyMode false -OutputLevel 1 -CompareFilesBy Timestamp -LicenseKey xxxxxxxxxxxx -LogFilePath "C:\Temp\S3 Sync <BucketName> <S3FolderKeyName> <<yyyy-MM-dd HH.mm.ss>> <SyncDirection>.txt"

@if %ERRORLEVEL% neq 0 goto error

@echo S3 upload successful.
RMDIR "C:\myfolder" /Q /S
REM pause
exit 0

:error
@echo S3 upload error. Local delete not executed.
REM pause
exit 1
  • The first line in the script above uploads files with S3 Sync.
  • The %ERRORLEVEL% line checks the error level returned from S3 Sync. If the error level is not 0, the application skips the next block of code and goes to the :error section.
  • The @echo line writes a message to the console window.
  • The line with RMDIR deletes the local folder. This line will only be executed if error level from S3 Sync was 0 which means success.
  • The REM pause line would wait for the use to press a key to continue. The REM statement comments out the line and it will not be executed. Remove REM if you would like to test the script and want to be notified before continuing.
  • The exit 0 line exits the batch script with a success code. The remainder of the script will not be executed.
  • The :error line is a section marker of the script.
  • The @echo line writes a message to the console window.
  • The REM pause line can be uncommented during testing.
  • The exit 1 line exits the batch script with an error code.

The following is an example of downloading files with S3 Sync then deleting those files with S3 Delete. You can get S3 Delete for free at http://sprightlysoft.com/S3Delete/

Download files from S3 then delete files on S3.

"C:\Program Files (x86)\SprightlySoft\S3 Sync\S3Sync.exe" -AWSAccessKeyId xxxxxxxxxxxx -AWSSecretAccessKey xxxxxxxxxxxx -BucketName mybucket -S3FolderKeyName "myfolder/" -SyncDirection Download -LocalFolderPath "C:\myfolder" -DeleteLocalItems true -LogOnlyMode false -OutputLevel 1 -CompareFilesBy Timestamp -LicenseKey xxxxxxxxxxxx -LogFilePath "C:\Temp\S3 Sync <BucketName> <S3FolderKeyName> <<yyyy-MM-dd HH.mm.ss>> <SyncDirection>.txt"

@if %ERRORLEVEL% neq 0 goto error

@echo S3 download successful.
"C:\Program Files (x86)\SprightlySoft\S3 Delete\S3Delete.exe" -AWSAccessKeyId xxxxxxxxxxxx -AWSSecretAccessKey xxxxxxxxxxxx -BucketName mybucket -S3KeyName "myfolder/" -LogFilePath "C:\Temp\S3 Sync delete.txt" -OutputLevel 2
REM pause
exit 0

:error
@echo S3 download error. S3 delete not executed.
REM pause
exit 1
  • The first line in the script above downloads files with S3 Sync.
  • The %ERRORLEVEL% line checks the error level returned from S3 Sync. If the error level is not 0, the application skips the next block of code and goes to the :error section.
  • The @echo line writes a message to the console window.
  • The line with S3Delete.exe deletes the files from S3. This line will only be executed if error level from S3 Sync was 0 which means success.
  • The REM pause line would wait for the use to press a key to continue. The REM statement comments out the line and it will not be executed. Remove REM if you would like to test the script and want to be notified before continuing.
  • The exit 0 line exits the batch script with a success code. The remainder of the script will not be executed.
  • The :error line is a section marker of the script.
  • The @echo line writes a message to the console window.
  • The REM pause line can be uncommented during testing.
  • The exit 1 line exits the batch script with an error code.

To use the batch files above copy the code an place them in a text file using a tool such as Notepad. Change the parameters such as AWSAccessKeyId to your values. Save the file with a .bat extension. An example of the file name is “DownloadThenDelete.bat”. You can double click the bat file to execute it.

{ Comments on this entry are closed }

S3 Sync and Server-Side Encryption

March 30, 2015

S3 Sync now supports server-side encryption using Amazon KMS-Managed Keys and Customer-Provided Keys. Server-side encryption is the process where Amazon encrypts files after you upload them. If you provide the correct credentials when retrieving a file, Amazon decrypts the file and returns it to you. Server-side encryption makes storing files on Amazon more secure. Server-Side Encryption using AWS KMS-Managed Keys […]

Read the full article →

How to Create a Scheduled Task to Automate S3 Sync

March 20, 2014

Microsoft Windows has a built in task scheduler. It allows you to automatically run a program at a certain time. Task scheduler is great for automating backups. You can configure a task to send files to Amazon S3 using SprightlySoft S3 Sync. The following article will take you step by step through the process of […]

Read the full article →

How to use Amazon Glacier in S3

February 24, 2014

Amazon Glacier is a service that allows you to archive your files on Amazon’s cloud infrastructure. It is designed for large files that are accessed infrequently such as video and picture backups. The main advantage of Glacier is the price. It costs $1 for 100 GB of storage space per month. This is 88% cheaper […]

Read the full article →

Restricting an IAM User to a Sub Folder in Amazon S3

September 13, 2013

Do you want to use multiple IAM users with a single S3 bucket but don’t want the users to access each other’s files? You can craft a S3 bucket policy to limit a user to a specific S3 sub folder. The following will show you how to create a bucket policy and use SprightlySoft S3 […]

Read the full article →

S3 Sync v4 Supports Restore from Glacier

January 24, 2013

S3 Sync version 4 has been released. The latest version supports downloading files that have been moved to the Glacier storage class. Amazon Glacier is a file archiving service. Storing a file in Glacier costs 1/10th the cost of storing it in S3. The drawback of Glacier is that if you want to retrieve a […]

Read the full article →

SprightlySoft Releases Glacier Sync

December 19, 2012

SprightlySoft is pleased to announce the release of Glacier Sync. Glacier Sync is a Windows application that allows you to synchronize a folder on your computer with a vault on Amazon Glacier. Amazon Glacier is an extremely inexpensive file archiving service. It is designed to store files for a long period of time where the […]

Read the full article →

Uploading Files to S3 with Plupload and ASP.NET

November 16, 2012

Plupload is a component that makes it easy to upload files through a web browser. When you add Plupload  to a web page, users can select multiple files to upload and users can see the progress while an upload is taking place. To read more about Plupload see http://www.plupload.com/. Plupload can be configured to upload files to […]

Read the full article →

S3 Sync Now Supports AWS GovCloud, GreenQloud, and Seeweb

May 11, 2012

SprightlySoft is pleased to announce S3 Sync now supports Amzon’s AWS GovCloud service. GovCloud provides compute and storage services for US government agencies. Data in this environment is segregated and complies with specific requirements for hosting government information.  To read more about GovCloud see http://aws.amazon.com/govcloud-us/. To use S3 Sync with GovCloud you must first have a […]

Read the full article →