SprightlySoft - Connecting you to the cloud.
 

Glacier Sync - Documentation

SprightlySoft Glacier Sync synchronizes a folder on your computer with a folder on Amazon Glacier.

Requirements

The application requires a Microsoft Windows computer running the Microsoft .NET Framework 4.0 or later. You will also need an Amazon AWS account with Glacier enabled.

How to Run the Program

Open a command prompt window where GlacierSync.exe is located. Do this by holding down the Shift key on your keyboard and right clicking the GlacierSync folder. From the context menu click Open command window here. A command prompt window will open. Type GlacierSync.exe and the command line parameters you require to synchronize your files. Here is an example of what you would type:

GlacierSync.exe -AWSAccessKeyId xxxxxxxxxxxx -AWSSecretAccessKey xxxxxxxxxxxxxx -SyncDirection upload -LocalFolderPath "C:\myfolder" -VaultName myvault -DeleteGlacierItemsWhereNotInLocalList true

Command Line Parameters

The following describes the parameters of Glacier Sync.

-SyncDirection (required) - The direction of the synchronization.

  • Upload - Upload files from your computer to Amazon Glacier. Your computer will be the source.
  • Download - Download files from Amazon Glacier to your computer. Amazon Glacier will be the source.

-AWSAccessKeyId (required) - Your AWS access key ID provided by Amazon.

-AWSSecretAccessKey (required) - Your AWS secret access key provided by Amazon.

-AWSAccountId (optional) - Your AWS Account ID. This value must match the AWS Account ID associated with your AWSAccessKeyId and AWSSecretAccessKey. If you leave the value blank Amazon Glacier uses the AWS Account ID associated with your AWSAccessKeyId.

-UseSSL (optional) - Specifies whether SSL should be used when communicating with Amazon. The default value is true.

  • True - Request URLs to Amazon will begin with https://.
  • False - Request URLs will begin with http://.

-VaultName (required) - The name of the Amazon Glacier vault which you will be synchronizing files with.

-Region (optional) - The region in which your vault is located. This value can be us-east-1, us-west-2, us-west-1, eu-west-1, or ap-northeast-1. If no value is specified the application will determine which region your vault is located by listing all your vaults in all the regions.

-LicenseKey (optional) -Your license key value obtained from SprightlySoft. A license key is required if you are using the application for commercial or non-individual use. A license key is not required for non-commercial individual use.

-LocalFolderPath (required) - The folder on your computer that you will be synchronizing with. This can be any valid path. The path will look like "C:\MyFolder".

-LogOnlyMode (optional) - Specifies whether changes should be applied to the destination or if changes should only be logged and no actions will be performed. The default value is false.

  • True - the program will run, check what changes are required, and report these changes. The program will not make any changes to the destination.
  • False - The program will check which files need to be deleted and transferred, and then it will delete and transfer the required files.

-OutputLevel (optional) - Specifies what level of logging the program should output. The default value is 2.

  • 1 - Minimal logging. Only show what will is deleted and transferred.
  • 2 - Detailed logging. Show all files that exist between both systems and show debug information in the event of an error.
  • 3 - HTTP logging. Show all information of detailed logging and all HTTP requests to anf from Amazon.

-LogFilePath (optional) - The path and file name where a log file will be generated. Leave this value empty if you do not want to create a text log file with log information when the program runs. The path will look like "C:\Temp\GlacierSyncLog.txt".

-OverwriteExistingLogFile (optional) - Specifies whether an existing log file should be overwritten when the program runs. The default value is true.

  • True - Delete the existing log file when the program starts. The log file will only contain data about the current execution of the program.
  • False - Append to the existing log file and do not overwrite data.

-CompareFilesBy (optional) - When a file exist on the source and destination with the same name different methods can be used to determine if the data in the files are different and if the source file needs to be copied over the destination file. The methods to determine if files are the different are listed below. The default value is SHA256TreeHash.

  • SHA256TreeHash - Compares the source file's tree hash with the destination file's tree hash. This is the most accurate method but it takes time to calculate the tree hash of a local file.
  • Size - Compare the size of the source file to the size of the destination file. If both files are the same size, the files are considered the same. This method is quicker but is not as accurate. Two files may have the identical size but not the identical content.
  • FileName - If the source file and destination file have the same file name, the files are considered the same. The content of the files are not compared.

-TransferThreads (optional) - Sets the number of concurrent uploads or downloads. Use this to transfer multiple files at the same time which will improve your bandwidth utilization. The default value is 2.

-MultipartThreads (optional) - Sets the number of concurrent transfers of parts during a multipart upload. Upload multiple parts at the same time to improve your bandwidth utilization. The default value is 2.

-ThreadSpeedLimitKBps (optional) - Sets the maximum transfer speed in kilobytes per second. Set this value to 0 if you do not wish to limit the speed of an upload or a download. This setting applies to each transfer thread and is not a global setting. The default value is 0.

-ShowTransferProgress (optional) - Indicates weather progress for a transfer should be displayed in the console. When running Glacier Sync from an application that reads the console output, such as SQL Agent, you may need to disable the progress update by setting this parameter to false. The default is true.

  • True - Show progress during a transfer.
  • False - Do not show progress during a transfer.

-AWSErrorRetries (optional) - The number of times a failure communicating with Amazon should be retried. The delay between retries increases as the number of retries increases. Only certain errors are retried such as connection timeout errors and 500/408 responses from Amazon. The default value is 5.

-WaitDelaySeconds (optional) - The number of seconds to wait before checking if an operation on Amazon is complete. You need to wait for an inventory job to complete and for an archieve to become available for download. The default value is 60.

-RefreshInventoryCache (optional) - Specifies if the local inventory cache should be discarded and the cache filled from Amazon's inventory list. Amazon creates a list of all archives in your vault once a day. You can request this list from Amazon but it takes 4 hours for the list to become available. To prevent long delays in listing your vault, Glacier Sync stores a list of all your archives locally. If you add or delete an archive from another computer or another application, your local Glacier Sync cache will not match what is on Amazon. Use the RefreshInventoryCache parameter to discard the local cache and fill it from Amazon's list. The default value is 0.

  • 0 - Do not refresh the local cache. Use the cached inventory.
  • 1 - Use the inventory list from Amazon's most recent inventory. If no job exists that lists this inventory, one will be initiated. You may need to wait up to 4 hours if initiating a new job.
  • 2 - Wait for the next inventory listing to complete on Amazon and use that list. If you added or deleted any archives since the most recent inventory took place, Amazon's most recent inventory will not include those changes. You will need to wait for Amazon's next inventory to complete in order for those changes to be included in Amazon's list. You may need to wait up to a day for the next inventory to complete.


The following parameters only apply when the SyncDirection is "Upload":

-UseMultipartUpload (optional) - Specifies if Amazon's multipart upload feature should be used when uploading large files to Glacier. Multipart upload is beneficial since you can resume an upload if it gets interrupted. If a file is smaller than the MultipartPartSizeMB value, multipart upload will not be used on that file regardless of this setting. The default value is true.

  • True - Use multipart upload.
  • False - Do not use multipart upload.

-MultipartPartSizeMB (optional) - The size of a part in megabytes when using multipart upload. This value must be between 1 and 4096 and must be a multiple of the power of 2. For example, 1, 2, 4, 8, and so on. The default value is 4.

-IncludeLocalSubFolders (optional) - Specifies whether sub folders of your LocalFolderPath should be synchronized to Glacier. The default value is true.

  • True - Synchronize sub folders to Glacier.
  • False - Do not synchronize sub folders to Glacier. Only files in the LocalFolderPath path will be synchronized.

-ExcludeLocalFolders (optional) - A list of local folder paths that will be excluded from synchronization. Multiple paths should be separated with the vertical bar character. Example; use "C:\MySync\DoNotSyncA|C:\MySync\DoNotSyncB" to exclude the folders C:\MySync\DoNotSyncA and C:\MySync\DoNotSyncB.

-IncludeOnlyLocalFilesRegularExpression (optional) - Only synchronize files where the file names match the regular expression. Example; use ".*\.txt$" to only synchronize files that end with .txt.

-ExcludeLocalFilesRegularExpression (optional) - Do not synchronize files that match the regular expression. Example; use ".*\.(jpg|gif|jpeg)$" to exclude files that end with . jpg, .gif, or . jpeg from synchronization. Use ".*\\(Downloads|My Music)\\.*" to exclude files where \Downloads\ or \My Music\ is in the path.

-DeleteGlacierItemsWhereNotInLocalList (optional) - Delete files on Glacier that do not exist in the local list of files. Any extra files that do not exist on your computer will be deleted from Glacier. The default value is false.

  • True - Delete files on Glacier that do not exist on your local system.
  • False - Leave extra files on Glacier. Do not delete them.


The following parameters only apply when the SyncDirection is "Download":

-RestoreTimestamps (optional) - Indicated weather the time stamp of files downloaded should be set to the timestamp values that were saves in the archive's description. The default value is true.

-IncludeOnlyGlacierFilesRegularExpression (optional) - Only synchronize files where the file names match the regular expression. Example; use ".*\.txt$" to only synchronize files that end with .txt.

-ExcludeGlacierFilesRegularExpression (optional) - Do not synchronize files that match the regular expression. Example; use ".*\.(jpg|gif|jpeg)$" to exclude files that end with . jpg, .gif, or . jpeg from synchronization.

-DeleteLocalItemsWhereNotInGlacierList (optional) - Delete local files that do not exist in Glacier. Any extra files that exist on your computer but do not exist on Glacier will be deleted. The default value is true.

  • True - Delete local files that do not exist on Glacier.
  • False - Leave extra local files. Do not delete them.

-DeleteLocalItemsToRecycleBin (optional) - When deleting a local file, send it to the recycle bin. The default value is true.

  • True - Send deleted files to the recycle bin.
  • False - Permanently delete local files. Do not use the recycle bin.


-EmailLogCondition (optional) - Specifies under what conditions an email should be sent when the program completes. The email will contain the log of the program's operation. The default value is 0.

  • 0 - Never send an email.
  • 1 - Send an email when there is an error.
  • 2 - Always send an email when the program completes.

-EmailSmtpHost (optional) - The SMTP server used to send the email.

-EmailSmtpPort (optional) - The port of the SMTP server.

-EmailSmtpEnableSsl (optional) - If SSL should be used with the SMTP server. The default value is false.

  • True - Use a SSL connection.
  • False - Do not use a SSL connection.

-EmailSmtpUserName (optional) - The user name to log into the SMTP server. Keep this value empty is your SMTP server does not require authentication.

-EmailSmtpPassword (optional) - The password used to log into the SMTP server.

-EmailFromAddress (optional) - The from address to be used in the email.

-EmailRecipients (optional) - The email addresses the email will be sent to. Separate multiple addresses with a comma.

Setting Parameters In the .Config File

Parameters can be set in the .config file so you do not have to pass them as command line parameters. This is useful for AWSAccessKeyId and AWSSecretAccessKey as they do not change often. Open the GlacierSync.exe.config file in a text editor such as Notepad and enter your parameter values.

Exit Codes

The application returns an exit code when it completes. The exit code is visible at the bottom of the log file, in the console window, and from other programs calling the application. The following is a list of exit codes the application can return.

  • 0 - The application completed successfully.
  • -1 - An error occurred and no further actions were performed.
  • -2 - An error occurred and the application continued. This error is generated when a file or folder cannot be accessed. Possible reasons may be the file is in use, you do not have permission to the folder, or the file is invalid. Check the log file for the full error message.

Running the Application As a Scheduled Task

If you would like to run the application regularly you can use Window's built in Task Scheduler. Read the following articles on how to configure a scheduled task. http://windows.microsoft.com/en-US/windows7/schedule-a-task

 

 
 
© 2024  SprightlySoft