Deploying StoreGrid 4.4 Online Backup on Amazon EC2 / S3

A 'How To' guide that helps you run your online backup service on Amazon's Cloud Platform

Section 1 - Introduction

Vembu StoreGrid Cloud AMI which facilitates deploying StoreGrid on Amazon's Cloud Computing Infrastructure is available to partners for production use. The StoreGrid Cloud AMI is like a virtual appliance which can be instantiated as a backup or a replication server and run on Amazon Elastic Cloud Computing (EC2) platform. The StoreGrid Cloud AMI will use Amazon Simple Storage Service (S3) to store the backup data from StoreGrid Clients.

So far, we have primarily worked with partners who are willing to host StoreGrid in their own data centers and offer online backup services to their customers. With the StoreGrid Cloud AMI Virtual Appliance, any IT solution provider can now start an online backup service using Amazon Web Services without any upfront capital investment on servers and storage infrastructure. Besides the current practice of deploying the StoreGrid backup server and replication server in their own data centers, service providers now have the added options of:

  • Running their StoreGrid backup server and replication server in the Amazon Web Services cloud computing infrastructure. The (Windows or Linux) servers will run on Amazon EC2 instances with Amazon S3 as the primary data storage backend. The Amazon EBS volumes are used for the MySQL database storage and as a temporary local cache for the backup data.

  • Deploying a StoreGrid backup server in their own data center and having the backup server replicate the backup data to a StoreGrid replication server running on Amazon EC2.

  • Running StoreGrid backup servers 'on-premise' at customer locations so that there is local copy of the backup data for quick restores. The StoreGrid Cloud AMI can function as a replication server to replicate the various 'on-premise backup servers' to the Amazon Cloud - in effect, providing an additional disaster recovery option.

The StoreGrid Cloud AMI virtual appliance is currently available for Windows Server and CentOS Linux server. Other requisite modules like the MySQL 5.5 back-end database are bundled together in the StoreGrid Cloud AMI to facilitate ease of deployment for partners. However, as detailed below, the Amazon deployment will currently require some work on your part.

If you have questions, please email us at storegrid-cloud@vembu.com

Section 2 - Signing up for your Amazon AWS account

  1. Point your browser to http://aws.amazon.com and click on the button ‘Sign Up Now’

  2. Enter your Amazon account details, if you have one (or select ‘I am a new user’)

  3. Enter your name and email address to register for an Amazon Web Services Account and click ‘Continue’

  4. Please enter your contact information and accept the AWS Customer Agreement

  5. You will see this screen below if you have successfully created your AWS account.

  6. Login into Amazon Web Services and select Your Account > Access Identifiers from the top menu. Note down your Amazon AWS account number from the top-right corner.

  7. You will see the ‘Access Key ID’ and your ‘Secret Access Key’ listed below. Please note them down for future use.

To proceed further, you can use AWS Management Console, ElasticFox or Amazon Command Line tools. In this guide, we will proceed with the AWS Management Console to manage your Amazon Cloud instances.

Section 3 - Launching the StoreGrid Cloud AMI Instance

  • Login to AWS Management Console from the URL, https://console.aws.amazon.com

  • Step 2: Select the EC2( Virtual Server in the Cloud) from the Amazon Web Services home page.

  • Step 3: Click on the EC2 ( Virtual Server in the cloud ), It will direct you to EC2 dashboard. At top right corner, you can choose your desired AMI region. By default, US East region will be selected.

Section 3.1 - Generating a Secure Key Pair for Authentication

In order to remotely login to your StoreGrid Cloud AMI instance running in Amazon EC2 through SSH or Remote Desktop (RDP) you need to generate a Secure Key-Pair for secure communication. Please note, launching public AMIs (such as StoreGrid Cloud AMI) without a key pair ID will leave them inaccessible. The Secure Key-Pair is a 2048 bit RSA key pair generated with a specific name. The generated key-pair should be specified while launching an Amazon EC2 instance and will be used for authenticating TCP connections such as SSH and RDP.

In Linux, the key pair content is used as an identity [SSH Authorization Key] to authenticate the root user connecting to the EC2 Instance.

In Windows, the key pair is used to generate and encrypt the initial random password for the EC2 Instance, such that the password is not present in a public image. To fetch and decrypt the initial random password which is set in the EC2 instance, you have to use the Key-Pair. After fetching the password, you will be able to access the EC2 instance with UserName: Administrator and Password: <as fetched>

In Linux, the key pair content is used as an identity [SSH Authorization Key] to authenticate the root user connecting to the EC2 Instance.

In Windows, the key pair is used to generate and encrypt the initial random password for the EC2 Instance, such that the password is not preset in a public image. To fetch and decrypt the initial random password which is set in the EC2 instance, you have to use the Key-Pair. After fetching the password, you would be able to access the EC2 instance with UserName: Administrator and Password: <as fetched>

Create a key pair by following the below steps in AWS Management Console (If you have already created the key pair, you can ignore these steps)

Step 1: In AWS Management Console, click 'Key Pairs' link under 'Navigation' panel.

Step 2: Click 'Create Key Pair' button ( Shown top) and enter the Key Pair Name and click 'Create'.

After clicking 'Create' button, you will be asked to save the key pair file. Save the key-pair file in a secured location in your machine.

NOTE: This key-pair file will be used for fetching password in a Windows instance & authenticating root user in Linux installation. Hence, this key-pair file needs to be preserved.

Section 3.2 - Launch StoreGrid EC2 instance

You can launch the below listed AMI ID to install StoreGrid Agent V4.4 only. The AMI ID of Vembu StoreGrid Cloud AMI are as follows:

StoreGrid Cloud AMIs for the US East Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-9c094ef5

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-ca0140a3

StoreGrid Cloud AMIs for the US West North California Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-205d7765

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-38654f7d

StoreGrid Cloud AMIs for the EU Ireland Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-6b4a511f

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-e7e6fe93

StoreGrid Cloud AMIs for the Asia Pacific (Singapore) Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-90d59dc2

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-f6c088a4

StoreGrid Cloud AMIs for the Asia Pacific(Sydney) Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-c91a88f3

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-2b61f311

StoreGrid Cloud AMIs for the Asia Pacific(Tokyo) Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-738e1d72

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-ffd043fe

StoreGrid Cloud AMIs for the US West Oregon Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-c21a87f2

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-ae79e49e

StoreGrid Cloud AMIs for the South America (São Paulo) Region

Vembu StoreGrid Cloud AMI ID for Windows [64 bit] :: ami-5353f74e

Vembu StoreGrid Cloud AMI ID for CentOS [64 bit] :: ami-7b4ce866

Launch a StoreGrid EC2 instance by following the below steps :

Step 1: Click on 'AMIs' link under 'Navigation' panel in AWS Management Console. All the public images will be listed. Select the StoreGrid public AMI under 'Amazon Machine Images' (StoreGrid AMI ID for corresponding regions are listed in Section 3.2).

Step 2: Right click on the StoreGrid public AMI ID and select "Launch Instance'.

Step 3: On the Select an Instance Type page, you can select the hardware configuration of your instance. The t1.micro instance is selected by default. Click on "All instance types" tab to choose the instance type as small/medium/large from the instance list.

Step 4: After selecting the instance type, Click on "Configure Instance Details" and Enter the number of instance(s) to start. Leave the other settings as Default Values.

Step 5: In Add storage page, you can view the storage device settings of your instance and increase the size of root device as required.

Step 6: In the tag instance page, you can provide a name for your instance.

Step 7: In the configure security group page, Click on the radio button for "Select an existing security group" and choose the "default group".

Step 8: Click on ‘Review and Launch “ button to review the instance configurations settings and then click on ‘Launch’. (In the Select an existing key pair or create a new key pair dialog box, select Choose an existing key pair, then select the key pair you created when getting set up in Section 3.1 ).

You will get a window with a message that your instance is running.

NOTE: Make sure the instance has entered running mode, as the Public DNS Name will be available only after the instance has entered running mode. The Public DNS Name will enable you to remotely access the EC2 instance.

Section 3.3 - Firewall Configuration

To allow the remote access to your EC2 instance, you need to grant permission to the 'default' security group which determines the ports opened up for communication in your EC2 instance.

Enable the firewall configurations by following the below steps:

Step 1: Click on 'Security Groups' link under 'Navigation' panel.

Step 2: Click on default Security Group.

Step 3: Add the necessary ports under 'Allowed Connections' in default group.

Step 4: The TCP port for which the permission should be granted is,

Port 22 for opening up SSH access in Linux

Port 3389 for opening up Remote Desktop (RDP) access in Windows

Section 3.4 - Logging in to the instance

For Linux instance, login to the Public DNS of the instance through SSH by executing the below command:

ssh -i root@

where is the key-pair file to authenticate the SSH Connection with the EC2 Instance.

For Windows instance, to get the instance password, please follow the below steps:

Step 1: Click on 'Instances' link under 'Navigation' panel.

Step 2: Right click on the particular instance and select 'Get Windows Password'.

Step 4: Copy and paste your key pair file content in 'Private Key' field and click 'Decrypt Password'. You will get the decrypted RDP Password.

Step 5: Now, you can connect to the Windows instance using RDP (Remote Desktop or the Microsoft Terminal Services Client) with the UserName: Administrator and Password:

It is strongly recommended to change the password in the running instance after you first login to the instance.

NOTE: Once the password is changed, you will not be able to login with the password fetched using 'Get Default Administrator Password'.

Section 3.5 - Mounting EBS Volumes

To store the backup data of your clients in Amazon S3 and the metadata information of the StoreGrid Backup/Replication Server, you need to create EBS Volumes with a storage size as required. The EBS volume will be used by StoreGrid as a temporary cache location before uploading the clients' data to Amazon S3. After creating the EBS volumes, you can mount them to the EC2 instance that you have started. The attached EBS Volume also stores the MySQL Database data.

To do this, please follow the below steps:

Step 1: Go to 'Volumes' link under 'Navigation' panel.

Step 2: To create a new volume click 'Create Volume' button. Enter the size of the Volume and select the availability zone. Please make sure that the availability zone you select here is same as the availability zone of the instance. (Ignore this step if you have already created the EBS volume)

Step 3: Attach the EBS volume to the instance by right clicking the volume and by selecting the instance ID and the device name.

Once the EBS volume is attached,

For Linux Machine,

Step 1: Login to instance via SSH and format (only for new EBS volume) & mount the volume by execute the following commands,

yes | mkfs -t xfs /dev/sdh

mount /dev/sdh /storegrid

where /dev/sdh is the device name and /storegrid is the mount point. Device name can be identified by executing the command “parted -l” in the terminal. This command will list out all the volumes available in the machine. As shown below.

Also mount point should be named as /storegrid for generic builds and for Branded builds, mount point name should be as product name. Eg: /productname. So that the StoreGrid automated script, recognizes the configurations.

Now, execute command 'df -lh' and you will find the EBS mounted in /storegrid location

For Windows,

Step 1: On the taskbar, click Start, and then click Run.

Step 2: Type diskmgmt.msc and click OK. The Disk Management utility opens.

Step 3: Right-click the Amazon EBS volume, select New Volume, and follow the onscreen prompts

Convert the EBS volume attached to the EC2 instance into a new partition using the New Partition Wizard on Windows

Once you complete the wizard, you will find the EBS volume listed as a new drive in the EC2 instance.

NOTE: If the EBS Volume mounted is an existing volume which already contains data(StoreGrid Cloud AMI configuration xml files), the EBS Volume need not be formatted and file system created, instead EBS Volume can be just attached to the instance.

Section 3.6 - Firewall Configurations for StoreGrid

To allow StoreGrid clients to connect to the StoreGrid Backup Server, you need to grant permission to the 'default' security group which determines the ports opened up for communication in your EC2 instance. The TCP ports for which the permission should be granted are 32004, 32007. For accessing StoreGrid Web Console remotely, you can open up 6060 and 6061 TCP Ports

Enable the firewall configurations by following the below steps:

Step 1: Click on 'Security Groups' link under 'Navigation' panel.

Step 2: Next, click on default security group.

Step 3: And, add the necessary ports under 'Allowed Connections' in default group.

The TCP port for which the permission should be granted is,

Port Number Description
32004 32007
6060 6061
For default Backup Port For default SSL Backup Port
For default HTTP Port of StoreGrid WebConsole For default HTTPS Port of StoreGrid WebConsole
Section 3.7 - Bucket creation for StoreGrid Data Storage

You need to create the specified bucket (should be in lowercase) in S3 for your S3 storage before starting StoreGrid. You can create the bucket by using the AWS web console or S3Fox (Firefox addon) or Bucket Explorer. For more information on Bucket Restrictions and Limitations

Step 1: Select the S3 (Scalable Storage in the Cloud) from the Amazon Web Services home page.

Step 2: You can create a new bucket in the required region. Click “Create Bucket” button shown at top. Enter your Bucket name and select the desired region. And Click “ Create”.

NOTE: S3 Upload will take more than 2 hours for a freshly created bucket in any region. Refer the Amazon forum thread.

Hurray ! Now the machine is ready for StoreGrid Installation. You can install the StoreGrid Agent. For Installation you can refer these installation guides for Windows and Linux.

NOTE: By default the MySQL 5.5 and 5.2 ODBC Connector will be installed in this AMI. So there is no need to install any packages in this AMI. Once the StoreGrid Agent installation is successful, MySQL will be started automatically.

X