Backup & Beyond

I should say I am seriously impressed with EMC. I am talking about their announcement of a new subsidiary called Decho, which combines the two acquisitions they had made in the last one year: the online backup services startup, Mozy, and the Personal Information Management startup, Pi Corp. When EMC acquired Mozy I had thought EMC would use Mozy’s technology to come up with some cloud storage initiative for the enterprise and the mid-market segment. I also felt that would take away Mozy’s focus on consumer and small and medium business segment. Of course, it was probably wishful thinking too-because with our StoreGrid online backup solution we focus on the SMB market segment too.

We actually do not compete with Mozy head on as our focus has been on enabling MSPs and IT Solution providers to host and offer their own online backup service to their SMB customers. Now that EMC is creating a new subsidiary, Decho, which will exclusively focus on the consumer (and the SMB ???) segment we need to take note of that and be prepared to start competing with them sometime in the future. But it is always good to have a formidable competitor. That will help us motivate ourselves to think better and work harder to make StoreGrid a better platform for our partners to offer an online backup service.

Coming back to why I am impressed with EMC! Being such a large company primarily focusing on the enterprise and mid-market segment, it would have been an execution disaster if they had tried to keep Mozy ‘in house’ and focus on the consumer/SMB segment. Chuck Hollis, EMC’s VP, Global Marketing, puts it succinctly in his blog  post - as to why this is a great move by EMC.

“I think the decision to create a separate standalone entity speaks volumes as to how EMC’s thinking has matured: this is a market that’s important to EMC, we really don’t have this sort of thing in our DNA, better leave to people who DO understand this space, and give them what they need to be successful.”

I think it is next to impossible for EMC to position themselves in the SMB market given that the company was built on a model of selling to large corporations. With a separate business which will have its own management, organization & business model, they can now be a formidable force to reckon with in the consumer/SMB market segment.

Needless to say,  we are quite positive about the general growth in the market for online backup services and our ability to do well  (in a niche of our own, at the very least) by building a great online backup platform with StoreGrid. Our recent Amazon Cloud support reaffirms our commitment to keeping you at the cutting edge of technology.

Not that we are not worried about EMC….I’d rather say that it helps to have a ‘target Goliath’ - to stay focused and put up a good fight!

The initial interest in our StoreGrid Cloud AMI solution for Amazon Web Services has been extremely encouraging. As we have more and more partners & customers testing it and discussing their ‘use cases’, we have been getting fantastic insights into the myriad possibilities the cloud throws up for all types of users. This includes MSPs and VARs offering online backup services and businesses running custom applications in the Amazon Cloud.

Backup to the cloud

The typical use case and the premise on which we started our Cloud AMI, this deployment  has service providers running the StoreGrid Cloud AMI as a backup server in Amazon EC2 by provisioning their own Amazon EBS storage. Thereafter, they install the StoreGrid Client in their customer PCs & Servers and configure it to backup to the StoreGrid Cloud AMI. This deployment is very compelling for service providers who do not want to host their customers’ backup data in their own data center. Compelling as it is, it is worth mentioning a few disadvantages with this approach…

One of the advantages a local service provider enjoys is her proximity to the customer. This is especially important when a customer has large amount of data, say 100 GB or more, to be backed up. The initial seed backup for that kind of data over the internet is going to take a long time. To circumvent this problem, StoreGrid supports a feature called “Local to Remote Server Migration (L2R)” which allows the service provider to go on-premise and do a local backup of the first full backup to an external drive. The service provider then manually copies the data from the external drive to the StoreGrid backup server deployed in her data center and then runs the “L2R” module in StoreGrid. This will ensure that the subsequent backups are done incrementally, i.e. only changed blocks are sent over the wire on subsequent backups. With an Amazon deployment, using this L2R feature would not be possible simply because one does not have physical access to the Amazon cloud. Hence, the first full backup has to be done over the internet, regardless of however long that will take. The same is the case when you have to do large restores. A StoreGrid online backup Service Provider with her own data center can do a quick server side restore to an external disk and deliver it to her customer. With the Amazon deployment, the restores have to be done only over the internet even if it is 100s of GB of data.

By no means am I trying to discourage service providers from using the Amazon cloud as the data center for their online backup service business. But it is best to take decisions after analyzing all pros and cons along with what exactly the customers’ needs are. It is also best to set the expectations of the end customer upfront so that the customer is fully aware of, and educated on what she is signing up to. That way you won’t have a “but I thought you’d ship me my data in 1 hour” kind of situation.

A hybrid approach - backup locally and to the cloud

In light of the above discussion, service providers who want to leverage the Amazon Cloud but have the benefit of quick on-site restores could explore a hybrid option wherein the StoreGrid backup server is deployed locally in a customer site and the StoreGrid Cloud AMI is run as a replication server in Amazon EC2. In this deployment model, the on-premise backup servers would be replicating the backup data to the replication server in the Amazon Cloud. A single replication server can receive data from multiple backup servers running across multiple customer sites. We have many service provider partners using the hybrid approach already with the StoreGrid replication server deployed in their own data center.

Backup within the cloud - backing up data from custom applications running in Amazon EC2

Very interestingly, there are also a few service providers and some end users who are deploying the StoreGrid Client in Amazon EC2 along with their custom applications (which are already running in EC2). We did not think about this use case initially but in retrospect its a fairly obvious opportunity…

Considering that many businesses are looking at running their custom applications in Amazon EC2, backing up application data (which is typically stored in the Amazon EBS volumes) from these custom applications are extremely important too! Even though Amazon supports backing up the EBS storage to Amazon S3 as a snapshot, this is not always sufficient. The reason being the snapshot backup of a whole EBS volume does not provide the granularity required for a partial data restore. With snapshot backup, you can only restore the whole volume data into a new EBS volume. However, with a StoreGrid client deployed in an Amazon EC2 instance running a custom application, businesses and IT solution providers, now have the option of configuring file level backups of the EBS volumes. This also applies to backing up data from any application which uses a relational database back end like MySQL or Microsoft SQL Server - since these database backups are supported by StoreGrid!

Where would these clients backup to? Typically, to a StoreGrid Cloud AMI deployed as a backup server - ideally, running in a different availability zone in the Amazon Cloud. The backing up of EBS volume at a granular file level would give enormous flexibility while trying to restore data partially. No wonder we are already generating some interest with this deployment option.

A reversal of roles - backup from the cloud to on-premise storage

Honestly, we didn’t see this coming…

An end user mentioned that they wanted to backup all their data in Amazon EBS to their on-premise storage. Read that again - from Amazon to their office!!! I was initially not convinced and wondered why someone would want to do that? Here’s why! Though he (the customer) liked running his applications in Amazon EC2 because of the benefits it offered, he was not wholly comfortable with all his customer data present only in the Amazon Cloud.

“What if the Amazon Cloud goes down or what if Amazon itself loses my application data because of some bug or an issue?”, he said. He asked me if he could deploy StoreGrid Client along with his application in Amazon EC2, have a StoreGrid backup server on-premise in his office, and simply backup the application data (stored in Amazon EBS) to the on-premise StoreGrid backup server. “Why Not?”, I thought to myself, and asked him to try it out - there’s no reason StoreGrid shouldn’t work for this kind of a deployment!

On top of this he also told me he would backup the backed up data to tape periodically and ship it for off-site storage. While I personally believe (and have told him so) him to have ‘data paranoia’, I fully understand that this is the nature of the beast! It all depends on the value you attribute to your data!

Needless to say, we are excited about all these possibilities. We are especially excited with the challenge of enhancing StoreGrid to seamlessly support such possibilities. We are gearing ourselves to explore these new frontiers!

I’d love to hear from our (current and prospective) partners and customers about their views and experiences. Got an Amazon story of your own? Do let us know.

Hot on the heels of Amazon removing the Beta tag and releasing Amazon EC2 for production, we are excited to announce the Beta release of Vembu StoreGrid Cloud AMI, which facilitates deploying StoreGrid in Amazon cloud computing infrastructure. This has been a long pending demand from our partner base, who are MSPs, VARs and IT Solution providers offering online backup services using StoreGrid.

StoreGrid Cloud AMI Beta is available for both Microsoft Windows Server and CentOS Linux Server. Also, the StoreGrid backup server uses the MySQL 5.0 database. All these are bundled together in the StoreGrid Cloud AMI to facilitate ease of deployment for our partners. Of course, we are working on lot more automation as we try to move into production release before the end of 2008.

Why is StoreGrid Cloud AMI relevant for our partners?

Our primary target market segment is Small and Medium Businesses. Considering the growing complexity of IT infrastructure it is our strong belief that it is not easy for software vendors to directly service SMB customers. Close proximity to the customer is extremely important when you service SMB customers. Hence the local VAR or an MSP is in the best position to provide IT services to a small and medium business customer. This is especially relevant when it comes to data backups and more specifically online backups. As we work with large number of partners servicing different types of small and medium businesses with different sets of requirements, it is an absolute must that any IT product or solution we build should provide the maximum flexibility when it comes to deployment options or other relevant functionality.

Given this context, we have always focused on giving as much choice to our partners as they go about augmenting their business with an online backup service powered by StoreGrid. Specifically, as cloud computing as a framework gains momentum, as an aspiring leader in the online backup category, we recognize the need to provide the choice of deploying StoreGrid in a leading cloud computing infrastructure - and nothing beats Amazon EC2 and Amazon S3 for a start.

Moreover, for the last two years we have primarily worked with partners who are willing to host StoreGrid in their own data center and offer online backup services to their customers. Many of our prospective partners had expressed interest in having a solution which they can host in a cloud computing environment like Amazon EC2/S3. With the release of StoreGrid Cloud AMI, we are responding to a long under-served market demand.

With StoreGrid Cloud AMI, any IT solution provider (MSPs, VARs) can now start an online backup service without any capital investment. All they have to do is to get an account in Amazon Web Services, instantiate an instance of StoreGrid Cloud AMI, create and mount the Amazon Elastic Block Store (EBS) volume as a backup storage and start offering online backup service to their customers. It is as simple as that. The backup data stored in Amazon EBS is periodically backed up as a snapshot to Amazon S3 for redundancy. On top of this partners who require another level of redundancy can instantiate StoreGrid Cloud AMI as a replication server and replicate the backup data to another Amazon EBS volume. This again can be backed up as a snapshot to Amazon S3.

Our existing partners or partners who prefer to deploy StoreGrid in their own data center can now use Amazon cloud infrastructure as a redundant storage for the backup data in their data center. All they have to do is to deploy StoreGrid Cloud AMI as a Replication Server in Amazon EC2 and configure their internally deployed StoreGrid backup server to replicate the backup data to the StoreGrid replication server running in Amazon EC2.

As I said, choice and flexibility of deployment is what we provide our partners. To summarize, with StoreGrid, our partners now can offer an online backup service in the following ways:

1. StoreGrid backup server and StoreGrid replication server deployed in their own data center with their own local storage.

2. StoreGrid backup server and StoreGrid replication server in Amazon EC2 with Amazon EBS volume as the mounted storage. And for additional redundancy data in the EBS volume is backed up as a snapshot to Amazon S3 storage.

3. StoreGrid backup server deployed in their own data center with local storage and StoreGrid replication server deployed in Amazon EC2 with Amazon EBS volume as the mounted storage for the replication data. Again for additional redundancy data in the EBS volume is backed up as a snapshot to Amazon S3 storage.

4. Another deployment which is also popular amongst some partners is to deploy StoreGrid backup server on-premise in the end customer location so that there is local copy of the backup data for quick restores. And these partners can now deploy StoreGrid Cloud AMI as a replication server and replicate the on-premise backup server to the Amazon EC2 deployed replication server.

You can learn about more technical details on using the StoreGrid Cloud AMI at http://www.vembu.com/storegrid/amazon-ec2-s3-cloud-online-backup.html

At times, I’m a ‘to do list’ freak!

I’ve even been guilty of writing up ‘to do lists’ consisting of tasks, some of which have already been completed - simply so that I can have the juvenile pleasure of ticking them as soon as I write them!

However, nothing beats the pleasure of ‘ticking off’ tasks that have been languishing in a to do list for far longer than you’d care to remember. We had the opportunity to do this earlier this week - with one of the (many) to do lists for the Vembu StoreGrid Online Backup Partner Zone…

You can now afford to Forget Your Password!

Everyday, we get up to 3 emails from partners who have forgotten their Partner Zone login name/password!

The (long pending) forgot your password feature is now up and allows you to immediately change your password, if required.

What’s more, we’ll also email you details on other members from your firm who are using the same partner zone account - so you can be nice, and let them know that they need to remember another obscure password!

The bigger piece of news: You can now Edit Build-O-Matic on Demand!

For various reasons, we’ve (so far) been ‘manually throttling’ the custom build creation process. As many of you know, each time you needed to make an edit to your custom build or create a new custom build - you needed to email us to get your form back to ‘edit’ mode. In fact, we get an average of two such requests every day - to set the form back into edit mode!

You can now do this yourself, i.e., set your form back to edit mode, as required!

Although the post title says ‘Edit on Demand’, please use your discretion when requesting new builds. We don’t have a formal cap on the number of edits you can do but since it ‘costs us’ a fair amount of bandwidth and storage each time Build-O-Matic churns out a new set of builds, we request you to be judicious about requesting new builds - that way, we’ll (hopefully) never need to place a cap on how many edits partners do.

Partners who have never forgoten their passwords and/or never edited their build settings probably won’t perceive these features to be even mildly interesting. I’m sure the others are probably cursing us for not introducing this earlier!