.

Two days ago,  Amazon announced Amazon Web Services (AWS) calculator on their blog. I can see this becoming quite useful, and am, in fact, surprised that they didn’t do this earlier.

You can access the AWS Simple Monthly Calculator here. The calculator, currently in Beta, allows you to get an idea of the kind of costs you can expect for different services that you might use. They’ve even had a stab at typical figures you might be seeing for some ‘typical use cases’.

Don’t see the use cases? You need to scroll to the right if your resolution is 1024 x 768 (or lower)  – see the screenshot above.

The five use cases they have are (comments in brackets are my own):

• A marketing website (bandwidth intensive; requests from visitors)
• A Web App (would use more of computing resources, including DB)
• A Media application (storage & bandwidth intensive)
• HPC Cluster (CPU intensive)
• Disaster Recovery & Backup (storage intensive and computing to a lesser degree)

Take the numbers with a pinch of salt (an ounce, in the case of DR & Backup). But that’s not the point – I think it is pretty bold of them to hazard a guess knowing fully well that one size won’t fit all…

Rather than take the numbers at face value, you should focus on the AWS services in the ‘mix’, i.e. the different services that AWS assume you will use for each scenario. It serves to give you a good idea of the services you should be looking at.

We’ll take a more detailed look and provide some data that’s more relevant to StoreGrid + AWS deployments. Stay tuned.

The above post was written by Lakshmanan (Lux) Narayan of Vembu Technologies. Vembu Technologies is a backup software vendor whose product, StoreGrid, powers the online backup services of a large number of service providers across the globe. Besides remote backup, StoreGrid is also used for on premise backups of workstations and servers at various companies & universities.

Vembu Home is the only FREE consumer backup solution for free local backups and optional Amazon Cloud backups. Get your FREE COPY now.

Google is key to our business! We spend a lot of time studying how people looking for online backup solutions found us – through the organic & paid results on Google.

Besides this affinity for many things ‘Google’, three other events inspired this post:

a) Google announced its quarterly results earlier this month. Though revenues were almost stagnant, Google managed to deliver a higher net income of $ 1.48 bn (Qtr ending June 30, 2009)

b) Techcrunch says Bing users are twice as likely to click on ads than Google users.

c) I clicked on the ‘I’m feeling lucky’ button on Google today – more out of curiosity than out of habit! This got me thinking…

Who clicks the ‘I’m feeling Lucky’ button anyway?

So, I googled it up (note the subtle irony here) and came across a very interesting article from Nov 2007 ; here’s an excerpt:

Google cofounder Sergey Brin told public radio’s Marketplace that around one percent of all Google searches go through the “I’m Feeling Lucky” button. Because the button takes users directly to the top search result, Google doesn’t get to show search ads on one percent of all its searches. That costs the company around $110 million in annual revenue, according to Rapt’s Tom Chavez. So why does Google keep such a costly button around? “It’s possible to become too dry, too corporate, too much about making money. I think what’s delightful about ‘I’m Feeling Lucky’ is that it reminds you there are real people here,” Google exec Marissa Mayer explained, or at least tried to.

So Google left $110 million on the table two years ago! Let’s try and recalculate the figure now…

a) Google’s annual revenue in 2008 was $22 billion – about twice its revenues in 2006. 66% of Google’s revenues are from Google owned sites. I’m simply assuming that Google.com (and other country sites) account for about 50%, and properties like Gmail and other Google sites account for the balance 16%. In effect, I’m assuming that search from Google homepage(s) contributes to $11bn of the total $22bn revenue.

b) Lets assume that the ‘1% of people click on I’m Feeling Lucky’ still holds. In fact, I suspect that this number would be higher considering the increasing number of newbie net users especially in developing countries! You’d expect the people who click on “I’m Feling Lucky” to be net newbies and unable to really differentiate between organic and paid results. Those well crafted & well placed ads (like ours!) would attract their attention a lot more.

c) If the average Bing user is twice as likely as the average Google user to click on an ad, I’d argue that the average ‘I’m feeling lucky’ clicker, had he clicked on the money making ‘Google Search’ button instead, would be at least thrice as likely to click on an ad. Hence the ‘I’m feeling Lucky’ guys could account for (almost) 3% of ‘opportunity’ revenues

So, let’s do the math…

$11bn in annual revenues x a conservative 3% =

$ 330mn in increased revenues – simply by removing the ‘I’m feeling lucky’ button!

What does this do for the bottom-line?

There is no direct cost (of revenue/goods sold) for this ad inventory – unlike the ad inventory from the content network where about 75% of the revenues are paid out to the content network partner. Hence this is $330mn of PURE Gross Profit! Google typically spends 30% of its revenues in R&D & G&A expenses. Assuming this ratio applies for the incremental revenues, and taking that expense figure of 30% out ($100mn) , you’d expect them to add a whopping $230mn annually to their net income – simply by removing the ‘I’m feeling lucky’ button!

This $230mn translates to an increase (in net income) of over 5%

I believe this is too tempting an opportunity for Google to pass up! When the going is good, its easy to say stuff like “It’s possible to become too dry, too corporate, too much about making money… “ . In the current economic climate, you can expect Google to milk every opportunity they have. After all, they still ‘answer’ to Wall Street!

Considering all this, the “I’m Feeling Lucky” opportunity is too much of a potential cash cow (or sitting duck, or low hanging fruit – pick your metaphor) to pass by. Remember, it translates into a direct increase of 5% in net income!

Of course, when it happens, Google won’t tell you the real reason! Expect to hear something like…

“…extensive research conducted by Google showed that the absence of the I’m Feeling Lucky button translated to a user taking 6 milliseconds less – to reach the results page he was looking for. As much as the ‘I’m Feeling Lucky’ is a part of Google history, removing it makes for an enhanced user experience, and finally, Google is all about the user’s experience. So, with a heavy heart…”

As some people say, It’s all about the Money, Honey!

Remember, you saw it here first!

The above post was written by Lakshmanan (Lux) Narayan of Vembu Technologies. Vembu Technologies is a backup software vendor whose product, StoreGrid, powers the online backup services of a large number of service providers across the globe. Besides remote backup, StoreGrid is also used for on premise backups of workstations and servers at various companies & universities.

Credit : Wikimedia Commons

Ah, the price of fame!

Tim McGuire of  “The Backup Plan” has been offering his customers ‘Vembu powered’ online backup services since January 2007. His company’s website is http://thebackupplan.com

All’s been well, and Tim has been steadily signing up new customers and backing them up!

However, in recent times, in spite of more business, and a huge traffic surge, a smaller proportion of his website visitors are really keen on his backup services!What happened? Well, Hollywood happened…

I have Google Alerts set up for various terms, and ‘backup’ is obviously one of them. As a consequence, I get snippets on everything from generator backups to data backups. Last month, I noticed a couple of alerts for ‘The Backup Plan’. I clicked on the link expecting to see something about Tim’s firm, but was surprised to see JLo’s smiling face instead! Apparently, “The Backup Plan” is the title of a new movie starring JLo. The movie is scheduled for a 2010 release but the spin doctors are already hard at work, and legions of crazed fans are typing thebackupplan into their browsers & search engines – hoping to see more on Jlo!

Not one to disappoint, Tim has even created a link for fans – that takes them to another site that focuses on the movie!

I’d emailed Tim last month asking him if he was being wooed by Hollywood, and this is what he had to say…

“…I am actually getting a brisk amount of traffic coming from people searching for J-Lo.  Unfortunately, not a lot of them are interested in backing up.  I did get a call from the movie people themselves, but they are mostly interested in seeing if I can link to their site, but they actually do not have one set up yet.  thebackupplanmovie.com is actually my site also; I started that when the traffic was spiking to my main site.  I wanted to try to divert the movie traffic to a site just for the movie, and leave my customers, prospects, and partners as the ones hitting my server.  So, on that site, I just started blogging about the movie and actors, and giving the fans of the actors a place to chat about the movie….”

In fact, when I last checked, Tim has even written an ode to The Backup Plan & JLo! He also blogged about the launch of the movie’s website and called it ‘cute’. He almost seems to be enjoying his new role; it certainly looks like Tim’s crossing over to the dark side! You could help avoid this by visiting his site; it would be great for him to get more people looking for the real backup plan!

It will be interesting to see how his traffic jumps when The Backup Plan (movie) releases in 2010! I’ll save that for a new post…next year!

The above post was written by Lakshmanan (Lux) Narayan of Vembu Technologies. Vembu Technologies is a backup software vendor whose product, StoreGrid, powers the online backup services of a large number of service providers across the globe. Besides remote backup, StoreGrid is also used for on premise backups of workstations and servers at various companies & universities.

Unless you’ve been living under a rock for the last few months or mistook it for a new version of Wolf, you’ve probably heard of the Wolfram Alpha engine and the (wrong) comparisons to Google.

For starters, Wolfram Alpha is NOT a search engine – its (what they call) a “computational knowledge engine” – it generates output by doing computations from its own internal knowledge base, instead of searching the web and returning links.

Slight detour to StoreGrid…

StoreGrid has a feature called “Local to Remote Migration” (LtoR) wherein you can ’seed’ a backup server with the 1st full backup and can therefore start with doing only incremental backups from day 1 itself. You’d typically take advantage of this feature for ‘first full backups’ that you think would take a long time.

How much time will the first backup take? Ask Wolfram Alpha…

For example, if you had 10GB and wanted to know how long it would take over a 1Mbps line (For asymmetric broadband plans, remember that you should need to look for the upload speed – NOT download), simply enter “data transfer time 10GB, 1Mb/s” – try it out using the box above.

See the answer, and if you don’t have the time, go the LtoR way! Go ahead, give it a spin.

Click here for more examples of “Web & Computer” stuff you can use Wolfram Alpha for. A larger list of possibilities is here – and includes things like knowing how many calories you had for breakfast and getting information about satellites.

What’s more, it even knows the meaning of life!

Have fun!

The above post was written by Lakshmanan (Lux) Narayan of Vembu Technologies. Vembu Technologies is a backup software vendor whose product, StoreGrid, powers the online backup services of a large number of service providers across the globe. Besides remote backup, StoreGrid is also used for on premise backups of workstations and servers at various companies & universities.

Talk about data deduplication (in the backup and archiving domain) seems to be gaining a fair amount of momentum in the last few years! Most enterprise backup software vendors like Symantec (Veritas), EMC (Avamar) etc. support deduplication in some form or the other – some do deduplication in the source system (that is being backed up) and others do deduplication at the target (backup/storage server). There are also pure “deduplication based storage hardware vendors” like Data Domain who have gained considerable traction in the enterprise.

I am actually quite surprised by the hype around deduplication and the adoption it seems to have gained in the enterprise. The reason I am surprised is similar to the one I articulated in my previous blog post: “Synthetic Full Backup in the online backup world – Are we inviting trouble?“. The crux of my argument is that backup and archiving is about building redundancy to the data and not about eliminating redundancy in the name of efficiency of storage or network bandwidth. So it is my contention that wherever feasible we should have as much redundancy to the data (that needs backing up) and only under unavoidable circumstances should we resort to using synthetic full backup or deduplication. Actually, let me state this more strongly: “avoid falling for the synthetic full backup or deduplication hype if you can!”

But who am I to say this. I am neither an “industry expert” nor am I Steve Jobs to say “this is what is good for you; take it or leave it”. Given that we are a niche company trying to grow (and growing) in the face of industry giants, we are actually contemplating building deduplication support in our data backup software, StoreGrid. While not many of our customers/partners are asking for it, we do get the occasional prospect saying that deduplication (rather, the lack of it) is a show stopper feature for them!

As we started thinking about and designing the best way to support deduplication in StoreGrid, we encountered many options to consider and many complexities to be handled. But at the end, we were left with a fundamental question – whether a full-fledged deduplication is indeed possible in the online backup world! Before I explain some of the options and the complexities, and why we think a full-fledged de-duplication may not be feasible in a pure online backup scenario, let me first get into a broad overview of the two deduplication approaches…

Deduplication at the source (client) vs. at the target (backup server) : There are vendors who claim they do the deduplication at the source (i.e. the client system that is being backed up) as opposed to others who claim that they do deduplication at the target (i.e. at the backup server). If deduplication is done at the source then it is easy to deduplicate data at a block level across all files within the source system. If deduplication is done at the target then it is equally easy to deduplicate data at a block level across all files across all the client systems backing up to the backup server. Quite obviously doing deduplication across all files across all clients will be much more effective than doing deduplication only at a client system level. It is theoretically possible to do deduplication at the source system and still be able to deduplicate across all systems backing up to the backup server. In this case, each client (source) has to continuously update itself with the meta-data of the blocks that are being stored in the backup server. The meta-data in this case would simply be the checksums of the blocks. These checksums are looked up to identify similar blocks of data. I have not personally tested such a product myself – i.e. the ones doing deduplication at the source system and still being able to deduplicate across all systems backing up to the backup server. But this may not be as efficient in terms of performance as compared to doing the deduplication at the backup server end, especially if the backup/storage server resides at a remote data center (and the meta-data needs to be downloaded each time from the remote server).

Armed with this background, lets dive deeper into the implications of these ‘approaches’ in the online backup context…

Option 1: Deduplication at target
One of the most important requirements in the online backup domain is that the data that is backed up is encrypted before the data leaves the source system and is sent over the internet to the remote data center (where the data is stored). Deduplication works by finding similar blocks across all the files and physically storing only one copy of the block in the storage system. And encryption works by destroying all patterns in a given data and making the data random. Because of the way encryption eliminates all patterns, trying to do deduplication on a set of encrypted files will have no effect – i.e. finding similar blocks of data across encrypted data will not be of much use as encryption would have eliminated all patterns. That means doing deduplication at the remote storage end, where all the data from different clients systems are encrypted and stored, is technically not possible. The option of not encrypting the data that is being backed up to the remote data center is not really an option in the online backup world.  Another point to note is that deduplication at target doesn’t really help much in the case of an online backup scenario – clients still send all data across and hence don’t save anything on bandwidth! Of course, you save on ’server side storage’ but optimizing this, I’d assume, comes a distant second to optimizing bandwidth utilization – for online backups!

Option 2: Deduplication at source – with a common encryption key
As I said before it is theoretically possible to do deduplication at source and still be able to deduplicate across all client systems in an organization. In order to do that, either the data should not be encrypted during backup or all the client systems will have to use a common encryption key to encrypt the data. Not encrypting the data is not really an option with online backups. Using a common encryption key would mean that for each block of data that is backed up the checksum signature of the unencrypted block is also sent to the backup server where it is stored. Every client that is backed up should look up this database of checksums stored in the backup server before sending a block of data to the backup server. Though this can be done efficiently, I am not really fond of this option, because of the performance penalty, considering that the backup server is at a remote location in the case of online backups.

Option 3: Deduplication at local target backup server – with offsite replication
The only practical option I can think of is to have a deployment model where all clients in an organization backup to a local backup server – without encryption. The backed up data is deduplicated at the local backup server and then encrypted and sent to a remote backup or replication server. This deployment model will ensure that the deduplication is done on data from across all clients backing up to the local backup server.  Depending upon a customer’s preference, the local backup server can either keep a copy of the deduplicated backed up data (for quicker restores) or the backed up data at the local backup server can be purged (not recommended) once the data is moved to the remote backup/replication server.

In summary, we prefer the last approach, viz. doing the deduplication at the target backup server which is deployed locally at the site where clients systems are. This would allow the client to backup to the local backup server without encrypting the data – thus facilitating  deduplication at the target. And for offsite storage, the data from the local backup server would be deduplicated, encrypted and sent to the remote backup or replication server.  This would also ensure that the benefits of bandwidth savings associated with deduplication are also achieved.

I look forward to feedback & suggestions on other ‘better’ ways of implementing deduplication in the online backup domain!

The above post was written by Sekar Vembu of Vembu Technologies. Vembu Technologies is a backup software vendor whose product, StoreGrid, powers the online backup services of a large number of service providers across the globe. Besides remote backup, StoreGrid is also used for on premise backups of workstations and servers at various companies & universities.