The Value of Sharing

How valuable is a robust, secure, scalable and reliable computing platform that allows you to store and retrieve any kind of digital data from anywhere in the world at anytime of the day? While your average high school student doesn't have much use for one, technology visionaries and corporate titans have been willing to pay big money to build such platforms. Groove Networks ran through approximately $120 million of investment in order to build and convince the world that theirs was robust, secure, scalable and reliable. Microsoft, which purchased Groove in 2005, is spending upwards of $2 billion in order to do the same thing. Google's entire business model requires them to reinvest huge amounts of capital into their storage platform. Throw in Yahoo and Baidu and it's clear that such computing power is seen as a technological edge.

However a high school student can rent one of the best, most proven, storage platforms of them all -- for a little less than $.50 per month. If he or she is willing to lay off the occaissional candy bar, he/she can be on an equal web infrastructure footing with the richest technology companies in the world. If I was an investor in MSFT, I might even ask if investing $200 million in Amazon makes more sense than $2 billion on your own technology and starting from scratch. In fact, taking it one step further, I might even be worried that Amazon's examples treat .NET as just another language - alongside Java, Ruby, PHP, Python and Perl. Another $1.8 billion into .NET and Visual Studio might take care of that, especially for those prized corporate programmers.

So Easy a High Schooler Can Do It

Like anything worthwhile, it takes a bit of time to figure out the best way to exploit it. The obvious comes first: use Amazon's storage capacity and scalability to store data such as images or globs of backup data. Indeed, a simple scan of Amazon's Simple Storage Service solutions shows many backup, photo sharing and large email attachment services. Furthermore, most of these services are web-based and so their single greatest cost is usually bandwidth. Hence Amazon's marketing focus on web 2.0 apps as their clientele. This makes perfect sense, as it fits the world's current multi-tiered architecture: the simple browser on the client, the business logic on a web server and now the robust data store on the backend.

Many well known and successful companies are exploiting this today: SmugMug, 37Signals and MyBlogLog to name a few. Would you know it buy using these services? Probably not, unless you have a keen eye reading the urls flash by as elements on your web page are downloaded. Does it matter? Not one bit. Each of these services uses Amazon S3 in order to offload the work of sending images and documents off their servers and onto Amazon's. As a result, each company can focus on their distinctiveness and bring it to market as quickly as possible, without worrying so much about infrastructure. In some cases it's enough to prove the concept and sell to a large company in just a few months.

The Next Fortunate Event

In the US there is a saying: "Only Nixon could go to China.". While Amazon has positioned itself as a proven resource for eager online entrepreneurs, they have also (accidentally?) created a solution that makes the client chic again - actually both the Client and the Server. Indeed, Amazon's Simple Storage Service takes us back to the 1980s, resurrects Client/Server architecture and provides it on such a scale that it actually works.

In the late 1980s, when C/S became a holy grail, it quickly sprang a leak at the supper table as large deployments grappled with networking protocol decisions, scaling issues, big capital investments for servers and, of course, maintenance. Amazon has moved these giant obstacles aside and resurrected something most thought dead. And this from an online department store!

While Amazon is still very much a consumer site selling anything they can display on a web page, at some point people will see the light and realize that a lot of C/S architecture implemented, deployed and wheezing along in corporations around the world can be refactored with services like Amazon S3. By providing the ultimate server, Amazon has made it possible for programmers to build corporate client software that a) actually scales and b) actually cuts costs - no more corporate servers to procure, house and maintain. A corporate programmer can create a client that can scale for the entire organization by using a good GUI tool and something only moderately more complicated than File.Open. Web application stacks of functionality might not look as appealing compared to this new reality. (Have you noticed that programming stacks have evolved like razor blades? We started with one, then two blades, then three solved the problem, until four blades really did the trick, and so on - in fact, aren't servers even called blades now?)

But while Amazon has provided the fuel, it's still up to the developers to provide the ignition; and right now we're dealing with sticks. By that I mean that Amazon isn't offering a database, nor server side scripting, interpreted languages, web server plugins or anything else to add complexity. Instead, at its core, is the world's greatest file server.

Groaning from the Peanut Gallery

The collective moan you just heard was from everyone who actually lived through the client/server days rolling their eyes and recalling how difficult it was to merge everyone's changes into a single file on the server. That was indeed a huge problem and yes it did lead to C/S being discredited. HOWEVER, the problem was twofold: 1) tackling multi-user scalability issues for the server and 2) tackling multi-user data issues for the client. In the 1980s, personal computing was still in its infancy, and nascent applications struggled to meet the needs of a single user, much less two.

If we look at the path highly available and scalable servers have travelled the last twenty years, we see a single machine-- even a really big one-- was simply inadequate for the task-- it's just too prone to failure. Furthermore, data needs have exploded past workgroups or departments in a single location-- people across the globe may need access to specific data now. Although it will take a post in the future to adequately describe how Amazon and others are building scalable and reliable data services, they do require multiple data centers located around the world connected by fiber optics, with each data center housing thousands of redundant systems built for quick switching in case of any single point of failure. In other words, a far cry from a PC running IBM OS/2 with an Intel 386 and 32 megs of RAM locked in a closet.

But while the server side of the equation has advanced over the last twenty years, can the same be said for the client? Or, more specifically, for the data clients are producing? No. While operating systems are more powerful today and prettier to look at, the state of application data today is not so different than it was back in the 1980s-- a stream of data supporting the state of the application as last saved by a single user. In this day and age, multiple people need to work in single context to produce a deliverable. Unfortuantely up till now this has been dealt with using the checkout, checkin, 'hey, you overwrote my data!' design pattern. The server has improved, now it's time for clients to respond in kind.

One More Event Before The Big Bang

The solution? How client applications read and write data needs to make the same advancements that server platforms made to make data storage reliable and delivery scalable. As application's become decoupled from the desktop, they should also decouple from the notion that a single user updates data at one time. Does only one person work ever work on a document? Does only one person ever participate in a project? Only if they wish to remain in the past.

Now that a superserver like Amazon S3 is almost a given (just a bit more reliability and maybe one more feature), applications can pull 'multi-user' files off, perform merge operations using local computing power and then place the file back in the cloud with an updated, combined view - ready for the next user to come along. Save doesn't have to mean 'write out my view of the data only' - it can also mean 'merge my view of the data with the group'. It might sound like a database, but it's really functionality that all apps working with groups, or in a group context, should have. The applications themselves will have the knowledge to merge files they recognize together - an excercise left to the developers of the next generation of client software.

There's a lot of refactoring to do, but this time the benefits are tangible. Entrepreneurs seem to be embracing Amazon web services, and the next wave can't be far behind. If Amazon manages to catch that next wave, they could be in for a great ride. And however it works out, in their quest for the ultimate online department store, Amazon might have finally solved the server side of the equation in Client/Server architecture. If they manage to attract the corporate programmers ready to build the next generation client pieces, then not only would Amazon again enjoy first mover advantage, but it would be for something worth a lot more than books. Even James Burke would have been proud of that connection.