Got an Hour? Create a Server in the Cloud
Dave Winer yesterday announced EC2 for Poets, a step-by-step guide to help you create a server on Amazon's EC2. His how-to is so easy to understand that we had our own server up and running within the hour. Sure, it may not seem like much that this fairly uninteresting page is sitting out there somewhere, but for this writer, it was an amazing coup.
"It's time to stop thinking about these servers as being things for geeks and start thinking about them as things for people with ideas," Winer said in a podcast roadmap he created for this work. The technology available today is enabling anyone with even the slightest technical bent to get out there and create amazing new things; often taking the technology in directions than the company which created it could have ever imagined.
EC2 for Poets is named after a class that Winer took at the University of Wisconsin called Computer Science for Poets, where the idea of "taking something that's inherently technical, and instead of doing something that technologists like to do with it - which is make it more mysterious - is to try and take as much of the mystery out of it as possible, and make it easy."
Much like Winer, many people find it difficult getting involved with Amazon's EC2 because of the new ideas and/or new terminology it introduces: "running instances," "EBS Volumes," Key Pairs," Elastic IPs," "Security Groups," 'AMIs." Even the definitions are more often than not difficult to understand. However, once you realize that much of the wording is just in a different form to the names of ideas we're familiar with, it becomes a lot more interesting. For instance, the term "security groups," Winer explains, "is essentially a firewall."
We went through Winer's HowTo: EC2 for Poets, and within 20 minutes had set most of it up. But for two instances, it went smoothly.
In step 12 of launch your server, Winer explains
"You should see a single entry whose status is "starting." We're now waiting for it to change to "running." This could take as much as 10 or 15 minutes, depending on how busy the angels and elves at Amazon are."
When we got to this step, our status of "pending" took almost 30 minutes to resolve. No biggie, just useful to note that it may take longer.
The second time we got a little stuck was when trying to wake the server up.
"First, locate the Key-Pair file (mine is called Tahoe), open it with a text editor (Notepad on Windows, TextEdit on the Mac). Select-All. Copy. Close the file."
It took a few minutes to realize this is the file we had downloaded earlier in the process; most likely due to this writer getting hyper excited of having become, as Dave Winer points out in good humor "a cloud computing expert."
Our 'instance' or 'server' took 20 minutes to set up; then another 30 minutes was spent waiting for the Amazon gods to make it a reality. Amazingly, a virtual server in under an hour.
EC2 for Poets is a must read if you're interested in utilizing the cloud for your server needs. And even if the cloud doesn't excite you in the slightest, we highly recommend spending an hour with Winer, if for no other reason than to truly see how technology today really is made for everyone.
As Winer points out in his podcast, "there is nothing inherently more difficult about installing software on a server than there is of installing software on a Mac or PC - it's exactly the same thing."
"Where we need to be," says Winer, "and this happens early on in every technology, we end up relying on the tech companies too much. It makes sense at first; we need someone to set it up for us and make it simple, but then, it has its downside because we end up being controlled by them and they may be taking us someplace we don't want to go."
"Then the users break out and they do it on their own," he adds. And this is what EC2 for Poets can do for us. It can help us get out there and do it on our own a lot faster.
An important point to remember: if you do decide to try this out, make sure you shut it off when you're done; you'll be paying about $1 for every eight hours you run the server.
But now, having created this server in the cloud, the question remains, what next?
We have a server which is hosting a single Web page, but we're fairly sure the ReadWriteWeb community can think of far more exciting uses for this service now that Dave Winer has made it easy for all of us to understand. So, what would you build? Let us know in the comments.
HowTo: EC2 for Poets
EC2 for Poets Roadmap (22 minute Podcast)
Share
Facebook
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
I want a virtual desktop. Something where I can have my email, favorites, word processing and web tools.
Wait. $1 per 8-hours? That's $90/month? Isn't that expensive? I pay $50/mo. for all the server service I can handle, supporting dozens of domains.
Nick, you're billed per minute of server uptime. If it's up 24/7, that works out to around $90 per month. Someone who was using it only to serve to themselves would presumably turn it off between accesses.
its too costly ..no to this
Its way too costly. But thanks for sharing the idea i never know about this concept.
Thanks for sharing... It's costly though..
If it's up 24/7, that works out to around $90 per month. Someone who was using it only to serve to themselves would presumably turn it off between accesses.
I did this a couple of days ago and found it to be a valuable experience. I highly recommend this to anyone remotely interested in in setting up a virtual server on Amazon's EC2. I'm not sure how far Dave is going to take this, but there is a follow up lesson for those who are interested. I've wanted to learn about this for some time now.As long as Dave's willing to help, then I'm willing to learn. It's an opportunity for me that's too valuable to pass by!
Humm useful technology but expensive as of today. But will be in thing in near future.
Michael -- that's the way to look at it, it costs very little to learn what EC2 is about. Now when someone writes about "cloud computing" you'll know, at a concrete and practical level, what they're talking about. And you may think of things that developers can do with this. I think of EC2 as a platform, and that people ultimately will look at the $90 not as how much it costs to run a server, but how much it costs to run a groupware application.
The next step is setting up ElasticFox which makes it really easy to work with EC2. Amazon itself has a pretty good write up as well, here is your
target="_blank">Getting Started Guide &
target="_blank">Setting up an Account to EC2.
We have just launched for beta testing a desktop client of our Yambi Unified Messaging for consumer and SMB. http://www.yambi.com. One of the thing I would like to do is to create a virtual server and upload my Yambi server and client on there. That will allow me to have Mail and IM aggregation and files sharing on my own server that I control and manage. It will be like having my own hosted Exchange, hosted OCS, Jabber server, Asterisk for $90 a month. It cannot get better than that. Controlling my own cloud. Imagine! Amazing!
I don't know what we would buid, but I would sure tell our webmaster to check it out. Thanks.
Great information to know. Our users will be glad to read this.
This is great. Amazon EC2 needs some more airtime, as it's come on in leaps and bounds recently with persistent storage volumes and backup solutions. It makes secure, scalable and redundant deployment seriously easy.
For anyone interested in who is using Amazon EC2 already, there is a solutions catalog here: http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=89
Why the heck to expensive. While I agree that we, as an industry, need to make it easier for non technical people to get a server up and running. But this guy is really selling a bill of goods that is anti-poet, and really a business model that is intended to create additional revenue by confusion.
We took a long look at all of these cloud services and truth be told instead of making it less costly than running your own server, it made it more costly. Hidden in a confusingly convoluted array of charges and costs. It's a great way to make money from the unsuspecting. But until they get their cost model down, it's cheaper and easier and MUCH less confusing (which isn't saying much) to run your own by using a shared provider or renting a dedicated server. I am not entirely sure what problem these guys were trying to solve.
Wake up and Smell the Coffee...
To those that feel EC2 is expensive, I would argue that its purpose (at the moment) is not to be a 24/7 web server. A write-up about how Smugmug uses EC2 outlines a more logical use of the EC2 service. They have their own webserver front end, and use EC2 for things like data processing. Their backend can start up instances and run jobs (and then shut them down) throughout the day. The kind of work they want to do happens to be highly distributable.
http://blogs.smugmug.com/don/2008/06/03/skynet-lives-aka-ec2-smugmug/
It is costly if your just hosting a single blog. Stick to a VPS or, god forbid, shared hosting. If you have moderate to high traffic that's unpredictable, then the cloud might be for you.
Running in the cloud is like managing multiple unmanaged dedicated servers. The only benefit I'm getting is being able to scale up or down to meet demand. Which, theoretically, should save me money.
Most of my sites are on either dedicated servers or VPS instances. I do have a few applications that sit in the EC2 cloud. I've also used the cloud for transcoding some very large HD video files. Worked great and cost me less than $15.
The pricing was a side issue for me that I wanted clarification or confirmation on. It's also transient. But enabling a run-of-the-mill person to set up and run a server in the cloud, on demand, is not only enduring, it's revolutionary, disrupting, freeing, and fantastically exciting. A little like watching blog technology develop in late 90s.
@Matt, good point, and one that I didn't mention. Yes if you have unpredictable traffic the cloud is an awesome place. The ability to absorb a hit and not drop to your knees is worth the pric.
@Nick, umm you are not witnessing anything revolutionary. NOBODY has created the product that you would like to see, the one that will let me mother or any average tech person to setup and manage their own site. I think that's what Dave was trying to show is that we don't have that yet. And twelves steps into confusion is just as bad as everything else we have out there. So you haven't seen anything yet that is revolutionary.
Clouds are nice for companies who need to expand quickly or have temp needs that can be meet within the cloud. But to run my service on the cloud, or to expect my mother to setup her own server etc.. IS pretty crazy. All due respect to Dave Winer about this, but he's a techy and still brings the techy view to the world. My mom would just stand there and go ok I really don't get that. And she's smart has a degree been a high level manager etc... So even she's not average, but she still wouldn't understand what the hell you just did.
So what I am trying to say is. You just watched an ad for Cloud services done by someone who wants to make money off of selling that service. You just bought into the sales pitch. The idea is good, the thinking is in the right direction, but execution is still a long way off.
Wake up and Smell the Coffee....
I think it's a great initiative to make Cloud Computing less "cloudy" than SOA. I agree the intention is to make things easier in terms of using those services, but it's still hard for me to believe if such concepts or procedures are for non-technical people in these days, even though in the future I believe it's gonna be achieved. What I see is that it's much easier for developers to start coding without concern about infrastructure.
@FireBrand, agree that Amazon doesn't work for everyone - it's expensive for a blog for example. For a startup service like mine it's very cost effective, and enables scale.
A lot of startups use Rackspace. However, we calculated that Amazon is cheaper than Rackspace for what we want to do.
Being able to tear down and add new servers at anytime brings many benefits. For example, when we upgrade the service we just bring up new servers on the new build and drop the old servers. no need to take the service or reduce capacity to upgrade.
After your critism of Amazon it's ironic that the second post on your blog demonstrates the risk of a cheap hosting solution. Amazon makes backup easy by the way ;)
@Duncan. LOL how true. I went cheap and got wacked. The blog is my rantlings collected. Not much traffic, so don't need much, but I didn't count on losing everything. Seems a trend these days, I have heard of a few other sites losing everything due to no backup. Shame on me. :-)
Amazon and Google are VERY expensive for a blog. But what I was talking about is for a real site, one that is growing it's not cost effective to stay on that service. We calculated it for a site that I am involved with and frankly buying equipment each month to keep up with the growth was cheaper than paying the monthly fee for the Cloud. Now it DOES make sense to have it as a buffer for when you get bursty in your growth.
But some sites, I think it's b0x.net are totally relying on it. Big mistake, but then again they are VC backed and idiots toss money at anything that looks like the next wave. On the other hand not having to do the midnight mad panic of bringing up new machines isn't fun while you watch your traffic bang on the one remaining server you have with free space/cycles. In the end you are paying each month after you get stable for them doing, what? Running your data center for you? That's something that is key and core to your business. It just doesn't make sense to me other than in the cases I have mentioned. And it certainly doesn't make sense for my Mother to use the Cloud.
I think when you look at a stable population of users, you are going to want to have that equipment in your own rack in your own cage under your own control, if you are a serious company. One has to wonder about the viability long term of a company who is stuck in the Cloud. Those costs aren't going to go away. Meantime a company that only uses the Cloud to deal with the bursty part of growing will have built out it's data center and the cost of running that will drop down to just the bandwidth and the space. Which as I said earlier, is less expensive than the Cloud, even doing backups right it's less. So how can a real company justify using the Cloud with it's current costs long term? I understand in startup mode, or to take on overflow traffic. I just don't see it as a viable way to run a business long term.
NOW if the costs drop, then it might make sense. But what I see are costs that essentially cover their growth allowing them to build their asset base while using your money to do so... One good hack into that cloud and you are down and most likely gone, which is the same issue if you are running it on your own, but you are a much smaller target than the "Cloud".
Wake up and Smell the Coffee...
@FireBrand: The product of revolutions don't spring forth fully formed like Athena from the Head of Zeus. The unfocused image of our future is slowly resolving here, I think.
@Nick, I would agree if there was something different than what we have now. That's the issue. It's nothing new, and really does nothing that we can't do now, and it's much more expensive than how it's done now. Cost is king. If they can't get it lower then it won't succeed in the long term. History is littered with technology that was cool and had promise but in the end wasn't cost effective. Anyone recall bubble memory, or any of a thousand other things that were very neat but just couldn't get the costs down to where it needed to be.
But on this one I think it goes past that. It's just nothing new or cool or interesting even. This is at best very expensive rooms outside of the hotel you own that you can use to put more people in WHEN/IF you overbook your rooms. Wow, that's great. That's earth shaking, that's what you need to hold lots of conventions about and anoint those who did great marketing to "create" a new thing. It's just not.
Please prove me wrong. I am not aiming this at you Nick. I am just trying to make the point.
I need something like this for the website I'm developing which runs a live auction for several hours once a week. During these few hours I could easily put a dozen instances to work handling the traffic. Later I don't need any of them as my main site is hosted elsewhere.
What's this going to cost me?
Well, let's say 4 hours per instance X 12 instances, that's 48 hours. At a rate of $1 per 8 hours that's only $6 per week -- a HUGE bargain compared with the cost of running a dozen dedicated servers.
@Fred (creative name by the way), Did you factor in the bandwidth and the storage? Course you might not need storage other than on your site.
But yes. This is a perfect use of it. It's still not the Cloud for Poets. But you have one decent example. But again that's not revolutionary.
Wake up and Smell the Coffee...
@tivrious Programming without thought of infrastructure? This is a real pet peeve for me. It is what separates engineers and IT programmers. You must think about your infrastructure when programming. It makes the difference in performance, later enhancement, and scalability. A big issue I've seen is that a lack of thought on infrastructure hobbles real ROI. Even bloggers have to think about their infrastructure.
Overall, I think EC2 is a step in the right direction. Sure, there is a question around support and management. But, if this really is that simple and de-mystifies another realm of computing for the masses, I'm for that. The cost of ~$1095 per year is relative considering what you do and who is doing it. If this lets webmasters in marketing departments more freedom, great. If this allows prototyping before making a large investment, even better. If this allows internal data crunchers statistical sandboxes, why not? What a great sandbox.
The big issue I see is how will IT handle the loss of control? I guess that is a bit of what I see in the comments.
IT has nothing to worry about loss of control. The Amazon EC2 service sucks. Nice job trying to market the gunk, not to mention shameless PR from Winer. Wake me up when stuff like this is ready for prime time, beyond the tinkering of ear-pierced toddlers at startups.
@Michele hmmm. Don't take such a sentence as an axiom. Of course, there will be always some issues regarding infrastructure that developers or software engineers must concern about.
What I would really like to express is the obvious opinion that with such Cloud Computing services, those who like to write doesn't need to manufacture the paper, or better, to buy an unknown amount of paper to write an unknown amount of words. I mean, as I learned here, for example, Smugmug focuses on what really matters, the system evolution regarding software. I don't think it is about losing control. Taking another example, there is Magnolia one, that if I am not making a mistake, tried to incorporate infrastructure processes that weren't their focus and culminated with a bad job.
IT has nothing to worry about loss of control. The Amazon EC2 service sucks. Nice job trying to market the gunk, not to mention shameless PR from Winer. Wake me up when stuff like this is ready for prime time, beyond the tinkering of ear-pierced toddlers at startups.