ec2 - ReadWriteWeb http://www.readwriteweb.com/feeds/tag/ec2 en Copyright 2009 Richard MacManus readwriteweb@gmail.com Mon, 23 Nov 2009 16:43:23 -0800 http://www.sixapart.com/movabletype/?v=4.23-en http://blogs.law.harvard.edu/tech/rss Got an Hour? Create a Server in the Cloud davewiner_mar_09.jpgDave Winer yesterday announced EC2 for Poets, a step-by-step guide to help you create a server on Amazon's EC2. His how-to is so easy to understand that we had our own server up and running within the hour. Sure, it may not seem like much that this fairly uninteresting page is sitting out there somewhere, but for this writer, it was an amazing coup.

"It's time to stop thinking about these servers as being things for geeks and start thinking about them as things for people with ideas," Winer said in a podcast roadmap he created for this work. The technology available today is enabling anyone with even the slightest technical bent to get out there and create amazing new things; often taking the technology in directions than the company which created it could have ever imagined.

]]>Sponsor

]]> Aplus.net

EC2 for Poets is named after a class that Winer took at the University of Wisconsin called Computer Science for Poets, where the idea of "taking something that's inherently technical, and instead of doing something that technologists like to do with it - which is make it more mysterious - is to try and take as much of the mystery out of it as possible, and make it easy."

Much like Winer, many people find it difficult getting involved with Amazon's EC2 because of the new ideas and/or new terminology it introduces: "running instances," "EBS Volumes," Key Pairs," Elastic IPs," "Security Groups," 'AMIs." Even the definitions are more often than not difficult to understand. However, once you realize that much of the wording is just in a different form to the names of ideas we're familiar with, it becomes a lot more interesting. For instance, the term "security groups," Winer explains, "is essentially a firewall."

We went through Winer's HowTo: EC2 for Poets, and within 20 minutes had set most of it up. But for two instances, it went smoothly.

In step 12 of launch your server, Winer explains

"You should see a single entry whose status is "starting." We're now waiting for it to change to "running." This could take as much as 10 or 15 minutes, depending on how busy the angels and elves at Amazon are."

When we got to this step, our status of "pending" took almost 30 minutes to resolve. No biggie, just useful to note that it may take longer.

The second time we got a little stuck was when trying to wake the server up.

"First, locate the Key-Pair file (mine is called Tahoe), open it with a text editor (Notepad on Windows, TextEdit on the Mac). Select-All. Copy. Close the file."

It took a few minutes to realize this is the file we had downloaded earlier in the process; most likely due to this writer getting hyper excited of having become, as Dave Winer points out in good humor "a cloud computing expert."

ec2rww_mar_09.jpg

Our 'instance' or 'server' took 20 minutes to set up; then another 30 minutes was spent waiting for the Amazon gods to make it a reality. Amazingly, a virtual server in under an hour.

EC2 for Poets is a must read if you're interested in utilizing the cloud for your server needs. And even if the cloud doesn't excite you in the slightest, we highly recommend spending an hour with Winer, if for no other reason than to truly see how technology today really is made for everyone.

As Winer points out in his podcast, "there is nothing inherently more difficult about installing software on a server than there is of installing software on a Mac or PC - it's exactly the same thing."

"Where we need to be," says Winer, "and this happens early on in every technology, we end up relying on the tech companies too much. It makes sense at first; we need someone to set it up for us and make it simple, but then, it has its downside because we end up being controlled by them and they may be taking us someplace we don't want to go."

"Then the users break out and they do it on their own," he adds. And this is what EC2 for Poets can do for us. It can help us get out there and do it on our own a lot faster.

An important point to remember: if you do decide to try this out, make sure you shut it off when you're done; you'll be paying about $1 for every eight hours you run the server.

But now, having created this server in the cloud, the question remains, what next?

We have a server which is hosting a single Web page, but we're fairly sure the ReadWriteWeb community can think of far more exciting uses for this service now that Dave Winer has made it easy for all of us to understand. So, what would you build? Let us know in the comments.

HowTo: EC2 for Poets
EC2 for Poets Roadmap (22 minute Podcast)

]]>Discuss]]>
http://www.readwriteweb.com/archives/got_an_hour_create_a_server_in_the_cloud.php http://www.readwriteweb.com/archives/got_an_hour_create_a_server_in_the_cloud.php NYT Sat, 21 Mar 2009 16:20:16 -0800 Lidija Davis
Cloud Computing Is More Than a Computer in the Cloud It is quite a remarkable feeling to watch as the pieces fall into place and the picture, anticipated for so long, is finally revealed in all its splendour. As with any jigsaw that lacked a guiding picture on the box, the final result is that inevitable mix of vindication and surprise. Some areas of the picture are wholly unexpected, some look as one predicted, while across most of the image there are new facets to explore in familiar places, anticipated scenes to compare with long-held expectations, and assumptions to challenge or validate.

]]>Sponsor

]]> Recent advances in the business of cloud computing form just such a picture and reach out to encompass previously unrelated aspects of Web 2.0, the semantic web, platform computing, software as a service (SaaS), and the economics of disruption.

This is a guest post by Paul Miller, a Semantic Web and Cloud Computing expert who was most recently a Technology Evangelist at UK technology company, Talis.

Not merely some game of buzzword bingo on an unprecedented scale, cloud computing is coming into its own, and it is becoming increasingly easy to see the opportunities for a significant shift in the way we access computational resources and to recognize that the walls separating organizations from their peers, partners, competitors, and customers will become ever-more permeable to the flow of data through which those distant machines will compute.

There are many areas to understand that have already been ascertained in related fields, and many ideas unique to this space to discover. One early challenge is to carve a distinct niche for the place we are moving towards with such rapidity. Far more than "just" a cloud, it is an evolutionary cycle beyond the playful flippancy that diminishes so many of Web 2.0's poster children, and it is difficult to relate to mainstream misconceptions of the semantic web's complexity. Yet this new place is greater than the sum of its parts. So do we sustain the already ephemeral notion of cloud computing? Do we appropriate the "next big thing" label of Web 3.0? Or do we need a fresh attitude towards business computing's apparently insatiable desire to apply labels?

First, though, let us consider the shape of this thing that is taking on more substance with each passing day.

Reporting on last month's Web 2.0 Summit in San Francisco, CNET's Dan Farber notes that "the cloud was omnipresent," before closing his report with the observation that "cloud computing won't be very compelling without what is variously called Web 3.0 or the semantic web."

Indeed.

For too long, the emphasis in cloud computing circles has been almost exclusively on the provision of rapidly scalable and ad hoc remote computing on top of cost-effective commodity hardware. The cloud play by Salesforce, Amazon's EC2, and the rest has been dominated by the implicit assumption that these cloud-based resources are an extension of the corporate data center; a way to simply reduce the costs of enterprise computing.

There is value down this road, but there are bigger opportunities.

Nick Carr is among those who fear that a small number of players may come to dominate the provision of cloud resources. He outlines many of these arguments in his latest book, The Big Switch, and more recently had an interesting discussion with Tim O'Reilly on the topic. Justin Leavesley shares some of Talis' views on the economics behind all of this over on Nodalities, broadly agreeing with Tim O'Reilly:

"It's pretty clear that utility cloud computing is highly capital intensive so it should come as no surprise that there are powerful economies of scale to be had. But the bottom line is that you are talking about plant and power. These are rival goods, scarce resources that are created and consumed. This is not different from many utility industries with one exception: the distribution network has global reach, already exists and is very cheap compared to existing utility distribution networks. It is a lot cheaper to access a computing resource on the other side of the planet than it is to send electricity or gas across the globe... [So] what is to stop economies of scale turning this into a global natural monopoly?

"Actually, unless there are some large network effects, quite a lot stops single companies ruling entire industries. For a start, without network effects, economies of scale tend to run out: the curve is usually U-shaped. Telecoms, gas, rail companies have strong network effects from their infrastructure -- it makes little sense to have duplicate rail networks or gas networks in a country. Utility computing does not have this advantage because the distribution network is not owned by them."

Continuing the conversation, Carr summarizes the usual widely held perception of cloud computing nicely:

"The history of computing has been a history of falling prices (and consequently expanding uses). But the arrival of cloud computing -- which transforms computer processing, data storage, and software applications into utilities served up by central plants -- marks a fundamental change in the economics of computing. It pushes down the price and expands the availability of computing in a way that effectively removes, or at least radically diminishes, capacity constraints on users. A PC suddenly becomes a terminal through which you can access and manipulate a mammoth computer that literally expands to meet your needs. What used to be hard or even impossible suddenly becomes easy."

This is quite true, but it continues and further entrenches the misapprehension that the cloud is little more than an adjunct to the corporate data centre, a misapprehension that we shall get down to challenging in a moment.

First, though, there is a growing recognition that today's market leaders will inevitably need to become more interoperable if this business segment, and they, are to grow. The proprietary nature of their offerings today may allow them to innovate ahead of the standards process (which will be shaped in large part by the lessons they learn), and the relatively high cost of switching to a competitor today may give each the critical mass on which to invest and grow; but the characteristics of the current market are clearly the characteristics of a nascent market: computing's new Wild West. As so often before, standardization, true competition, mainstream adoption, and commoditization will all follow as we move towards phases 2 and 3 of Gartner analyst Thomas Bittman's intriguing analysis of the "evolution of the cloud computing market." Similarly, Erica Naone offered a useful overview of cloud computing's open-source component in Technology Review last month. None of the projects she covers are a significant challenge to Amazon's EC2, Microsoft's Azure, Salesforce's Force.com or Google's App Engine... yet. But together, they help to keep these commercial entrants honest and remind all of us that switching costs can be brought very low indeed if the pain of the status quo becomes too great.

Writing "Welcome to the Data Cloud?" for ZDNet in October, I began to explore the important role that data could and should play in the cloud:

"Just as 'we' used to duplicate and under-utilize computational resources, so we do something very similar with our data. We expensively enter and re-enter the same facts, over and over again. We over-engineer data capture forms and schemas, making collection exorbitantly expensive, whilst often appearing to do all we can to limit opportunities for re-use. Under the all-too-easy banners of 'security' and 'privacy' we secure individual data stores and fail to exploit connections with other sources, whether inside or outside the enterprise.

"In a small way, the efforts of the Linked Data Project's enthusiasts have demonstrated how different things should be. The cloud of contributing data sets grows from month to month, and the number of double-headed arrows denoting a two-way linkage is on the rise. Even the one-way relationships that currently dominate the diagram are a marked improvement on 'business as usual' elsewhere on the data web; even in these cases, data from a third party is being re-used (by means of a link across the web) rather than replicated or re-invented. Costs fall. Opportunities open up. Both resources, potentially, improve. The strands of the web grow stronger."

It is here, in the use and reuse of data, that the potential of the cloud will be realized. Back to the previously cited conversation between Nick Carr and Tim O'Reilly, O'Reilly himself comes very close to saying so:

"In short, Google is the ultimate network effects machine. 'Harnessing collective intelligence' isn't a different idea from network effects, as Nick argues. It is in fact the science of network effects -- understanding and applying the implications of networks.

"I want to emphasize one more point: the heart of my argument about Web 2.0 is that the network effects that matter today are network effects in data. My thought process (outlined in 'The Open Source Paradigm Shift' and then in 'What is Web 2.0?,' went something like this:

  1. The consequence of IBM's design of a personal computer made out of commodity, off-the-shelf parts was to drive attractive margins out of hardware and into software, via Clayton Christensen's 'law of conservation of attractive profits.' Hardware became a low margin business; software became a very high margin business.
  2. Open-source software and the standardized protocols of the Internet are doing the same thing to software. Margins will go down in software, but per the law of conservation of attractive profits, this means that they will go up somewhere else. Where?
  3. The next layer of attractive profits will accrue to companies that build data-backed applications in which the data gets better the more people use the system. This is what I've called Web 2.0.

It's network effects (perhaps more simply described as virtuous circles) in data that ultimately matter, not network effects per se."
(my emphasis)

Talis CTO Ian Davis would appear to agree, commenting:

"People need to be investing in their data as the long-term carrier of value, not the applications around them... The data is more likely to persist than the software, so it's important to get the data right and take care of it."

Salesforce CEO Marc Benioff, too, used his Dreamforce User Conference this month to move a company long associated with the "data-centre extending cloud" firmly in the direction of embracing data and the network. As Krishnan Subramanian noted on Cloud Ave before the keynote:

"Till now, the Force.com platform served business users to develop apps that can be used internally within an organization. They have to tap into Force.com APIs from outside platforms to offer customer-facing web apps. With the new initiative, it becomes easy for customers to allow the Internet users to 'interact' with their data."

Over on VentureBeat, Anthony Ha had more:

"Salesforce.com wants to become an even big player in the cloud computing market with a new service called Force.com Sites, which allows companies to host public-facing web applications in the Force.com platform. That means Salesforce --- nominally a maker of customer relationship management (CRM) software, but also an increasingly important platform for business-related applications --- is moving closer to direct competition with cloud giants like Amazon Web Services and the Google App Engine."

Locked away within an organization and only accessed by that organization's applications, data cannot be put to full use. Much of the value in each individual datum lies in comparing it to other measurements, in delving into detail, and in pulling back to observe the bigger picture.

Organizations that believe that either the big picture or the detail resides in their own systems alone are woefully misguided. Even the most specialized, proprietary, and confidential of data only reveal their true value when put in context, and that context is all the richer when informed by numerous perspectives.

Cloud computing, and the various SaaS movements, have finally brought us to a place where the fiercely guarded and tightly delineated boundaries between the organization and those outside it may become permeable in ways that should benefit the organization rather than threaten it. Data is just a resource. In the terminology of Geoffrey Moore, most data are often mere context, and there are savings to be made both in reusing the data of others and in re-selling necessary context to those prepared to pay. Some data, of course, is core to the business, and this may continue to receive the same reverence and protection that we misguidedly apply to the entire database today. Even here, though, the opportunities afforded by (controlled?) sharing may outweigh any desire to maintain data protectionism.

The language of Groundswell offers opportunities to go further, to embrace and exploit the behaviors and motivations of customers and the wider web.

There is clearly far more to say in clarifying this view of both the components and the whole, but at over 2,000 words, this post has perhaps gone on long enough.

For now, then, we should conclude by asking what role the semantic web has to play in any of this. The semantic web, with its unadulterated recognition of the primacy of the web's hyperlink? The semantic web, designed from the outset to convey context and relationships derived from data spread across the web? The semantic web, supported by technologies that operate openly and on the scale of the web?

Isn't it obvious yet?

Returning to the Web 2.0 Summit with which we began, another presentation was from Kevin Kelly, founding editor of Wired Magazine. Steve Gillmor and Nicole Ferraro reported on his presentation at the time, and the video was subsequently shared online, echoing Kelly's earlier presentation (which I greatly enjoyed), in which he argued:

"You have to be open to having your data shared... which is a much bigger step than just sharing your web pages or your computer."

Yep, here we go, on a journey toward Kevin Kelly's "World Wide Database," which will take in a lot of the shifts facing enterprise computing along the way.

This is a guest post by Paul Miller, a Semantic Web and Cloud Computing expert who was most recently a Technology Evangelist at UK technology company, Talis.

]]>Discuss]]>
http://www.readwriteweb.com/archives/cloud_computing_is_more_than_a_computer_in_the_cloud.php http://www.readwriteweb.com/archives/cloud_computing_is_more_than_a_computer_in_the_cloud.php Cloud computing Sun, 14 Dec 2008 10:45:10 -0800 Guest Author
Amazon Makes it Easier to Charge for AWS-based Apps with DevPay A few days after Amazon released their SimpleDB service, they also added the new DevPay service to their web application infrastructure stack, which makes it easier for developers to charge for apps built on Amazon's family of web services. "You can think of DevPay as an enabling technology for our other services. As as developer you will spend most of your time working with the other AWS services while counting on DevPay to allow you to monetize your hard work," wrote Jeff Barr, Amazon's Web Services Evangelist, on the AWS blog.

]]>Sponsor

]]> DevPay basically hooks into Amazon's storage (S3) and computing (EC2) services and allows developers to measure and charge for usage. For example, if you run a service that hosts files for people, you can use DevPay to automatically charge if they go over a certain limit and need to use more resources.

DevPay offers a lot of flexibility for developers, allowing users to create their own pricing plan "using any combination of one-time charges, recurring monthly charges, and metered Amazon Web Service usage." Beyond automatically determining usage, sending out bills, and handling payment processing, DevPay also handles the courtesies such as email notifications -- all of which are customizable via the API.

Amazon's first customer for their new service is Red Hat, who is offering Red Hat Enterprise Linux on Amazon EC2 via DevPay.

There was some initial confusion about the difference between DevPay and Amazon's Flexible Payment Service, which allows web sites to receive payments. "I'm still not clear on what the difference is from Amazon FPS. Is it specifically the ability to meter AWS usage? If so, why not simply build this into FPS?" wrote commenter A. Logan Murray on Amazon's announcement post.

The difference, according to Amazon, is that while FPS allows developers to charge for any sort of service, DevPay specifically hooks into the AWS. When you use DevPay to charge users for S3 usage, for example, the payments are made directly to Amazon and then deducted from your S3 bill (with the remainder being deposited into your Amazon account). DevPay cuts out the middleman -- your customers are paying Amazon directly for the web services they are using.

DevPay is priced at $.30 per transaction + 3% of the total amount billed. That's a bit higher than PayPal, but because it can also do the legwork of calculating how much your customers owe you based on how much they use, DevPay should be a very attractive way for developers to charge customers for AWS usage. Like other Amazon services, this one is only available in the US right now.

]]>Discuss]]>
http://www.readwriteweb.com/archives/amazon_devpay.php http://www.readwriteweb.com/archives/amazon_devpay.php Products Wed, 02 Jan 2008 12:57:20 -0800 Josh Catone