A big part of the announcement is Facebook's vision of a consumer Semantic Web. In this new world, publishers have an incentive to annotate pages by marking up activities, events, people, movies, books, music and more. The proper markup, would in turn, lead to a much more interconnected Web - people would be connected with each other across websites and around the things they are interested in.

As you will see from the rest of the post, it appears that getting semantics right has not been a big priority for Facebook, at least not prior to the announcement. Here are the issues we identify:

1. Open Graph Protocol does not support object disambiguation
2. Open Graph Protocol does not support multiple objects on the page
3. Launch partners have not implemented Open Graph Protocol correctly on their sites
4. Facebook does not have the markup on its own pages that it asks the world to adopt
5. A growing amount of user profile data is full of duplicates and ambiguity

Concerns with Open Graph Protocol

A week ago, we complimented Open Graph Protocol for its simplicity, but upon closer look we are seeing a couple of flaws. First of all, there is no way to disambiguate objects. For example, two movies that have the same name would be considered to be the same movie. A proper way to deal with this sort of thing is to introduce secondary attributes like director or a year that can help identify specific object, but the protocol does not define secondary attributes.

The second issue is that there is no way to markup the objects inside the page. In its current version, the protocol only supports declaring that entire page is about a person, a news event, a musician or a movie, but there is no way to identify objects inside the page. This is a big use case for bloggers and review sites - each blog post typically mentions many entities, and it would be nice to support this use case from the start.

Both of these shortcomings are easy to correct. The nice thing is that the protocol is simple and minimalistic, so adding the bits to handle disambiguation and multiple entities is straightforward. The other things that we are going to discuss, are much more troublesome

Launch Partners - Why No Markup?

The truism of making the Web more structured is adding more markup. No matter how limited, having markup on the pages is always better than not having it. When Facebook announced the Open Graph Protocol, it highlighted several sites that are already using it. Among them were Yelp, IMDB and Pandora. We took a look to see how exactly these sites are marking up their pages. What we found is rather surprising - none of these sites implemented markup correctly. We looked at the How to Train Your Dragon movie on IMDB, Brad Pitt's page on IMDB, the Muse page on Pandora and the Acquagrill page on Yelp.

This is what Facebook defines as required properties:

And this is what we found on the actual partner pages:

So what does this mean? It means that Facebook implemented special handling for these sites. When a user likes a movie on IMDB or when she likes a movie star, Facebook can't really tell the difference since IMDB is not passing correct information via the protocol. The only reason it works is because IMDB is explicitly hard coded by Facebook.

The roll-out for launch was not generic, but custom, targeted more towards PR than correctness. Why would Facebook allow this instead of having partners implement correct markup is unclear. It is so easy to implement, and the partner pages already have all the necessary information. We conclude that enforcing correctness was simply not a priority for the launch.

Eating Your Own Dog Food

As it turns out, not only did publishers not markup pages - neither did Facebook. At the time of writing of this post, none of the entity pages on Facebook.com have Open Graph markup. So much for being open - Facebook's own pages remain closed. Ironically, it might not be because the company does not want to markup the pages, but instead that it can't. At least not yet.

Figuring out what is on the page is actually not a trivial problem. This is what semantic technologies that Freebase, Powerset, Open Calais, Evri, Zemanta and GetGlue, among others, have been building over the past several years. To be able to markup the pages correctly, especially the ones created by the users, Facebook needs to run them through a sort of semantic processing and disambiguation. This isn't a trivial matter.

Unstructured data on user profiles

All of this comes full circle to impact the users. As the Like buttons spread through the Web, so is the unstructured, duplicated data spreading through user profiles. Absence of semantics creates fragmented connections and noise around the Web.

Below is the listing of movies that I liked and fetched via Facebook Open Graph API. How to Train Your Dragon shows up twice, because I liked it once on IMDB and then also on Fandango. Friends that are see on Fandango page are different from the ones I see on IMDB. And worst of all, all this uncleaned data is showing up on my profile - movie title contains a year in one case and the originating site in the other case.

So right now Facebook does not correlate things across sites. Instead, it just captures the information as is, hoping to maybe clean it up later.

Conclusion: A different goal?

All of these facts when added together lead to the obvious conclusion: Facebook's goal is not to create a better, more structured Web. Instead, it appears that semantics is an afterthought in the race to capture user identity and information, in exchange for sending publishers the traffic.

As more and more data flows into Facebook via the Like buttons, Facebook and publishers are getting the benefit of recycling friends through the content on sites around the Web. But at the same time, the data in user profiles is becoming more and more noisy. Since not as many users are paying attention yet, it just looks silly under a closer inspection.

But to be able to power recommendations, to make social plugins a success and to facilitate good user experience, Facebook will literally need clean up its act. Duplicate and dirty data will be a big turn off for the users, and the longer this problem goes on the more difficult it is going to be for Facebook to deal with it.

We will see in coming weeks and months how the social networking giant will handle this issue. In the mean time, we'd like to reverse the tables. Please tell us what do you think about Facebook's semantic Web ambitions. Should they have gotten the core bits right first before the launch, or is this fine and they will be able to quickly catch up?

Disclaimer: Alex Iskold is a founder and CEO of GetGlue.com, a social network for entertainment. GetGlue developed the ability to connect users across different sites through a combination of browser addons and semantic databases in the cloud.

The bits of this platform bring together the visions of a social, personalized and semantic Web that have been discussed since del.icio.us pioneered Web 2.0 back in 2004. Facebook's vision is both minimalistic and encompassing - but its ambition is to kill off its competition and use 500 million users to take over entire Web.

Facebook Open Graph: Publisher Plugins

The Open Graph is a set combination of publisher plugins, semantic markup and a developer API.

Login with Faces & Facepile: The simpler publisher plugins enhance Facebook Connect. They makes it easy and compelling to sign in by leveraging Facebook cookies and showing faces of Facebook friends who are already members of the service.

Like Button and Like Box: These plugins add the liking feature to any content, typically the whole page. Both can be enhanced with semantic markup, described below. But the very basic intent for these is to get users to Like on the site and post a link to Facebook, which is then permanently stored on a user's profile and points back to the original site.

Activity Feed and Live Stream: These plugins show static and dynamic activity on the site. Activity Feed lists recent likes and comments from the site, while Live Stream shows a real-time view of activity on the site and is intended for interactive events.

Recommendations: This plugin surfaces personalized recommendations for the user based on what friends and everyone else is liking on the site. It is intended to drive the users to other pages on the site.

Facebook Open Graph: Semantic Markup

Facebook announced simple, RDF-based markup to make the plugins smarter. In a nutshell, the markup enables publishers to say what object is on the page - a movie, a book, a recording artist, an event, a sports team, etc. This automatically enables semantics, that is, an understanding that the user is not just interacting with a webpage, but that he or she is liking a specific kind of thing. Semantics then leads to bucketing of the objects into categories like books, movies, music, etc., and gives rise to all sort of applications, including personalized recommendations.

Perhaps even more importantly, the markup helps Facebook connect the users across common interests across different websites. For example, if both Pandora and Last.fm annotate a page about The Beatles using Facebook's markup, then users will be able to see their friends, who like the Beatles across different sites. This is very significant, because the data around friends is sparse and scattered around the sites. Previously, Facebook would surface this data in the stream without persisting it. Now, the information about a friend's likes of movies, music, books, recording artists, events, sports team, etc. will be permanent on Facebook profiles and readily available in context around the Web.

Facebook Open Graph: New API

The new Facebook API is elegant and streamlined. It makes it easy to access user information (with permission of course) such as profile, friends, etc. All of the calls are REST based and return JSON objects. For example, my profile information can be fetched like this: http://graph.facebook.com/alexiskold. The authentication is based on OAuth 2.0 protocol and makes it simple not only to connect, but to also prompt for permissions to access user information.

This new API turns Facebook into a read/write storage of users' tastes. And not just one user - all Facebook users.

Implications for the Users

With this release, Facebook asks users if they are willing to trade off privacy for personalization. To be clear, no personalization is ever possible without users telling a system about their tastes. What Facebook is asking for is necessary in order to then create personalized Web experience. Whether users want this sort of thing is a different question, but assuming that you want to know more about your friends you will.

Friends' interests around entertainment, sports, travel, etc. will be categorized and available. It will be easy to figure out what your friends are into both on Facebook and around the Web. In addition, Facebook is going to be using its own engine to bring you recommendations for related content. This will further accelerate the discovery and cross linking between friends. This will likely further impact the amount of search people do around the Web. As Fred Wilson pointed out - passed links replace search.

Yet, the crux of user implications is neither of the above, but one single issue: privacy. It is unclear at this point that this issue is a concern for actual Facebook users, but it is clear that tech world is raising its eyebrows: Marshall Kirkpatrick, Dave Winer, Jeff Jarvis and many others expressed their concerns. People are saying that not only Facebook will know too much about us (because Google is already there today), but that it will be able to control too much.

Personally, I am skeptical that the average Facebook user is going to care all that much. People are notoriously naive about being watched on the Web, and this is likely to be no exception. More likely than not, Facebook users will enjoy the personalization aspects of the new platform and won't think much about it - until Facebook starts openly targeting them.

This was not been part of f8 of course, but Facebook is likely to use the information for targeting. After all, advertising is a major part of its monetization already so why won't it make it even better? If this targeting is too spot on, lots of users will probably get annoyed. Facebook is likely to sooth them via Facebook credits and heavy discounts, negotiated because of their massive volume.

How exactly users react remains to be seen, but they will probably like the new Facebook more because of increased relevancy and interaction with friends around the Web.

Implications for Publishers

On the surface, this Facebook offering is a no-brainer for publishers. Who does not want more social activity on their site? However, in reality this is far from a slam dunk. To understand why, consider two types of sites: sites that are either social networks or have social networking integrated, and the sites that have their own commenting and ratings systems. In the first camp you will find Last.fm, Flixster, Goodreads, etc. None of these sites were a launch partner, understandably so. Social connections around music, movies and books are their bread and butter as are the ratings, reviews and recommendations. If they switch to Facebook for all of this, what do they have left?

So any site that already has social networking built in has to decide to abandon that before jumping into the Facebook Open Graph. The even worse problem is the ownership of ratings and comments. Are publishers really ready to give that up? Nobody seriously thinks that users are going to be rating through Facebook and then through the site again. So how is this going to work? It is unclear at this point, but it's likely publishers will ask for ways to replicate or export comments and likes that users sent to Facebook via their site. Perhaps an open API that allows publishers to manipulate the data is the answer, but it is easy to see how some publishers would be very concerned.

However, if you run a website like eCommerce or a blog or a service like Pandora that currently does not have a lot of social built-in, this offering is a no-brainer as it will instantly start recycling your pages through the massive Facebook power of passed links.

Implications for Competitors

This is aggressive and brilliant move by Facebook - and Twitter, Google, Yahoo, MySpace, AOL, eBay, Amazon and others, except for Microsoft, should be really worried. It appears that Microsoft is content with just partnering with Facebook, perhaps rightly so. Possibly a Bing deal is in the works, which would make a lot of sense.

For all other players on the Web, the worry is that Facebook is trying to close the loop in exclusively owning user eyeballs. Apparently Facebook is not content with just connecting people; it wants to connect people and things. And not only that, it wants to do it around the Web. And not just any people - friends. You don't need to look too closely to see that Facebook is creating a feedback loop, which includes it, users and the rest of the Web and excludes its competitors.

There are several things that other big players might try to do, the worst of which is to try to mimic Facebook. The "me too" that we've seen way too many times recently has not worked, and will not work now. The second best choice is to try to block it. As strange as it sounds it might just work. Between publisher and user issues there are a lot of concerns, and a carefully orchestrated and coordinated campaign may seriously hurt this initiative. Remember, Beacon was brought down fairly quickly by a combination of user backlash and derogatory press.

The third option - to embrace and extend this platform, to innovate on top of it - is likely to be the best move. Innovation has always trumped stagnation on the Web. The problem is that it might not be that easy to embrace this initiative. After all, it does not look like Facebook asked everyone to gather around the table and cooperate on this. It might not be open to cooperation, but if it is then this is the way forward.

Technically speaking, what Facebook has done is elegant and correct. From markup, to plugins, to API, all of it is modern and awesome. The missing bit is that Facebook appears to be the only repository of data in this equation - and that makes the whole offering seriously closed. Publishers and users don't have a choice as to where to store the data. It is going to Facebook and Facebook alone. Perhaps there is a way to rework the system in a way that fixes that. We will look forward to see how this unfolds.

Implications for Facebook

Clearly this announcement is yet another turning point for Facebook. Before the conference Facebook was the biggest social network on the planet. If its vision actually happens, Facebook will be the biggest network of people and things on the planet- or to put it differently, it will be the taste graph of the planet.

Obviously there is a different technology that Facebook will need to be building. It already perfected the social networking part, but semantic analysis, recommendation systems, vertical categories like movies and books, as well as having completely open read/write storage of tastes is completely new to the team. The biggest challenge that Facebook will face is to inject, re-deliver and most importantly make use of the data that is flowing into it.

Facebook will be doing some serious number crunching and UI revamps to prepare for this next phase of its life. But perhaps the biggest experiment and test will be delivering relevancy. Google succeed with this in search; Facebook will now have the challenge to bring relevancy to the recommendations and taste-based advertising arena.

Implications For the Semantic Web

One of the most exciting parts of the Facebook announcement to me personally is the possible breakthrough in semanticizing the Web. We've written previously about the Semantic Web here, and it has been a personal passion of mine. What Facebook has done has a chance to make vast parts of the consumer Web including movies, books, music, events, sports, and news semantically tagged. Publishers and websites finally have a strong incentive to mark things up and get return traffic from Facebook.

The actual protocol that Facebook suggested is very simple. To describe the object on the page, the site owner needs to specify the title, type of the object, image, url and the name of the site using simple meta tags. The format is extensible and additional tags can be added. For example, for a book a site can add an isbn number. This format leaves room for ambiguity. The goal of classic semantic markups traditionally has been to refer to entities precisely; for example adding the director to a movie, or a year to remakes. The Facebook protocol does not seem to have this.

There were lots of previous efforts to markup the Web. To name a few, RDF, microformats, Google Rich Snippets, Yahoo's Search Monkey (based on RDF and microformats), and lastly, abmeta, which was developed by me with help from Peter Mika at Yahoo. Of all these formats, Facebook's is most similar to abmeta because the markup is placed into meta tags, and is simple and human readable. This simplicity is the key to broad adoption.

So all around, this is a great chance for the Semantic Web to finally hit consumer verticals and become real.

Implications for Developers

Every new rich platform that has been rolled out in the past couple of years presented a big opportunity for developers and this one will be no exception. While we do not know exactly what sort of applications will be build on top of new Facebook, we know that they will be very powerful. This platform has the potential to give rise to to new kind of personalization and attention economy that people have been talking about for years. It has of course, a chance to majorly backfire, but I am optimistic.

This will be a gold rush for application that is likely to last for at least a year, like the last one did. It's too early to tell whether this will be a platform that survives and does not hurt is participants. However, it is very likely that the best applications built on this platform will be owned by Facebook. Still, there is a huge new opportunity here for developers and the sky is the limit.

Checkmate?

Facebook made a major chess move. It might have checkmated its competitors, or perhaps it might have to lose another piece like it lost Beacon. Whichever is the case, right now there are deep implications for Facebook and its competitors, publishers, users and the Web at large. What Facebook has announced cannot be ignored and can not be undone. Everyone needs to figure out the next steps and understand what to do.

Time will tell where we land, but my gut is that positive things will come out of this. If nothing else, let's give Facebook credit for innovation and re-imagination the Web.

This is where the book review come in. Every publicist and publisher's dream is to land a positive review with an authoritative source. A good review in The New York Times or the L.A. Times used to be a pass to big figure sales. Sounds like it still should be, but it is not, because most book reviews are poorly formatted and cannot be recognized by Google and other software.

Lets take a look at this edgy review of the Manhood, by the L.A. Times. It is a pure joy to read - it is elegant, clever and gets to the heart of the issue. There is only one problem with it - nobody is going to read it, because Google can't find it.

Try running this very specific Google search - "Manhood" by Mels van Driel review - and you will not find the L.A. Times among the results - at least not within first three pages that humans would care to flip through. How come might you ask? Well the answer is simple - there is nothing whatsoever that tells Google that this post is a book review about this particular book.

And this is not just an isolated problem with this book review from this particular newspaper. The issue is widespread across all major U.S. and international media outlets. Either due to lack of tools or lack of understanding how search engines and other software works, people notoriously don't make their content discoverable.

A Simple Way to Please Google

So how should be the book reviews tagged?

To start with, the title needs to make it clear, that this is a book review. Of course humans may find a more subtle title more enticing, but for the sake of machine: Book Review: Manhood by Mels has to be present. It would be even better to mark up that this is a book review, and here is the book title and here is the author.

Next, the post needs to be adorned with the right tags and keywords. L.A. Times' reviews are certainly very clever, but again, Google does not get humor. A better tag would the title of the book, the name of the author and the non-conspicuous phrase "book review".

A Better Way to Please Google and Tim Berners-Lee

The tagging system described above is still error prone. A computer might not interpret it correctly and would miss this post in the search results. This is because that kind of description is not structured. Humans enjoy a wonderful ability to deal with fuzzy things; computers simply can't do it.

For a computer to understand content, it needs to be described using a markup language. This is a broad and complex topic that has been a focus of the so-called Semantic Web and structured data.

The right way of marking up content so that it can be understood by Google, other search engines and semantic technologies is by using a structured format such as ePub, hReview Microformat, abmeta or one of the other structured formats. Using a structured format removes the ambiguity and enables computer to "know" what the review is about.

Making the content discoverable by Google in turn makes it discoverable by humans.

Tagging: It's All About the Money

Could it just be that book reviewers in major newspapers would get more page views if they did a better job tagging content? And then in turn, could it also be that if more people discovered clever and elegant reviews then more books would be sold? Even if you don't think so, there is way too much risk of getting this one wrong.

Doing appropriate, standard tagging and markup of book reviews is cheap and simple and should be part of the daily publishing routine. Each media company needs to invest in standards and guidelines around content markup. This is not just a matter of being good citizen of the Web, it is a matter of making money.

Photo credit: Ivan Petrov]]> Discuss]]> http://www.readwriteweb.com/archives/why_google_and_other_humans_dont_read_your_book_reviews.php http://www.readwriteweb.com/archives/why_google_and_other_humans_dont_read_your_book_reviews.php Semantic Web Sun, 14 Feb 2010 18:00:00 -0800 Alex Iskold FriendFeed has recently launched a search feature, and so Facebook search must be coming soon.

Real-time Web search (of streams of activities) is a hot topic right now. Everyone, including Google and Microsoft, recognizes the value of using trusted contacts as filters. What was once called social search is now called real-time search, but this time it will really happen. First, it will be applied to streams and then to the Web in general.

What we are about to get is a Social Relevancy Rank. Whenever you search streams of activity, the results will be ordered not chronologically but by how relevant each is to you based on your social graph. That is, people who matter more to you will bubble up. How does this work? Well, there will be a formula, just as there is a formula for Page Rank.

Here is an idea so obvious that it is surprising Twitter has not implemented it already: front-load search results with people you follow. When you search for, say, "Wilco" on Twitter today, the results are in the chronological order. That is not really relevant because you do not know who most of these people are. But if instead you could see people you follow, the search results would be much more useful.

This is not possible on Twitter today, but it already works great on FriendFeed. There, results are filtered or ranked based your social graph. This is not difficult for FriendFeed to do because, on the one hand, it knows who you care about and, on the other, it applies its advanced feed search technology to your social graph:

This sounds awesome, but there is a problem. "Wilco" works well as a query because the band has just released a new album, but many other queries would return no results. Simply put, your friends on Facebook and people you follow on Twitter can't possibly have an opinion on every topic you may be interested in. This is a problem of sparse data: trusted opinions are scarce.

Small Worlds and Taste Neighbors

To solve the problem of sparse data, we need more data... obviously. One possible solution is to incorporate other sources that you trust (i.e. broaden your social graph). As a next step, search results could rank people you may not be directly following but who are being followed by people you follow. Or in Facebook-speak, friends of friends. You could argue that you are not familiar with their opinions and so cannot yet trust them, but given the small world phenomenon, their contributions are often just as valuable.

Another step could be to include people with similar tastes, so-called taste neighbors. This approach is common among vertical social networks such as Last.fm, Flixster, and Goodreads. These networks have ideas about which people, other than your friends, are like you. However, this is a costly calculation and takes time. In order for Twitter to do something like this, it would have to compare people based on links or perform semantic analyses of tweets over time. Yet even though this is a difficult problem, it will be solved in time.

The Influencers and the Crowd

Aside from using the "second degree" of your social graph or taste neighbors, a Social Relevancy Rank could front-load influencers. In the absence of any other metric, someone who is followed by hundreds of thousands of users is likely more relevant to you than someone you don't know at all. Using number of followers as a weight might be a good way to order the rest of the activity stream.

In general, combing through countless tweets from strangers is not terribly useful anyway. Just as people have stopped looking at anything beyond the first page of results on Google, sifting through pages of tweets in chronological order gets tedious quickly. What needs to be incorporated into the Social Relevancy Rank is the aggregate sentiment of the crowd: a score that tells you yay or nay and gives you an opportunity to drill into more results if you choose.

The Quest for the Perfect Filter

There is no such thing as a perfect formula. Even Page Rank isn't perfect. Yet we all use it and find it useful. Much as Page Rank has been adapted and tuned to search the web, Social Relevancy Rank will evolve over time to help us make sense of endless streams of activity. This ranking will have a profound impact on how we tap into our friends' opinions.

It will change the face of general Web searches in time, too. Today, results are automatically ranked by relevancy and freshness. Once Social Relevancy Rank is factored in, search results will be re-ordered based on social relevancy.

And now, as always, please tell us what you think? What would you expect from a search engine with Social Relevancy Rank built in?

Mike Masnick followed on TechDirt with an insightful post in which he attributes some of Gladwell's confusion to the way that Anderson wrote the book. Masnick says that the book does not provide enough details on the mechanics and applications of Free. (I haven't read the book, so I can't comment on that.) Fred Wilson joined the conversation with a sharply delivered post on Freemium and Freeconomics. He gives examples of the kinds of Free that actually work.

So, as Albert Wenger wrote recently, there is "continuous confusion about free."

This is because the topic is broad, and everyone is taking a different angle. In this post, we will break down Free into three separate classes: the kind that actually works, another that struggles, and the last that can be dangerous.

Freemium: When Free Really Works

Fred Wilson nails it on the head when he identifies the two instances when Free actually works. The first instance is the service or software that offers a free trial and then converts users into paying customers. There are different flavors of this approach, the most popular being, give the basic version for free and charge for the advanced version.

An early example of this model was online email, where you got a certain amount of storage for free and had to pay for more (see more about this, though, in the section on when Free is dangerous). Other examples in this category include project management software, like 37signals, and online photo collections, like Flickr.

The second instance that Wilson identifies is the consumer service that manages to build a massive audience. Citing Facebook as an example, Wilson says, "Free gets you to a place where you can ask to get paid." He argues that because Facebook has managed to amass such a valuable asset, it is able to monetize in any number of ways. Citing Business Insider, he lists Facebook's revenue:

• $125 million from brand ads,
• $150 million from its ad deal with Microsoft,
• $75 million from virtual goods,
• $200 million from self-service ads.

Interesting that all but one revenue source here (the virtual goods) is advertising. The only thing that consumers of this Free service were willing to pay for was a supplemental service in the form of virtual goods.

In any case, the main point is that, given a truly massive audience, monetization opportunities present themselves, at the very least in the form of advertising.

Old Media: When Free Cries

It is ironic that the very thing that makes large consumer services successful also makes old media cry. Online advertising does not seem able to deliver the kind of revenue that old-fashioned subscription services did. The culprit? A drastic drop in the cost of publishing, and complete destruction of barriers to entry. Even at the turn of the century, publishing was a closed game. Today, anyone can be a publisher, thanks to the read/write Web (no pun intended).

What really angered Gladwell was Anderson's verdict on journalists. Gladwell writes:

"It is not entirely clear what distinction is being marked between 'paying people to get other people to write' and paying people to write. If you can afford to pay someone to get other people to write, why can't you pay people to write? It would be nice to know, as well, just how a business goes about reorganizing itself around getting people to work for 'non-monetary rewards.' Does he mean that the New York Times should be staffed by volunteers, like Meals on Wheels?"

While this question is valid, it misses the point. It does not matter whether journalism should be free or not. The issue is that those old media profit margins are nowhere to be found anymore. And so the money dissipates, the way that the big VC money from the '90s can no longer be deployed in tech.

To answer Gladwell's question, journalists will still get paid, but they will get paid to work at smaller outfits, like ReadWriteWeb.

Free, abundant content and more nimble, agile news sources from the blogosphere and Twitter are striking a deadly blow to old media. Old media cries because it can't figure out how exactly to remain the way it was. Ultimately, it can't.

Monopoly: When Free Bites

Most of the discussion around Free focuses on the freemium model and media. When we wrote about Free earlier, we focused on a different side of it: how Free can be dangerous.

The problem is that large companies can exploit Free in a way that is essentially monopolistic. A large company could enter a brand new market to undermine competition. Consider Google Docs, a completely free consumer product that serves no ads and competes with Microsoft Office. A subtler example is Gmail, which does display ads (even if they don't attract many clicks) and has limited storage, but the limit is so high (2.5 GB) that the product is essentially free.

Free can also be used to kill off competition and create a barrier to entry. IBM was the main player behind the open-source project called Eclipse, a platform for building software applications. Seemingly innocent and even good for the world, the initiative managed to kill off all of the small and mid-sized players in the market within five years. In doing so, it killed innovation and became the de facto tool for building Java applications.

When I spoke about the danger of Free during a recent summit on Freeconomics, I brought up a point that did not seem to resonate with the audience. I wondered, what are the moral implications of Free, and what specific impact does Free have on children? For example, what is it like to grow up in a world in which most software is Free? Does Free create a sense of entitlement? Does it lead people to wonder why they should pay for anything at all? Where do we draw the line on what should and should not be free? These questions are not simple and are certainly far from being answered.

Conclusion

Clearly, Free and Freeconomics are broad and complex topics. No single post could begin to address all of the issues involved. Anderson's book is timely and important. While we need to be careful, Free is also inevitable. Not only is it our future, it is already our present. So we need to understand what it is and what impact it has on the Internet, our lives, and our children.

The debate that is unfolding around Free is fascinating to follow and even more fascinating to participate in. So join the conversation with your posts, comments, and tweets!

True, Twitter traffic has gone through the roof. True, Twitter is the new killer app, the new cool kid on the block. And yes, even Oprah now loves Twitter. But does this mean Facebook should be worried? Well, maybe yes, but likely no, because Twitter and Facebook are two very different services.

It is always hard not to be the cool one anymore. Whether you're a movie star or an NBA player, going from #1 to #2 is hard. It is even harder not to be near the top at all. Aging is pretty hard and something we all have to deal with. But change is inevitable, part of the cycle of life. The new comes in and replaces the old.

The history of the software industry is a classic illustration of this kind of transition. IBM was replaced by Microsoft, which reigned for decades. Its strong grip was taken over by Google, and for a while Google was cool. With the rise of the social web, things have changed again. MySpace, Facebook, and now Twitter are taking over in people's minds as the newest, coolest kids on the block.

But IBM is still around and doing well. So is Microsoft, which now looks more like IBM. And, of course, Google, despite not being so cool anymore, is still king of the web. Not so cool, perhaps, but certainly very solid and with enviable revenue.

So maybe Facebook should not feel the need to be so cool and look like Twitter.

Why Did Facebook Take Off to Begin With?

Facebook emerged out of Harvard and was initially a network for college students. It started out as a simple way to keep in touch, to see what was going on around campus. By the time its doors opened to everyone, Facebook had a few things going for it:

- Unlike MySpace, it had clean and elegant profiles.
- It made sharing pictures easy.
- It made sending private messages to friends easy.
- It made posting public messages on walls easy.

In short, Facebook solved basic problems of communication between friends. And it solved them very well.

But it felt compelled to continue evolving. Perhaps it felt threatened by Google's foray into the social space. Perhaps it was enticed by the prospect of being bigger than Google. Or perhaps it was the $15 billion valuation offered by Microsoft that set the bar too high. Whatever it was, it kept rolling out features, including the Facebook platform and Beacon, which aimed to make the web revolve around it.

Facebook blew past MySpace and managed to keep Google at bay. It firmly won the race for the social web. But now it has begun a brand new race, this one against Twitter.

Twitter and Facebook Are Just Different

The thing is, though, this race makes no sense. Facebook and Twitter are simply two different services that need to co-exist on today's web. The only thing they have in common is that their users have a limited number of hours in the day in which to socialize.

At its core, Facebook is about closed sharing between a group of friends. That is why my sister, one of Facebook's first users, felt so compelled to use it, not because of apps or Beacon or tabs within tabs within tabs. It was simply about photos and messages on walls between friends.

Unlike Facebook, Twitter has not added new features. It has stuck to its core product: connecting people via short messages. And unlike Facebook, Twitter has allowed uni-directional connections: if you want to follow someone without him or her following you, you can. Twitter was never about sharing between friends in the first place, but rather about sharing news. And if you look at Twitter today, it has clearly changed the way the world consumes news.

So, Facebook is chasing a rival that is playing, in Gartner-speak, in a different Magic Quadrant. And that does not make sense because even if it refocuses on streams, Facebook at its core is about friends, not news. Even if it had public pages for celebrities that everyone could follow, Facebook would still not be about news. It's just different.

Focus on What You Do Best

Chasing Twitter could be costly. Facebook likely won't overtake Twitter and what it has built up today. Twitter has won that race already. But if Facebook continues to spend too much time trying to re-position itself, its core business (i.e. connecting friends) is in danger of becoming vulnerable.

Clearly, MySpace, with its brand new management, is not wasting time. AOL is cooking up some interesting new stuff itself in the social networking space. And Google may just decide to make Chrome more social than other browsers.

So, it seems that Facebook's best path to preserving its strength is to not waste energy chasing Twitter. Instead, it should return to its roots and core strength: being the #1 social networking site that makes it easy to network with your friends.

If Obama and his team are going to use the Internet to help change America, so should we. The Internet is the most powerful information technology ever invented, and it has already changed the way we elect the President of the United States. Next, it just may help us restore America to its true self. How? One person at a time, starting with you and then passing it along. Here are 7 things you can do on the Internet to help President Obama turn this great country around.

1. Read About and Know the Issues

The least you can do is be aware. In this day and age, ignorance cannot be excused. All the information is out there, and whether you are Democrat or Republican or independent, the least you can do is find it. Maybe you prefer the Huffington Post or the New York Times or Fox News or CNN or independent local media; whatever your channel, spend at least 1 hour a week learning what is going on in America and around the world.

2. Join the Conversation

Obama is not a one-way president. If he was, he would never have assembled such a diverse, opinionated cabinet. Mr. Obama knows that our strength is in our diversity, and he wants to hear from you. We've been living in the age of the read/write web for the past 5+ years, and we have recently seen a decline in user participation. The time is now to re-engage, particularly around the conversation on how to turn this country around. You can directly help shape the future of America, one comment at a time. Find the blogs and online newspapers that you like and comment on the issues that matter.

3. Spread the News

The social explosion on the web created a wonderful way to disseminate news. Before, news was broadcast through centralized hubs; now, through Twitter, Facebook and other social networks, the news is spread by individuals. The implication is that your tweet matters. When you tweet news, it does not matter if it reaches thousands or dozens of people. What matters is that it reaches the people who want to hear the news from you. By spreading the news, you are leveraging our social fabric, helping the right information travel through the right channels.

4. Learn American History

To change the future, it is important to know the past. Even if you are well versed in American history, there is always more to learn. Learning history is both educational and patriotic. It is the subject that all of us should turn to in tough times. A good starting point is the Wikipedia page on American history, packed with links to historical figures and events. About.com features a whole subdomain dedicated to American history. You can find articles on subjects ranging from the Civil War to presidential elections to biographies. For more in-depth learning, head over to Kindle books on Amazon. Over four thousand books on American history come up.

5. Get Your Finances in Order

Let's face it, the latest financial crisis is partially our fault. Sure, Wall Street execs were greedy and went too far, but so did we. Many people were not disciplined and lived beyond their means. If you haven't yet, start using financial software such as Mint and Wesabe, and get your finances under control. Getting organized is the most important step to getting your finances in order. And like news dissemination, fixing the financial crisis is also done one person at a time.

6. Learn about Green Tech

The issues of global warming and green technologies are going to be at the forefront of Obama's presidency. Yes, the current economic crisis and the needless war in Iraq are the issues of the day, but global warming and green tech are the issues for decades to come. The first step you need to take in 2009 is to educate yourself. What is this all about? What is my energy consumption? What are "green homes"? Which products are green? What is the state of the solar energy industry? The web is full of green tech information; you just need to pick a source to start learning. And for a deeper look at the profound issues facing us, read Thomas Friedmans' Hot, Flat, and Crowded.

7. Engage Locally

There is one place where Obama needs your help more than anywhere else: your community. Federal government can't reach out to each corner of America, and frankly, that is not its job. This is why you can really make a difference by engaging in local politics, buying regional produce, taking your kids to a local history museum. Every township is online these days. You can connect through the Internet and engage in local issues. From spring to fall, you can shop at your local farmers' market, helping make things a little greener. Stepping up and doing just a little locally really goes a long way.

Conclusion

We are living in troubled yet exciting times. Electing Barack Obama was only a first step to turning America around. Now the tough work, the heavy lifting, begins. Mr. Obama is ready, but he needs your help. Take the time to think about things you can do on the Internet to help The Change. It's patriotic, it is right, and it is a lot of fun. Ready? Let's do this together!

See also: ReadWriteWeb Guide to Celebrating Inauguration Day

In general, I am just not good with physical things, because I am a software person. I've always been fascinated by people who can easily make sense of physical objects, because for me it takes a lot of effort even to put together children's toys. My brain is wired differently, to see patterns in software, not in hardware. But most people are the other way around.

A Brief History of Making Physical Things

Before we dive into the realm of software, let's look briefly at how we got here. Clearly, the history of humanity is the history of hardware. Since the Stone Age, when we invented the hammer, the world around us has been, so to speak, a nail. For the past several thousand years, we figured out how to use physics and chemistry to make physical objects. For a long time, we were craftsmen, until the industrial revolution brought scale and systematic manufacturing.

The main thing we learned is patterns in physical objects. We know that we can bend them under certain conditions. We know that there is friction. We know that things react differently to heat. The millions of facts that occupy our lives have to do with how we interact with the physical world and make physical things. These patterns get wired into our brains and help us live our daily lives.

Software 1.0: Why Computer-Phobia Happened to The Mainstream

Computers have software inside that does not behave like physical objects do. The key thing about software is that it is soft and that the conventional laws of physics do not apply to it. As a result, it's hard for people to picture what is going inside. It is hard for people with brains trained to deal with physical things to understand how software works. Because there are no physical laws we are used to, what are we to expect?

I constantly meet really smart non-technical people -- doctors, lawyers, teachers -- who have no clue how computers work. They treat them as magic black boxes that randomly break and never make sense. Why? Because software is a fundamentally different kind of system. It does not behave like the other things around us that we are used to.

On top of being different, software was really hard to use as recently as 5 years ago. PCs and Windows are to blame for the computer-phobia that still dominates the mainstream. My wife works for a large pharmaceutical company, and her Dell with its Windows 2000 is a monster. I do not blame her for being confused. In addition to being unbearably slow, the computer just makes no sense with its crazy administration scripts, incompatible drivers, annoying popups, endless choices, and daily crashes.

There is nothing in the physical world as complicated or confusing as her computer. All of the systems in our home work with the push of a button; all her daily routines are much simpler than figuring out how to turn off auto-spellcheck in Microsoft Word. Like most people, my wife is perplexed by the complexity of her work computer. But thankfully, things are turning around.

Software 2.0: Apple, Ajax, Video Games, and the New Laws of Physics

We changed our household computer to an Apple in 2004 and later on got one of the first iPhones when they came out. Obviously, my wife has no trouble using Apple products, because they are simple. More important, they, like physical things, behave as expected. Every user action results in a reaction that makes sense. There are way fewer choices, and things do not pop up out of nowhere.

Even better, this new kind of software proves to the user that the digital world can be superior to the physical one. There is no friction; things move around effortlessly; and they do work. This is not happening just with Apple products; it is happening all over the web. With the advent of AJAX, a new breed of web applications is teaching users that software can indeed work better than hardware. And that it can be way more fun.

It's the fun element that is likely going to be the tipping point, because it involves kids, our future. The latest video games are simply stunning. Like Apple software, they feature a lot of physical realism and behave as users expect. But in addition, they add a lot of special behavior only possible in the digital world. And it's not just about flying or instantly transporting from one place to another. Video games are creating a whole new reality that is an intricate blend of physical and digital.

Our kids are growing up native to this new digital world. To them, the new rules of digital physics are what the rules of physical physics are to us. They take these new rules for granted, because that is just how all our brains work.

Techies + Mainstream + Kids = Software Revolution

I never had trouble grasping software. To me, its inner workings and abstractions always made more sense than the physical things around me. But this is not so for many other people. Yet, the shift is on its way. Good software is becoming an integral part of our lives. Apple is one of the first and most visible examples of how software is taking over the hearts and minds of mainstream consumers. Its software is beautiful, simple, it works the way we expect it to, and people are starting to love it.

And this love is the first step to truly getting it, to understanding how something works. Because this new software is simple and fun, people are becoming attached to it and are willing to spend more time with it and, ultimately, to understand it well. This new software has a shot at making people get it, just like they get hardware.

And kids, of course, are already there. For them, all of this novelty is effortless and natural. It's a ton of fun, and they do not think twice about which buttons to press; they just know. So, with techies, the mainstream, and kids on board, we are poised to enter this brave new world dominated by software. The impact of this new world is going to be huge because, whether we like it or not, we will no longer be the same. Our brains, while never forgetting the laws of physics, will increasingly adapt to the laws of this new, digital physics.

Editor's note: Looking back over 2008, there were some posts on ReadWriteWeb that did not get the attention we felt they deserved - whether because of timing, competing news stories, etc. So in this end-of-year series, called Redux, we're resurrecting some of those hidden gems. This is one of them, we hope you enjoy (re)reading it!

Web sites are only one kind of presence that companies can have today. Social Networks like Facebook and MySpace, mobile platforms like iPhone and Blackberry, browser extensions and RIA Applications all have an equal - sometimes considerably larger - share of users attention. Figuring out which presence should be delivered by the website vs. a third party is an important question that each company should ask.

User Experience is King

User interface innovation is a major part of the ongoing web revolution. As we've recently written in The Rise of Contextual User Interfaces post, static user interfaces are 'dead'. The new interfaces are simpler and more contextual. Instead of revealing choices upfront, they present them based on user gestures and context. The new user experience is about fluidity.

The innovations have set the bar for UI high. Users demand simplicity and elegance and want to know how to use the product without a manual. They expect the software to work perfectly, for it to be helpful and smart. No company can afford to ignore usability, or it will lose users to someone doing the same product with a better UI.

Today the user experience is not just a set of widgets or a website design. As Leander Kahney explained in his book Inside Steve's Brain, for Steve Jobs design is the function.

This is increasingly true about any modern web application. Users perceive all elements of the service as the service itself. They don't distinguish particular widgets inside Twitter or Twitterific; the vertical conversational faceroll defines Twitter. The way the service is delivered is why users like it.

Why Controlling User Experience is Important?

Each service that we love, whether Twitter or Digg or Flickr or del.icio.us, has its particular look, feel and philosophy. Passionate users enjoy these services because of the elements, choices and collective experience that the services deliver. The clients built on top of the API would not necessarily channel the secret sauce. For example, RIA applications for Twitter are built for people who don't work for Twitter and don't regularly communicate with the Twitter team. They're not going to preserve the user experience philosophy.

Third party clients create new user experiences, which are at times confusing. As a user, on web, desktop and iPhone, ideally you'd like to experience the service the same way, but if the iPhone application is delivered by someone else the experience might not be the same.

In addition to user experience, there's the issue of branding. Larger companies are strict about their identity. When a couple of guys build an Amazon application for iPhone, they won't pay close attention to Amazon branding. Some will argue it doesn't matter as long as it drives transactions for Amazon. Yes and no. Yes because the users will buy. No because the users will accumulate imperfect user experience and associate this with Amazon, which might add up to a big negative.

Monetization Factor

A strong reason for investing in user experience is monetization. Many consumer services today are monetized via advertising. Having additional presence on different platforms increases the potential volume of advertising.

Put simply, many Twitter clients, like Twitterrific, are already monetizing the service by adding a single ad on top of each result set. If Twitter owned the RIA client, it would be able to monetize it in the same way.

Any service that is transactional or advertising driven benefits from multiple interfaces. Whatever it takes to reach the user to deliver value and drive the transactions is what services have to do. In the world of APIs, we at times forget that service should tap into all its major channels to build the business.

Which Platforms Are Critical?

Which platforms are important to tap into? There are 4 major venues for companies to consider seriously: iPhone, RIA, Facebook and Browser Extensions. All these platform plug into the same audience, but in a different context.

iPhone is great on the go. With the opening of the App Store, increasingly iPhone is going to be our personal computer. RIA clients are popular, particularly among early adopters who want richer, snappier experience compared to the web. Facebook, despite its recent scaling back on the platform, remains a major way to reach mainstream audience. Browser extensions enable the user to access the service from around the web.

Tapping into these platforms is not cheap. Building a specific and correct solution for each platform requires product management, development and testing resources.

If the company has correct API, the exercise is simpler. Instead of duplicating the application, the company builds a client for each platform and benefits from common API and common back-end architecture. It is not trivial to maintain presence in all these places, but it's likely to pay off.

Conclusion

APIs offer an amazing way for companies to scale, to create an ecosystem of innovation and tap into a wider audience. Companies should consider building and managing their presence on major platforms like iPhone, Facebook, RIA and Browsers. The way that people perceive and interact with the service is increasingly important; just rolling out an API and having a third party take care of the client could be dangerous. In addition an opportunity of being in front of the audience driving monetization could be missed.

And now tell us which of your favortire services you want to see build presence on different platforms?

In a recent interview Tim Berners-Lee pointed out that the infrastructure to power the Semantic Web is already here. ReadWriteWeb's founder, Richard MacManus, even picked it to be the number one trend in 2008. And rightly so. Not only are the bits of infrastructure now in place, but we are also seeing startups and larger corporations working hard to deliver end user value on top of this sophisticated set of technologies.

The Semantic Web means many things to different people, because there are a lot of pieces to it. To some, the Semantic Web is the web of data, where information is represented in RDF and OWL. Some people replace RDF with Microformats. Others think that the Semantic Web is about web services, while for many it is about artificial intelligence - computer programs solving complex optimization problems that are out of our reach. And business people always redefine the problem in terms of end user value, saying that whatever it is, it needs to have simple and tangible applications for consumers and enterprises.

The disagreement is not accidental, because the technology and concepts are broad. Much is possible and much is to be imagined.

1. Bottom-Up and Top-Down

We have written a lot about the different approaches to the Semantic Web - the classic bottom-up approach and the new top-down one. The bottom-up approach is focused on annotating information in pages, using RDF, so that it is machine readable. The top-down approach is focused on leveraging information in existing web pages, as is, to derive meaning automatically. Both approaches are making good progress.

A big win for the bottom-up approach was recent announcement from Yahoo! that their search engine is going to support RDF and microformats. This is a win-win-win for publishers, for Yahoo!, and for customers - publishers now have an incentive to annotate information because Yahoo! Search will be taking advantage of it, and users will then see better, more precise results.

Another recent win for the bottom-up approach was the announcement of the Semantify web service from Dapper (previous coverage). This offering will enable publishers to add semantic annotations to existing web pages. The more tools like Semantify that pop up, the easier it will be for publishers to annotate pages. Automatic annotation tools combined with the incentive to annotate the pages is going to make the bottom-up approach more compelling.

But even if the tools and incentive exist, to make the bottom-up approach widespread is difficult. Today, the magic of Google is that it can understand information as is, without asking people to fully comply with W3C standards of SEO optimization techniques. Similarly, top-down semantic tools are focused on dealing with imperfections in existing information. Among them are the natural language processing tools that do entity extraction - such as the Calais and TextWise APIs that recognize people, companies, places, etc. in documents; vertical search engines, like ZoomInfo and Spock, which mine the web for people; technologies like Dapper and BlueOrganizer, which recognize objects in web pages; and Yahoo! Shortcuts, Snap and SmartLinks, which recognize objects in text and links.

[Disclosure: Alex Iskold is founder and CEO of AdaptiveBlue, which makes BlueOrganizer and SmartLinks.]

Top-down technologies are racing forward despite imperfect information. And, of course, they benefit from the bottom-up annotations as well. The more annotations there are, the more precise top-down technologies will get - because they will be able to take advantage of structured information as well.

2. Annotation Technologies: RDF, Microformats, and Meta Headers

Within the bottom-up approach to annotation of data, there are several choices for annotation. They are not equally powerful, and in fact each approach is a trade off between simplicity and completeness. The most comprehensive approach is RDF - a powerful, graph-based language for declaring things, and attributes and relationships between things. In a simplistic way, one can think of RDF as the language that allows expressing truths like: Alex IS human (type expression), Alex HAS a brain (attribute expression), and Alex IS the father of Alice, Lilly, and Sofia (relationship expression). RDF is powerful, but because it is highly recursive, precise, and mathematically sound, it is also complex.

At present, most use of RDF is for interoperability. For example, the medical community uses RDF to describe genomic databases. Because the information is normalized, the databases that were previously silos can now be queried together and correlated. In general, in addition to semantic soundness, the major benefit of RDF is interoperability and standardization, particularly for enterprises, as we will discuss below.

Microformats offer a simpler approach by adding semantics to existing HTML documents using specific CSS styles. The metadata is compact and is embedded inside the actual HTML. Popular microformats are hCard, which describes personal and company contact information, hReview, which adds meta information to review pages, and hCalendar, which is used to describe events.

Microformats are gaining popularity because of their simplicity, but they are still quite limiting. There is no way to describe type hierarchies, which the classic semantic community would say is critical. The other issue is that microformats are somewhat cryptic, because the focus is to keep the annotations to a minimum. This, in turn, brings up another question of whether embedding metadata into the view (HTML) is a good idea. The question is: what happens if the underlying data changes when someone makes a copy of the HTML document? Nevertheless, despite these issues, microformats are gaining popularity because they are simple. Microformats are currently used by Flickr, Eventful, and LinkedIn; and many other companies are looking to adopt microformats, particularly because of the recent Yahoo! announcement.

An even simpler approach is to put meta data into the meta headers. This approach has been around for a while and it is a shame that it has not been widely adopted. As an example, the New York Times recently launched extended annotations for its news pages. The benefit of this approach is that it works great for pages that are focused on a topic or a thing. For example, a news page can be described with a set of keywords, geo location, date, time, people, and categories. Another example would be for book pages. O'Reilly.com has been putting book information into the meta headers, describing the author, ISBN, and category of the book.

Despite the fact that all these approaches are different, they are also somewhat complementary; and each of them is helpful. The more annotations there are in web pages, the more standards are implemented, and the more discoverable and powerful the information becomes.

3. Consumer and Enterprise

Yet another dimension of the conversation about the Semantic Web is the focus on consumer and enterprise applications. In the consumer arena we have been looking for a Killer App - something that delivers tangible and simple consumer value. People simply do not care that a product is built on the Semantic Web; all they are looking for is utility and usefulness.

Up until recently, the challenge has been that the Semantic Web focused on rather academic issues - like annotating information to make it machine-readable. The promise was that once the information is annotated and the web becomes one big giant RDF database, then exciting consumer applications would come. The skeptics, however, have been pointing out that first there needs to be a compelling use case.

Some consumer applications based on the Semantic Web: generic and vertical search, contextual shortcuts and previews, personal information management systems, semantic browsing tools. All of these applications are in their early days and have a long way to go before being truly compelling for the average web user. Still, even if these applications succeed, consumers will not be interested in knowing about the underlying technology - so there is really no marketing play for the Semantic Web in the consumer space.

Enterprises are a different story for a couple of reasons. First, enterprises are much more used to techno speak. To them utilizing semantic technologies translates into being intelligent and that, in turn, is good marketing. 'Our products are better and smarter because we use the Semantic Web' sounds like a good value proposition for the enterprise.

But even above the marketing speak, RDF solves a problem of data interoperability and standards. This "Tower of Babel" situation has been in existence since the early days of software. Forget semantics; just a standard protocol, a standard way to pass around information between two programs, is hugely valuable in the enterprise.

RDF offers a way to communicate using XML-based language, which on top of it has sound mathematical elements to enable semantics. This sounds great, and even the complexity of RDF is not going to stop enterprises from using it. However, there is another problem that might stop it - scalability. Unlike relational databases, which have been around for ages and have been optimized and tuned, XML-based databases are still not widespread. In general, the problem is in the scale and querying capabilities. Like object-oriented database technologies of the late '90s, XML-based databases hold a lot of promise, but we have yet to see them in action in a big way.

4. Semantic APIs

With the rise of Semantic Web applications, we are also seeing the rise of Semantic APIs. In general, these web services take as an input unstructured information and find entities and relationships. One way to think of these services is mini natural language processing tools, which are only concerned with a subset of the language.

The first example is the Open Calais API from Reuters that we have covered in two articles here and here. This service accepts raw text and returns information about people, places, and companies found in the document. The output not only returns the list of found matches, but also specifies places in the document where the information is found. Behind Calais is a powerful natural language processing technology developed by Clear Forest (now owned by Reuters), which relies on algorithms and databases to extract entities out of text. According to Reuters, Calais is extensible, and it is just a matter of time before new entities will be added.

Another example is the SemanticHacker API from TextWise, which is offering a one million dollar prize for the best commercial semantic web application developed on top of it. This API classifies information in documents into categories called semantic signatures. Given a document, it outputs entities or topics that the document is about. It is kind of like Calais, but also delivers a topical hierarchy, where the actual objects are leafs.

Another semantic API is offered by Dapper - a web service which facilitates the extraction of structure from unstructured HTML pages. Dapper works by enabling users to define attributes of an object based on the bits of the page. For example, a book publisher might define where the information about author, ISBN and number of pages is on a typical book page and the Dapper application would then create a recognizer for any page on the publisher site and enable access to it via REST API.

While this seems backwards from an engineering point of view, Dapper's technology is remarkably useful in the real world. In a typical scenario, for websites that do not have clean APIs to access their information, even non-technical people can build an API in minutes with Dapper. This is a powerful way of quickly turning websites into web services.

5. Search Technologies

Perhaps the first significant blow to the Semantic Web has been the inability thus far to improve search. The premise that a semantic understanding of pages leads to vastly better search has yet to be validated. The two main contenders, Hakia and PowerSet, have made some progress, but not enough. The problem is that Google's algorithm, which is based on statistical analysis, deals just fine with semantic entities like people, cities, and companies. When asked What is the capital of France? Google returns a good enough answer.

There is a growing realization that marginal improvement in search might not be enough to beat Google or to declare search the killer app for the Semantic Web. Likely, understanding semantics is helpful but not sufficient to build a better search engine. A combination of semantics, innovative presentation, and memory of who the user is, will be necessary to power the next generation search experience.

Alternative approaches also attempt to overlay semantics on top of the search results. Even Google ventures into verticals by partitioning the results into different categories. The consumer can then decide which type of answer they are interested in.

Yet search is a game that is far from won and a lot of semantic companies are really trying to raise the bar. There may be another twist to the whole search play - contextual technologies, as well as semantic databases, could lead to qualitatively better results. And so we turn to these next.

6. Contextual Technologies

We are seeing an increasing number of contextual tools entering the consumer market. Contextual navigation does not just improve search, but rather shortcuts it. Applications like Snap or Yahoo! Shortcuts, and SmartLinks "understand" the objects inside text and links and bring relevant information right into the user's context. The result is that the user does not need to search at all.

Thinking about this more deeply, one realizes that contextual tools leverage semantics in a much more interesting way. Instead of trying to parse what a user types into the search box, contextual technologies rely on analyzing the content. So the meaning is derived in a much more precise way - or rather, there is less guessing. The contextual tools then offer the users relevant choices, each of which leads to a correct result. This is fundamentally different from trying to pull the right results from a myriad of possible choices resulting from a web search.

We are also seeing an increasing number of contextual technologies make their way into the browser. Top-down semantic technologies need to work without publishers doing anything; and so to infer context, contextual technologies integrate into the browser. Firefox's recommended extensions page features a number of contextual browsing solutions - Interclue, ThumbStrips, Cooliris, and BlueOrganizer (from my own company).

The common theme among these tools is the recognition of information and the creation of specific micro contexts for the users to interact with that information.

7. Semantic Databases

Semantic databases are another breed of semantic applications focused on annotating web information to be more structured. Twine, a product of Radar Networks and currently in private beta, focuses on building a personal knowledge base. Twine works by absorbing unstructured content in various forms and building a personal database of people, companies, things, locations, etc. The content is sent to Twine via a bookmarklet, via email, or manually. The technology needs to evolve more, but one can see how such databases can be useful once the kinks are worked out. One of the very powerful applications that could be built on top of Twine, for example, is personalized search - a way to filter the results of any search engine based on a particular individual.

It is worth noting that Radar Networks has spent a lot of time getting the infrastructure right. The underlying representation is RDF and is ready to be consumed by other semantic web services. But a big chunk of the core algorithms, the ones that are dealing with entity extraction, are being commoditized by Semantic Web APIs. Reuters offers this as an API call, for example, and so moving forward, Twine won't need to be concerned with how to do that.

Another big player in the semantic databases space is a company called Metaweb, which created Freebase. In its present form, Freebase is just a fancier and more structured version of Wikipedia - with RDF inside and less information in total. The overall goal of Freebase, however, is to build a Wikipedia equivalent of the world's information. Such a database would be enormously powerful because it could be queried exactly - much like relational databases. So once again the promise is to build much better search.

But the problem is, how can Freebase keep up with the world? Google indexes the Internet daily and grows together with the web. Freebase currently allows editing of information by individuals and has bootstrapped by taking in parts of Wikipedia and other databases, but in order to scale this approach, it needs to perfect the art of continuously taking in unstructured information from the world, parsing it, and updating its database.

The problem of keeping up with the world is common to all database approaches, which are effectively silos. In the case of Twine, there needs to be continuous influx of user data, and in the case of Freebase there needs to be influx of data from the web. These problems are far from trivial and need to be solved successfully in order for the databases to be useful.

Conclusion

With any new technology it is important to define and classify things. The Semantic Web is offering an exciting promise: improved information discoverability, automation of complex searches, and innovative web browsing. Yet the Semantic Web means different things to different people. Indeed, its definitions in the enterprise and consumer spaces are different, and there are different means to a common end - top-down vs. bottom-up and microformats vs. RDF. In addition to these patterns, we are observing the rise of semantic APIs and contextual browsing tools. All of these are in their early days but hold a big promise to fundamentally change the way we interact with information on the web.

What do you think about Semantic Web Patterns? What trends are you seeing and which applications are you waiting for? And if you work with semantic technologies in the enterprise, please share your experiences with us in the comments below.

Companies are being forced to cut costs and let people go. Some smart people aren't sitting around waiting to be downsized - instead they're jumping ship and hopping aboard another.

Whether you are let go or you leave on your own, there is an impact. Of course conventional wisdom says that everyone is replaceable. That may still be true, but the really important question is: at what cost? Is the cost of replacing someone today the same as it was 10 years ago? Just because technology is cheaper and more abundant it does not mean that it is cheap to replace people.

The fact is that we are increasingly becoming a real-time information processing society. Because of that, each one of us processes an increasingly large amount of unique information on a daily basis. Knowledge-based workers are very different from workers on a factory line, and the cost of replacing them is also very different. While companies save money in the short term, the longer term impact of losing a person is not so clear. In this post we look at the impact cutting jobs has on modern companies and try to figure out: are YOU replaceable?

Somewhere In Corporate America 20 Years Ago...

Story 1: Sorry Bob, you've been wonderful, but we will not be needing your services any longer. Here is Jack, fresh out of college, his salary is 25% of yours. Please train him in the next two weeks to do what you do.

Story 2: Hey boss, you've been wonderful, but I found a job that pays twice as much. Here is Jack, fresh out of college, his salary is 25% of mine. I will train him in the next two weeks to do what I do.

The aftermath of both stories is the same: Bob trains Jack in two weeks and then leaves. And even though Jack is very bright and had a 4.0 GPA in college, it is impossible to learn Bob's job in two weeks. It takes about two months for him to become productive. And this is quite a find. The long term cost is substantially lower and company benefits from the employee churn.

Why Today is Different

The problem is that today, two months ramp up time is not acceptable. It is just way too long. For a startup, two months is an eternity, but even for large companies two months is a long time. Today, people need to be replaced real-time - one is out and the next one is in full-speed, day one. This is difficult, particularly because of the incredible amount of information that we end up processing daily.

Increasingly, modern business is becoming a complex, distributed information processing system. The nodes of this system are employees, tirelessly passing bits around to each other, crunching and filtering with the goal to compute, to gain competitive advantage, and to help the business survive.

The problem is that unlike factories or boxes in the computing cloud, employees in the modern company are not identical. Each one knows a unique piece of the information puzzle that makes a company tick. Two weeks is not enough to do the transition and two months is way too long to waste training the new guy. This is why the old adage that everyone is replaceable may need some re-thinking.

The Days of Slackers are Over

Certainly, there are still plenty of examples where slackers are growing old getting paid to work their 9-5 jobs while getting little done. But safe havens for slackers are rapidly diminishing, because they are losing out to smarter, more agile, and faster competition.

A few years ago my wife, who is a clinical pharmacist, worked with a character that ultimately managed to get a pink slip in a huge company that had probably only ever fired one person - him. He started his day by searching the Internet for a rich relative. Yes, you read that correctly, the guy was searching for wealthy family members. At 11am, he walked around the office and asked what people thought would be offered in the cafeteria for lunch. Between 1pm and 2pm, he spent time reading the news, and then typically called his wife to discuss the dinner menu. This is funny, absurd, and sad at the same time. But think about it, can someone like that work in your company? No way! Companies can not afford to have people like this anymore.

The Emergence of the Digital Elite

In a way, the pressure of real-time information is polarizing - the hard-working people are becoming harder to replace, while slackers and perhaps less knowledgeable people are just not needed. We have seen this trend in software engineering for a while - a handful of smart people can accomplish much more than an army of mediocre workers. A skilled, quick professional stands out these days. The people who shine are the people who get the new world - a no-nonsense approach, courtesy, and most importantly, speed.

Recently, my insurance broker switched companies. He quickly contacted me, offered an attractive new package, and then drove 1.5 hours from his office to my home to sign the papers. His commission would not want warrant the trip, but he was smart to make the investment of his time because he won me as a client. On the other hand, the cost of losing a talented employee for his old company just increased - they also lost a client, and I am sure I was not the only one.

Although my insurance agent lives in the technical world, he is part of new breed of folks that I call the digital elite. He uses Facebook to keep in touch with his friends, he was savvy enough to look up my company on the web, and he knows all the cool financial websites. In other words, he is on top of what's going on. He knows all about the speed of information in our world. And this makes him a serious and important player, of the type that is really hard to replace.

Are Leaders and Visionaries Replaceable?

Clearly, after Bradley Horowitz moves to Google, Yahoo! survives. He will be replaced by someone else just as talented and as passionate and the ship will sail forward. But saying that he will be replaced is very different from computing the cost of his departure. Losing leaders and visionaries is very, very costly. The knowledge, the vision, and the game plan that was in his head is unique and can not be replicated.

Great companies are defined by the great people behind them. There are no great companies without visionary leaders. And if you agree that all knowledge workers are becoming increasingly more valuable, the leaders are then 10 times more valuable. Retention of key leaders and managers is paramount to the success of modern large companies. So I am sure that Bradley, who grew through the ranks at Yahoo! and was one of the faces of the company, will be greatly missed.

Conclusion

And yet, churn is such a huge part of nature! Our world is based on transition and change. Changing jobs is an integral part of your career path. When people move around, society benefits from knowledge sharing and new alliances that lead to great new ideas. Remixing is good for both individuals and companies, so there is no way that churn will ever stop.

But still, it is now becoming more costly for the companies. Because of the increasing amount of information processing done by individuals and the uniqueness of each, getting replacements up to speed is more costly. Retaining and motivating the digital elite should be recognized as a high priority for any company.

So, tell us how you feel about all of this? Do you feel insecure in your current position? Are you looking for a new job? What is your company doing to motivate and retain key people?

Top image credit: New York Times

One of these evolutions started quietly in 2008. We are witnessing the rise of a new kind of web: contextual. You might not have heard or thought about it much yet, but you are already using it today. Search remains the killer app on the web, but context is quickly become a viable contender. Why? Because context is what happens instead of search.

Until recently on the web, most sites have not been software - only data, a bunch of flat HTML pages. The software that you used to look at them was, of course, the web browser. The problem was that the browser had no idea what the pages contained, and it did not know what you were doing. Because the browser could not infer your context, it could not help you explore related and relevant information.

The contextual web experience is fundamentally different because there is an understanding of what the user is doing. The combination of the information on the page and the user's behavior creates the context. Once you understand the user's context, you can be more helpful. So, contextual technologies have the potential to bite into the pie that today belongs to search, because it is able to bypass search.

Consider the difference between looking at the IBM home page and a movie page on Netflix. The context is very different, and so likely is the user's intention. The user may be looking at IBM because she is looking for a job or researching IBM products. When the user is looking up a movie, it is because she is thinking about renting it.

The Key Properties of The Contextual Web

The Contextual web will happen when browsers and websites evolve to recognize what users are trying to do. It is the web with less choice and more meaning, where instead of Googling all the time, we Google once and then the rest of the information is available to us automatically, based on our current context.

Here are the key properties of the contextual web experience:

• Relevancy: understanding the user's context better drives content relevancy.
• Shortcuts: contextual shortcuts reduce the need for raw search.
• Personalization: context is based on user intentions and history.
• Remixing: relevant information from around the web is instantly available.

What are some specific examples of contextual technologies that are improving our web experience right now?

Markup Technologies

One of the keys to inferring user context is understanding the underlying information that the user is looking at. This is why the contextual web is related to -- and, to be more precise, is powered by -- semantic web. We have written a lot here on ReadWriteWeb about semantic technologies. Notably, we discussed the difference between the top-down and bottom-up approaches to semantic web, both of which are important for understanding the contextual web.

The bottom-up approach to context is about annotating pages. For example, all modern browsers can detect if the page you are looking at contains an RSS feed. This happens because the browser looks for a tag in the head of the page, which declares type . The fact that the page offers an RSS feed creates an obvious context: subscription. So, the browser then invites you to subscribe using your favorite RSS reader.

Another form of markup that has been discussed recently is microformats, which offer an XHTML-compliant way of embedding metadata about people, places, events, and reviews in existing web pages. Even though microformats are not ubiquitous today, there are clear benefits to using them. The image below is from an excellent post by Mozilla UI lead Alex Faaborg about leveraging microformats in the browser.

Building on the hAtom microformat are Web Slices, introduced by Microsoft in Internet Explorer 8. Web Slices enable publishers to notify users when the information in their web pages changes. For example, Weather.com can create a Web Slice that tells the user when a local weather update is available. eBay can deliver a Web Slice that notifies the user when the price of an auction changes. Similar in concept to RSS, Web Slices focus on updates in part of the web page, enabling publishers and users to communicate directly via the browser.

There are other markup formats that help provide context. For example, popular add-on Cooliris offers a markup format for signaling that a site contains images. By placing a bit of XML code in their home directory, site owners enable users to experience their images using the stunning 3D visualization developed by Cooliris. Another markup format, developed by AdaptiveBlue [disclosure: this is the company I founded], is called ABMeta. This format allows publishers to annotate pages that contain information about books, music, movies, wine, restaurants, stocks, and other everyday things.

All of these markup-based approaches face the same issue: publishers have to do the work of actually annotating the pages. And while the semantic web community has been very vocal about the benefits of annotation, the majority of the web is still flat HTML.

Page 2: Widgets

Widgets

In the meantime, we are seeing a surge in applications that deploy a top-down approach to inferring user context and being helpful. Instead of relying on markup in pages, these applications use heuristics and APIs to recognize the information that the user is interacting with. Based on their recognition, these smart tools then offer contextual shortcuts to the relevant bits of information. Speaking broadly, today there are two primary categories of top-down contextual applications: blog plugins and browser add-ons.

Blog plugins offering a contextual experience have been around for some time, starting with preview technologies. One of the first recent unsuccessful attempts at a preview technology was delivered by Browster. Despite its failure, its successors, CoolPreviews, SnapShots, and Apture, have done much better. While CoolPreviews literally focuses on a preview of the page, SnapShots and Apture deliver a compact summary of the information behind the link. The basic premise behind the previews is sound: if the user is not interested in what is behind the link, the preview can save the user an unnecessary click. Because the user context (in this case, the link) is known, relevant information can be brought to the surface and delivered to the user instantly.

Closely related to previews are widgets that offer contextual shortcuts. Like previews, these widgets work by being anchored to links. Examples of this technology include Yahoo! Shortcuts and SmartLinks from my company AdaptiveBlue. Instead of providing a preview of the underlying content, these technologies offer links to related content around the web. The example below shows a SmartLink on a New York Times movie page, which invites the user, for example, to buy the movie on Amazon or rent it on Netflix.

Our last example of contextual widget technology comes from Colorado-based Lijit. Lijit has developed a search technology that focuses on the individual. You claim your presence around the web, on blogs, Twitter, Flickr, etc., and then others can search your pages on specific topics. Say you are on someone's blog and want to do a search on that person; it makes the most sense to search through this person's pages first; after all, you are in this person's context.

Remarkably, Lijit offers another contextual nugget, called Re-search. It is simple, but a great example of the power of context. If you search for a term on Google and end up on a blog that has a Lijit widget, a header appears with additional results and a prompt to search for more. Lijit automatically infers your context, does a search, and offers additional helpful shortcuts.

Page 3: Browser Add-Ons and Conclusion

Browser Add-Ons

Widgets and markup technologies are making inroads into the contextual web, but an even bigger inroad is being made by browser add-ons. Since Firefox introduced its platform for delivering additional features to the browser, many thousands of browser extensions have been developed. A lot of these extension focus on enhancing the browsing experience by leveraging context. Probably the granddaddy of them all is Greasemonkey, an extension that allows users to install bits of JavaScript that alter the look and content of web pages they visit. We wrote an introduction to this popular add-on and recently followed up with a post on 7 New Greasemonkey Tweaks.

While Greasemonkey scripts are mainly regarded as experimental tools for power users, quite a few other extensions are aimed at building a business around contextual enhancements to the browser. One example is a company called WebMynd, which focuses on enhancing Google search results. WebMynd's extension automatically pushes down Google ads (clever!) to insert its contexual gadget. It allows users to simultaneously search posts on Twitter, book matches on Amazon, video matches on YouTube, and many other sources. An example of what shows up when searching for "semantic web" using the gadget is shown below.

We have mentioned Cooliris several times already. The most popular Cooliris product is the browser add-on that shows stunning 3D views of photos and videos around the web. Cooliris works contextually by automatically recognizing popular photo sites like Flickr, Google Images and Photobucket. Another contextual add-on that works by recognizing content on pages is Glue. This add-on shows you friends and other users who have visited the same book, music, movie, restaurant, wine, and other pages around the web. The context here crosses the content you are viewing with your social graph. Unlike lifestreaming applications that deliver you information about your friends out of context, Glue only brings to the surface information that is relevant to your current context.

A very different example of a contextual web add-on comes from Slovenia-based Zemanta. Its add-on makes it easy for publishers to add contextually relevant links, photos, and video to their posts. Zemanta works by applying its semantic engine to the body of the post and automatically recommending related content. Using Zemanta, bloggers are able to instantly add relevant content to their blog posts, in turn creating a relevant web experience for their users.

Our last two examples of contextual add-ons are Zentact and App Discover, both recently launched. Zentact -- developed by the folks who brought us another contextual phenomenon, MyBlogLog -- aims to solve a problem of staying in touch with people who matter to you. First, you import and tag your email contacts with different tags that reflect their interests. Then, as you browse, Zentact pops up a reminder to contact a person if it deems the page relevant based on the tags. It is a simple yet powerful example of the contextual web.

App Discover works by recommending related applications to the sites that you are visiting. Today, it requires publishers to add markup to their sites, but one can easily imagine how this technology could work top-down. The example below shows App Discover recommending TweetDeck when the user navigates to Twitter.

Browsers

Without a doubt, web browsers are in the best position to deliver the user's contextual web experience. While add-ons are taken in a variety of directions, the two dominant browsers, Internet Explorer and Firefox, have already incorporated the basic contextual experience: shortcuts. Internet Explorer 8 features technology called Accelerators.

According to Microsoft, Accelerators give you ready access to the online services you use everyday, from any page you visit. Accelerators are defined as little chunks of XML based on pre-defined variables and defined by the browser. Some examples of variables available to Accelerator are the active URL, the active domain, and selected text. The most common action that Accelerators do is perform contextual search based on the user's selection. Another common Accelerator function is the in-place lookup, such as looking up a map based on a given address.

The problem with Accelerators is that they are not really based on a selection; that is, they lack semantics. When you highlight an address, you still have to decide which Accelerator to use. If you have dozens of Accelerators installed, this quickly becomes hard to manage. Firefox has recognized the issue with the menu-driven approach and instead offers its contextual technology via text. Called Ubiquity, this contextual technology is still only available as an add-on today but is likely to be a part of Firefox core soon.

Marketed as user-generated mashups, Ubiquity is actually a contextual technology based on language. As with Accelerators, the user is able to select a piece of text and then invoke Ubiquity and type a command. In the screenshot above, Ubiquity is used to insert a map into an email. Hundreds of Ubiquity commands have been implemented to date. ReadWriteWeb has already written about Ubiquity commands.

Will Context be the Future of the Web?

So where is all of this heading? Are these technologies signal or noise? We believe we are witnessing the birth of a fundamentally new kind of web, a smarter one, a contextual one. Unlike the old web we are used to, this one understands what we are doing and helps us. It is a web in which we search less and find relevant content faster. This new contextual web is still very young and unevenly distributed, but it is definitely here.

The fact that these contextual technologies are springing up is not accidental. The Contextual web is made possible by our push into semantic web and the rise of web services/API culture. The combination of basic semantics and API is fueling all of these contextual applications. Bit by bit, the web is getting smarter, friendlier, and more enjoyable.

Web browsers are in the best position to deliver these new contextual experiences to users because of their wide reach. The fact that Microsoft made Accelerators its flagship feature for Internet Explorer 8 and that Mozilla is putting much effort into Ubiquity tells us that contexual browsing is a priority. This is really good news, because these technologies bring great benefits to the user.

It is great to see that in these tough economic times, evolution is brewing. The tremendous effort that all of these companies are making with contextual technologies is beginning to come to fruition. It may be that we are seeing glimpses of what the next generation of the web will be like.

And now, as usual, let's turn the tables. Tell us examples of your favorite contextual browsing technologies. Do you think context will be play significant part in the next evolution of the web?

For big Internet companies, 2009 is going to be a very bad year for sure. Advertising profits are going to plunge, and consumers will spend less money overall, particularly on the web. There is little that can be done to change that. But what big companies can do is invest in innovation and killer moves that will bear fruit in the years to follow. Here is what we think would be cool for various big web companies to do in 2009.

Despite the downturn, Google remains a lean, mean revenue-generating machine. The search king has a strong hold on this area of the web, and as soon as things get better, its revenues will go up as well. So it makes sense to invest in its next strongest assets: video and its new adventure, the web browser.

1. Integrate Web-Wide Video Search in YouTube

We recently wrote here about YouTube and its rising popularity among kids. In a couple of years, a lot of web content is going to evolve from text to video, making YouTube a much bigger asset for Google. Even today, YouTube is already the second-largest search engine on the web.

However, it is a portal featuring only user-generated content. Google should seamlessly plug its video search into YouTube and bring videos from around the web to the surface using this same user-friendly interface. This would make YouTube the destination for people searching for any video content on the web.

2. Make Firefox Extensions Compatible with Chrome

Google Chrome made a big spash when it was launched earlier this year. Surely Google's foray into the browser war is very deliberate. Chrome looks like an excellent modern browser and already has a solid base of followers. But the road to mass adoption is quite steep, even if Google promotes Chrome via its home page and makes bundling deals with computer manufacturers.

One of the things that can definitely help Chrome gain adoption is extension support. Most early adopters use Firefox these days and love the Mozilla extensions that allow them to make the browser their own. A killer move would be for Google to support Firefox extensions. This would save a whole lot of time for all developers and users and would show that Google respects the existing web eco-system.

Tips for Microsoft

Everyone knows that Google is the new Microsoft and that Microsoft is the new IBM. Ever since it lost the search war to Google, the Redmond giant has been playing an agonizing catch-up game. And it has yet to make a dent in Google's market. Perhaps its most successful recent move has been investing $250 million in Facebook, which was valued at $15 billion. But while it was a clever strategic play, there is no impressive technology here. And this is really what Microsoft needs to invest in: execution and innovation.

1. Execute Faster and Better

This problem nags large companies. Bureaucracy takes over for common sense and introduces rot. Endless meetings, hierarchies of approval, and "what if" scenarios turn any large company into a beast. Microsoft needs to fundamentally restructure its approach to delivering products: reduce the coupling between products, shrink timelines, throw away old rotten code. In short, Microsoft needs to be more like Google in order to compete with Google.

2. Innovate

Sure, Microsoft has a lot of things going on in research. But every time we hear of a new product announcement, it sounds like "Me too" envy. First, there is the rivalry with Google in search. Many people think that advertising is the killer app for the web. Sure, it is, at least today. But it does not mean that search is the only way to deliver it. Microsoft is stuck in a battle to build a better search engine, while the answer may be to deliver advertsing in a fundamentally different way.

Similarly, Amazon has made a big push into cloud computing. Microsoft followed a year later with yet another "Me too." Zune was another fiasco, an attempt to chase the beloved Apple product. The problem is that the age of "Me too" is over. The only way to own the future is to define it. Microsoft needs to truly embrace innovation in order to have a chance of coming back.

Tips for Amazon

Amazon has executed its web services play remarkably well. The company took its core infrastructure, which runs the biggest shopping store online, and turned it into a product. More importantly, Amazon's foray into web services marks the true beginning of the cloud computing era. But at the same time, strange things are happening with the store, such as advertising appearing on product pages. Amazon should continue to accelerate its web services strategy, but it also needs to go back to the store and clean up the way it looks.

1. Continue the Rapid Push into Web Services

The first-mover advantage sometimes works, and sometimes doesn't. Many first movers in the market are overtaken and ultimately defeated by the second comers who know how to improve on what has been done. But if the first move is brilliant (think Apple kind of brilliant), then it is becomes hard for others to keep up. Amazon has rapidly innovated in the web services market and has a very substantial lead.

Its delivery to date has been nearly perfect, and the company needs to continue to invest in its core stack. The more useful the blocks that are there, the less reason users will have to switch to something else. Cost and reliability are already there, so it boils down to adding a few more blocks and then re-focusing on customer support and enablement.

2. Simplify and Clean Up the Product Pages

Amazon pages, despite the recent cleanup, still feel so 1990s. There is just too much going on. First of all, the banner ads need to go. Sure, they make money, but it is insulting to see completely irrelevant truck ads when a user is looking at Freedman's latest book.

Secondly, the pages are difficult to read because of both the design and the amount of information on the pages. Most of the information is not needed and will not enhance transactions. Instead of geeky bar charts, give users a simple popularity indicator. People do not need to see 400 reviews; five should do. The lists promoted in the sidebar are distracting. And the list goes on; there is a lot of room for clean-up.

Tips for eBay

eBay has just not been an exciting company since the 1990s. It has not executed well, its site has not evolved, and it failed to make much of its acquisitions, such as Skype and StumbleUpon. What eBay needs to do is reinvent itself through relevant acquistions.

1. Buy Etsy

Etsy is a rapidly growing online marketplace for hand-made goods. With sustainability and green poised to be on people's minds for the next decade, hand-made is the new black. Etsy has got a stellar team behind it, a user-centric culture, and a knack for innovation. If Esty were to be unleashed on eBay's site, the user experience would likely be much more improved and fun.

2. Buy Craigslist

If one company other than Google has cracked the secret of simplicity on the web, it's Craigslist. It has discovered the most effective and simple way to do online classifieds, buy and sell things, find a job, and make a hire. Cloning the same simple service for many markets was genius, and Craigslist just nailed it. It is unclear whether they would be willing to sell, but if there is a transaction there, eBay's coolness level would shoot up high. And the revenue would likely follow, because Craigslist has been rather conservative when it comes to charging users.

Tips for Yahoo!

Yahoo! is in a really tough spot. It seems the only way out is through a focus on innovation (that is, if it really doesn't want to sell to Microsoft). First, Yahoo! needs to figure out what business it wants to be in, and then it needs to execute flawlessly in the direction it chooses.

1. Invest in Best Directions, Cut Out the Rest

Seems like a lot of internal projects were coming out of the brick house and then didn't go anywhere. The problem with incubating products, launching them, and then dropping the ball with user adoption and marketing is that it is not good for business. Take Yahoo! Shortcuts as an example: great idea, competitive space, well executed. Where is it now? Lanched in 2007, it works only on WordPress, hasn't really been promoted much, and is unlikely to go anywhere.

We see a similar pattern with acquistions. Too much time was lost getting the ball rolling with MyBlogLog and del.icio.us. The result? Momentum was lost, and so were users and opportunity. Yahoo! can't afford misses like these. Execution needs to be quick and flawless -- and relevant to the business. Yahoo! needs to assess its entire product line, pick the things to invest in, and then put both engineering and marketing dollars behind them.

2. Compete in Search

Not all is lost, especially when you are number two in the space. Sure the gap to get to number one is big, but you are only one spot away. Google attracts users with its simple user interface, speed, and relevancy. Yahoo! needs to fix the speed and relevancy of its search engine and then invest in a different user interface. I really think that the Search Monkey project is on to something. Out of all the recent innovations in the search space, this one stands out with its simple yet powerful approach.

And the recent idea to open up its search engine as a web service is great too. It just needs to be done better and faster in order to really undermine Google. Yahoo! should look to Seattle for the model of how to do this sort of thing quickly and elegantly. If Yahoo! can create a simple way for everyone to build vertical search applications, Google will surely take a hit and have to start playing catch-up.

Conclusion

2009 will surely be a tough year for everyone. Yet it is the year to revamp and invest in the future. How exactly the big web companies will execute really matters. What do you think they should be doing?

During the Web 2.0 Summit recently, the Mayor of San Francisco, Gavin Newsom, pointed out that one of the reasons the world is no longer the same is YouTube. For better or worse, said Mr. Newsom, we are now always on the record. Every significant and insignificant conversation is being recorded, and the videos are available on YouTube.

The panel was indeed thought provoking, but there was another conversation at the Summit that got me thinking that the second largest search engine may be on its way to becoming the largest.

The story of one 9-year-old boy

During one of the conference breaks, I met Ian Kennedy, one of the heads of Service Innovation at Nokia. (You've seen a lot of Ian around the web because he used to be the product manager for MyBlogLog at Yahoo!.) Ian and I started talking, and he mentioned that his son accesses the web through YouTube. At first, I didn't get it and thought Ian was making a joke.

But then I realized he was not. Whenever his son needed any information, he would open up YouTube, type in the search term and then just watch the videos that showed up as matches. He never Googled anything; he never went to any other site; his entire web experience was confined to YouTube videos. It was rather puzzling, I thought. Could it be that there are YouTube videos on any topic? My curiosity was piqued, and I decided to run a little experiment.

The YouTube experiment

To get an idea, I ran several different searches:

• Chicken Noodle Soup (bad results)
• George Washington (bad results)
• Astrophysics (bad results)
• Donkey from Shrek (good results)
• Lance Wataru (good results)
• vacuuming carpet (good results)
• Gavin Newsom (good results)
• Bakugan (good results)

Like any search engine, YouTube is not perfect. In general, short queries on generic topics, like Astrophysics, do not work very well. Somewhat surprisingly, George Washington and Chicken Noodle Soup do not yield good results either. But some searches do work very well. Ian said that his son frequently searches for episodes of Bakugan, which come up perfectly. Another likely search for a 9-year-old, Lance Wataru (a Pokemon character), works well, too. More specific searches, such as Donkey from Shrek, work even better.

Anything that can be a video will be a video

I walked away with the impression that we are not quite there yet, but was intrigued. Clearly a lot of things lend themselves to video, not just movies and music clips, but educational videos, tourism and a lot of other things. If video content continues to grow, could video eventually replace text?

Most likely not. The main reason that text rules the web today is because of hyperlinks. Linking pages via text links is what makes the web possible. Hyperlinking videos would be a harder thing to do. Not impossible, of course, because you can link objects and insert text in videos, but it's just not as elegant as text. Besides the linking issue, not everything would be an effective video. For example, a research paper could be made into a video but would not be as easy to follow as the text.

But there is definitely a shift. Because video was not possible before, the web was dominated by text. Now that video cameras and broadband are cheap, information that is better served by video is getting converted. As a result, YouTube is now the second largest search engine, and traffic is through the roof. And because kids like Ian's son are video natives, this is just the beginning.

Generation YouTube

Imagine a whole generation of kids growing up and learning about the world through YouTube. In the first half of the 20th century, people grew up reading books and newspapers. Then there was a generation that grew up on movies and television. The last shift was to the Internet. And now web video is creating yet another generation.

Kids no longer learn about the world by reading text. Like the television generation, they are absorbing the world through their visual sense. But there is a big difference. Television was programmed and inflexible. YouTube is completely micro-chunked and on demand. Kids can search for what they need anytime. This is different, and powerful.

True, the current model of YouTube is still raw and still skewed to entertainment. But imagine online video 5 years from now, geared to kids, where entertainment, games, education, travel -- everything for kids -- is mixed and delivered via searchable channels. This would be a big change on the Internet and in the world. Just as we no longer think twice about Googling, kids of the future will be consuming huge volumes of information via video.

And now tell us your stories. Are you seeing your kids use video more than text? Do you yourself use YouTube to find information? Let us know. We want to hear from you!

Until recently LinkedIn's website resembled something circa 1994. The latest overhaul of the UI makes it more accessible and useful, but still not fun - it's dominated by links, not people. What if LinkedIn refocused? What if there were elements of entertainment, story telling and human feel to the whole experience? In this post we ponder how the brand could become more fun.

Hint at Humanity - the iPhone App

When the LinkedIn App for iPhone came out I was struck by how a minor difference in user interface represents a big difference in perception. On the website, the list of contacts is dull and hard to sift through. iPhone implementation leveraged the standard widget for scrolling through lists and was spectacular.

As names of my contacts flew by, my brain started reminiscing: Oh I remember this person? Man, that was a fun project! I wonder where this person is now?.

I was compelled to click on some entries to see people's faces, to check their resumes.

The playful iPhone UI made the same LinkedIn information much more engaging. It instantly brought into the spotlight the most interesting aspect of LinkedIn - the people.

LinkedIn as Entertainment

LinkedIn is a great business tool - you can use it to find jobs or suitable candidates. It's helpful for introductions and lead generation, but it underplays human aspects of business connections. If LinkedIn were more interesting and entertaining, imagine what it could do?

The opportunity lies in a better user interface, lending itself to more exploration. A visualization like the one PicLens provides would be great fun - e.g. the ability to see peoples timelines visualized, such as 5 previous co-workers now working for Apple. It's essentially slicing and dicing information that LinkedIn has, though in a way that is playful and useful.

In addition to playfulness, there's a sentimental factor here. Enabling people to tap into their memories and recall co-workers. Imagine a flashback - your job at Yahoo. Remember Jane, John, Kate and Mike? Here's where they are now. Jane and Mike are still at Yahoo, John is engineer at Google by way of Microsoft, and Kate is working for a startup in Colorado.

Will Entertainment Pay?

Would this sort of thing help LinkedIn's bottom line? Not directly, because people are unlikely to pay for such entertainment as they pay for other LinkedIn features. But the pages would surely generate traffic and bring CPM advertising dollars. Facebook grew big because of entertainment.

It's cool to see pictures of friends, to know what they're up to, and to stay in touch. By adding this human focus, LinkedIn could become 'cool' in addition to useful.

What do you think about adding the entertainment dimension to LinkedIn? Is this something you'd like the company to do? What other features do you wish LinkedIn had?