Now information continually bursts through our Google Readers, our cell phones, our tablets, display screens in elevators and grocery stores. Do we really need to read all 88,731 articles on the Bernie Madoff trial? Probably not. And that's the dilemma for news organizations.

In the old metaphor, column-inches was the constraint. In the new metaphor, reader attention span becomes the constraint.

The dilemma is that relevant is in the mind of the beholder. Emerging application - news readers such as Flipboard, Pulse and Feedly - are struggling to deliver relevancy. One proxy is to use a reader's social graph to curate stories. Another is user profiles and preferences.

These current approaches are doomed. With respect to social graph curation, people have different roles at during different times. On the weekend, a reader might be interested in arts, entertainment and sports news based on a friends and family. During the week, this same person may be interested in business news based on recommendations from trading partners in the capital markets. How do readers seamlessly reconcile this?

I am not able to predict when viable applications will emerge, but I do believe the industry will struggle for several years. The fate of these first generation news readers may be similar to first generation social networks. Remember Friendster and Orkut?

However, there is some clarity on key underlying technologies that will provide the scaffolding for next generation news consumption. Here are some of them.

Tagging and Semantic Extraction

Tagging or semantic extraction engines, process news articles and return structured metadata, to provide insight into the underlying text. A trivial application that uses tags is the tag cloud, where users click tags displayed on a web page to uncover underlying content. Tag clouds miss the point.

Disambiguation

Disambiguation is a technique to uniquely identify named entities: people, cities, and subjects. Disambiguation can identify that one article is about George Herbert Walker Bush, the 41st President of the US, and another article is about George Walker Bush, number 43. Similarly, the technology can distinguish between Lincoln Continental, the car, and Lincoln, Nebraska, the town. As part of the metadata, many tagging engines that disambiguate return unique identifiers called Uniform Resource Identifiers (URI). A URI is a pointer into a database.

If tagging creates machine readable assets, disambiguation is the connective tissue between these assets. Leveraging tagging and disambiguation technologies, applications can now connect content with very disparate origins. Today's article on George W. Bush can be automatically linked to an article he wrote when he owned the Texas Ranger's baseball team. Similarly the online bio of Bill Gates can be automatically tied to his online New Mexico arrest record in April 1975.

Linked Data Structures

Typically, a URI entry holds some content related to the entity. URIs are linked to form a database called a Linked Data Structure. For instance, a URI on Barack Obama may contain a reference to his current position, former jobs, marital status, spouse's name, children, education and schools.

The URI Michelle Obama may similarly contain information on her spouse, children, college, graduate school, etc. The URI on Michele Obama's law school will contain information about the school, such as current and past deans.

For instance, one could build an application to retrieve articles on the Supreme Court and determine if the article mentions any justices who previously headed a law school attended by the wife of any U.S. President. This application would identify all articles mentioning Justice Elena Kagan, previously Dean of Harvard Law School, from which Michelle Obama graduated.

That example, per se, may be nonsensical, but the power of the technology is immense. For instance are there any CEOs of government contractors who spouses happen to sit on philanthropic boards along with lawmakers on the House Ways and Means committee?

Linked Data Cloud

The metaphor becomes even more powerful with the federation of Linked Data Structures into the Linked Data Cloud. This level of abstraction allows different owners (such as dbPedia, IMDb, the New York Times) to link their data to create a powerful ecosystem. The power here is the power of the network effect.

Utilizing these underlying technologies emerging applications will drive a completely news-reading metaphor.

Conclusion

Semantic infrastructure technologies will propel next generation news consumption. The fluidity and deluge of online news overwhelms us, but smart readers will tame this flow and enable new consumption models and insights.

Newspaper box photo by George Kelly

We asked three people building cutting-edge social software what they think the relationship between relevance, popularity and filtering is going to be in the future. They offered three very different responses. What do you think the future of information filtering will look like?

Nick Halstead is the founder of Twitter-tracking service Tweetmeme and data mining startup DataSift. He thinks that both relevance and popularity will be important for filtering information in the future, but that what these concepts mean is changing dramatically.

"Relevance is changing," he says.

"We used to think of relevance as matching a query with a result, in most part via search engines performing keyword searches. And popularity was based on single metrics (such as Pagerank) that were based on global behavior.

Nick Halstead

"What is evolving is the next evolution of both, and both are being changed by the social graph. Relevance is defined now by an understanding of a person's likes and dislikes - Facebook and Twitter both have immense data on behavioral patterns around what you like. And content is now understood and categorized semantically. So relevance in the future will be determined by how your likes match content.

"Popularity is changing as well - services like Klout, PeerIndex etc. are proof that social authority is becoming important. The first company to find that magic 'rank' that suddenly means 'ad-targeting' (and therefore cash) will become huge.

"So the same combination of relevance plus popularity will still drive the future. See what Quora wrote about how they plan to choose content, for example. But it is going to be based on new social metrics - and fundamentally you won't 'search' - you will 'follow' content that is matched to you. With or without needing to tell the computer.

Relevance vs Popularity in Filtering, Throughout History

Pre-press days: Religious leaders kept the content on-topic by selecting it themselves. But at the same time, they had to keep their audiences in mind. As Sy Safransky said in the most recent Sun magazine, if Jesus had been a sourpuss, he wouldn't have been able to draw a crowd.

Days of the Printing Press: The battle between editorial responsibility and the need to sell papers pitted relevance against popularity. The most popular content subsidized the most relevant, though.

The Early Web: Easy publishing enabled so much niche content that observers worried people would become insulated in self-justifying info-ghettos. Hyper-personalized, with no counter-balance of concern for popularity?

Early Social Web: The promise of mega-distribution through Facebook and Twitter has prompted publishers to emphasize link-bait and other content that's all about popularity.

The Future?: Nearly infinite data, through easily published content, exhaust data from online behavior and meta-content built by people and machines, based on the patterns manifested online. That's a whole lot of content to choose from in serving Web users. How should relevance and popularity be used in deciding which of it to deliver?

"I think Google knows it needs to adapt, quick."

Edwin Khodabakchian is the creator of Feedly, a magazine interface for content subscribed-to from around the web. Feedly released its iPhone app this week and we called it possibly the best iPhone feed reader on the market.

Khodabakchian says that there are different kinds of web users and they need different kinds of filtering. Feedly uses relevance for discovery of sources and popularity of items for filtering and prioritization.

"We break users into 3 categories," Edwin told us.

• Facebookers - they spend hours in Facebook. They love to connect with friends and content is just an excuse to interact, be cool, feel part of the tribe.
• Passionate Users - They care about topics, they have a specific connection with the author and brand they read. They love predictability.
• Twitter Users and Bloggers - who live in information and crave for real time.

"I think filtering is different for these different classes of users of information. For Facebookers, relevance and popularity is about: can I find something really funny or different which I can share with my friends and be cool. For passionate people, they have already sources or sites they trust - this is what make it predictable. Filtering for them is a mix of their favorite sources, what is popular, and suggestions.

"Twitter is more interesting because it has an implicit interest graph under the hood and it is time based. It is great to know what is happening right now...and it is great to get content from sources you might already know and feel passionately about.

"We focus on passionate people who want to feed their minds. Who look for a predictable experience...with a tiny bit of discovery.

"When you are on a source page in Feedly desktop, there is a 'you might also like' list of sites. This is collaborative filtering...similar to the Amazon you might also like.

"In doing that, we are trying to create a relationship between a user and a source or author, not a one-off article display. About 70% of the new Feedly users do not know about RSS or Google Reader. The hard part of the personalization nut is that you have 30 seconds to help them build something relevant and you can not ask them because people go blank. And it has to not feel like work."

Ouriel Ohayon is a serial entrepreneur, most recently the founder of mobile app recommendation service Appsfire. He argues that this emphasis on user experience is so important that it overrides both relevance and popularity as a concern when it comes to filtering information.

"The problem is not algorithmic or metrics," Ohayon says.

"The problem is experience. You can build the best filters [but] it does not matter if the tools for experimenting them and UI for displaying them are awful. For example: Google Reader vs Flipboard.

"People care more about presentation and packaging than the actual set of parameters behind the info feed. It requires of course elements of contextual serendipity and personalization: social graph, geography (very important), freshness, interestingness (rated if you will). But eventually users don't want to hear about that - they want a nice experience. Especially on mobile."

As a technologist, I find that a little bit depressing. I don't think I agree with it. I like parts of all three of these responses, but feel most closely myself like reality lies somewhere in between what Halstead said and what Khodabakchian said. I'm drawn to processes that determine quality sources of information through relevance detection, personalized as appropriate, but then filtered on the item-level by popularity or variable weight of sources.

I also want to be able to lift the hood and fiddle with the calculations as appropriate. Perhaps that points to the subjectivity of this question - because most people certainly don't want to do that.

I expect that information overload will be dealt with in a variety of ways in the future, and for most people the end result will be to kick back and have the best content sent to them automatically, however what's best is determined.

Do you think there's a relationship between relevance, popularity and filtering content? What do you think it will look like in the future?

Trash photo by ericortner; Nick Halstead photo by Loic Le Meur.

Because of Android's relative openness, Aro is able to completely integrate its PIM solution onto the Android mobile platform. But now, as the company prepares to launch its iPhone version, compromises had to be made. This begs the question: can innovation around core apps even work on iPhone?

Aro Mobile, to catch you up, is a semantic technology software company led by CEO John Lazarus, a former Microsoft employee, now senior advisor to Allen's Vulcan Capital and board member at Evri, another notable semantic tech company, which recently went mobile too. Simply put, Aro's goal is to make our so-called "smartphones" much, much smarter by using machine intelligence to interconnect the apps we use the most, the core apps used in communications.

Semantic technology is a difficult concept to explain to mainstream users, but Aro demonstrates its potential by innovating on top of the PIM, the core communications applications that mobile users access daily. The software understands the language of mobile devices, including the way you chat via SMS, the way you email, even the way you Tweet. It can identify people, places and dates referenced in your emails, messages and calendar appointments and the importance of those items to you. Within Aro's graph of your social connections, it understands that there's a stronger connection between someone that you've emailed multiple times and someone who's only listed as an address book entry. That's the power of semantics, actually: the ability to understand.

We've seen semantic technology cropping up here and there lately, but mostly on Android devices. Keyboard replacement apps like SwiftKey and Siine, for example, are using machine intelligence to help you type more quickly with word suggestions - but not just suggestions based on a database of words in a given language, but those delivered through an understanding of how you - you personally - communicate. If you start typing "cy" on your phone, SwiftKey or Siine might suggest "cya!!!!," if that's the way you normally complete a text message conversation. Siine plans to take things a step further, with support for translation between languages and integration with other mobile applications, too.

Aro, along these same lines, wants to function as a conduit between the phone and the "cloud." Although it focuses on email for now, it will soon integrate with Facebook and Twitter to add another layer of data and understanding to your communications.

That's an explanation of Aro at a very high level, however. In practice, the app, or rather, the suite of apps installed function as your phone book, contact lists, calendar and email client. Even in its raw, early format, the app is smart. Very smart. It can identify a person, place or date, and then, when you tap on the item (identified objects are circled), a list of actions appear. If you tap a name, Aro may suggest that you edit the contact, add to your contacts, send an email, place a call, etc. And if you were to proceed, it knows what information to use - it knows the phone number to call or it could automatically fill in the contact details. The exchange of data between the apps is seamless - you don't have to copy and paste information from one app to another and you don't have to constantly toggle between apps, either. Everything talks to everything else.

For now, Aro serves as a useful personal assistant of sorts - in addition to its interoperable apps, you can use Aro search against your entire archive of data, the results custom-tailored to you. In a later version of the app, a location service will be added, allowing you to further personalize your search results. For example, a search for a "coffee" in a given city could return recommendations based on where you had been recently, as opposed to the sort of default list that appears in Google.

Design Challenges

As innovative as the technology is, the interface still feels too technical, as if built by engineers not designers. The objects Aro recognizes are circled so as to highlight them, like a teacher correcting a student's homework. Aro is training wheels for learning semantic technology. Here's a person, here's a date, it tells you. Tap this here. See what happens.

The text is circled because we, as Web users, can't quite grasp the concept of actionable data that's not highlighted in some way. We expect hyperlinks, colored and underlined, to direct our clicks. But links are going away, says Andrew Hickl, Aro's CTO. Semantic technology will eventually lead to their demise. In a decade or so, any object, any piece of data, anything that you can touch will be able to do anything and no one will need training to know that's the case - it will just be the way it is.

On iPhone, Limitations

Android was the perfect place for new, experimental technology like this to launch. Because Android isn't as tightly locked down as iPhone, there's room for software like this to take hold. There, Aro can behave more like individual applications and can more deeply integrate into various Android menus and functions.

As Aro prepares to launch on the iPhone, however, there are limitations. The iPhone Aro app when launched opens up the Aro suite of apps, the icons appearing as if in a folder. While the "apps" can still talk to each other, on iPhone they aren't really individual apps. The email "app" can't become the default mail app on the iPhone and the calendar "app" can't become the default calendar. This limits how deeply Aro's integration can be. You can't, for example, email a picture using Aro's email app - you still have to use the Apple-provided mail client.

For iPhone users, the level of control and consistency Apple provides is one of the key selling points of the device. Everything works as it should. But it's all Steve Jobs' vision - and while that's nothing to sniff at, not by any means - there's a reason why many early adopters are going to Android. The platform allows others outside of Apple to fully develop ideas of their own, too. And we get to watch them as they do, we get to test the apps in early alpha and beta formats, and we get to participate in the innovation that occurs.

Comparatively to iPhone's own core apps, Aro may not be as pretty, but its underlying technology needs a place to flourish and develop. Like other semantic apps, there needs to be a real-world playground where people can try these new things. For now, that playground is Android.

Aro is only one example of this. Siine and SwiftKey (mentioned above) are another. ON, a new company with an innovative take on the address book is yet a third. ON lets you maintain different profiles for different groups of users - one status message for friends, another for colleagues; one voicemail message for the boss, another for the spouse, and so forth. Like Aro, ON launched on Android, then dumbed itself down for iPhone. SwiftkKey hasn't even bothered with iPhone app. It's unclear how Siine will manage.

iPhone users wishing to get a taste of what's possible on Android can try Aro, when the app becomes available in Q1 2011. A lot of the functionality will still be present. But for iPhone users, trying Aro may feel like toying around with a demo. For full-speed immersion, try the Android version instead.

But this is no mere keyboard replacement "app," it's much more. It's a communication platform. A universal translator. A system that learns how you speak and then speaks for you. Siine is the future of text-based communication - or, at least that's what the company says.

Not Just a Keyboard Replacement App

Several companies have already begun re-thinking the keyboard, including Swype, a keyboard replacement app which lets you drag your finger across the screen from letter to letter, and SwiftKey, which smartly suggests the word you may enter next (e.g. You type "t" and it suggests "the.") Similar to Siine, SwiftKey also makes use of semantics and machine learning to suggest the words you may type - that is the sorts of things you typically say in a conversation.

When Siine first enters the market, it may be confused as a direct competitor to apps like these, because it will initially arrive as an Android Market application. But Android is just the beginning - Siine aims to be an app you use on all your screens, coming next to the iPhone, the iPad, tablet PCs and the Web shortly after its Android launch. And according to the company, it will more than just a keyboard replacement application.

So what is Siine, exactly? Well, it's a little curious. Operating in stealth mode until recently, the company is still short on details, leaving much to be revealed during its launch at November's Under the Radar conference, an event showcasing new startups.

What We Do Know: Semantics, Understanding, Translation

Here's what we do know: Siine is a communication tool that will eventually exist for any platform where you enter text into a little white box. Whether you're SMS'ing on your Android, typing on an iPad, tweeting from a basic messaging phone or surfing the Web.

Siine doesn't just suggest words, it suggests the words you would say. For example, if you always sign off your text messages with "Cya!!!!!!!!!!!", Siine knows. If you say "like" all the time, Siine knows. If you use acronyms, Siine knows. It even knows what those acronyms mean. It even knows what acronyms mean in different languages. And it doesn't need to have giant dictionary-based databases on the backend to do all this because it uses semantics - a technology that enables the understanding of meanings and context of the words and phrases you use.

In short, Siine combines intelligence, semantics and the ability to understand you.

Business Model Still Flexible

For now, the company's business model is flexible - the company is in talks with several big-name handset manufacturers and mobile carriers, as well as VCs.

Siine's CEO Ed Maklouf's educational background plays well into the creation of a startup like this. While at Stanford, he studied Communications and Linguistics, so he has a better understanding of how people communicate with each other than the average entrepreneur. He and his small team have now put that knowledge into action in Siine and we're excited to take a look at what they've come up with.

Are you excited too? We thought so. In a couple of weeks, Siine will launch into private beta and we'll offer invites to our readers - stay tuned.

Above is the first and only image of Siine that the company is providing at the moment. It's notable for the fact that it doesn't look remarkably different than a standard keyboard. How does Siine turn that keyboard into a "revolutionary" new interface, exactly? We'll keep you posted.

Update! New Video!

Siine has just released a new YouTube video detailing its service a bit more. Although it's still vague on the specifics, you'll see there are some hints about how it integrates with applications - a feature that may position it as being both a virtual assistant and a keyboard replacement application.

Invites available! Click through for link.

Focusing on the looks, though, is missing the big picture. Meshin is different from other email-based contact management systems including not only Xobni, but analysis engines like Gist, too. Where those companies hinge on the person - here's their title, where they work, their emails, attachments, their blog posts, their last Twitter update, etc. - Meshin actually analyzes the information found in the information streams it examines. It then extracts related conversations, related messages, related people and other semantically understood data. And it does so by looking beyond keywords. It knows what things mean. It knows if a word is referring to a person, place or thing. It can also surface related links and news from the Web for any given entity.

Already, the engine behind Meshin isn't limited to email messages alone. For example, if you subscribe to RSS feeds within Outlook, those are also understood as being a part of the relationship map with another person. If you subscribe to Twitter feeds within Outlook, again, those are analyzed, along with the other streams.

Meshin arose from a Xerox-funded project inside PARC whose goal is to commercialize older PARC IP for a broader audience. The project has been in development for only a year, with a small core team and support from PARC researchers. The long-term goal for Meshin is to extend itself beyond Outlook, in order to connect other types of information streams together. 

The researchers are contemplating where they should take the technology next - another email platform? An RSS reader? A standalone product? Should they open up Meshin APIs (application programming interfaces) for developers to use within their own applications and services? All these models are a possibility, but first the engineers wanted to just get the technology out there, in the hands of users.

We're helping them with that by distributing invites to the private beta. For access, click here.

]]> Discuss]]> http://www.readwriteweb.com/archives/parc_releases_new_semantic_technology_in_form_of_an_outlook_plugin.php http://www.readwriteweb.com/archives/parc_releases_new_semantic_technology_in_form_of_an_outlook_plugin.php Product Reviews Mon, 06 Sep 2010 07:45:52 -0800 Sarah Perez Facebook just shook the tech world by announcing several major initiatives that collectively constitute an aggressive move to weave the social net on top of the existing Web.The rumors were that the leading social network would launch a "Like" button for the entire Web. Instead, Zuckerberg & Co. unveiled a bold and visionary new platform that cannot be ignored.

The bits of this platform bring together the visions of a social, personalized and semantic Web that have been discussed since del.icio.us pioneered Web 2.0 back in 2004. Facebook's vision is both minimalistic and encompassing - but its ambition is to kill off its competition and use 500 million users to take over entire Web.

Facebook Open Graph: Publisher Plugins

The Open Graph is a set combination of publisher plugins, semantic markup and a developer API.

Login with Faces & Facepile: The simpler publisher plugins enhance Facebook Connect. They makes it easy and compelling to sign in by leveraging Facebook cookies and showing faces of Facebook friends who are already members of the service.

Like Button and Like Box: These plugins add the liking feature to any content, typically the whole page. Both can be enhanced with semantic markup, described below. But the very basic intent for these is to get users to Like on the site and post a link to Facebook, which is then permanently stored on a user's profile and points back to the original site.

Activity Feed and Live Stream: These plugins show static and dynamic activity on the site. Activity Feed lists recent likes and comments from the site, while Live Stream shows a real-time view of activity on the site and is intended for interactive events.

Recommendations: This plugin surfaces personalized recommendations for the user based on what friends and everyone else is liking on the site. It is intended to drive the users to other pages on the site.

Facebook Open Graph: Semantic Markup

Facebook announced simple, RDF-based markup to make the plugins smarter. In a nutshell, the markup enables publishers to say what object is on the page - a movie, a book, a recording artist, an event, a sports team, etc. This automatically enables semantics, that is, an understanding that the user is not just interacting with a webpage, but that he or she is liking a specific kind of thing. Semantics then leads to bucketing of the objects into categories like books, movies, music, etc., and gives rise to all sort of applications, including personalized recommendations.

Perhaps even more importantly, the markup helps Facebook connect the users across common interests across different websites. For example, if both Pandora and Last.fm annotate a page about The Beatles using Facebook's markup, then users will be able to see their friends, who like the Beatles across different sites. This is very significant, because the data around friends is sparse and scattered around the sites. Previously, Facebook would surface this data in the stream without persisting it. Now, the information about a friend's likes of movies, music, books, recording artists, events, sports team, etc. will be permanent on Facebook profiles and readily available in context around the Web.

Facebook Open Graph: New API

The new Facebook API is elegant and streamlined. It makes it easy to access user information (with permission of course) such as profile, friends, etc. All of the calls are REST based and return JSON objects. For example, my profile information can be fetched like this: http://graph.facebook.com/alexiskold. The authentication is based on OAuth 2.0 protocol and makes it simple not only to connect, but to also prompt for permissions to access user information.

This new API turns Facebook into a read/write storage of users' tastes. And not just one user - all Facebook users.

Implications for the Users

With this release, Facebook asks users if they are willing to trade off privacy for personalization. To be clear, no personalization is ever possible without users telling a system about their tastes. What Facebook is asking for is necessary in order to then create personalized Web experience. Whether users want this sort of thing is a different question, but assuming that you want to know more about your friends you will.

Friends' interests around entertainment, sports, travel, etc. will be categorized and available. It will be easy to figure out what your friends are into both on Facebook and around the Web. In addition, Facebook is going to be using its own engine to bring you recommendations for related content. This will further accelerate the discovery and cross linking between friends. This will likely further impact the amount of search people do around the Web. As Fred Wilson pointed out - passed links replace search.

Yet, the crux of user implications is neither of the above, but one single issue: privacy. It is unclear at this point that this issue is a concern for actual Facebook users, but it is clear that tech world is raising its eyebrows: Marshall Kirkpatrick, Dave Winer, Jeff Jarvis and many others expressed their concerns. People are saying that not only Facebook will know too much about us (because Google is already there today), but that it will be able to control too much.

Personally, I am skeptical that the average Facebook user is going to care all that much. People are notoriously naive about being watched on the Web, and this is likely to be no exception. More likely than not, Facebook users will enjoy the personalization aspects of the new platform and won't think much about it - until Facebook starts openly targeting them.

This was not been part of f8 of course, but Facebook is likely to use the information for targeting. After all, advertising is a major part of its monetization already so why won't it make it even better? If this targeting is too spot on, lots of users will probably get annoyed. Facebook is likely to sooth them via Facebook credits and heavy discounts, negotiated because of their massive volume.

How exactly users react remains to be seen, but they will probably like the new Facebook more because of increased relevancy and interaction with friends around the Web.

Implications for Publishers

On the surface, this Facebook offering is a no-brainer for publishers. Who does not want more social activity on their site? However, in reality this is far from a slam dunk. To understand why, consider two types of sites: sites that are either social networks or have social networking integrated, and the sites that have their own commenting and ratings systems. In the first camp you will find Last.fm, Flixster, Goodreads, etc. None of these sites were a launch partner, understandably so. Social connections around music, movies and books are their bread and butter as are the ratings, reviews and recommendations. If they switch to Facebook for all of this, what do they have left?

So any site that already has social networking built in has to decide to abandon that before jumping into the Facebook Open Graph. The even worse problem is the ownership of ratings and comments. Are publishers really ready to give that up? Nobody seriously thinks that users are going to be rating through Facebook and then through the site again. So how is this going to work? It is unclear at this point, but it's likely publishers will ask for ways to replicate or export comments and likes that users sent to Facebook via their site. Perhaps an open API that allows publishers to manipulate the data is the answer, but it is easy to see how some publishers would be very concerned.

However, if you run a website like eCommerce or a blog or a service like Pandora that currently does not have a lot of social built-in, this offering is a no-brainer as it will instantly start recycling your pages through the massive Facebook power of passed links.

Implications for Competitors

This is aggressive and brilliant move by Facebook - and Twitter, Google, Yahoo, MySpace, AOL, eBay, Amazon and others, except for Microsoft, should be really worried. It appears that Microsoft is content with just partnering with Facebook, perhaps rightly so. Possibly a Bing deal is in the works, which would make a lot of sense.

For all other players on the Web, the worry is that Facebook is trying to close the loop in exclusively owning user eyeballs. Apparently Facebook is not content with just connecting people; it wants to connect people and things. And not only that, it wants to do it around the Web. And not just any people - friends. You don't need to look too closely to see that Facebook is creating a feedback loop, which includes it, users and the rest of the Web and excludes its competitors.

There are several things that other big players might try to do, the worst of which is to try to mimic Facebook. The "me too" that we've seen way too many times recently has not worked, and will not work now. The second best choice is to try to block it. As strange as it sounds it might just work. Between publisher and user issues there are a lot of concerns, and a carefully orchestrated and coordinated campaign may seriously hurt this initiative. Remember, Beacon was brought down fairly quickly by a combination of user backlash and derogatory press.

The third option - to embrace and extend this platform, to innovate on top of it - is likely to be the best move. Innovation has always trumped stagnation on the Web. The problem is that it might not be that easy to embrace this initiative. After all, it does not look like Facebook asked everyone to gather around the table and cooperate on this. It might not be open to cooperation, but if it is then this is the way forward.

Technically speaking, what Facebook has done is elegant and correct. From markup, to plugins, to API, all of it is modern and awesome. The missing bit is that Facebook appears to be the only repository of data in this equation - and that makes the whole offering seriously closed. Publishers and users don't have a choice as to where to store the data. It is going to Facebook and Facebook alone. Perhaps there is a way to rework the system in a way that fixes that. We will look forward to see how this unfolds.

Implications for Facebook

Clearly this announcement is yet another turning point for Facebook. Before the conference Facebook was the biggest social network on the planet. If its vision actually happens, Facebook will be the biggest network of people and things on the planet- or to put it differently, it will be the taste graph of the planet.

Obviously there is a different technology that Facebook will need to be building. It already perfected the social networking part, but semantic analysis, recommendation systems, vertical categories like movies and books, as well as having completely open read/write storage of tastes is completely new to the team. The biggest challenge that Facebook will face is to inject, re-deliver and most importantly make use of the data that is flowing into it.

Facebook will be doing some serious number crunching and UI revamps to prepare for this next phase of its life. But perhaps the biggest experiment and test will be delivering relevancy. Google succeed with this in search; Facebook will now have the challenge to bring relevancy to the recommendations and taste-based advertising arena.

Implications For the Semantic Web

One of the most exciting parts of the Facebook announcement to me personally is the possible breakthrough in semanticizing the Web. We've written previously about the Semantic Web here, and it has been a personal passion of mine. What Facebook has done has a chance to make vast parts of the consumer Web including movies, books, music, events, sports, and news semantically tagged. Publishers and websites finally have a strong incentive to mark things up and get return traffic from Facebook.

The actual protocol that Facebook suggested is very simple. To describe the object on the page, the site owner needs to specify the title, type of the object, image, url and the name of the site using simple meta tags. The format is extensible and additional tags can be added. For example, for a book a site can add an isbn number. This format leaves room for ambiguity. The goal of classic semantic markups traditionally has been to refer to entities precisely; for example adding the director to a movie, or a year to remakes. The Facebook protocol does not seem to have this.

There were lots of previous efforts to markup the Web. To name a few, RDF, microformats, Google Rich Snippets, Yahoo's Search Monkey (based on RDF and microformats), and lastly, abmeta, which was developed by me with help from Peter Mika at Yahoo. Of all these formats, Facebook's is most similar to abmeta because the markup is placed into meta tags, and is simple and human readable. This simplicity is the key to broad adoption.

So all around, this is a great chance for the Semantic Web to finally hit consumer verticals and become real.

Implications for Developers

Every new rich platform that has been rolled out in the past couple of years presented a big opportunity for developers and this one will be no exception. While we do not know exactly what sort of applications will be build on top of new Facebook, we know that they will be very powerful. This platform has the potential to give rise to to new kind of personalization and attention economy that people have been talking about for years. It has of course, a chance to majorly backfire, but I am optimistic.

This will be a gold rush for application that is likely to last for at least a year, like the last one did. It's too early to tell whether this will be a platform that survives and does not hurt is participants. However, it is very likely that the best applications built on this platform will be owned by Facebook. Still, there is a huge new opportunity here for developers and the sky is the limit.

Checkmate?

Facebook made a major chess move. It might have checkmated its competitors, or perhaps it might have to lose another piece like it lost Beacon. Whichever is the case, right now there are deep implications for Facebook and its competitors, publishers, users and the Web at large. What Facebook has announced cannot be ignored and can not be undone. Everyone needs to figure out the next steps and understand what to do.

Time will tell where we land, but my gut is that positive things will come out of this. If nothing else, let's give Facebook credit for innovation and re-imagination the Web.

Two such "divining" projects include CoPhIR (Content-based Photo Image Retrieval) Test-Collection and IBM's MUFIN (Multi-Feature Indexing Network). These projects tie into SAPIR's back end by extracting data from the Flickr archive and indexing features such as scalable color, color structure, color layout, shape edges and texture.

As shown in the video of Madrid's Plaza de España, SAPIR identifies matching media in the same way that humans derive intrinsic value from visual and sensory clues. Users can also choose to combine search terms with additional text to further drill down in search results. As is the case with regular search, if you already know the city where your image was taken, you're one step closer to finding your result. Additionally, SAPIR also has the ability to index sound and video files.

While the catalogue of media is still very limited, theoretically we may one day be able to search for almost anything using this technique. If Ashton Kutcher wears a pair of sunglasses we like, we can scan the image and search for the storefronts stocking them. If we're looking for the name of a town square, we can find it in the tags of similar images. And finally, if we're looking to self-diagnose we can compare photos of ourselves against jaundice or malaria patients.

The advantage of this tool is that we may one day have a chance to collect up the disparate bits in the digital ether and identify them as useful points of information. To test SAPIR in its early research stage, visit the homepage. You may also want to test out MUFIN to compare results.

NetBase employs the same principals across a variety of enterprise tools, but healthBase is its first foray into consumer-facing products. While the company is used to powering corporate, federal and market research, healthBase allows NetBase to show off its content intelligence tool in a way that gives us insight into our selves and our bodies.

Because NetBase is not reliant on manual annotation or custom taxonomies, the system is also very scalable. It took roughly 2 days to produce all of the data in healthBase - a feat that would never be possible by a combination human and machine system.

"It's important for us to address real issues with semantic technologies outside of a lab," Says Tellefsen. To try healthBase visit healthbase.netbase.com

2. Cuil

3. Hakia

4. Worio

5. Ubiquity

Ubiquity for Firefox from Aza Raskin on Vimeo.

6. Semanti

If you've got more examples of semantic data collection tools, list them in the comments below.

Common Tag is a new tagging format that creates references to concretely defined concepts with their own metadata and URLs. With Common Tag, site owners can simply topic hubs, cross-promote content, and enrich pages with data, images, and widgets.

According to the Common Tag website, "The Common Tag format was developed to address the current shortcomings of tagging and help everyone - including end users, publishers, and developers - get more out of Web content. With Common Tag, content is tagged with unique, well-defined concepts - everything about New York City is tagged with one concept for New York City and everything about jaguar the animal is tagged with one concept for jaguar the animal. Common Tag also provides access to useful metadata that defines each concept and describes how the concepts relate to one another. For example, metadata for the Barack Obama Common Tag indicates that he's the President of the United States and that he's married to Michelle Obama."

The project aims to help make content as discoverable and connected as could reasonably be assumed. The creators also hope to make content more engaging. When a web app can determine what a piece of content is actually about, the UX improves exponentially. The website gives the example of a developer creating an app that uses an article about the most recent Star Trek movie and lets users purchase tickets on the same page. The site reads, "Since both the publisher and ticket service use Common Tag, the application is able to easily make the connection without having to guess at what the content of the two services is about."

Tags are expressed using RDFa, a standard format for defining data in HTML. Relevant code can be found in the Common Tag Quick Start Guide. Interested parties can learn more in the Yahoo! Common Tag group.

According to a Q&A with partner company Zemanta, CTO, Andraz Tori, said the idea for Common Tags "started in informal discussion with Peter Mika from Yahoo! and research about what would be the easiest way to let publishers get more out of their content by semantically marking it up." We've seen Common Tag as a vehicle to make Web content more discoverable, connected, and engaging.

"We then learned on previous efforts and decided that we need a full-blown ecosystem from day one. Not just academic support, but web industry support. As you can see the idea was well received."

In terms of adoption, Tori stated, "This is the first time that this number of web companies have stepped together from day one to introduce a tagging standard. We tried to build on previous academic efforts. Over that we added business incentive to participate."

You can also interact with the items being tracked via the Glue plugin which features a "like" button and another "2 Cents" button which lets you leave a comment about whatever it is you're viewing.

As with the previously released Firefox plugin, the Glue IE plugin also delivers the same type of interactions as you would expect: the connected conversations around everyday things, recommendations, and web-wide "top lists" that include the top items across the entire Glue network.

You can grab the Glue IE plugin from the main page of the Glue web site here. Note: the "Download" button still features the Firefox logo only at this time, but clicking the button reveals the IE download is available as well.

Disclosure: Alex Iskold (@alexiskold) is the founder of AdaptiveBlue, the company behind Glue, and occasional RWW feature writer.

In testing LinkWithin on our own WordPress blog, the suggested posts were always spot-on, but obviously, your mileage may vary.

A Few Caveats

Because LinkWithin does most of the computing on its own servers, it can take a little while before its results appear on your blog (usually around one hour). The plugin also doesn't play nice with every theme, though the team promises to fix any problem you might encounter within a few hours after you contact them.

Verdict

LinkWithin is still a bit rough around the edges. It would be great, for example, if you could actually customize how the related items are displayed on your blog. For now, you are restricted to showing three related stories, for example, and the related stories will appear on both your homepage and on individual posts.

If you are looking for a different way to showcase more of your blogposts on your blog, however, LinkWithin is definitely worth a try.

Thanks to Marjolein Hoekstra (@cleverclogs) for telling us about this new service.

At this month's DEMO 09 conference, one of the most apparent trends was the emergence of several new intelligent web services. In this transitional period between Web 2.0 and Web 3.0 (or whatever it is that comes next), the tools of the future are just now being revealed. Although at first glance some of these services and applications may seem somewhat incomplete, in many cases they actually represent years' worth of work to have reached the point they're at now. These are no simple Web 2.0 applications; these are highly complex and intelligent tools of tomorrow's smarter web.

A.I.-Powered Shopping (Gazaro)

Gazaro is a new service that lets you make what they call "personal sales fliers." Instead of sifting through the local paper to find the latest deals, you just tell Gazaro what sorts of products you're interested in. The service then scours the web for the best deals and presents its findings in a clean, easy-to-read interface. But Gazaro isn't simply a price comparison engine. It's a really smart one.

Gazaro knows that a "camera" is a "camera" or that an "LCD" is an "LCD." It's not doing simple keyword matching, it really understands the difference. In other words, you'll never get results for a camera lens or camera accessories when you're searching for just a camera because Gazaro knows those are not the same things.

The reason it can differentiate between items is because it's powered by Artificial Intelligence (A.I.) on the back end. In this case, "A.I." is no buzzword - the company was incubated by Apption Software who had developed A.I. technology for use in the enterprise. They realized that the same technology could deliver value in a consumer application as well, and from there came Gazaro.

When Gazaro goes out and crawls the internet, it compares the items it finds to the items it already knows in order to determine what exactly the new items are. If it encounters something it doesn't know, it makes an educated guess using its A.I. "brain." And the more it crawls, the more it learns.

After identifying what an item is, Gazaro then determines if the item found is actually a good deal. How good of a deal it is or not is represented with the "Gazaro Deal Score." These deal ratings are based on Gazaro's knowledge of historical prices, how often an item goes on sale, what other retailers are selling it for now and what they've sold it for in the past. All that analysis is done using the A.I. technology in order to rate the deal on a scale from 1 to 10, with 10 being the best deal.

To the consumer using the system, the complexity of what the A.I. is doing is all hidden behind the scenes. The end user only sees a simple interface where they can enter in the items they're shopping for and then find the best prices. Gazaro can also alert users to new sales and deals using email, RSS, or Twitter. At the moment, Gazaro is for consumer electronics shopping only, but in time the system could expand and learn more product categories.

Understanding Intentions (Primal Fusion)

Another company of interest is Primal Fusion whose new "thought networking" service is a semantic technology platform designed to help you research the subjects that interest you. Unfortunately, "thought networking" is a buzzword-sounding phrase that doesn't really convey what the system does. Primal Fusion essentially is an alternative to doing traditional web searches when you want to learn about a particular topic.

Once signed up for the Primal Fusion service, you enter in your topic in the search box provided and you'll see a tag cloud of words appear which are relevant to the word you initially searched on. You can either select those words by checking them or you can click on the individual words to further drill down into a more specific aspect of the original topic.

In the example they demonstrated today, a student researching climate change might see a tag cloud featuring words and phrases like "pollution," "co2," "greenhouse gases," etc. In addition, the service can also return relevant photos to your topic from sites like Flickr.

Initially, Primal Fusion searches Wikipedia to deliver the tag cloud, but once you have your specific interests checkmarked you can then change a drop-down box to search the web instead. This web searching is done courtesy of a Yahoo BOSS integration and it's here where Primal Fusion one-ups a normal search engine. Instead of just returning the top 5 or 10 results on the original keyword, it sifts through all the results found and returns only those relevant to your specific interests - even if those results would have been pages deep on a normal search query. Whatever Primal Fusion retrieves can then be extracted to a web page, document, or RSS feed. At the moment, Primal Fusion only extracts to web pages - files and feeds will come later. The web pages created by the service are public sites representing your research around a particular topic and are filled with links and images relevant to your query.

Because Primal Fusion comes off as somewhat of a confusing mind-mapping tool, many folks will probably miss the point: Primal Fusion is infrastructure, not an application. The way it understands the relationships between words and phrases and how it can then extract the most relevant search results based on that understanding is what's most important about the company's technology.

Remember: This Is Only the Beginning

If you go out and try most of the services we've profiled in this series, you might walk away feeling a bit disappointed. You'll probably be thinking of all the things the service doesn't do but that you wished it could. Or perhaps you'll find the UI unappealing or the recommendations provided somewhat incomplete. However, It's important to understand that many of these services aren't ready for mainstream use just yet. Instead, they represent the beginnings of tomorrow's web - a web that better understands the data it contains. And by better understanding itself, the new intelligent web of the future can then better understand and serve you.

According to MIT's Technology Review, the program allows users to select email recipients by creating a search query as opposed to typing in names, addresses, or the name of a mailing list. For example, a user could use SEAmail to send a message to a group - created on the fly - like "all professors who graduated from Harvard University since 1960."

Being able to pinpoint your recipients in this way would not only be helpful, it could also stem the overflow of email that creates information overload by making sure that only the exact recipients who need to get the message are contacted.

A Semantic Email Chooser

In SEAmail, a email addressing interface (as shown here) could be used to build a list on the fly, without necessarily needing to know a single name. Instead, all an email user would need to do is fill in the parts of the form using the drop-down boxes to guide their selections along the way. The system would then take care of the rest.

Pulling from Databases

Obviously, in order for a semantically-based email system to work, there needs to be a rich database on the back-end that contains relevant information about the people sending email to each other and their interests. In the Stanford tests, this data is being retrieved from already existing databases which are then integrated into the system.

While such a thing may work well at a university like Stanford, it may be less useful to real-world businesses where there aren't numerous existing databases to draw upon, only Active Directory or some other resource management system.

Drawbacks and Concerns

1) Does this solve real problems?

There's also the small question as to whether SEAmail is solving a problem that really needs to be solved. Take for instance one of the touted benefits of the system: name resolution. The article provided an example where people wanting to send a message to "Michael Genesereth" could simply type his name as a recipient, and his most recent email address would automatically be selected. Sounds excellent, right? Except for one minor problem - that technology has existed in most email systems, and certainly within Microsoft Exchange, for years on end.

To continue the comparison with Microsoft Exchange, the advancements SEAmail makes have more to do with putting the power of creating these queries into the hands of users, who often don't get involved with the business of creating distribution lists, leaving it up to I.T. to do it for them. Exchange's built in ability to create query-based distribution lists already make it dead simple for admins to create lists based on almost any requirement you can dream up: city, state, organization, company, department, title, floor, supervisor, etc. In order for SEAmail to be revolutionary, it would need to do more than make it easy to create relevant lists - it would need to so while using much more detailed data about the intended recipients than any system allows for today.

2) Or Does it Create More Problems (like spam)?

Although in theory, the system's ability to pinpoint users could cut back on unintended email, there's also the possibility that the system could lead to more email. Oren Etzioni, director of the Turing Center at the University of Washington, has some concerns about the potential for misuse of the system. "This technology has clear benefits, but it's also ripe for misuse," he says. "The technical issues are solvable. The tricky things are the social issues. How do we create a workable system, given the vagaries of human nature?" Etzioni worries that a system that makes it too easy would lead to some people receiving overwhelming amounts of mail and no good way to limit it.

3) What about Social Networking?

Technology analyst Craig Roth thinks that the most glaring issue here is that the system doesn't take into account how social networks are being used by those who need to contact, market to, or inform others. He notes that consumers today can use LinkedIn, Facebook, MySpace, and Xing while enterprise customers have IBM Lotus Connections, MySites in Microsoft SharePoint, and the social search capabilities that were in BEA Pathways were merged into Oracle's secure enterprise search.  

Arriving Later this Year

Of course, it's hard to get a real feel for the potential of a system like this until it's actually put into practice. As it turns out, that will happen very soon. SEAmail will be launched at Stanford later this year, initially in the computer science department. It will later be rolled out to the rest of the university over time.

That dynamic backend technology - semantic markup - adds additional context to data about the artist which can include anything from previous bands, past collaborators, venues played, and more. The metadata is then linked together to create relationships that you may not have even known about before. 

Most of the information for the project comes from MusicBrainz, an open content music "metadatabase" that lists information for over 400,000 artists. To make a BBC music page, the contextual information surrounding the artist is imported to their BBC page. By using the artist's "MusicBrainzID," web page creators can integrate the artist's Wikipedia biography, too. Reusing this content is a better use of their time and energy, says Shorter, because the content is already available on the public domain.

As more projects like this take advantage of the publicly available metadata available, the beginnings of a real semantic web can finally take root.