This week I spoke with Barney Pell, CEO of Powerset; and Melek Pulatkonak, COO of hakia. In both (separate) conversations we discussed how the Semantic Web is getting very close. The Semantic Web as defined by Tim Berners-Lee is: "a universal platform for the exchange of data, information and knowledge." I think Barney and Melek would agree, that the only thing preventing the Semantic Web so far has been an inefficient use of horsepower - or a lack of it.
Semantics is expressed meaning in language, code or "other" representations of information. My discussions with Barney and Melek revealed the fundamental differences in architecture and philosophy between hakia and Powerset. The index systems of the two companies are fundamentally different, as is their philosophy - but their goals and visions are remarkably similar. They are also different in the way they apply what I term horsepower to natural language search. Like the symbolism of Shelby vs. Ferrari,– it is possible for different approaches to achieve a desired result - given enough horsepower.
Hakia has built their search in-house, refining and sculpting the QDex indexing system (like an Enzo Ferrari). Their view is that processing power should be maximized with super efficiency, via fuzzy logic and advanced semantics. Powerset, on the other hand, utilizes basically the same inverted indexing system as Google - but backed by natural language and immensely powerful processing that essentially “overpowers” the long tail query (like the GT 500). This is a vast oversimplification, but the elements involved reveal the larger story.
Technology (horsepower), communication (language) and people make up the semantic Web. The Web has not been lacking "language", but the adequate application of processing power. As Barney said: "Even five years ago we did not have the processing capability to even attempt this, but five years from now these answers will seem elementary." Google's system below, currently consumes massive horsepower with comparatively limited results - at least according to hakia and Powerset!
Diagram of Google's inverted index and search (courtesy -changturtle)
Barney described the relationship between people and computers as people being "bent" around or adapted to technology in order to utilize it. With the advent of services like Facebook, programs and applications are beginning to “understand” each other. Everyone reading this has been “forced” by technology to conform to varied “bending events”, in order to use it. Barney explained this idea by calling Facebook and the iPhone true innovations approaching total “community engagement.” Barney also said that “Facebook will become one of the primary communications platforms of the future.” Given this new perspective, I could not agree more because Facebook is one heck of a representation of information for a social network. Essentially, hakia, Powerset, Facebook and others are bending the machines to engage humans. And in a way, Facebook is the semantic Web in a microcosm - but in it's infancy.
Search is a critical part of our daily lives, but the interface has changed very little over the years. We define search as the act of typing in a query on Google and getting results. This is a type of search, but how many other kinds of “searches” do we perform? In an earlier article, Josh Catone wrote about Yahoo!’s contention that search will not determine the future of the Web. Josh rightly asked if Facebook and MySpace might be better positioned if “personalization” was to be the future of the Web.
I should make it clear that neither Barney nor Melek really consider themselves as "Google Killers". Powerset and hakia are not in a race either against each other or to overtake Google, but they are on a quest for better Web communication and engagement. Both efforts emphasize the necessity for “the system” to be able to universally understand and handle data without ambiguity. Viewing Facebook and others as functional repositories of semantic data is essential in seeing the long view. Whether we are talking about object oriented data, textual semantics or complex algorithms, the semantic Web is about making people “bend” less for technology.
Listed below are links to blogs that reference this entry: Powerset and hakia - Quest For The Semantic Web.
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/1467
Let‚Äôs start from the premise that Search is ‚Äúgame-over‚Ä?. Google has won. The best explanation of Google‚Äôs dominance is in this Read/WriteWeb post, ironically from someone who works for a Google competitor. But if there is a Google-killer out there,... Read More
Comments
Subscribe to comments for this post OR Subscribe to comments for all Read/WriteWeb posts
Interesting. I tried a few terms at Hakia.com and their results are so different from Google, and even Yahoo! This is a manifestation of the interminable challenge the online businesses are going to face: how to consistently appear on every new and old emerging search engine and widget.
Posted by: Amrit Hallan - Content Blog | July 20, 2007 3:21 AMDude the Semantic web is already here, have you not checked out freebase.
They are only in alpha but the stuff that they have done so far is amazing, because they are building a framework from the ground up and making it accessible to everyone for free. This in the long run will kick Google’s ass because big corporations can’t keep up with the open source community.
Check out Cinespin it has to be one of the coolest app on the net. That was created by just one user just using films just think what else can be created.
Posted by: john | July 20, 2007 5:33 AMI came away from this post feeling that the Semantic Web could somehow develop in a vacuum.
Hakia, Powerset, Freebase etc. have all jumped the gun. As I understand it, the Semantic Web calls for a fundamental rethink and rebuild of the Web which as we all know has not happened.
For the kind of universality intended, in my opinion, any vacuum-based approach is towards achieving some other goal and not the true Semantic Web...
Posted by: Adrian Keys | July 20, 2007 6:10 AMPhil, you are saying: "Powerset and hakia are not in a race either against each other or to overtake Google." But aren't both trying to better the same thing Google does? There are a few other similar semantics/NLP-based efforts out there, and rest assured that Google is not asleep as well. As long as there is no real paradigm shift, it still looks like a competition in the same pool.
Posted by: Dmitri | July 20, 2007 7:29 AMGreat article Phil, and an interestingly mixed reaction to the "semantic web" from your readers.
I don't think there's a problem with semantic web oriented projects developing in a vacuum. Some of us are working on platforms which will connect the everyday web user to these semantic tools in a way that they feed off each other, and so that no one has to stop innovation going on around them.
Keep it up Hakia and Powerset. Your innovation is needed!
Posted by: Nate Westheimer | July 20, 2007 10:21 AMNot so sure freebase is the semantic web. They are really providing a mechanism for structured data to be stored consistently. The challenge of a semantic search system is to structure the currently unstructed mess that is most web pages - the content people want to find. Freebase doesn't do semantical language analysis like hakia and powerset are trying to do.
The semantic web is really a reflection of the structuring of loose data. That'll come from two directions: web sites wanting to do it (formatting their information with microformats is the most likely course I think) and semantical conversion engines, like these two, trying to brute force that conversion. Ultimately you need both, but it's going to take a while to see real results.
Posted by: Martin Wells | July 20, 2007 4:13 PMThanks for the great comments. I cannot respond better than Marty did about the essence of what these (and other) semantic engines are doing to connect us.
Both Riza and Melek mailed me to tell me I did a good job of delivering the essence of their vision ( they are too kind really) - these ideas are as difficult to reveal as the are to actualize really.
Powerset and hakia are not "creating" a semantic web - there is already a great deal of meaning here. The inter connective fabric is what is missing. The semantics are crude like the language of a child. The kind of capability these companies (and as Marty mentioned the counter dynamic of sites improving semantics) is Web 3.0 or whatever we want to call it.
This type of discourse is part and parcel the engine that will drive this thing to the next level.
Posted by: Phil Butler | July 20, 2007 6:18 PMThe idea with the sports cars is really funny. But you are kidding about the simple google diagram, aren't you? LOL!Dugg!
Posted by: Mihaela Lica | July 21, 2007 12:48 PMI see that you are interested in semantic web. You should take a look at www.linguisticagents.com. It’s a start-up company that has developed a natural language understanding technology that will be used in many applications in addition to search. This technology uses a deep parsing algorithm that is based on nano-syntax technology.
Posted by: Andy | July 25, 2007 2:33 AM