Alexander Korth - ReadWriteWeb

The Web of Services: Machine-Accessible Services

Alexander Korth — Fri, 16 Oct 2009 12:00:39 -0800

In the last two posts in this series, we discussed the Web of data, which makes structured interlinked data sets machine-accessible, and the Web of identities, which makes data about people machine-accessible while addressing privacy and data volatility.

This time, we'll focus on the Web of services, which makes services accessible to and processable for machines. These Webs all have a semantic architecture in common and follow basic Web principles, such as being decentralized, modular, simple, addressable via URIs, and built for machines.

]]> Sponsor

]]> The services sector has become the world's biggest business sector, accounting for 64% of the worldwide gross domestic product. The sector has pressure on it to make its services easier and more widely accessible, as well as to quickly adapt to ever faster changes in the market environment.

The effort to standardize such things as service-oriented architectures (SOA) and Web services has taken years, but still we have no clear definition of what constitutes a service at a conceptual level. The interface, which is the format of what goes in and out of the service, is often described formally, but what the service is actually doing, semantically speaking, is not. While there are a number of different approaches to semantically describing Web services, such as OWL-S, WSMO and WDSL-S, none so far has managed to break out of its academic confines.

Today, there are already all kinds of services with different levels of complexity, and their number is expected to grow exponentially. The services follow different standards, and a lot of them are proprietary, uni-directional and designed to be used by humans to mash up something new. Editorial catalogs such as ProgrammableWeb and search engines for Web services such as seekda are designed for humans who are searching for a particular service for that reason. For tasks that are unsolvable for machines, there are even Web services such as Amazon's Mechanical Turk, which have humans in the back end answering tricky queries.

The problem with all of this is that each of the tens of thousands of services is accessible but not findable by a machine without a machine-understandable description. Thus, every service nowadays has to be wired to a machine by hand. So, what would machines be capable of if services were annotated with semantic descriptions?

Service discovery
Given an index of Web services, a machine charged with finding the right service for a particular problem could choose one among those that have been indexed.
Contracting and execution
Once a service has been selected, a machine could look up its terms and decide on contracting and execution details. How often would the service be needed? And what would be the cheapest contract then?
Billing or revenue sharing
Depending on the autonomy of the machine, one could imagine something like an Autonomous Agent, which automatically makes the best deal with the service provider on such things as billing or revenue sharing for service usage.
Replacement on failure, based on experience
Of course, the machine would be able to replace a failing service with an equivalent one. It could also rate a service and publish it.
Service orchestration
A machine could, given enough intelligence, split a task into sub-tasks and then discover, contract and orchestrate services to solve these sub-tasks. And after the sub-tasks have been addressed, the main task would be solved. Such orchestration could involve the parallelization of tasks, for speeding up or redundancy purposes, or chaining services (whereby the output of one service is inputted into the next).

Research projects such as TripCom, SUPER, SHAPE and SOA4All are dealing with these ideas and scenarios.

Future scenarios are limited only by our imagination: machines could autonomously pursue goals on behalf of their master user or company, according to a specified level of freedom. These agents could solve increasingly complex problems and be granted increasingly more autonomy (finally ending up as Skynet).

In the next and final post in this series, we will discuss how all of these scenarios could become a reality with the arrival of all three Webs: a revolution in the ability of machines to access, process and apply information.

Do you also count the Web of services as a third Web? Where do you see its limits?

(Photo by zorro-art.)

]]> Discuss

The Web of Identities: Making Machine-Accessible People Data

Alexander Korth — Sat, 11 Jul 2009 14:04:57 -0800

In a previous article, we discussed the Web of data, which is about inter-linking open data sets and, thus, turning them into machine-accessible structured data. In this post, we'll draw a picture of how the emerging social Web could serve as a Web of identities, which is essentially a people-data version of the Web of data.

]]> Sponsor

]]> W3C's Linking Open Data (LOD) project has gotten quite a bit of attention for the good job it does with the Web of data. Currently, all participating data sets are accessible free of charge and can be used without constraints. The project focuses on growth for now. In an email, Chris Bizer hinted that a payment model to charge for particular content may come in future.

The LOD approach is very good for static and encyclopedic knowledge, but what about accessing our personal data? Technically, modeling our identity, profile data, social graph, groups, activity stream, assets, and other kinds of personal data is straightforward. But empowering machines to access this data could present challenges to the LOD approach, because it comes with all sorts of constraints and peculiarities, such as privacy and data volatility. People want control over who has access to their data or parts of their data and want to be able to block access for any reason. And issues such as rapidly changing and outdated data remain unaddressed.

This is where the social Web can help.

The Emerging Social Web

There was a time when we had to create a new digital identity for each social application we wanted to use. A social application provides features based on social attributes. Every application provider implemented its own proprietary ID management to authorize users to log on and implemented its own proprietary user profile system to manage information about its users. Application providers were judged by the size of their user and content base and so erected endless walled gardens to protect their properties.

The most significant issues people had were:

Low conversion rate for user registration,
Users had to register for many accounts,
Users had to re-enter and synchronize profile data,
Privacy, data ownership, and inability to export.

Not much has changed, unfortunately. Most remarkable, perhaps, is the growing number of single sign-on (SSO) solutions that address the first issue for application providers and the second issue for users. New application providers can now outsource this functionality to a third-party SSO provider. Some of the biggest application providers became ID providers themselves to allow their users to log on to third-party applications with the same ID, and this has gained traction beyond these few providers. This has led us to an era of identity wars between the big providers.

Many ID providers, such as Google, Yahoo!, MySpace, and Facebook, have added the OpenID SSO to their own proprietary mechanisms over time. Because of the open nature of OpenID, many third-party providers have found it easy to integrate with the bigger providers, giving them more traction because users are able to access their services so easily using their OpenID credentials. Now, these ID providers can offer read-only access to fragments of profile data that users can look up or copy to third-party applications. Like SSO and OpenID, this began with proprietary solutions, but now exchange formats and protocols are emerging whose open language allows applications to easily exchange and synchronize data. These include:

API access authorization protocol OAuth,
Social graph exchange format FOAF ("friend of a friend"),
Updates exchange format activity streams,
Address book exchange format Portable Contacts.

In the future, ID providers will loosen their connection to social applications and start taking over management of users' social attributes. Users will be able to log in to applications using credentials hosted by their ID providers of choice and grant permissions to these applications to read or even sync selected fragments of their profile data. The borders of these walled gardens will thus blur, and the social Web will become more of a weave than a patchwork quilt.

The Web of Identities

The Web of data is a distributed web of interconnected sets of semantically annotated data. A connection is achieved as a result of data pointing to data contained in another set through a URI, just as websites point to each other with URIs. This way, machines can crawl the sets to read the data. ID providers will most likely refer to their users via URIs in the future as well. A social connection will consist of one user's URI pointing to another user's URI or ID provider. If permitted by users, a machine may very well accomplish its tasks by jumping through the Web of identities from user to user, the way it does through the Web of data.

Why is this needed? The Web of identities is actually a super-social graph that spans multiple ID providers. If we come across walled gardens, this infrastructure would be needed for all of the social-related search functions we perform. The following examples are thus far provided only (if at all) within individual applications:

"What is the best book read by friends in my circle?"
This query might retrieve book purchases and book-related status updates that your friends have made accessible through their privacy settings and then rank the books in a set.
"Notify me if a close friend visits Berlin."
This permanent task repeatedly looks up your friends' geo-locations. You may also have granted your close friends access to this data, too. This task could even be combined with the Web of data to look up the meaning and location of Berlin.
"Sync my address book."
This permanent task continually synchronizes my friends' addresses and numbers with my personal address book.

Now it's your turn. In what ways do you think the social Web and Web of identities are evolving?

(Diagrams by alexkorth)

]]> Discuss

The Web of Data: Creating Machine-Accessible Information

Alexander Korth — Sat, 18 Apr 2009 10:00:00 -0800

In the coming years, we will see a revolution in the ability of machines to access, process, and apply information. This revolution will emerge from three distinct areas of activity connected to the Semantic Web: the Web of Data, the Web of Services, and the Web of Identity providers. These webs aim to make semantic knowledge of data accessible, semantic services available and connectable, and semantic knowledge of individuals processable, respectively. In this post, we will look at the first of these Webs (of Data) and see how making information accessible to machines will transform how we find information.

]]> Sponsor

]]> The amount of information and services available is growing exponentially. Every day, it is getting harder to find the information we are actually looking for. Still, we have to learn how to tell machines what we want. Why can't a machine understand which website, recent tweet, Flickr photo, Facebook message, or restaurant we are currently looking for?

Because it can't. It does not understand. It has no access to most sources. It lacks the semantic understanding and common sense to build bridges between information.

It is critical that machines gain a new level of understanding. Instead of statistically computing how well a search term matches a document, a machine must literally be able to understand. Therefore, knowledge bases are needed to look things up. Examples of these knowledge bases include:

an encyclopedia containing knowledge to look up the semantic meaning and context of a particular term (e.g. to understand that Berlin is a city, how many people live there, and where it is),
Yellow Pages or a service pool to query often-changing and more complex information (e.g. a route from Berlin to Porto by car, or the current temperature of Porto in Celsius),
a people database to look up profile information, with user permissions, which could improve personalization and recommendations.

The Web of Data

The idea of the Web of Data originated with the Semantic Web. People tried to solve the problem of the inherent inability of machines to understand web pages. Initially, the aim of the Semantic Web was to invisibly annotate web pages with a set of meta-attributes and categories to enable machines to interpret text and put it in some kind of context. This approach did not succeed because the annotation was too complicated for humans who had no technical background. Similar approaches, like microformats, simplify the markup process and thus help bootstrap this chicken-egg problem.

These approaches have in common the effort to improve the machine-accessibility of knowledge on web pages that were designed to be consumed by humans. Furthermore, these sites contain a lot of information that is irrelevant to machines and that needs to be filtered. What is needed is a knowledge base for machines to look up "noiseless" information. But wait! Who said that machines and us humans need to share one web anyway?

The idea of the Web of Data came about as a result of both this limitation and the existence of countless structured data sets distributed all over the world and containing all kinds of information. These data sets are the property of companies that trend to make them accessible. Typically, a data set contains knowledge about a particular domain, like books, music, encyclopedic data, companies, you name it. If these data sets were interconnected (i.e. link to each other like websites), a machine could traverse this independent web of noiseless, structured information to gather semantic knowledge of arbitrary entities and domains. The result would be a massive, freely accessible knowledge base forming the foundation of a new generation of applications and services.

Linking Open Data

One promising approach is W3C's Linking Open Data (LOD) project. The above image illustrates participating data sets. The data sets themselves are set up to re-use existing ontologies such as WordNet, FOAF, and SKOS and interconnect them.

The data sets all grant access to their knowledge bases and link to items of other data sets. The project follows basic design principles of the World Wide Web: simplicity, tolerance, modular design, and decentralization. The LOD project currently counts more than 2 billion RDF triples, which is a lot of knowledge. (A triple is a piece of information that consists of a subject, predicate, and object to express a particular subject's property or relationship to another subject.) Also, the number of participating data sets is rapidly growing. The data sets currently can be accessed in heterogeneous ways; for example, through a semantic web browser or by being crawled by a semantic search engine.

To get a feeling of how this machine Web of Data feels like, you may want to look up:

the company Yahoo! on CrunchBase,
the city of Berlin or the game Tetris on DBpedia,
the book iPhone: The Missing Manual on O'Reilly Media.

With every fact available on the Web of Data, more general and specific knowledge is made accessible to machines that will enable a whole new generation of services to be created. Highly sophisticated queries become machine-processable and accessible to the next generation of, say, search services.

Check out Tim Berners-Lee's talk at TED about the Web of Data. How do you think about it? Do you encounter the same issues being overloaded by information or too much noise?

(Photo by zorro-art. Graph by the Linking Open Data project.)

]]> Discuss