That a serious trend is on the rise would not be doubted by those watching Twitter's rise in usage and media popularity. In fact, the debate this summer has centered not on whether something is afoot but rather on what to call it. Ron Conway favors "now media" in the belief that it's a media phenomenon. But most commenters, led by several bloggers and lead investors, prefer to call it "real-time Web" ("real-time stream" is also popular).

The trend is not with Twitter alone. Just as the social Web was more than Friendster, then MySpace, and now Facebook, the real-time Web is more than just 140 characters bursts about what your friends and acquaintances are thinking and doing. The number of people using Twitter (44.5 million in June 2009) and the purchase of FriendFeed by Facebook for $47.5 million are eye-opening figures alone, but the number of independent developers building on top of and alongside the Twitter platform make it something worthy of close attention. Unlike the social networking space, these are not "Me too" networks or a mess of widget applications. The depth and breadth of the problems that independent developers are addressing are clear evidence that a serious trend has formed, one with significant implications for both the technological infrastructure of the Web and for the companies that rely on the Web.

As with other recent waves of innovation (Web 2.0 and cloud computing, for example) there is no single definition of what the term "real-time Web" means. As a result, it is used as a catch-all phrase for a number of developments underway. At this point, we can identify that the real-time Web...

1. is a new form of communication,
2. creates a new body of content,
3. is real time,
4. is public and has an explicit social graph associated with it,
5. carries an implicit model of federation.

A New Form of Communication

One obvious way of looking at Twitter is as a new form of communication, with its own protocols and ways of doing things, and with similarities to instant messaging (IM) and email. The timing of communications on it is near synchronous (i.e. it is a continuous stream of up-to-date messages), and its tone is conversational and authentic (marketing messages and ghost tweeting are frowned upon, at least for now). Like IM, email, texting, and even the telephone, technical constraints (in this case, a 140-character limit) create a rather special conversational structure, giving Twitter-speak its own distinct mode of communication.

This arbitrary limit also simplifies its usage, which has had a tremendous effect on the adoption rate. For one, the limit makes it easy for adopters to pick up the general etiquette, and thus the barrier to participate is much lower than it is for most things tech. Plus, the technology threshold is low: no authoring software or templates (as one might have for a blog) are needed, nor do you have to create an extensive profile, as you do with social networking. Figures suggests that many independent bloggers, especially in technology, media, and political sectors, have dramatically reduced their blogging schedules, preferring instead to publish their thoughts, or refer to the thoughts of others, in 140-character segments throughout the day, rather than regularly come up with 500-word blog posts.

One consequence of the 140-character limit (and a key reason why the real-time stream is so easily adaptable to other uses) is that messages are largely atomic in nature. Each refers to an individual item: a thought, link, event, product, person, or company. They also typically contain some evaluative or emotional component, such as "Look at X because it's cool or interesting" or "I support Y" or "I disagree" or "this is no good."

This discrete nature of Twitter messages means that items, and the sentiments attached to them, can be extracted and then aggregated, allowing us to measure the activity being generated around a particular subject and, in some cases, the general feeling about it. This is not unlike the way buy-and-sell orders signal interest in a stock; but in this case, any popular subject matter can be tracked. In combination with other factors, this discrete nature and emotional component allow for some interesting usage and applications.

This deconstruction of content is not limited to Twitter. The movement to expose underlying data and make it more actionable is gaining momentum across industries and platforms. One example is the move to report financial data in XBRL format (eXtensible Business Reporting Language). Another is the growing use of microformats and RDFa, which are small patterns of HTML that represent data on commonly published subjects on Web pages, such as people, events, blog topics, reviews, and tags. Twitter's character limit and accessibility, however, are the simplest and most recognized example of how elements of connected data can provide value both individually and in aggregate.

The Power of Constraints

One of the more magical aspects of Twitter is that it reminds us that arbitrary constraints can have a liberating and profound effect on creativity. It sounds counter-intuitive, but coming up with a host of examples doesn't take long. If a group of high school students were given the choice between writing an essay about their summer vacation or writing a 300-word essay on the funniest thing that happened in the last three weeks, we could easily guess which would get their pens moving faster and lead to more imaginative results.

Poetry shows a similar relationship between constraint and inspiration. Whether having to keep to a certain meter or follow a particular rhyming scheme, poets come up with turns of phrases and ideas that they might not have otherwise happened upon if they did not need to fit words into a pattern. The limitations also give them license to play with language in a way that would not make sense or be valued in other modes. The same goes for music, with its meter and form, and even TV shows and movies, whose time restrictions and story constraints can make for enjoyable, funny, scary, or moving experiences.

A New Body of Content

Another characteristic of the real-time Web is that it gives the world a new body of content, one that, unlike IM or email's, is largely public. Plus the underlying APIs allow third parties to make use of the data through programs, thus extending the reach of the content. (Only 20% of Twitter traffic comes from the site itself. The other 80% comes from users accessing the platform through APIs.) Ron Conway and other proponents of the real-time Web see this new body of content as a great opportunity for investment, with the potential for companies to shape, extend, present, and amplify it in any number of ways.

On the surface, people consume this body of content simply by reading messages from people they follow. Much like a stock ticker, these messages scroll across whatever client they use to access it. When it first launched -- without the scale, celebrities, and business leaders -- many people failed to see the value of this mode of interaction. It was interesting, but not compelling. Now with its scale, the personalities using it, and a better general understanding of how to use it (less about your breakfast, more insight, reaction, and commentary), the channels have become fascinating -- overwhelming if you follow a lot of people, but fascinating nonetheless.

The tipping point in Twitter's adoption rate came when its stream became searchable. This happened in July 2008, when Twitter purchased a tiny search company called Summize and renamed it Twitter Search. The acquisition made it easier for users and third parties to pull specific words and tags from the Twitter stream.

This new capability revealed another layer of value, because it enabled people to access particular threads of information. Users could now search for other users, words, and specific topics. An example from earlier this summer was the aggregation of the steady stream of messages about the green revolution in Iran. With APIs, we could create filters to keep constant track of a person, item, or topic. Real-time search and filtering are still primitive, though, and a tremendous effort is being made to improve them. As one investor puts it, a lot of investment is being made "to build filters that give you only the portion of the firehose that makes sense to you."

Content in digital format is not really new. We saw this with early Web pages, then MP3s, blogs, videos, social network profiles, and so on. The difference is accessibility. Web pages have to be crawled and indexed, which limit the derivative use and retransmission of their data. RSS provided a revolutionary way to syndicate content and made it much easier to process by machine. The accessibility of the Twitter stream via APIs extends this syndication idea even further by providing much greater immediacy and fidelity. As Twitter and third parties introduce better filtering mechanisms, that stream and, by extension, other content formats on the Web will be able to be more effectively harnessed and extended.

Inside Baseball Twitter

More advanced uses of Twitter, such as retweeting, direct messaging, and thread tagging, make it a bit more of an insider's game, but even their limits makes them not all that difficult to pick up. The interesting thing is that these uses (RT for retweet, @username for mentions, and #keyword for hash tags) can be followed mechanically and used to capture these derivative streams.

Read part 2 of this series.

Guest author: Ken Fromm is a serial entrepreneur who has been active during both the Internet and Web 2.0 innovation cycles. He co-founded two companies, Vivid Studios, one of the first interactive agencies, and Loomia, one of the top recommendation, discovery, and personalization companies. He has worked at the leading edge of recommendations and personalization, interactive development, e-commerce and online advertising, semantic technologies and information interoperability, digital publishing, and digital telephony. He is currently advising a number of startups and looking at the next big thing in Web 3.0. He can be found on Twitter at @frommww.