real time - ReadWriteWeb http://www.readwriteweb.com/feeds/tag/real time en Copyright 2009 Richard MacManus readwriteweb@gmail.com Sat, 21 Nov 2009 05:00:00 -0800 http://www.sixapart.com/movabletype/?v=4.23-en http://blogs.law.harvard.edu/tech/rss YouTube Launches Real-Time Discussion Search and Tracking Real-time information is red hot all around the web but it made a surprise appearance on YouTube tonight in the form of real-time search for comments, of all things. YouTube comments are notoriously not worth reading, but now you can search their full text...in real time. There are some very real, potential use-cases crying out for a tool like this. Companies in particular are likely to want to know what people are saying about their names in the comments on YouTube. You name your topic, though: it's now available for real-time search across viewer discussion.

Real-time search appears to have been rolled out very recently, with no mention, on this page. In addition to search results continuously updated ala Facebook's newsfeed ("3 new results") there's also a frequently-updated list of "trending topics" on the search page.

]]>Sponsor

]]> youtuberealtime.jpgUnfortunately, there are no feeds being published to syndicate these search results into a reader off-site. The regular search on YouTube now has RSS feeds and Google Wonder Wheel data being published, so perhaps comment search will have feeds added soon as well.

Proper nouns will likely be of interest to searchers watching YouTube comments. This could be a popular addition to the toolkits of social media watchers everywhere.

What's the benefit of serving those results up in real time? For certain search queries you don't want to wait around to find out there's new results.

What could be next? Presuming this feature is as real as it looks and goes live to the public soon, we'd love to see YouTube support something like the Salmon comment aggregation protocol and publish updates for this and other GData feeds through in a real-time syndication format.

Thanks to Tikva Morowati for the tip. Tikva is the Community Platform Director for KGBWeb, a stealth startup made up of ex-Googlers and others in New York City that will likely make a splash among web-watchers later this year.

]]>Discuss]]>
http://www.readwriteweb.com/archives/youtube_launches_real-time_discussion_search_and_t.php http://www.readwriteweb.com/archives/youtube_launches_real-time_discussion_search_and_t.php News Fri, 16 Oct 2009 03:39:27 -0800 Marshall Kirkpatrick
TinyChat Goes P2P - Leverages Adobe's Real Time Media Flow Protocol tinychat_logo_may09.pngTinyChat, the popular Twitter-centric video chatroom solution, just launched a P2P-enabled version of its service. While the regular TinyChat routes its videos through the company's servers, the P2P version uses the Real Time Media Flow Protocol that Adobe builds into the Flash platform and Flash Player 10. As these video streams require a lot of bandwidth, this current version is limited to two active participants per room. For now, this version is more of a demo than a full-blown product, though the company plans to roll it into the regular TinyChat experience in the next few months.

]]>Sponsor

]]> Adobe introduced this P2P technology, the Real Time Media Flow Protocol, last December, but developers haven't really latched on to it yet as enabling this technology is rather involved.

Flash-based P2P Video

tinychat_homepage_small_oct09.jpgFor TinyChat, the ability to route these videos around its servers means reduced bandwidth costs. Once this feature becomes part of the default TinyChat setup, only calls with more than 2 participants will have to go through the company's servers and as TinyChat's founder Dan Blake told us earlier today, the experience of switching between the P2P chat and the server-based version should be completely seamless.

Unlike other P2P solutions, TinyChat is able to leverage a plugin that virtually all users already have on their machines and users don't have to download another plugin. All a user needs is a webcam and a microphone. Currently, when you are chatting on the P2P server, your privacy is also protected, as the service simply won't allow a third user to listen in or join the call.

]]>Discuss]]>
http://www.readwriteweb.com/archives/tinychat_goes_p2p_-_leverages_adobes_real_time_med.php http://www.readwriteweb.com/archives/tinychat_goes_p2p_-_leverages_adobes_real_time_med.php News Fri, 02 Oct 2009 13:30:56 -0800 Frederic Lardinois
Sponsor Post: The Limits of Tweet-Based Web Search Editor's note: we offer our long-term sponsors the opportunity to write 'Sponsor Posts' and tell their story. These posts are clearly marked as written by sponsors, but we also want them to be useful and interesting to our readers. We hope you like the posts and we encourage you to support our sponsors by trying out their products.

Many of the recent real-time search engines are based on Twitter. They use the URLs enclosed in tweets to discover and rank new and popular pages. In this post, we'll take a look at the quantitative structure of the underlying foundation, to determine the feasibility and limits of this approach. We'll also look at how to overcome these limitations by using the implicit Web.

]]>Sponsor

]]> You may have seen recently the interesting visualization of Twitter statistics. It essentially proves that, as with other social services, only a small fraction of users actively contribute.

But it also shows another fact: that those people who contribute publish an even smaller fraction of the information they know.

Both of these factors account for the huge difference in efficiency between implicit and explicit voting. Explicit voting, as the name implies, requires users to actively express interest in a page; for example, by tweeting a link. Implicit voting requires no deliberate action on the part of the user; a simple visit to a Web page would count as a vote.

A Quick Calculation

Twitter now has 44.5 million users and delivers about 20,000 tweets per minute. If every second tweet contained a URL, that would be 10,000 URLs shared per minute.

According to Nielsen, the number of visited Web pages per person per month is 1,591.

Twitter's 44.5 million users visit 1.6 million Web pages per minute and explicitly vote for only 10,000 per minute. That is to say, implicit voting and discovery generates 160 times more attention-getting data than explicit voting.

This means that 280,000 implicit votes could provide as much information as 44.5 million explicit votes. Put another way, as many Web pages are implicitly discovered during one day as there are Web pages explicitly discovered during half a year.

This dramatically shows the limits of Web searches based solely on explicit votes and mentions, searches whose potential could be leveraged by using the implicit Web.

Beyond the Mainstream

This becomes even more important if we look beyond mainstream topics and the English language. Then it becomes simply impossible to achieve the critical mass of explicit votes needed to have statistically significant attention-based ranking or popularity-based discovery.

Time and Votes Are Precious

Time is also a crucial factor, especially with real-time search. We want to be able to discover new pages as soon as possible. And we want to assess almost instantly how popular those new pages are. If we fail to reliably rank a page quickly, it will get buried in the noise. But the goals of speed and votes conflict with the fact that the number of votes a page gets is inversely proportional to the time it took to be viewed.

Again a much higher frequency of implicit votes would help.

Relevance vs. Equality

We could also improve on explicit votes. But we should not treat them as being equal because they are not. We trust some of them more than others, and our interests overlap with some more than others, for the very same reason that we follow some people and not others. This helps us get more value and meaning out of that very first vote.

FAROO is moving in this direction by combining real-time search with a peer-to-peer infrastructure.

A Holistic Approach

Discovering topical, fresh, and novel information has always been an important aspect of search. But the perception of what "recent" is has changed dramatically with the popularity of services such as Twitter, and it has led to the emergence of real-time search engines.

Real-time search shouldn't be a silo, but rather should be part of a unified and distributed approach to Web search.

The era of purely document-centered search is over. The equally important roles of user and conversation, both as targets of search and as contributors to discovery and ranking, should be reflected in the infrastructure.

A Distributed Infrastructure

As long as both source and recipient of information are distributed, then the natural design of search is distributed, too. P2P offers an efficient alternative to the ubiquitous concentration and centralization of search we find today.

A peer-to-peer client allows every visited Web page to be implicitly discovered and ranked according to attention received. This is important, because the majority of pages in a real-time search are in the long tail. They appear once or not at all in the Twitter stream and can't be discovered or ranked through explicit votes.

With real-time search, the amount of indexed data is limited, because only recent documents (those that have gained a lot of attention and a high reputation) are accounted for in the index. This allows for a centralized infrastructure at a moderate cost. But as soon as search moves beyond the short head of real-time search and aims to fully index the long tail of the entire Web, then a distributed peer-to-peer architecture provides a huge cost advantage.

]]>Discuss]]>
http://www.readwriteweb.com/archives/the_limits_of_tweet-based_web_search.php http://www.readwriteweb.com/archives/the_limits_of_tweet-based_web_search.php Sponsors Tue, 22 Sep 2009 05:00:01 -0800 RWW Sponsor
Where Is the Real-Time Web Message Bus? Real-time computing is not new. This is the third generation of real-time:

• First generation: was done on a single processor, usually for process control in military systems.

• Second generation: within a Local Area Network, usually for a financial trading room.

• Third generation: applied across the whole Web/Internet, what we call the real-time Web.

In each generation a stack has emerged, and secure messaging has been key to that stack. The names change and the scale of the prize and challenges certainly changes, but the basic issue remains the same: delivering messages reliably and quickly. In this post, we trace the steps from the second generation to the third generation to see how the real-time Web might play out.

]]>Sponsor

]]>

Generation 2's Winner: The Teknekron Information Bus Story

I worked with the technology's second generation. My company took on the challenge of delivering market prices from multiple sources to hundreds of traders in a trading room, enabling new apps based on real-time calculation. The solution had to scale to maybe 2,000 traders, which is laughably small by the standards of the real-time Web. But you did not want to face a roomful of furious traders if your system was down for three minutes or delayed prices by 15 seconds. The reliability and latency requirements were very stringent.

The company that emerged as leader, Teknekron, coined the term "Information Bus." It did very well by coining the term, because it established Teknekron as a market leader; it followed the "get mindshare first, and then go for market share" strategy. Vivek Ranadive, the visionary founder and CEO, describes in his 1999 book "The Power Of Now" how the term came to him. He was schooled as a hardware engineer and was appalled by how much more chaotic (i.e. error- and delay-prone) software projects were compared to hardware projects. Looking at why, he realized that the concept of a "bus," which is the device that all hardware components connect to, could be applied to software.

Ranadive understood that real-time, event-driven systems would not be limited to financial trading, and he executed brilliantly on that insight. He sold Teknekron to Reuters, the company's main rival in trading systems, but retained the right to use the technology outside of financial markets. That offshoot became TIBCO (The Information Bus Company), which at the time of writing is a publicly traded company valued at $1.6 billion, with annual revenue of over $600 million.

But TIBCO did not leverage its position to dominate the third generation. Another company tried that and blew a lot of money in the attempt.

KnowNow: Lessons of a Blow-Out

KnowNow worked through $50 million in funding before throwing in the towel in July 2008. It was going to update the message bus to the Web using RSS. It was still an enterprise play.

It had a strong management team, top-tier VC (Kleiner Perkins), and strong technology. Why did it fail? Three big reasons stand out:

  • Generation 3 is a tougher technical challenge. Generation 2's state of technology was perfect for local area networks using Ethernet, where the user device was always connected and where we measured users by the thousands. With generation 3, we are dealing with millions of users who are offline some of the time, and the network bandwidth available for messages is not under their control. In other words, it is one big bear of a technical challenge!
  • Companies are reluctant to invest millions of dollars in closed software when open standards clearly always win on the Web. Companies can instead experiment very cheaply using consumer-centric RSS tools and open source.
  • KnowHow was early. The hype on real-time enterprise did not last long enough, and it hit the trough of disillusionment.

Contenders for the Real-Time Web Message Bus

We have probably missed a few, so please tell us about them in the comments. These are very different types of solutions, but they are working towards a similar objective. First, we will list them and then attempt a bit of categorization:

  • Gnip: an independent venture whose monetization model relies on it being a hub that different sites can connect to.
  • Tornado (FriendFeed's Python-based Web server technology, which was open sourced by Facebook): this uses PubSubHubbub.
  • RSS Cloud: promoted by Dave Winer, with WordPress as a marquee partner.
  • PubSubHubbub: promoted by Google (and FriendFeed/Facebook), with SixApart as a marquee partner.
  • XMPP The technology with which IM clients interoperate. Being used by Yammer, Present.ly, and Drop.io.
  • Twitter.
  • Facebook.
  • TIB: an enterprise solution from TIBCO.
  • MQ: an enterprise solution from IBM.
  • Sonic: an enterprise solution from Progress Software.

Here are the categories they fall into:

  1. Enterprise
    TIB, MQ, and Sonic all fall into this category, and there are more. They will find it hard to make the transition to the real-time Web for two reasons. The first is technical. Connecting millions of users over a low bandwidth network via HTTP is very different from connecting a few thousand users over a corporate network. The second is commercial: they monetize by selling licensing fees to enterprises. But that is not the primary way in which the real-time Web will be monetized. Still, these are big profitable companies with the right tech chops, so they cannot be dismissed.
  2. Open source and open standards
    Tornado, RSS Cloud, PubSubHubbub, and XMPP fall into this category. They have differing purposes but also a lot of overlap. XMPP is low level and not affiliated with any big company. RSS Cloud and PubSubHubbub are most similar to each other. Tornado is a Web server. But they all face the issue that an open standard takes a long time to evolve and consolidate into a winner. Before that happens, we will likely see more entrants, making the choice even more confusing for developers. So, developers may hedge their bets and go for commercial/de facto standards.
  3. Commercial hub
    Gnip is the purest example of this. It aims to get traction by being simpler to implement than the open-source/open-standards offerings. It is a bold strategy from a strong team of entrepreneurs and investors. For something as big as this - basic plumbing for the Web - it feels like a bit too much reliance on one company.
  4. Traffic plays
    Twitter and Facebook are the major players here. They might become de facto standards because the biggest issue is traffic. Developers will build where there is traffic as long as two conditions are met:
    • It works technically (i.e. latency at scale). Twitter has some credibility issues here.
    • It is open and transparent - i.e. anyone can connect without fear of the game changing on them. Facebook has some credibility issues here.

The Technical Challenges

First, we have to address the question of "What does real time mean?"

In each generation, the gurus of the previous generation get sniffy about the definition of real time ("What do you mean one second is real time? We measure in micro-seconds!").

It depends on the usage case and technical possibilities. Real time within a single processor is obviously faster than real time over a LAN, which is faster than real time over HTTP for the whole Web.

Real time, practically speaking, means "orders of magnitude faster than how data has been delivered in the past, and faster than most people think they need today." So, RSS Cloud is faster than RSS, and Twitter search is faster than Google search, and IM is faster than email.

In generation 2, the thinking evolved through three approaches:

  1. Polling. It soon became obvious that this would not scale.
  2. Broadcast, which put too much load on the client.
  3. Multicast, which became PubSub.

Most current Web HTTP-based real-time systems are still in the polling phase (i.e. subscribers poll publishers to see if they have an update), which clearly won't scale. The newer approaches fundamentally enable a publisher to say, "I will tell you, the subscriber, when I have an update, so you don't need to poll me." It's a geek's way of saying "Don't call us. We'll call you."

For a good technical primer on how this works and a discussion of the differences between RSS Cloud and PubSubHubbub (mercifully shortened to PuSH), check out this post.

Sticking My Neck Out to Guess the Winner

Twitter. Why? It has the traffic and is open and transparent enough to win confidence. But two big problems remain:

  1. It has to reveal its business model. Only then will partners feel confident that they understand its strategic direction.
  2. It has to prove it can get good latency at scale. Its early history of fail whales makes people justifiably skeptical.

The winner will likely be the platform that hosts the killer app. Most platforms get traction through a killer app. In the second generation of real time, that killer app was market data for financial traders. What will it be in the third generation? We will explore this in a future post.

What to Name This Generation?

"Message/Information Bus" worked for generation 2. It took a key concept from generation 1 - the hardware bus - and applied it to a local area network.

That does not work so well for generation 3, the real-time Web. For now, we talk about a Web-wide real-time message bus because there is no better alternative.

The new term may have something to do with "status," because the status update is a central concept in social media. Perhaps something like "StatusFabric" or "StatusNet" or "StatusWeb" will emerge.

]]>Discuss]]>
http://www.readwriteweb.com/archives/where_is_the_real_time_web_message_bus.php http://www.readwriteweb.com/archives/where_is_the_real_time_web_message_bus.php Real-Time Web Fri, 18 Sep 2009 10:14:07 -0800 Bernard Lunn
The Real-Time Web Is Not Hype: We Are All Traders Now Hype cycles, like all cycles, are getting shorter. People want to be the first to say, "You heard it here first, folks: this or that hot thing you hear about all the time is a bunch of hot air."

We love to debunk myths and prick bubbles as much as the next set of pundits, but we think the real-time Web is for real. Financial traders have lived in a real-time world for a while, but only within the confines of the trading floor. When they left work, they entered a batch world. Most other people work in a batch world. That is changing. We are all entering the real-time world of the trader. Some of us are getting there faster, but we are all heading there. And relax, there is an "Off" button!

]]>Sponsor

]]> Twitter and the Mass-Market Real-Time News Business

One reason you see so much news about Twitter is that it is a news business, and journalists love to write about their own business!

Calling Twitter a news business may be a bit limiting. It is more than that. But it certainly is affecting the news business dramatically:

  • As a source of stories. One of the millions of just-in-time stringers twittering away may witness a revolution at any second;
  • As a backchannel for other media, replacing the dial-in line used for TV and radio;
  • As a traffic source;
  • As a source to check stories. Traditional journalists can tap into their Twitter network faster than using their Rolodex or speed dial.

Twitter makes it much easier for journalists to tap into real-time news sources. Mastering Twitter is part of new journalism school.

Traders have always worked in a real-time news business. Information services such as Bloomberg and Reuters compete to deliver news a few seconds faster, and that time difference is important to traders.

Okay, so traders and journalists live in a real-time world. What about the rest of us?

Advertising Exchanges

A friend who runs a financial trading systems business told me that Google was constantly trying to poach his best engineers. Why? Because those employees were very good at real-time engineering, and advertising is also a real-time market.

Advertising markets are just like financial markets: available space is matched by available demand. Today, we have two totally different worlds:

  1. Very personal: the "Let's do lunch" style of selling, with long-term commitment. This is a batch world.
  2. Totally automated: no messy humans. Machines negotiate with machines in real time. This is the world of automated ad networks and exchanges, typically for sales of either remnant or long-tail inventory.

These worlds get more interesting as they move closer together and we see the gray areas in between where humans make quick judgment calls, inserting themselves into the real-time flow to, for example, approve creative or simply optimize (choosing one advertiser or publisher over another).

Buying and Selling Digital Goods

Anything that can be sold in digital form is becoming part of this trend towards real time. We are simply matching supply and demand. Information (or code or images or songs) that was not worth very much yesterday is suddenly very valuable. Or the opposite: its value suddenly drops. No matter what the digital artifact -- writing, spreadsheet numbers, code, design, images, music -- matching supply and demand is critical to realizing value.

You may be selling a digital artifact that costs money to create, but the marginal cost is zero, so you are quite happy to optimize supply and demand in real time and take whatever price the market offers at the time.

Or you may want real-time markets to help manage your spare work capacity. This is what ventures such as Turn Here focus on.

Real-Time Supply Networks for Physical Products

Dell revolutionized the PC industry by using a real-time supply chain to eliminate costs. Inventory risk is the biggest pain point for most businesses that trade physical goods. The real-time Web can have a similarly revolutionary impact on Main Street for businesses that no longer need multi-million dollar supply-chain systems. We are already seeing straws in the wind of this massive change with very simple uses of Titter, such as:

  • The bakery that tells its patrons/followers that "The bread is hot and fresh."
  • The Korean BBQ truck in LA that has 39,000 followers who want to know when it is going to roll into their neighborhood.

One can imagine this for anything that is either fresh or scarce: when your local farmer has freshly laid eggs, for example. These tools can be used to sell inventory, matching actual customer demand with bargain offers in real time.

Relaxed Concentration

Human beings naturally operate in real time. Anyone who has children knows about the "real-time interruption machine."

We also need our batch time, our quiet time for reflection and creation. We need to know when to switch off... literally switch off the devices that constantly ping us. Filtering tools and techniques have become critical. As Clay Shirky famously remarked: "It's not information overload. It's filter failure." But we also need to master the ability to deal with a lot of real-time information in a mode of relaxed concentration. In other words, we need to study how great traders work.

Photo credit: artemuestra.

]]>Discuss]]>
http://www.readwriteweb.com/archives/real_time_web_is_not_hype_we_are_all_traders_now.php http://www.readwriteweb.com/archives/real_time_web_is_not_hype_we_are_all_traders_now.php Analysis Fri, 14 Aug 2009 14:00:28 -0800 Bernard Lunn
Evolution of a Revolution: Visualizing Millions of Iran Tweets At its peak, a search for "Iran" on Twitter generated over 100,000 tweets per day and over 8,000 tweets per hour. The plot just below shows the growth in volume of information in the number of tweets per hour.

How does an Internet junkie, news organization, or political operative monitor rapidly evolving real-time events, from the crucial details to the bigger picture? More importantly, how can a data stream be turned into real-time action, reaching the people who need it, when they need it, and in a form they can easily digest?

]]>Sponsor

]]>

Overview

The proliferation of real-time search engines and trend monitors (sometimes referred to as "listening platforms") has thus far done little to address problems of this scale and importance. This is because they fail to provide context -- i.e. show how a new piece of information is relevant to what we've seen before and where it fits in the space of possibilities and relationships.

For instance, if you are a programming director at CNN trying to discriminate between significant news and Internet memes, simply knowing that #iranelection is a trending topic doesn't tell you its relation to other topics or which communities are driving it -- both critical factors.

One promising area is data-oriented user interfaces: data and algorithmic analysis in the back-end and direct visualization and navigation in the front-end. This the next stage of social information, slicing and dicing, mixing and matching, interpreting and analyzing, completely on demand. In this new landscape, the data is the interface.

It's not just about sitting back and looking at pretty pictures. It is about setting aside stale UI metaphors and getting as bare-to-the-bone a human interface as possible for computation. The recently launched Wolfram-Alpha applies this principle to structured data. (Disclosure: I was a member of the core Wolfram-Alpha team and may continue to consult with Wolfram Research.)

Real-time data streaming offers similar possibilities and opportunities. In this vein, let's outline some basic ideas and methods for giving context to the streams.

The Computational History of #iranelection

At the most abstract level, history and computation are the same thing: the evolution of systems over time. Twitter has several remarkable properties that allow us to finally leverage this correspondence in tangible ways. The simplicity of its data, the openness of its system, and its extreme time resolution make it possible for us to detect atoms of history, those moments when something is triggered and society is reconfigured ever so slightly.

Look at the bandwidth plots below. They represent the relative volume of different Iran-related phrases on Twitter over time. Their most striking characteristic is how discrete and spikey they are: a tell-tale sign of an organic computational system.

The first pair of terms compares the bandwidth of "Ahmadinejad" and "Khamenei" mentions, respectively. The evolution of the uprising at the very highest level of social abstraction is shown with remarkable clarity: moving from a dispute over the election process involving Ahmadinejad (shown in pink) to a dispute over authority involving the supreme leader Khamenei (shown in red).

Not only do we get the gist of the evolution, we also see its details and relationships to other social sub-structures. For example, looking at the second plot, we see a co-relation between mentions of the Basij militia and the reports of deaths; and that initial uptick in Khamenei mentions corresponds to the uptick in Basij mentions, foreshadowing the later crackdown.

This idea of computational history applies to events that Twitter not only reported but shaped and hosted as well. A plot further down below compares the Twitter-centric discussion of #cnnfail to the distribution of Twitter proxy IPs that allowed information to continue to flow out of Iran. Is it a coincidence that these two terms merge smoothly together? And what about the big spike in mentions of proxy distribution coinciding with the first reports of violence?

Computing with Social Structures

Simply tracking the volume of various phrases gives us a sense of what is happening on the street, literally and figuratively. But that signal is but a shadow of a far more complex and intricate reality, an interwoven web of individuals and actions.

Twitter allows these social structures to become data structures by means of the "RT" convention. And this in turn allows us to perform extremely powerful computations on the social structures that underlie the flow of information.

Network layout algorithms are a familiar, powerful, and fascinating example. They self-organize in your computer to reveal self-organization in the real world. And that is exactly the kind of tool we need to test our hypothesis about #cnnfail.

The plot below shows the network of people who re-tweeted mentions of IP proxies, with those who had tweeted earlier about #cnnfail highlighted. We see not only significant overlap among the people involved but also a considerable structure in the relationships between them. We have captured a real community at the moment of its birth.

Remember this as you look at the next plot below. Here, we see the re-tweet network that formed around the top five Iranian tweets. Its structure shows a very different phenomenon, capturing the emergence not of a community but of an elite. Despite massive interest, or perhaps because of it, most people did not discover more than one of the top Iranians. The network simply grew faster than the information could naturally propagate. But a small inner circle did succeed in identifying core sources of information.

The final plot below shows yet another community structure, as well as a new algorithmic technique. This plot does not show the emergence of a new community but rather shows the appropriation of a new topic by mature political factions. This re-tweet network has formed around Iranian tweets that mention Obama. Using graph theory, we can computationally extract the sub-communities and then use that information to color the network. The large blue mass on the right is the conservative Twittersphere, while the other structures are a less-organized collection of mainstream or progressive news outlets.

Algorithms and Social Change

Will future Presidents express strategic goals in terms of Twitter graph theory? That is almost a certainty.

The purpose of these computations is two-fold: first, to contextualize information from across time and space in terms that are accessible to humans; and secondly, to distill abstract ideas into actionable form.

Twitter is a platform for achieving both of these purposes in human affairs: detecting networks of information propagation and erecting new networks to reshape emerging social computations. This is the core of Twitter's social and business value. If we were to play that age-old game of "Guess the business model," we'd look here first.

How to algorithmically discover and deploy novel social structures is perhaps the billion, or trillion, dollar question. With Twitter, the data and API are in place. And if the history of computation is any guide, once programming a system becomes possible, progressing from a hack to an application to a platform is only a matter of time.

Guest author: Kovas Boguta is a co-founder of Infoharmoni, a stealth startup building computable knowledge interfaces for real-time data sets. He just returned from last week's Personal Democracy Forum, where discussion about the Iranian uprising took center-stage.

]]>Discuss]]>
http://www.readwriteweb.com/archives/evolution_revolution_visualizing_millions_iran_tweets.php http://www.readwriteweb.com/archives/evolution_revolution_visualizing_millions_iran_tweets.php Twitter Sat, 25 Jul 2009 09:20:29 -0800 Guest Author
Facebook Goes Real Time on Any Site with Live Stream Box Today, Facebook is launching a new feature for webmasters to post a stream of relevant Facebook updates in real time.

The new feature, called a Live Stream Box, can run on sites "next to live streaming videos of concerts, speeches, sporting events, webcasts, TV shows, presentations, or webinars," according to an announcement we received via email. "Sites can also run the Live Stream Box in multi-player games or with any other experience where many people are visiting a website at the same time."

]]>Sponsor

]]> According to a new post on the Facebook developer blog, the team at Facebook began testing this feature during the 2009 presidential inauguration, when they ran real-time status updates and comments alongside streaming video on CNN's website. They then implemented the feature for other events that generally cause a real-time traffic spike in microblogging and live video content, such as the NBA All-Star Game and the Academy Awards.

fb-realtime.png

This feature can now be added to any website where real-time UGC and user interaction are desired. According to the blog, "Users log in using Facebook Connect and share updates that appear both within the Live Stream Box and on their Facebook profiles and in their friends' home page streams. Each post includes a link back to the Live Stream Box on your site so users can discover the live event and immediately join based on their friends' recommendations."

The feature, Facebook says, is built to scale; they anticipate certain sites or events having so many real-time updates that not all users will be able to view or absorb all the content in the stream.

Developers and site owners can visit Facebook's Live Stream Box page to learn more. For websites that don't require real-time user interaction, Facebook still recommends their Comments Box, released earlier this year. The Comments Box also allows users to connect through Facebook Connect, but content generated is archived, asynchronous, and searchable.

]]>Discuss]]>
http://www.readwriteweb.com/archives/facebook_goes_real_time_on_any_site_with_live_stre.php http://www.readwriteweb.com/archives/facebook_goes_real_time_on_any_site_with_live_stre.php Facebook Wed, 24 Jun 2009 12:26:39 -0800 Jolie O'Dell
Make Google Real-Time With Twitter Search Add-on Some people say that "the real-time web" could be the next generation of post-Google search. Social media tools have greatly increased not just the number of people posting content online but also the speed with which they are able to do so. Do we need a new search paradigm that prioritizes publishing freshness higher than page rank?

Google backers say that Google is already capable of indexing anything online mere moments after it's been published - but the user experience in search doesn't really feel "real time" right now. Movable Type consultant Mark Carey came up with a simple solution this weekend that could change your use of Google more than anything else has in a while.

]]>Sponsor

]]> Carey has written a simple browser add-on that displays the 5 most recent Twitter search results for any search terms at the top of any Google search results page. It's so simple! Why hasn't anyone else done it before now? We don't know, but we installed Carey's script just as soon as we found out about it.

twittergoogle.jpg

Called "Realtime Twitter Search Results on Google," the service is actually a Greasemonkey script. Don't be intimidated: Carey explains on his blog how anyone running Firefox can add the script to their browser with less than 5 clicks of the mouse. (See our short video on learning how to use Greasemonkey in under 5 minutes, too.)

To be honest, we've been using Twitter search for discovery lately anyway and Google for retrieval of known info at URLs we can't remember.

Carey consults for ReadWriteWeb and also built our awesome FriendFeed comment integration, among other things.

If Twitter is down (as it was again this morning) or if there are no search results available, then the Google page will simply look unaltered. The Twitter results don't load as fast as the Google page does, which is a bit of a bummer, but hardly a deal breaker. That may be a Twitter API issue or it may be an implementation problem, but it's the only shortcoming we see to this otherwise straightforward approach. We'd love to see some additional little features, like a link to the RSS feed on Twitter for search results, some javascript to view more results if the first 5 are valuable and maybe an option to view Twellow user bio search results.

As it is, though, Carey's script delivers real-time search results to Google - just like people have asked for! Thanks Mark! You can follow Mark on Twitter at @mthacks, you can follow me at @marshallk and you can follow the activities of the whole ReadWriteWeb team at @readwriteweb.

]]>Discuss]]>
http://www.readwriteweb.com/archives/make_google_real_time_with_twitter_ad-on.php http://www.readwriteweb.com/archives/make_google_real_time_with_twitter_ad-on.php Google Mon, 02 Mar 2009 10:32:00 -0800 Marshall Kirkpatrick
EtherPad: Simple Real-Time Collaboration etherpad_logo_nov08.pngEtherPad is not likely to win a prize for its user interface design, but it may just be one of the most useful web apps we have seen in quite a while. EtherPad allows you to instantly create a workspace for text documents that you can then share with your colleagues, clients, or friends. Every edit to the document will immediately appear on your co-workers' screens in real-time.

EtherPad acknowledges that Google Docs already allows for a similar kind of collaboration, but compared to EtherPad, Google Docs is clunky and slow when you just want to collaborate on a simple text document.

]]>Sponsor

]]> Editor's note: Looking back over 2008, there were some posts on ReadWriteWeb that did not get the attention we felt they deserved - whether because of timing, competing news stories, etc. So in this end-of-year series, called Redux, we're resurrecting some of those hidden gems. This is one of them, we hope you enjoy (re)reading it!

EtherPad would be a great tool if you want to keep collaborative notes during a conference call or meeting, but you could also use it to draft or edit text for a press release or email collaboratively.

No Sign-Up

EtherPad, for example, doesn't require you to sign-up before you start working on your document. You can also just share your workspace's URL with your co-workers and they, too, don't have to sign up. Indeed, you can't even sign up for the service, which may become a bit of a problem if you want to go back to a document you worked on earlier but don't remember the randomly assigned URL.

Google Docs requires you to send an email invitation to all your collaborators, and updates to documents don't appear in real-time.

etherpad_sshot_nov08.png

Features

EtherPad also has a versioning system that allows you, or anybody else with access to your workspace, to save the document at any time.

Developers who want to share code might also find this a useful tool, as it can highlight JavaScript syntax. Looking at EtherPad's heritage, it becomes clear why the developers added this feature. The service was developed by AppJet, an online web programming platform, and is basically a showcase for the next version of AppJet's tools, but was mainly created because the developers at AppJet were looking for a tool that had EtherPad's functionality but weren't able to find one.

Verdict

As is often the case, the most useful tools are those that have a relatively restricted feature set but allow users the freedom to use them as they see fit. EtherPad is one of these services and it will probably become a standard tool for us very soon.

You can find a screencast of the product here, but given that you don't have to sign up for it, you may just as well try it out for yourself (or join this workspace we already created).

]]>Discuss]]>
http://www.readwriteweb.com/archives/etherpad_real_time_collaboration_redux.php http://www.readwriteweb.com/archives/etherpad_real_time_collaboration_redux.php Products Thu, 01 Jan 2009 13:00:00 -0800 Frederic Lardinois
EtherPad: Dead Simple Real Time Collaboration etherpad_logo_nov08.pngEtherPad is not likely to win a price for its user interface design, but it might just be one of the most useful web apps we have seen in quite a while. EtherPad allows you to instantly create a workspace for text documents that you can then share with your colleagues, clients, or friends. Every edit to the document will immediately appear on your co-workers' screens in real-time.

EtherPad acknowledges that Google Docs already allows for a similar kind of collaboration, but compared to EtherPad, Google Docs is clunky and slow when you just want to collaborate on a simple text document.

]]>Sponsor

]]> EtherPad would be a great tool if you want to keep collaborative notes during a conference call or meeting, but you could also use it to draft or edit text for a press release or email collaboratively.

No-Sign Up

EtherPad, for example, doesn't require you to sign-up before you can start working on your document. You can also just share your workspace's URL with your co-workers and they, too, don't have to sign up. Indeed, you can't even sign up for the service, which may become a bit of a problem if you want to go back to a document you worked on earlier, but don't remember the randomly assigned URL.

Google Docs requires you to send email invitation to all your collaborators and updates to documents don't appear in real-time.

etherpad_sshot_nov08.png

Features

EtherPad also has a versioning system that allows you, or anybody else with access to your workspace, to save the document at any time.

Developers who want to share code might also find this a useful tool, as it can highlight JavaScript syntax. Looking at EtherPad's heritage, it becomes clear why the developers added this feature. The service was developed by AppJet, an online web programming platform, and is basically a showcase for the next version of AppJet's tools, but was mainly created because the developers at AppJet were looking for a tool that had EtherPad's functionality but weren't able to find one.

Verdict

As is often the case, the most useful tools are those that have a relatively restricted feature set, but allow users the freedom to use them as they see fit. EtherPad is one of these services and it will probably become a standard tools for us very soon.

You can find a screencast of the product here, but given that you don't have to sign up for it, you could just as well try it out for yourself (or join this workspace we already created).

]]>Discuss]]>
http://www.readwriteweb.com/archives/etherpad_real_time_collaboration.php http://www.readwriteweb.com/archives/etherpad_real_time_collaboration.php Products Wed, 19 Nov 2008 09:16:35 -0800 Frederic Lardinois
Google Finance and the Real Time Web googfinancelogo.jpgGoogle is announcing this morning that after more than a year of work on the problem, Google Finance is now offering real-time price quotes for any stock traded on NASDAQ.

As Henry Blodget points out at Silicon Alley Insider, Yahoo! Finance has had real time quotes for a while now and the companies appear to be engaged in some minor squabbling about who's real time quotes are more real time. That's a secondary, albeit important, matter as far as we're concerned. Of more general interest is the increasing availability of and user expectation for real time information on the web.

]]>Sponsor

]]> The Growth of Real Time

In January we wrote about the potential for powering the web with the open source XMPP Jabber protocol used in many IM platforms. Since that time we've seen XMPP used to power real time communication on Facebook and Jive Software's enterprise social networking. Developer support network Qunu has implemented XMPP for real time crowdsourced technical assistance. We've seen Twitter struggle with the real time XMPP part of its infrastructure. The protocol is being explored by multi-media messaging service Seesmic. The soon-to-launch cloud computing project Vertebra from Engine Yard will use XMPP to enable the complex communication between apps in the cloud. (See presentation slides from Vertebra's Ezra Zygmuntowicz below.) More XMPP for machine to machine communication would be super hot. Real time photo sharing and discussion service PhotoPhlow uses Java and is still in closed beta. We wrote about real time news tracker Anothr last month, as well. SlingPage just launched 10 minutes ago and lets users share and discuss web pages with each other in real time. We expect to see more and more apps offering real time functionality through a variety of methods.

We don't know if Google Finance is using XMPP but we wouldn't be surprised. The point is, though, that the real-time web deserves a place in the "next big thing" column. From push-email to live video, from AJAX to social browsing experiences - a real time web can offer social connectedness, interface responsiveness and a smooth user experience that stands head and shoulders above the call-and-response web of the past.

Imagine a web characterized by standards based recommendations, filtering and easy collaboration - all in real time. That's going to be a thrilling new era and it's one that is foretold by the increasingly mainstream adoption of real time information delivery in places like Google Finance.

]]>Discuss]]>
http://www.readwriteweb.com/archives/google_finance_and_the_real_time_web.php http://www.readwriteweb.com/archives/google_finance_and_the_real_time_web.php Analysis Mon, 02 Jun 2008 09:20:57 -0800 Marshall Kirkpatrick