ReadWriteWeb

Written by Can Erten and edited by Richard MacManus. This is the second in a 2-part series exploring the world of P2P on the Web. Part 1 was a general introduction to P2P, along with some real-world applications of P2P. Part 2 (this post) discusses future applications.

As we mentioned in Part 1, broadband speeds are ever increasing and so the demand for peer-to-peer networks is also increasing. However many things that could be accomplished by P2P networks are still in development or research. There is huge potential that at least some of the resulting applications will go mainstream, just as Napster did in the late 90's or Skype in the early part of this century. In Part 2 of our series, we look at some of these potential future applications for P2P on the Internet.

Search Engines

Starting in the late 90's, a search engine company called Google changed the way we search the internet. Their idea was to index the web and get the top results, using their now famous Page Rank algorithm. However nowadays, indexing the web accurately has become a huge and seemingly impossible job to complete. So P2P search engines could be the next solution - where every node (user) is a crawler itself.

Metalink is a new technology that makes it easier and faster to get large files. It's kind of like a playlist for such files (they can be FTP, HTTP, or P2P), because it stores multiple file locations in a single file - and then the download proceeds according to a set of rules.

The Wikipedia page for metalink describes it like this:

"Metalink is an open standard/framework for programs that download (download managers, BitTorrent clients, Web browsers, FTP clients, & P2P programs). For increased convenience, it stores the many locations of files (FTP/HTTP/P2P) in a single file (a .metalink) for extra reliability in case one method fails and so chunks/segments of each file can be downloaded from multiple resources at the same time (known as accelerated/multi-threaded/segmented downloading)."

For developers, metalink files are made up of XML and they are extensible. They also work across multiple operating systems. For ordinary users, metalinks apparently make downloads "simpler, faster, and more reliable".

While looking through Mary Meeker's 2006 Web 2.0 Summit presentation, I was struck by the figures on page 19: "Peer-to-Peer (P2P) traffic was 60% (and rising) of Internet traffic in 2004, with BitTorrent accounting for 30% of traffic, per CacheLogic". You can definitely see why this is the case, as P2P is normally used to download very large media files - music, movies, etc. But still it makes you realise just how big P2P currently is on the Internet and, given the increasing amount of video coming onto the Web, how crucial it is going forward.

Source: Mary Meeker presentation, via CacheLogic

I followed up by checking out the CacheLogic webpage, which has an interesting research presentation on its homepage entitled True Picture of P2P Filesharing. Also note that there is an updated 2005 version of the report on the site.

Tamago recently launched a peer-to-peer commerce system that enables people to sell digital media directly to customers. It's designed for semi-professional and amateur people who publish music, videos, photos, e-books, etc. to earn royalties whenever their creative content is downloaded. Meanwhile the buyers can also earn commissions, for distributing media to others.

Tamago was founded in 2005 by Sony Music Executive Joel Floyd. On Joel's LinkedIn page, he is listed as being a past Principal Architect/Developer at Sony Music. Tamago is based in San Francisco.

While the pros and cons of P2P networks on the Web are still being debated, there are a number of private invite-only P2P networks (aka darknets) out there which enable users to get quality-approved media and software. They are decentralized, secret and almost certainly not very legal in their media-sharing activities. Many of them use the Bittorrent service. A reader who goes by the name The Rub of Clubs let me in on a few of the secrets...

The Rub says that invite-only P2P networks are more popular than most people realise. He uses them "to get the kinds of albums, film, and software that are almost impossible to find anywhere else, including Netflix." What's more, these private networks are being used by some marketing companies to 'leak' new music - in order to get new songs quickly into the hands of influencers. The Rub says this has been an informal practice for a couple years and is not far separated from mp3 blogs or myspace.

I asked what he meant by "quality-approved" media, a term he used to describe the types of media that can be obtained in these private networks. He said quality-approved means there are strict standards of quality. For example, "telesync" films or "cam" films can't be posted, but "telecine", DVD quality, or HDTV quality films can be posted. For music, usually there is a minimum of 220 bitrate, and often music is posted with lossless quality. This Wikipedia list of standards explains more about quality-approved.

Recently The Rub's friend Ariel made a YouTube video about private torrent communities:

In it Ariel explains the story of a former Torrent network called WDMA (Where Da Moviez At?), which apparently had a user base of around 20k.... until it abruptly ended around August 2005. Copycat sites popped up afterwards, but what happened to the original? Well, according to Ariel's story "those in the know won't tell".

So what kind of people are using these secret P2P networks? The Rub told me:

"It's difficult to know how many people use these networks, their age, their occupations, locations, etc. That's just the nature of these networks. I get the impression that many are in college. Almost all male. They seem fairly intelligent, and tech-savvy, as you can imagine. It's very difficult to gather conclusive data about darknets, and I'm sure there are more networks that I don't know about. I've wondered about the possibility of an "ASmallWorld" type ultra-exclusive torrent trading network. Intriguing idea, but almost impossible to verify."

These private networks thrive on the concept of exclusivity, so there's little chance that people not in the know will ever know much about them.

In my search for innovative web and Internet applications, I noted the buzz around new P2P company Skyrider at the end of last week. While it's hard to say whether Skyrider is truly innovative, seeing as they don't actually have a product on the market yet (it's due "in the fall"), they are making interesting noises about commercializing P2P. 

In particular Skyrider states in its press release that "P2P has the potential to be the dominant network architecture of this century" - which of course Skyrider is poised to take advantage of. This is echoed by Tim O'Reilly on his blog, in which he likens Skyrider's potential to that of Google:

"Just as Google crystallized a new vision of how to monetize the web, leading to the explosion of innovation that we now call Web 2.0, Skyrider is building technologies that could bring a rich new commercial ecosystem to P2P."

Those are big claims, especially as the dominant network architecture currently is the client-server model of the Web (i.e. one to many). P2P is about many-to-many distribution and has so far been most useful for distributing music and other large multimedia files. And when it comes to video and audio, I don't think many would argue that P2P is more efficient than a client-server model - as long as it's managed well.

Other P2P contenders

Perhaps Skyrider will be the platform to bring P2P to the fore again, but there are others in the game too - like Red Swoosh (TechCrunch review here), AllPeers (the Firefox plug-in), Enfra (a Korean app founded by Hyung Yong Jun, the guy behind Cyworld - hat-tip Danny for the info), open source Dijjer, and let's not forget the BitTorrent network.

Opera has also integrated BitTorrent into its browser and there are rumors it'll be integrated into Apple's new Leopard OS too (hat-tip Steve O'Hear).

Note that there is a difference between a P2P network (like BitTorrent) and a P2P client (like KaZaA). I can't claim to fully understand this market, but Zeropaid.com has more info if you're interested:

"P2P clients like Grokster, KaZaA, Linewire, Morpheus and eDonkey do not make a P2P network; they only allow the user to connect to an existing network by using common protocols. There are a large number of P2P networks out there: Gnutella, Gnutella 2, Ares, FastTrack, eDonkey2000, BitTorrent and even Freenet. Clients are applications used to connect to those networks. [...]"

Skype's P2P success

An example of a P2P success story in the '2.0' era of the Web is Skype (Tim referenced them in his post). Developed by the same people who created KaZaA, the music-sharing P2P network that recently went 'legit', Skype calls itself "the first P2P telephony network". They claim P2P has enabled them to raise call completion rates and call quality "to levels exceeding that of POTS (“Plain Old Telephony System”)".

Summary - untapped P2P market

I have no doubt that P2P networks will be vitally important in our increasingly networked world of media. Early P2P pioneers like Napster and KaZaA have now 'gone clean', after years of overbearing pressure from the music industry. Likewise the movie and tv industry has been keeping a close eye on video-sharing websites. But beyond all that fear and loathing by media companies, there is a largely untapped market for media search and distribution via the Internet.

So are we at a turning point on the Web, where P2P comes into serious play again? I think we are and this will be one area of innovation to look out for. Contact me if you know of a current innovative P2P technology doing big things on the Web.