Comments for The Shrinking Long Tail - Top 10 Web Domains Increasing in Reach

The Shrinking Long Tail - Top 10 Web Domains Increasing in Reach

2006-12-18T22:04:34Z

Last week I was following the De-Portalization of the Internet thread, started by Fred Wilson and then extended by Keith Teare. I was struck by one observation in particular by Fred:

"I don't have the data to prove it, but my guess is if you looked at the percent of all pageviews that are generated each month, a much smaller portion exist on the top 10 properties today than in 2000, at the height of the first Internet era."

Essentially Fred's theory is that the Long Tail of Internet content makes up a higher percentage of total Web page views today, than 6 years ago.

I asked Web Analytics firm Compete if they could come up with some data to prove or disprove Fred's hypothesis. Compete kindly provided me with some great data, which in a nutshell disproves Fred's theory. According to Compete's data, the top 10 domains are not shrinking - but proportionally increasing.

]]> Internet has grown by 77% in last 5 years

Looking first at the overall growth of the Internet, according to Compete's data the Internet has grown by 77% in the last five years to over 5 million unique domains. (Note: This count includes misspelled and unhosted domains people accidentally find themselves at).

Source: Compete

Top 10 account for larger % of total PV, not smaller

Contrary to Fred's theory and the larger theory of the Long Tail, the top internet properties are accounting for a larger percentage of total pageviews across the web. According to Compete, currently the Top 10 domains account for 40% of the total pageviews on the internet - a 29% increase over the last five years.

Source: Compete

Social Networks to blame

The driver of this Top Domain growth can be summed up in two words: "social networks". If you were to remove MySpace and Facebook from consideration in 2006 (also removing their pageviews from the total) Top Domains would only account for 33% of total pageviews - basically on par with 2001.

The following chart shows the difference in top 10 domains over the past 5 years.

Source: Compete

Note on The Long Tail

The Long Tail is a theory that has been popularized over the past year by Wired editor Chris Anderson - with his book and blog of the same name. The easiest way to define The Long Tail is to present this graph (c/o Wikipedia):

The long tail is the part colored in yellow.

Summary

The usual disclaimers apply about Internet statistics - you would probably get different stats from comScore and Alexa. But the overall trends are of most interest here. It does appear that the top 10 web properties are claiming proportionally more page views in 2006 than in 2001.

The impact of the Long Tail can still be felt strongly on the content creation side, on sites like MySpace and YouTube. So in one sense you could argue that The Long Tail has had an impact on the top 10 domains after all - in terms of who is creating the content. But the fact is that those properties are still owned and operated by big media (News Corp and Google in the above two cases). So in that respect, and according to Compete's stats, the top 10 domains have more reach now than 5 years ago.

Update: The Compete blog has further analysis.

Comment from ron pruett/offertrax on 2006-12-18

2006-12-18T22:21:31Z

This is a good read and I buy the stats. But, more importantly and interesting to me is the advent of the new names on the list, including Craigslist. Yes, you can pretend they don't exist or they were acquired by big media, but they do exist and there will be more. Now, how are revenues tracking with page views over time?

Comment from Ken Norton on 2006-12-18

2006-12-19T00:18:12Z

I'm not sure this is so much the shrinking of the long tail as it is the "heightening" of it. In other words, the head is getting taller as the largest few reach a growing percentage. That's typical of any maturing power law curve, I don't think it's any indication that the tail is shrinking.

Comment from Kingcob Bob IV on 2006-12-18

2006-12-19T00:44:50Z

Mmm, I'm not quite sure I believe those stats....

Comment from Casey Sackett on 2006-12-18

2006-12-19T00:49:55Z

I think it's more useful to see the long tail as a demand-side distribution. The fact that the 10 largest web sites have a larger % of pageviews now than 5 years ago says nothing about the shape of the long tail and what sections of the long tail those web sites are fulfilling. My guess is that the shape of the long tail hasn't changed much in five years: we are still a heterogeneous folk, I would think just as much as 5 years ago since there haven't been any major cultural revolutions (and, um, no web 2.0 doesn't count as a cultural revolution). The original argument is that the most successful online retailers are successful _because_ they service many parts along the tail. Maybe the larger companies are growing in size because they're good at servicing our needs, even those of us out in the tail.

Comment from Deepak on 2006-12-18

2006-12-19T02:52:18Z

It's funny when you think how much of an influence Google had on the concept of the long tail (being able to find it), that today, the company is considered big media.

Personally, I don't think the long tail has to do with who, but with what. You might be going to the same starting point, but where you end up is what counts. In the pre-long tail world, that end was limited. Today, it is a lot less so.

Comment from Randy Charles Morin on 2006-12-18

2006-12-19T03:33:08Z

I think Fred's assertion was correct, he just didn't qualify it properly.

Comment from anon on 2006-12-19

2006-12-19T10:20:51Z

I think Casey Sackett really sums up the main point. These top 10 sites are catering for the long tail, so just because there numbers go up doesn't indicate a shortening of the long tail on the demand side.

Comment from T.J. Crowder on 2006-12-19

2006-12-19T14:44:47Z

@anon & Casey - Exactly. Myspace, ebay, facebook, craigslist -- they're on that list *because* they cater to the long tail.

@Casey - Have to differ with you, I think we're right at the outset of a cultural revolution of sorts, and the next five years will be very, very interesting. Young people live their lives much, much more connectedly than we were able to. New ways of presenting yourself to the world are already evolving, and as social networking sites and communications technologies make the connections ever easier and ever more pervasive, that evolution will continue and increase in speed.

Comment from Adrian Keys on 2006-12-19

2006-12-19T14:46:03Z

Oh for some simple language...this reminds of the intellectual discussions Tim O'Reilly and company had about Web 2.0 which only served to fuel further confusion.

Some telling numbers though. Maybe we need to rethink the hype about Web 2.0...seems things are only growing in one direction...social networking...at least when you consider it from the angle of pageviews.

Also we constantly talk about the number of domains registered to reinforce our assertions about internet growth...but when you have a significant portion of Typo and Parked domains...is that considered growth?

www.jollyjo.org

Comment from Conrad on 2006-12-19

2006-12-19T15:47:03Z

Whether you believe the Long Tail is disappearing or not based on these figures depends on your interpretation of the Long Tail. I thought the whole point of the Long Tail was that it integrated the interests of people into the chart, so the sheer number of individual interests means that the top 10% can never have more attention than the remaining 90%. Surely then you could argue that with the introduction of Facebook and Myspace, the tail has become even longer - people on social networking sites are all taking part in extremely individual activities by messaging individual people. If you base the Long Tail purely on traffic figures, then yes, the introduction of Myspace and Facebook and other social networks has decreased its effect. But if you take into account what each person is doing (they're only messaging and communicating with tight groups with a larger network) then the Long Tail has become even longer with the introduction of social networks.

Comment from Arnie SEO Company on 2006-12-19

2006-12-19T16:23:00Z

I'm not sure I agree with the conclusion either. Plus most proponents of the long tail are simply saying it's your best chance to get better rankings. I personally go after the long tail because I am all about revenue and those people are closer to actually buying something.

Comment from TJ Mahony on 2006-12-19

2006-12-19T16:41:40Z

Compete published an expanded analysis this morning on its blog. The updated analysis provides a few additional data points and perspectives that you may find interesting.

blog.compete.com

Comment from Brian A. on 2006-12-19

2006-12-19T17:52:24Z

I agree with Adrian Key's sentiment, you can't simply look at the number of new sites as "growth", nor can you refer to page views as a metric for popularity. Without some sort of entry/exit point or visit duration statistics you can't be sure that anyone actually stayed on the site long enough to read the content.

That aside I do believe that social networking sites are the most popular on the net simply because their design encourages visitors to stick around. They also enjoy a nice growth of new users not simply because they rank well in Google search results, but because their users encourage others to visit the site as well.

Comment from Jakob Nielsen on 2006-12-19

2006-12-19T19:04:06Z

"Arnie SEO Company" makes an important point: there's hugely different value to pageviews, and some of the top sites in terms of traffic have pageviews of extremely low value. MySpace in particular has extraordinarily low value per PV.

In contrast, Google and the search sections of Yahoo and MSN have extremely high value. EBay and Craigslist have medium value (high value, of course, to the people selling there, but only medium value to the sites themselves because they only score a percentage).

Smaller sites will often have medium or high values per PV because they are more targeted. Often, they are B2B sites, meaning that they can have extremely high value per PV if they attract prospective customers in the market for million-dollar products or services.

I did an analysis of the value of big sites vs. small sites back in 1997 which I still believe: most of the value will eventually come from smaller sites because they can be more targeted (and because there are more of them, of course - the value per site will be higher for big sites).

Comment from Chungiz Mernt-Heinz on 2006-12-19

2006-12-19T20:58:38Z

1) Using GMail generates 1 pageview no matter how many messages I read and write
2) Using MySpace generates a pageview if I sneeze

Conclusion: page views are a shitty measure of web property tube time.

Comment from Danno on 2006-12-20

2006-12-20T13:31:16Z

Didn't anyone notice that most of those sites *serve* the Longtail?

MySpace, YouTube, Google, Yahoo, craigslist, and ebay are incontrovertibly Long-Tail powered sites.

Comment from Alan Andrew on 2006-12-21

2006-12-21T15:28:48Z

How many of these are aggregators--taking from the tail and bringing it to the head? Some I've never heard of or ever visited. I go to only 2 (now and 2 then): google and craigslist. Google is an aggregator; craigslist I use at work to help people find jobs. So the list is a mash-up of heads eating tails, to some degree!

Comment from stefano epifani on 2006-12-24

2006-12-24T19:10:30Z

Hi!
your trakback works? :-(