aggregation - ReadWriteWeb http://www.readwriteweb.com/feeds/search/aggregation en Copyright 2010 Richard MacManus readwriteweb@gmail.com Sat, 20 Mar 2010 10:30:00 -0800 http://www.sixapart.com/movabletype/?v=4.23-en http://blogs.law.harvard.edu/tech/rss Building on Google's aggregation services

In his article "Google Aggregation Strategy", Elwyn Jenkins from Microdot News reviews three Google "information aggregations" and asks which one will be moved from beta to live first - Blogger, Froogle or Google News. Microdot News argues that aggregation is at the heart of Google's business and that Google will build on the success of their search engine by offering similar aggregation services for weblogs, shopping and news - and more.

I will take that argument one step further, because I think that Google will start to provide syndication services as well as aggregation. The current crop of Google's aggregation services are "pull" rather than "push". People still need to go to Google's websites in order to find information. I suggest that Google will adopt a publish-subscribe model. People will subscribe to information and it will be automatically delivered to them on a regular basis.

The aggregation of information is the base for Google's success. But the value-add is to enable people to tap into Google's aggregated information base and create personalised "feeds". These feeds won't be the same as the results from traditional search queries. Instead of inputting a wide-ranging query, people will enter a somewhat more constrained "topic". For example rather than entering an esoteric query like "Dave Winer XML-RPC spicy noodles in Boston RSS sunset", a person will need to enter something that can be converted into a reasonably generic topic. A "topic" in this sense will be something between a Google search query and a Yahoo category. That is, it will be automatically generated but within a manually-defined framework (such as ENT perhaps).

]]>Sponsor

]]>
http://www.readwriteweb.com/archives/building_on_goo.php http://www.readwriteweb.com/archives/building_on_goo.php Info Architecture Wed, 07 May 2003 21:31:43 -0800 Richard MacManus
Why Filtering is the Next Step for Social Media If there's one thing to be learned from social media tools, it's that these services were not made to interact with one another. Complaints are rolling in and heated discussions are taking place about the noise levels within social media platforms. Here's a look at why noise levels are high and why filtering should be the next step for social media platforms.

]]>Sponsor

]]> Confusing Aggregation With Importation

With so many different platforms to aggregate, noise levels are surging. An underlying issue in the level of noise is that some of these services were not made to interact with one another. Users of social aggregation tools should understand that what you may consider noise is actually a side-effect of using a social aggregation platform. Users should also note when you may be confusing aggregation with importation.

Though importation may be a necessary step within aggregation, there is a difference between the two. Importation is usually more selective and limited to the inclusion of select items of other services. This may include common specifics such as names, ages, and locations. However, with aggregation a service attempts to conglomerate key features and entire contents of other services. This makes aggregation seemingly more flexible due to it's ability to encompass a wider scope of content.

Using Platforms for Different Purposes

The services that are aggregated are usually used for entirely different purposes. For example, some platforms are used to keep in touch with others such as family, friends, or business contacts. On the other hand, you have services that are used only for the purpose of finding more content and conversations pertaining to certain contents.

When you pull in an account from a platform that is completely unrelated to to the usage of another, you will inevitably create a small amount of noise. However, with social aggregation platforms it's hardly ever just one account. This can increase the noise level to an irritating high for other members of these services, including those within your personal network.

Services Cater To Various Audiences

Aggregated services are not only used for entirely different purposes, but also cater to different audiences. Consequently, who you may be catering to is dependent upon why you may be using the service. While some articles or content submitted to services may overlap, this is only because there are overlapping interests for the different audiences on these platforms.

How does this affect noise levels? If you're using a service to promote content, you may be considered noisy to those that are looking for conversations. If you're using a platform to keep in touch, then those looking for content and in depth conversation surrounding particular content would need a way to block out idle chatter.

Filters Are The Future Of Social Media

Filters are rapidly becoming a pertinent issue for developers of social media services. As a result, social aggregation platforms are in the perfect position to lead the pack. While this is no easy task and one that cannot be solved in its entirety, it would help resolve another issue social media users are facing: courtesy.

Instead of being able to freely add whatever service you wish, some users like myself are taking into account what others may consider noise on certain services as a courtesy to members. In essence, you are becoming our own filter. You may refrain from important other services for fear of being labeled as "noisy". With better filtering options, users can use these services to their fullest extent without becoming a nuisance to others or missing the benefits of aggregating all of their accounts.

]]>Discuss]]>
http://www.readwriteweb.com/archives/why_filtering_is_the_next_step.php http://www.readwriteweb.com/archives/why_filtering_is_the_next_step.php Social Web Sat, 10 May 2008 09:59:54 -0800 Corvida
5 News Aggregation Methods Compared News is delivered in many different ways via the Internet. With so many blogs and news sites, it is sometimes hard to keep up and know if you're really getting all the day's top headlines. That's where content aggregation comes into play. Aggregators are sites specifically designed to bring multiple news sources together and repackage them in a more convenient form. These sites make it possible to track and digest the day's top news with minimal effort. But there is more than one method of aggregation, so which is right for you?

Below, I will take a look at five different techniques for aggregating the day's top news: Single Stream Aggregation, Single Page Aggregation, Meme Aggregation, People Powered Aggregation, and Edited Aggregation. For this round up I will use only sports-related sites.

]]>Sponsor

]]> Single Stream Aggregation

Single stream aggregation, like SportSpyder takes multiple sources and combines them into a single stream. This is a great method for tracking more than one feed and keeping on top of breaking news. SportSpyder itself aggregates from a large number of varied sports news sources and filters them by team or player. The site also lets users create custom mashup feeds of news for multiple teams.

I use SportSpyer indirectly through a module on my MyYahoo! page to track Yankees team news. And while it certainly reports on-topic news from various papers and blogs well, there is one major drawback to this model: duplicates. Because a single stream aggregator simply mashes up a single stream from multiple feeds, there will often be duplicate stories reported from more than one source. Currently in my stream, for example, I have 3 or 4 different stories about Phil Hughes' rehab start from different sources.

Single Page Aggregation

A single page aggregator puts headlines from more than one source onto a single page. Sport Snipe, for example, lists 22 sources on its main headlines page (and also includes specialized areas by sport). Singe page aggregators keep feeds separate and let you sort out the news. These are basically simple RSS readers, and the benefit is that you get a lot of sources on a single, convenient page. However, that's also the drawback. This method of aggregation puts a lot of information on the page at once and it can be overwhelming.

Meme Aggregation

Meme aggregation attempts to cut down on the signal to noise ratio by figuring out what is the most talked about news (and thus, hopefully, the most important). Megite Sports is a good example of a meme aggregator. The day's news is distilled down into the top 20 stories based on what is being linked to. (Note that Google News is similar, but relies less on linking and likely more on keyword analysis and comparison to group stories into memes.) Gabe Rivera's BallBug is another example - it is similar to the tech news favorite Techmeme, but focused on baseball.

This is a great way to get a quick glimpse at what are very likely the top stories of the day. The biggest problem with this method is that it is not always super fast at catching breaking news. Meme aggregators will only update with new memes as fast as bloggers and news writers can create them -- in other words, breaking news won't be noticed until enough people are talking about it.

People Powered Aggregation

Yardbarker is an example of a sports-themed people powered aggregator -- which of course means a digg-clone for sports. People powered aggregation decides the top news by letting people submit and vote on stories. These sites are great for finding interesting and often overlooked content, and the biggest stories will usually get play fairly quickly after they break, but people powered aggregation is generally not great at reporting the day's news. The crowd can be fickle, and a lot of times selections on these sites seem random, if entertaining.

Edited Aggregation

Edited aggregation, like Yahoo! Sports News combines news from trusted sources and is vetted by human editors. Yahoo! for example, takes news from the wires -- the Associated Press, Reuters, AFP, and PA SportsTicker, among others -- and aggregates them into a single stream. Editors keep duplicates to a minimum and decide which stories are the most important to be listed as top headlines on the main sports page.

The benefit to edited aggregation is that the signal to noise ratio is great, and you don't have to do much sifting to find the top news. The drawback, of course, is that you are getting your news through a filter based on the opinions of a few people.

Conclusion

So which is the best? It's really very difficult to say. Each method above has both positive and negative characteristics. The best solution will be the one that you think gives you the best headlines according to your tastes and needs. That very likely could include more than one of the above types of aggregators.

]]>Discuss]]>
http://www.readwriteweb.com/archives/news_aggregation_methods.php http://www.readwriteweb.com/archives/news_aggregation_methods.php Analysis Tue, 10 Jul 2007 12:54:34 -0800 Josh Catone
The Art and Science of Creative Aggregation Jay Fienberg wrote a very interesting post regarding the future of RSS aggregators and blogs:

"Now (today), it's so easy to publish blogs that there are tons of them, and the effort to aggregate them is beginning to again attract editor-like and writer-like functions, i.e., merely mechanical aggregation of sites is seeming too read-only-passive, and folks are being attracted to more and more active, creative, interactions."

He terms this "creative aggregation" and lists Technorati tags, del.icio.us, Flickr, and Webjay as examples of this. He likens it to music DJ's, who are "aggregators of existing recordings" but have also "became creators in their development of playlists". This is all part of the Remix Culture we're currently in the middle of building.

He suggests that the developers of services like Bloglines, Flickr, delicious and Webjay need to continue to build on their creative aggregation functionality - in order to stay ahead of their competitors. I understood this better when I read Lucas Gonze's interpretation:

"...so what he's developing here is the dividing lines between manually generated content, content generated by bots reaping the manual content, and insight generated as bots become refined enough to perform a curatorial role."

I think a good role model for this type of editorial functionality is Amazon. Ever since they opened for business in 1995 (10 years ago, seems like an eternity in Web time!), Amazon has provided interactive functionality on their site and they raise the bar every year. Although their core task is to aggregate information about their products - e.g. books - what makes Amazon stand out from its competitors is their ability to creatively mine that aggregated data and enable users to do all sorts of things with it. Including, most importantly, contributing to the data (user reviews, etc). Which of course leads to more content/data to aggregate! That's an important point - creative aggregation feeds off itself, by creating ever more data for users to aggregate and remix.

The companies that most excite me in the Web 2.0 era are those that, like Amazon, are aggregating content and enabling users to do remixes of it and build on it. Letting users mix and dance to their own tunes, rather than serving up the usual top 40 fare. Flickr and del.icio.us are the two most obvious examples, but I think PubSub and Feedburner are two other examples where content is not only aggregated - but enhanced, remixed, personalized, composed.

]]>Sponsor

]]>
http://www.readwriteweb.com/archives/the_art_and_sci.php http://www.readwriteweb.com/archives/the_art_and_sci.php Web Development Thu, 27 Jan 2005 22:13:53 -0800 Richard MacManus
What Is The Data Matrix? Machines and Humans Coexist Andrew Nachison from The Media Centre raises some interesting issues regarding RSS content aggregation. He boils it down to these 3 questions:

1. Machines vs. humans? [in regards to aggregation]

I agree with Andrew's conclusion: "This isn't an “either/or” theory but a “both/and” reality." P.S. also check out The Media Center Matrix. Rich Skrenta from Topix.net has a similar view: "For comprehensiveness, algorithmic techniques will have to come into play. People-powered systems just don't scale to the long tail."

I too think there's a middle ground to machine and human/social aggregation. Currently I think we're probably nearing the peak of human/social feed aggregation, in percentage terms. What I mean by that is that the average blogger/reader probably subscribes to 80-90% human feeds and 10-20% machine feeds - and most of the latter would be egofeeds from the likes of PubSub, Technorati, Feedster (sometimes egofeeds of other people :-). So topic and tag RSS feeds are at a very early stage of adoption - companies like PubSub, Findory and Topix are leading the way.

From now on in, machine aggregation can really only increase its percentage of attention - while human feeds will decrease. But don't worry, because it's not a zero sum game. Aggregation as a market will continue to increase at a great rate. Even though the ratio of human-to-machine feeds will even up in the coming years, the whole pie will grow significantly.

2. Who profits from the exploding digital datastream?

Andrew says that "traditional media companies" have in the past "derived enormous profit" from controlling information. But fragmenting audience is quickly moving that control to content aggregators - he specifically cites Google and Yahoo.

The search and aggregation companies are set to profit. And provided a click-through is still required to access full content, niche publishers should also profit too.

One thing to watch is the brewing controversy (or browsting controversy in that case!) over full-content aggregation, which some companies are already attempting to profit from. I wonder also about excerpted content aggregation, or remixes of content - because the boundaries will surely be pushed in those areas too.

3. Who controls the datastream itself?

Andrew doesn't write much about number 3, so I'll take a punt at it. The control of content is in one sense moving very definitely towards the consumer, or reader (neither term seems to fit in this age of the read/write web!). This is something I've been exploring over the past months and which continues to fascinate me. RSS Aggregators and topic/tag feeds are two technologies that in a very real sense give power back to the user. I choose (by subscribing) what content flows into my Aggregator. I choose which of a million niche topics to track by RSS.

However as Andrew points out in his post, Google and Yahoo - and apps like Bloglines - are the main tools now for accessing the datastream. Their influence over the datastream is increasingly important - you can see evidence of this in Google's highly profitable advertising business.

]]>Sponsor

]]>
http://www.readwriteweb.com/archives/what_is_the_dat.php http://www.readwriteweb.com/archives/what_is_the_dat.php Web 2.0 Fri, 18 Feb 2005 21:48:20 -0800 Richard MacManus
Search, Aggregation, and Conversation: Keys to a Killer Web Service There are thousands of new services that pop up every day. Too many services imitate, and only a handful innovate. With all of these services, one wonders what their plans are for success. Competition on the web is stiff and users are demanding more from the services they join. While there's no formula for success, there are three keys to a killer web service: search, aggregation, and conversation. In this post, we take a look at successful services that have integrated these keys just right.

]]>Sponsor

]]> Facebook

While everyone has their opinions about Facebook's future and we've given recommendations on how to make Facebook useful again, there's no denying that Facebook is a success. Facebook's integration of search, aggregation, and conversation is a winner in anyone's book.

Search
Their search feature is very useful for exploring every nook and cranny on Facebook. These days, tracking someone on Facebook is as easy as finding that same person on Myspace. Facebook's search feature is stalking at its best.

Aggregation
Facebook has been great with aggregating personal content since its all exclusive launch. With the addition of a plethora of great Facebook Apps to help out, aggregation on Facebook has only gotten sweeter. They've recently opened up the Newsfeed to include web 2.0 services Flickr, Picasca, Yelp, and Del.icio.us. While we didn't think Facebook's news feed was open enough, it was a huge leap for Facebook. Coupled with the Facebook Apps that integrate Twitter, FriendFeed, and more into the service, aggregation is becoming a strong area for Facebook.

Conversation
There's plenty of conversation about Facebook and on Facebook. The Wall application is one of Facebook's most popular features and definitely enhances the service. Really, Facebook speaks volumes in this area, so we won't bore you with a rehash.

Twitter

Our beloved Twitter has received a lot of complaints in the past week for its downtime problems. Non-Twitter users even may wonder why we put up with the service. Three words: search, aggregation, conversation.

Search
The integrated search function on the Twitter site isn't all that useful. It took over a year for the feature to be implemented and like Facebook, Twitter apps do the job better. While search is basic on Twitter, search engines like Summize, Terraminds, and Tweetscan do more than enough to solve the various problems that Twitter's official search engine doesn't. With these search tools, you can better see what people say when they tweet, track your brand on Twitter, and more. In fact, two of these engines are Twitter's most popular tools, helping to make Twitter a killer service.

Aggregation
Twitter doesn't discriminate on what you can add to the service. Pulling in feeds from other sites and pushing them to Twitter is simple. Since Twitter doesn't do this by default, you'll have to use one of the many twitter tools available such as Twitterfeed. Grab the RSS feed of any service you'd like to aggregate on Twitter, plug it into Twitterfeed and you're good to go.

Conversation
Twitter is one massive conversation in itself. It's a communication tool so of course conversation is a focal point that Twitter is most successful at. However, what's so great about the conversations is the wide variety of topics and "tweets" being sent through the service. Content may be king for blogging, but conversation is the king for Twitter.

FriendFeed

FriendFeed is this year's Twitter, which was last years innovator. While FriendFeed may be a duplicator in some ways of other services before its inception, there's no denying that it's now the top social aggregator and a service that's gaining a lot of steam.

Search
FriendFeed is the child of some of Google's best engineers. Consequently, the search engine on FriendFeed is fantastic. Users have a multitude of ways to search for things such as going through your own aggregated items for something, searching your entire network of friends or one person, and even using the search to find something within a specific service.

Aggregation
FriendFeed is a social aggregator. What can't you import to FriendFeed if not through the various integrated services then via the "imaginary friend" feature? Enough said, don't you think?

Conversation
FriendFeed has some of the best conversations on the web. Not getting enough comments on your blog? Head over to FriendFeed to find them. Members of FriendFeed congregate not only around various popular blog articles, but also Twitter messages, shared items and more. In contrast to Twitter, FriendFeed's conversations are a lot more organized and logical. Where Twitter conversations can be noisy, FriendFeed conversations would be the exact opposite: valuable.

Continuing Success

Search, aggregation, and conversation are the focal points of today's web services. There are more blogs, alternative search engines, and aggregators then ever before. These are just a handful of services that have taken off. While these may be the keys to a great beginning, in our next post we'll follow up with the keys to expanding the success of a service. Until then, we'd love to hear your thoughts on what some other keys to success are and what keys help with the longevity of a service.

]]>Discuss]]>
http://www.readwriteweb.com/archives/search_aggregation_and_conversation.php http://www.readwriteweb.com/archives/search_aggregation_and_conversation.php Product Reviews Tue, 20 May 2008 20:23:00 -0800 Corvida
Web Aggregation: What Works, What Doesn't summitlogo_150wide.pngNo one is getting Web aggregation quite right. That's one of the big take-aways from "Web Aggregation: What Works, What Doesn't," one of the breakout sessions at the ReadWrite Real-Time Web Summit.

We first heard about the fire hose meme several years ago in discussions about RSS. It was often used as a way to describe how information comes to you in a feed. The context has changed as real-time data becomes pervasive, and the questions about its volume persists.

]]>Sponsor

]]> The fire hose conversation is often centered on Twitter these days, but it's an issue across the social Web. Perhaps most of all, we should be thinking about what are the subsets of the fire hose and, in particular, how we use data streams in our lives.

The real-time Web ebbs and flows. Most people find the real-time information well after it has been published in an activity stream. Thus, a proliferation of new search engines are coming to market, looking to capture this real-time data and making it relevant to users.

Another distinction made in the discussion centered on how we consume real-time information and the persistence with which we need to get it.

For instance, some information you do not need to be pinged on every 30 seconds. Instead, it may make more sense for it to be pushed to you when you need it. You may only get this information on rare occasions, such as an emergency. So when you do get that information, it is very relevant.

The real-time Web may be instant, but our lives do not work that way. One participant said that he may be interested in a photo of his son that appears in his stream but perhaps not the photo of his son's buddy, who happens to like Scandinavian death metal music.

So, the question becomes, how will the real-time Web fully develop. For Joseph Smarr of Plaxo, that's where open standards come into play. Interestingly, open standards are emerging as an oft-discussed issue at the Summit.

Smarr made the point that RSS and Atom were designed to share the titles and bodies of blog posts. What we are actually sharing in an activity stream is far richer. What we need is language that embodies the far richer meta data that comes in a real-time activity stream. Pubsubhubbub and RSSCloud are starts, but there is still a lot of work to do to put the pieces together.

]]>Discuss]]>
http://www.readwriteweb.com/archives/web_aggregation_what_works_what_doesnt.php http://www.readwriteweb.com/archives/web_aggregation_what_works_what_doesnt.php Thu, 15 Oct 2009 15:20:28 -0800 Alex Williams
Smashing Feeds: Get the Latest Twitter and FriendFeed News News aggregation sites aren't anything new to us. There are sites like Techmeme, Popurls, Techsted, Alltop, and more that get the job done. So when we took a look at Smashing Feeds we weren't all that excited about it, until we discovered something that the other sites don't have: Twitter and FriendFeed link aggregation.

]]>Sponsor

]]> Smashing Feeds

Smashing Feeds happens to be just like any other news aggregator. It uses the Google AJAX feed API for feed creation and management to aggregate news from a ton of various sources. It doesn't require users to signup, but it's not customizable and the UI isn't something we're goo-goo gah-gah over. The aggregated news coverage sites are separated into categories such as Technology, Business, Productivity, Web 2.0, and more. None of these sections are new, but their "Tweet" section is a definite eye catcher.

Twitter and FriendFeed News

The 'Tweet' section of Smashing Feeds has a host of Twitter and FriendFeed related news. This section sports links aggregated from FriendFeed with usernames included in the headlines, links to what seems to be random tweets from Twitter, the latest links to make it on TwitLinks, Intwition, Tweetmeme River, TwitBuzz, Twemes, and the Techmeme Firehose Twitter stream. Basically, you can get the hottest and most popular Twitter links from various services all on one page.

Not Much To Offer

Bouncing from site to site that aggregates the hottest or most popular links on Twitter can be a chore. Users usually end up just picking on site and sticking with it. The 'Tweet' section of Smashing Feeds solves this problem. However, the randomness of some of the aggregated sites are weird. Why are random twitter messages and FriendFeed messages being aggregated as news? I'd suggest they stick to the sites that do most of the aggregation for them and aggregate those sites for the 'Tweet' section. Beyond that, Smashing Feed isn't offering anything better than the competition.

]]>Discuss]]>
http://www.readwriteweb.com/archives/smashing_feeds_get_the_latest_twitter_friendfeed_news.php http://www.readwriteweb.com/archives/smashing_feeds_get_the_latest_twitter_friendfeed_news.php Product Reviews Sun, 15 Jun 2008 12:17:20 -0800 Corvida
Bloglines' Search-based Makeover - In The Big League Now A couple of weeks ago Microsoft's new web-based RSS Aggregator prototype was discovered. I wrote then (also in my weekly wrap-up) that even though technically the prototype wasn't that special, it did reveal Microsoft's business strategy for RSS. Which is that Microsoft intends to integrate RSS Aggregation with its MSN Search product.

To summarise my view, I think search is going to be the platform from which Microsoft will challenge its main rivals in the RSS space. And one of those rivals will be Bloglines/Ask Jeeves. More on that in a momemt...

In an email exchange with someone today I speculated that start.com will give Microsoft an advantage over its traditional rivals Google and Yahoo - at least until Google reveals what it's going to do with its Blogger technology. Or whatever else they are cooking up with Google News and similar customization products. Yahoo is also enhancing its My Yahoo platform by making RSS a key component.

Bloglines/Ask Jeeves Steps Up To The Plate

The one player that hasn't been talked about much yet, in this context, is Bloglines/Ask Jeeves. But they are definitely positioning themselves to join in the RSS aggregation/search integration game. Take a look at the new Bloglines homepage - notice anything different? Well it appears that search now gets top billing on the page, ahead of blog reading and subscription. The search box is at the centre-top of the page, the search component of the circular graphic is at the top of the circle, and most of all the introductory statement mentions search first:

"Bloglines is the most comprehensive, integrated service for searching, subscribing, publishing and sharing news feeds, blogs, and rich Web content. It's free and easy-to-use."

The interesting thing is they don't even mention the word "read" in that paragraph. I also think the word "integrated" is a key...

Now take a look at the Bloglines homepage from a year ago, c/o the wonderful archive.org. It's from January 2004 and this was the introductory paragraph back then:

"Bloglines is a free service that makes it easy to keep up with your favorite blogs and newsfeeds. With Bloglines, you can subscribe to the RSS feeds of your favorite blogs, and Bloglines will monitor updates to those sites. You can read the latest entries easily within Bloglines."

Notice that the word "search" didn't even rate a mention one year ago. In 2003/04 Bloglines was a service to "keep up with" blogs. Subscription and blog-reading was the focus, not search.

Aggregation/Search Integration - The Moneyball in '05

None of this is a great surprise. Ask Jeeves is a search company after all and they bought Bloglines in order to use it. It's natural that Bloglines is being influenced by the coach yelling in the dugouts.

But I think it's more than just Bloglines utilising Jeeves' search strength in its product. It's clear that Bloglines/Ask Jeeves is doing this in order to position itself to take on The Big 3 of Microsoft, Google and Yahoo. Because RSS Aggregation + Search is going to be a winning strategy when competing in the big leagues.

]]>Sponsor

]]>
http://www.readwriteweb.com/archives/bloglines_searc.php http://www.readwriteweb.com/archives/bloglines_searc.php Thu, 24 Mar 2005 23:31:51 -0800 Richard MacManus
Can Web Apps Get Too Drunk on Aggregation? There's a mind-numbing amount of conversations and transactions going on around the internet these days and quality aggregation of content is a very hot trend. When is more too much, though? Are some aggregation services shooting themselves in the foot by sacrificing quality for breadth? Is this madness and does it need to stop?

Call it feature creep, call it "so meta it hurts," it appears that a growing class of websites run the risk of aggregating too much. Maybe that's not the case, but there are some issues and we're going to write about them. We'll also offer collected examples of sites that take one strategy or another - you can let us know if our own aggregation here is too much.

]]>Sponsor

]]> Long Tail vs. User Experience

Sometimes it's a simple matter of priorities. Do you focus on developing the most effective user experience you can, or do you extend your service to as many niche user groups as possible? The best case scenario of course would not require such a choice to make, but for startups seeking to innovate - that choice often does have to be made.

Look at lifestreaming apps FriendFeed vs. awkwardly named competitor Profilactic. (profiles+disease prevention?).

FriendFeed has nailed user experience. It's simple, easy to use, recommends friends till the cows come home and is a great place to communicate about much of your friends' social media content. The number of sources you can associate with your profile is limited, however. At least compared to Profilactic it is.

FriendFeed lets you share content from 35 different services around the web and ironically, that looks like the focused option compared to Profilactic's 177 and growing.

What's not to love about Profilactic's support for super-awesome services like music mix sharing service Muxtape and the RSS community Toluu? That's awesome. Except not very many people use those services. In the mean time, I don't want to use Profilactic and neither do most of my friends. I could use it without them but that's not as much fun as using FriendFeed.

It seems to me that Profilactic has sacrificed user experience for long tail support. That's a sacrifice that probably won't serve them well. None the less, we wish them the best.

Breadth doesn't have to be sacrificed for ease of use, though. To test this theory out I just bought a t-shirt through the meta-T-shirt aggregation voting site experience Rumplo. It was just as easy as buying a shirt through one of a million T-shirt sites (which I had to do eventually, Rumplo is just a voted-on directory) and it was fun. Watch out for shipping and handling prices on some of those sites though, ouch.

Niche Content vs. Economies of Scale

In the brick and mortar world, commercial enterprises tend to have to choose between offering diverse niche goods on one hand or buying and selling the most popular goods in bulk and reduced rates. The lack of physical inventory requirements and the low incremental cost in offering most digital goods means this dilemma may not translate online (news flash!).

You can get as nichey as you want online and still scale fairly large, it appears. Is that the lesson of Last.fm and Pandora?

Perhaps the way this dynamic can still come into play is the aggregation of aggregations. Perhaps offering collections of obscure collections gets too complicated and mitigates the network effects of a large user base that can come from a more accessible user experience. Music playlist social network Imeem suggests that's not necessarily the case.

It doesn't appear that greater levels of aggregation has to lead to a loss of niche content for the sake of economies of scale.

Diverse Commenting Communities vs. Centralized Discussion

As Sarah Perez wrote here last month in her post The Conversation Has Left the Blogosphere there are now comments being left all over the place. Just as some people get frustrated when the full text of their content gets "aggregated" and they lose out on pageviews, others are growing frustrated that comments aren't being left all in one place.

Fortunately, there's people working on this problem as well. Check out Sarah's post yesterday on YackTrack, a service you can use to look up all the comments left around the web - on or about a particular URL. You can even use this bookmarklet: YackTrack It!.

Yesterday we wrote up BooRah, a company that aggregates restaurant reviews from all around the web and analyzes them for emotional content. That's one way that aggregation can help centralize comments.

Dispersed comments don't have to be a major problem coming out of continued aggregation. It's going to be ok.

Conclusion

After looking at the questions above, the only remaining problem left really unsolved may be user back backlash. You can aggregate all day long but users may feel like it's just too much.

We would argue, however, that a well executed user experience and perhaps some solid recommendation technology is going to be able to smooth most of the wrinkles that come from these increasing layers of aggregation.

Top image: jrhode

]]>Discuss]]>
http://www.readwriteweb.com/archives/can_web_apps_get_too_drunk_on.php http://www.readwriteweb.com/archives/can_web_apps_get_too_drunk_on.php Analysis Tue, 29 Apr 2008 11:33:01 -0800 Marshall Kirkpatrick
Newser: Human Edited News Aggregation The web has a new news aggregation site today with the launch of Newser, a start up with an impressive team behind it. Newser is run by Highbeam CEO Patrick Spain, Vanity Fair columnist Michael Wolff, and Caroline Miller, who was the editor in chief of New York Magazine. According paidContent.org, the site was "born with the idea of trying to reinvent the network TV news paradigm for the Web," meaning to keep news items short, snappy, and entertaining.

Early reviews of the site have not been so favorable. Gawker especially was unimpressed, calling Newser "the dullest thing I've seen all day." But I'm not sure it's that dull.

]]>Sponsor

]]> Newser uses a combination of human and algorithmic aggregation to determine the top stories, relying on their list of top 100 sources (which could presumably change over time -- for example, mlb.com ranks #32 right now, but when baseball season is over, I'd expect to see the Newser editors rely on it less). It's unclear what sort of machine aggregation Newser is using, but human editors ultimately decide what news makes it to the site.

Newser editors summarize news stories and publish the summaries to the site along with images from a variety of sources (including Flickr, Wikipedia, and the Associated Press), and sometimes with videos from YouTube and excerpted background content from non-news sources (see this topic page for the Chicago Cubs, for example). News stories on the site are accompanied by related articles picked by the editors, from the AP (presumably matched contextually), and via the Highbeam newswire, which draws on a number of sources.

Conclusion

The site's design is attractive, with 9 top stories displayed by default on the main page in a way that isn't overwhelming, and the summarized news does make it easier to consume. Newser's goal of bringing the "network TV news paradigm" to the Internet looks to me to have been realized. News is presented visually in short snippets and becomes easy to scan. However, the success of Newser will rely on the ability of their editors to deliver content that interests readers. Newser hopes to offer news that is "free of bias," but when you add a human element you unavoidably add a bias as well, and the ultimate success or failure of the site might hinge on what that bias is.

I actually like Newser, but whether I would use it regularly will depend on whether the editors can do a good job of not only keeping on top of breaking and important stories, but also picking stories that interest me. The site plans to offer a way for users to filter news based their interests -- which should help with my last point. However, it is baffling to me that a site like this doesn't offer some sort of social interaction (like rating or comments).

]]>Discuss]]>
http://www.readwriteweb.com/archives/newser_human_edited_news.php http://www.readwriteweb.com/archives/newser_human_edited_news.php Thu, 02 Aug 2007 14:10:49 -0800 Josh Catone
AOL is Getting Serious About Lifestreaming: Buys SocialThing socialthing-logo2.jpgSocialThing, a lifestreaming/social aggregation site, has been acquired by AOL, TechCrunch reports. We currently have no information about the final price of the acquisition, but given that SocialThing was still in private beta, we assume that it was relatively low. SocialThing was founded in 2007 with $15,000 in seed capital from TechStars. AOL seems to be rather interested in the lifestreaming and aggregation business these days. As AOL product manager Frank Gruber reported, AOL also just released its AIM BuddyUpdates yesterday.

]]>Sponsor

]]> While it is not unusual for a company to be bought up this quickly, it is interesting that SocialThing was acquired before it even came out of private beta. This could mean that AOL was less interested in the technology behind SocialThing and more in the team behind the service. SocialThing, after all, is still in such an early phase of its development that it doesn't even support Microsoft's Internet Explorer yet (though some people might, of course, consider that a good thing).

socialthing-sshot.jpg

While SocialThing does the things it does well, it never quite got the hype and user base that its nearest direct competitor Friendfeed has been getting for the last few months. While SocialThing CEO and founder Matt Galligan pointed out to us that he doesn't think SocialThing is actually competing with FriendFeed, the similarities between the two are just too striking.

It is true, though, that SocialThing is less focused on creating an internal community and puts more emphasis on sharing information back to the aggregated services than Friendfeed, especially since they just integrated ping.fm updates.

It will be interesting to see what AOL is going to do with this new property. Chances are that it will be integrated into AOL's new BuddyUpdate service or that the SocialThing team will move over to work on BuddyUpdates while SocialThing itself will become a thing of the past.

]]>Discuss]]>
http://www.readwriteweb.com/archives/aol_buys_socialthing.php http://www.readwriteweb.com/archives/aol_buys_socialthing.php News Fri, 01 Aug 2008 19:39:52 -0800 Frederic Lardinois
Microsoft's Web-based RSS Aggregator? SiliconBeat has a scoop. They point to a sandbox version of a working web-based RSS Aggregator - from Microsoft! I had a very quick play-round (note it doesn't work in Firefox) and it's similar to MyYahoo, but what's most interesting is the strong integration with MSN Search. Hmmmm.


Click to enlarge

NB: On the page it states "This site is not an officially supported site. It is an incubation experiment and doesn't represent any particular strategy or policy." But it doesn't take a brain surgeon to realise that a web-based RSS Aggregator with integrated search is going to seriously up the ante with Yahoo and Google. Interesting...

Update: Here's a screenshot that shows more clearly the RSS/search integration.

Update 2: The site (start.com) seems to be down now. Lucky I took those screenshots :-)

Also, thinking more about what this says about Microsoft's strategy for RSS... I think they're going to use MSN Search as the base for an RSS Aggregator, much as Yahoo uses the MyYahoo portal as the base for their RSS Aggregator. So while technically start.com may not be that impressive (it is a sandbox app remember), the business strategy behind this is significant. Microsoft are planning to integrate RSS into their search platform. Yahoo has already integrated RSS into their portal, so which direction will Google take I wonder?

Update 3: ZDNet: "Google began testing new customization features on its news aggregation site Wednesday in a salvo against competition from Microsoft." Here's the Google News beta. No RSS yet, but surely not far away. This makes my question above (which direction will Google take with RSS aggregation) even more intriguing...

Update 4: The Start.com prototype is back up now and they are inviting feedback. Also note there is a second version of the prototype - with blue colour scheme, but sans RSS. Here's a screenshot. Now I grok the name "start" - this is supposed to be a prototype portal... start page, geddit? :-)

Microsoft developer Dare Obasanjo now has more details about the project. He says: "This isn't a final product but instead is intended to show people some of the ideas we at MSN are exploring around providing a rich experience around Web-based RSS/Atom aggregation."

Update 5: The ASP.NET weblog has officially announced start.com: "The cat is out of the bag. We have been working on a new personalized start page."

]]>Sponsor

]]>
http://www.readwriteweb.com/archives/microsofts_webb.php http://www.readwriteweb.com/archives/microsofts_webb.php Blogging Thu, 10 Mar 2005 15:40:47 -0800 Richard MacManus
Netvibes to Share Items With ReadBurner The RSS Aggregation niche has been very busy this week. Numerous changes and enhancements were recently made to RSS aggregator RSSmeme.

However, another service that was once exclusively for Google Reader users is expanding to give users a more accurate analysis of what's popular on the web. Today, ReadBurner announced a partnership with the personalized homepage service Netvibes.

]]>Sponsor

]]> Adding Your Netvibes Account to ReadBurner

The new partnership will allow ReadBurner to analyze and aggregate the activity streams of Netvibes users to incorporate their shared items into ReadBurner. To get started, Netvibes users can head to ReadBurner's "Add A Feed" page and insert their Netvibes username.

Netvibes users can expect their shared items to start appearing in ReadBurner approximately 30 minutes after adding their account. Only the 10 most recent shared items will be immediately pulled in. However, there are plans to expand this to include the latest 30 items. While there will be no counter to display how many times an item was shared via Google Reader versus Netvibes, Netvibes users will be listed in the "Shared by" section in green to better separate the services.

Partnership Responses

It seems that to be the first service to publicly utilize the new APIs for the Netvibes Ginger platform. With the partnership being made possible through the upcoming APIs, ReadBurner's CEO Adam Ostrow has only positive words about working with Netvibes:

Netvibes Ginger is a revolutionary product that truly makes the start page a social experience. We are thrilled to be working with Netvibes to allow their millions of users to help us determine the most important content on the Web in a completely transparent and opt-in way.

VP Product Development of Netvibes, Franck Mahon, was also enthusiastic about partnering with ReadBurner:

We are very focused on making it easy for our users to share their favorite content with their friends or the public. It has really been a pleasure to partner with ReadBurner in testing our APIs and we're really excited about opening Netvibes Ginger activities streams to external applications.

What's in the Future?

There are other services that allow users to share items that have yet to be included in RSS aggregation. Are talks brewing? We sure hope so. We'd love to see shared items from RSS readers such as FeedDemon and the controversial Shftyr included in the aggregation for more accurate statistics and web analysis. For now, users can look forward to more accurate statistics about an item's popularity via ReadBurner's homepage.

]]>Discuss]]>
http://www.readwriteweb.com/archives/readburner_partners_with_netvi.php http://www.readwriteweb.com/archives/readburner_partners_with_netvi.php Product Reviews Tue, 13 May 2008 15:07:00 -0800 Corvida
Vicito News: Personalized News Aggregation Via IM Vicito News is a new personalized news aggregation service that operates over instant messenger using an IM robot. The service currently works with AIM, Google Talk, and Windows Live Messenger. Vicito is something akin to Google News alerts for IM -- you tell the service what to watch for, and it updates you at preset intervals via instant messenger when it finds new news matching your query.

]]>Sponsor

]]> Vicito would fall under the single stream aggregation category of news aggregators, in that it combines news from multiple sources about a specific topic into a single stream. You set up Vicito entirely through IM by telling the bot what keywords you want to track, how many stories you want maximum per update, and how frequently you want updates.

I told Vicito to track stories related to "baseball" and this afternoon I compared the results to those from SportsSpyder's MLB page. Not surprisingly, they were both dominated by news about the trade of Johan Santana -- baseball's top pitcher -- to the New York Mets. Somewhat surprisingly, though, I actually found Vicito's news to be more varied. Beyond Satana I was getting stories about Roger Clemens visits the Astros training camp, pitcher Andy Pettitte, and Major Leaue Baseball's umpire background check policy that were nowhere to be found amid the Santana noise (or were buried) on SportsSpyder.


Tracking "yankees" news on Vicito.

A lot of that might have to do with the source list. Vocito right now is drawing from 750 large and mainstream news sources that cover the gamut in terms of topic. SportsSpyder, on the other hand, is drawing from specialized sources that deal exclusively with baseball -- so certainly the overlap and number of reporters covering the same big story will be greater.

But the experience does highlight one major problem with single stream aggregators like these: there is a lot repetition. Because Vicito is just posting a river of news that matches your search query, you end up getting the same story from multiple source. Unlike with meme-style aggregators like Google News or Techmeme (or perhaps, more on topic, Ballbug), which group similar stories, Vicito's stream loses some appeal once you've seen the same story 5 times.

Nat Burke, founder of Vicito, is aware that eventually some sort of filtering will be necessary. "As we continue to grow our source volume, having something to weed out the noise will become essential," he told me. "What eventual form that takes is up in the air at this point."

Even with the noise problem, though, Vicito remains a useful service for getting breaking, topically filtered, news updates over IM. If instant messenger is your preferred method of communication, then you might find some utility in Vicito. Vicito is free to use, with a premium for-pay package that ups the limit on the number of news stories you can receive per update and how often.

]]>Discuss]]>
http://www.readwriteweb.com/archives/vicito_personalized_news_via_im.php http://www.readwriteweb.com/archives/vicito_personalized_news_via_im.php Product Reviews Wed, 30 Jan 2008 12:48:55 -0800 Josh Catone