Digg Filter, a Recommendation Engine for Digg - Interview with Founder
This is a guest post by Muhammad Saleem, a social media consultant and a top-ranked community member on multiple social news sites.
We have been hearing about an upcoming new way to discover content on
Digg for quite a while
now. The new Digg 'recommendation engine' will purportedly look
at your past submissions, Digging, and burying activity - and from all that recommend
other stories that you might like. This system won't be much different
from how Reddit's 'recommended' page works, or how StumbleUpon
generally functions. While we wait for the Digg recommendation engine,
which is perpetually 'coming soon', one enterprising Digg user has
taken the initiative and built one himself. After putting in 200 hours of his own time, Dmytro Mulyava has
come up with Digg Filter, an API based Digg story recommender.
We had the chance to talk to Dmytro and ask him some questions about Digg Filter.
Before we begin, can you tell our readers a little more about yourself. Who are you, what you do, and how you got involved with Digg?
Sure, I am student at a business school here in Toronto and I am really passionate about the Internet and the things that it allows people to do around the world. I first heard about Digg through an article in Business Week in August of 2006 and thought to myself “Hey, that’s a pretty neat idea!”. Since then I have found Digg to be a great source for things that wouldn’t be broadcasted by other media sources.
How active are you on Digg and what is your opinion on how the site has evolved since when you first joined and started participating in late 2006?
I lurk Digg a few hours a week and Digg some stories here and there, based on the content I find to be interesting. I am certainly not a power user – I never submit stories because I know my stuff is never going to make it to the front page. I love the comments – the top 5 (sorted by Diggs) for any story can be downright hilarious! Digg has been is growing at a very fast pace. While some feel that this is “bad” because content quality is declining as the user base expands, I haven't actually noticed this.
Digg has many different visualizations and ways to discover new content. Which would you say is your favorite?
I like Digg Spy. It is simple yet useful. There is something addicting about watching the stories scroll through and see how users are interacting with them (submitting, Digging, burying, and commenting).
But even given all these tools, you felt that there was something missing in the features Digg has and you decided to make a service of your own called Digg Filter. Tell us a little about this service and how you envision it being used.
The idea behind Digg Filter is very simple. I want to help people discover content based on their past preferences. Digg Filter looks at users’ Digging patterns and tries to “guess” what fresh content they will enjoy.
There are many, many stories submitted to Digg on a daily basis, more than anyone could ever sift through. Most of these stories never make it to the front page – and as such are undiscovered by the majority of the people. Hopefully the tool will help users find these “hidden gems” that the majority of the community moderated so they couldn't be promoted, but may be of interest to you.
I see the service being used by Diggers who are in a rush to find “the stories that matter” without flipping through many pages in the “Upcoming” section or relying completely on the front page of Digg.
Could you give us an idea of how these results are formulated? What specific data are you taking into account?
Digg Filter looks at pretty much everything. (grin)
There are a couple of things that worry me about the site though. First of all, you use Digg in the url and the name of your site, something that could get you a cease and desist from Digg, and second, Kevin Rose has already mentioned that an official Digg recommendation engine is coming soon. What impact do you think these two things may have on your service?
Those are two very valid concerns! If the folks at Digg want me to shut down the service, so be it. It (Digg.com) is their site and their trademark. I am obviously using their API to piece this together. They can unplug me at any minute and I understand that and hope that they will be a little more reasonable about it.
As far as the official Digg recommendation engine is concerned, I started work on my recommendation algorithm before I found out that their recommendation engine was in the works. When I did find out about their engine, my first instinct was “Oh damn, I've wasted so much time for something that will end up being useless!” [However] I decided to finish DiggFilter and put it online regardless.
How much time would you say you've spent on making this service?
Approximately 200 hours, give or take a few.
We think that the service is a great one, and we hope that Digg gives you some support and at least thanks you for your efforts. Thank you Dmytro for taking the time to answer our questions.
Share2 TrackBacks
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/1807
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
Interesting concept. Sometimes I like a recommendation instead of digging some of the things that I don't want. Good find.
Perhaps if he gets shut down at all, he could use his technology on another similar site. Something where you can upload the OPML file of your subscribed feeds and then got a list of "similar feeds" would be really cool, for example, and would have a similar underlying mechanism I'd guess (in terms of comparison, etc).
I doubt if Digg is going to give him any support. That's one of the problems with Digg. They never acknowledge the work of their users such as DigPicz or this Digg Filter. I think we should have better social media system where the community can helping in the building of the system itself. Read my post if you are interested: http://abcdefu.wordpress.com/2007/12/01/open-social-media
I thought Digg was supposed to be a community-based recommendation engine. Now we're building on top of it a personalized recommendation engine? It's a form of double-filtering.
There are times where this might be very useful but I wonder if it's very useful to be applied for Digg. Yes I gave it a go but it said I need to be a bit more active for it to figure me out.
Pierre
Very nice concept...digg is like a avalanche of links...sorting through the most usefull ones is a pretty neat idea. It also goes on to suggest that personlization is the next step towards adding richness to applications.
The only concern i have is of security. I wouldnt like a hacker/spammer to get its hands on the list of links/sites that i like.
I would say this is a neat idea but i will point out to you why it is a bad thing. If people come to rely on digg filter it will narrow the scope of the opinions they come across.
Say your a die hard republican, with digg filter you will end up getting very little opposing views. This will solidify the premises they hold and lead to less questioning.
If you are certain you are right about something then thats cool but keeping an open mind and trying to understand the viewpoint of others is essential to stay sane.
Individuals could end up becoming even more polarized than they are today and that is the last thing the internet was made for. The internet was made to share opinions ideas and pose questions not to solidify bias.
This is a dangerous road and i can see Digg actually taking this idea on as they will one day become establishment media. Diggs USP is slowly dying...
as some one who has received a C&D letter from digg for diggcard.com I hope that digg handle this differently to the way they handled me. When I heard about the first C&D letter for a domain using digg in the name I emailed them for advise and offered them the domain for free and all I got back was a C&D letter and noone at digg would return any contact from me. I actually sent them a few bug emails etc before and liked the site but now they have left a bad taste in my mouth.
Digg has been very "notorious" with sites using Digg as part of their app url and name, I think a C&D will come soon.
Digg Filter has a very nice idea, but when I use it I don't see "fresh" content, it just displays submissions that have have 300+ votes.
Given that the results are based on my past votes and such, this could be a problem specific for me. Though in the interview here the creator says:
"There are many, many stories submitted to Digg on a daily basis, more than anyone could ever sift through. Most of these stories never make it to the front page – and as such are undiscovered by the majority of the people."
So, why even display something that obviously has a spot on the frontpage?
I think a lot of the commenters are missing the point. This is a tool to help people find more relevant information. If it doesn't work for you, don't use it. That's pretty simple.
@Pierre, I love the idea of a double filter, I think that's the point. Take the web, filter it by popularity among digg users, then filter it by personal preferences. That equals, popular and personally interesting stories. Isn't that why we use the web in the first place, to find these?
@Random Vandal, I totally disagree. Digg isn't popular because it promotes open-mindedness, it's popular because it brings people interesting content that they might not discover otherwise. And I hope Digg isn't the only place people are gathering information from, from which they form and solidify their opinions on things. If so, that's an entirely greater problem than DiggFilter only showing them one-sided opinions. Finally, I think the filter would have to be pretty sophisticated to only show results on topics that are consistent with their own viewpoints. It may end up finding stories that are of interest, but being discussed from alternate viewpoints (the algo not knowing the difference between conservative or liberal), and open up the reader to more diversity than they'd find if they were left searching themselves trying to find other like-minded stories.
@Jeremiah, If I got to the point where DiggFilter was accurate enough to bring me quality stories, I'd immediately quit going to the frontpage of digg altogether, for that reason, I'd want front page stories included in my filter. Or possibly there can be a setting to turn off front-page stories...
As for C&D, I think Digg misses the boat here. Why anyone would actively fight against individuals passionate enough about their product to enhance it is beyond me. If someone were monetizing their site to the point it was making a significant impact on Digg's bottom line, then that makes sense, but I think Digg is far to quick to act in these cases, and should embrace their free, 3rd party developers. It's things like this that in the end create migrations from one system to another. Just look at how quick myspace started to lose ground to Facebook when Facebook decided to embrace outside developers while allowing them to monetize their efforts. Digg isn't the only kid on the block, and other social media sites could really jump ahead at moments like this by stealing the early adopter/passionate user/developer crowd and luring them to their sites with promises of recognition and support.
I actually think someone will sneak up and catch digg out in the next couple of years and they will be left with fighting it out with their clones :p