Build A Custom Search Engine Using Your Social Bookmarks
Last week, Yahoo finally unveiled the long-awaited new version of the social bookmarking site Delicious. Along with the new URL, simply delicious.com, the site got a revamped UI and added new features like selectable detail levels and alphabetical sorting of bookmarks. However, amid the delighted oohs and ahhs from the tech community over the new-and-improved site, some people were raising the valid question: "Who bookmarks anymore?" Besides bookmarking for the sake of making sure a site gets seen in your FriendFeed stream, the truth is that many people bookmark, but then turn to Google search when they actually want to find something.
If that's the case, then why are we bothering to bookmark at all? Instead, why not just build a Google Custom Search Engine that searches your Delicious bookmarks as well as any other sites you find interesting? The process is actually surprisingly simple.
Here at RWW, we're big fans of Custom Search Engines, and we use them regularly, so it just made sense to make one from our bookmarks.
Getting Started
The first step to building a custom search engine is to go to the Google Custom Search Engine site. Building a CSE is pretty simple - just click the button on the homepage to start and then fill in the search engine details (name, description, etc.). We're not going to hold your hand through this process step-by-step - it's very straightforward and Google provides documentation on how to do this. To get through the initial setup, you'll have to include at least one URL to search (we suggest www.readwriteweb.com, of course). You can remove it later on if you would like, though.
Once the engine has been configured, click the link to go into the engine's "Control Panel." You'll be on a page with links across the top that let you manage various aspects of your engine. Click the link that reads "Sites:"
Get Your Bookmarks
Now, in a separate tab of your browser, login to your social bookmarking account (delicious, ma.gnol.ia, etc.) and export your bookmarks through the option provided in your settings.
The reason we're doing this is because, in order to proceed, we need to get all the bookmarks to appear on one page on the web, so unless your particular service offers that as an option, just proceed with the export.
Once exported, you'll need to upload your bookmarks back to the web so that they all appear on one page. If you want to post them to your blog or use a simple page creator program to do so, that's your choice. However, I found that the absolute easiest way to get them on the web fast was to use the new service from Posterous.
Posterous is a lightweight blogging service that lets you blog without even setting up an account. You just email post@posterous.com. Compose an email to that account and copy-and-paste the content from the HTML file that contains your exported bookmarks. Posterous will quickly email you back with a link to that page on the web. (This is also a handy way to back up your bookmarks, too.)
Now click the link emailed to you to go to your Posterous page on the web. You'll want to actually click the permalink to the blog post you created, for example: http://myblogname.posterous.com/my-first-blog-post-1159
Add Your Links to Your CSE
The final step is to return to the tab where your CSE control panel is loaded. Click the "Add Sites" button and a dialog box will appear. Paste in the URL (the permalink) from Posterous and choose the 3rd bullet ("Dynamically extract links from this page..."). Then choose the first bullet underneath that ("Include all pages this page links to."). Click "Save." (Now you see why I made you put all the links on one page, right?)
You're done! You can now test out your engine. If you did everything correctly, your search engine will just search through the sites you've bookmarked.
But Wait...There's One More Thing!
OK, great, you now have your own custom search engine up-and-running, but what if you want to add to it in the future? Luckily, there's an easy browser bookmarklet you can use to do so: the Google Marker. As with any bookmarklet, you simply drag it to your bookmarks toolbar to install it.
Now, when you come across a site you want to add to your search engine, you just click the bookmarklet. If you have more than one engine, you can choose the one you want from the drop-down box provided. You also have to option to bookmark either everything on that site or just the page that you're currently on. When you're done, click "Save."
There you have it - instead of bookmarking and tagging, you can just search instead.
ShareRelated Entries
2 TrackBacks
TrackBack URL for this entry: http://www.readwriteweb.com/cgi-bin/mt/mt-tb.cgi/4601
Comments
Subscribe to comments for this post OR Subscribe to comments for all ReadWriteWeb posts
This is very innovative idea. Searching is easier. I am going to try this for some time and see if it is more productive habit.
Thanks for the information! I'll translate some of it and post in my blog.
why can we just use http://www.nsyght.com/
Posted by: thejeshgn.com
|
August 5, 2008 10:50 AM
Smart workaround, but I think it would be painful to not be able to easily add more bookmarks from delicious. I demoed NSyght too, though it doesn't do what I thought it would. Ideally, the coolest new search engine for me would be one that indexes all of my social properties (i.e. everything I've bookmarked with delicious, everything I've twittered about, etc.). The value in this is that I am indirectly recording everything I'm interested in. The ability to search back through that history would certainly be valuable.
With the enhanced search built into the latest rev of delicious, what's the point of this?
I've done this with a few of my websites. For example I own 3 different photo blogs on different niches, but my search engine searches all of them to give them cross-site exposure. Good tips
It would be great if I could add my Google Reader OPML file to this to just search the blogs I keep track of (or use it to manage a custom search on my site).
this tool already does this
and builds refinements into your cse
I'll give this a miss if that's okay.
I'll just keep clicking the icon in Firefox that lets me instantly tag any page to Delicious and use the time I've saved jumping through the hoops outlined here for some productive work.
...and here is another prototype doing exactly this, with some comments:
- proto: http://gadgets.kbac70.googlepages.com/csebuilder.html
- comments: http://kbac70.blogspot.com/2007/07/custom-search-engine-builder.html
Sahara,
You could also try Lijit. Rather then going through all the problem of creating and updating your CSE engine everytime something changes just let Lijit find all your content and index it. Lijit also leverages your network of people (and their blogs, bookmarks, photos, etc) to create a larger body of relevant content.
A search of my vacations at the Broadmoor in Colorado including del.icio.us and flickr photos..
http://www.lijit.com/search/toddvernon?q=Broadmoor&type=content
@Daniel Bachhuber: You should check out http:// Secondbrain.com. We import many of your social properties so that you can organize and search in everything.
You can use Twine.com to do this as well. We will soon enable you to import your bookmarks and search them in Twine. You can also email content into to Twine to quickly add it.
Will you marry me?
Great article, dugg.
That is a neat way to share all your bookmarks with visitors to your Website. Thanks for sharing this info!
I built this search engine http://www.google.com/coop/cse?cx=012637369834019564726:c0l3mrytnxu
on technology after I read your previous post on Search Engines and it helps me do my tech search very easily . Thanks
You not use Jamespot or Twine to do that ?
Wide-eyed again, Sarah and to all those who gave their comments... Thanks.
Things you can do with bookmarksIt just never ends
Best.
alain
www.mor.ph
Since 1 1/2 years the Focuss.Info Initiative has started the same idea. This initiative is a collaborative platform where, on one side, regional, national and international institutions are storing their favorite websites in social bookmark accounts (such as del.icio.us). On the other side we ask researcher, students and practitioners to do the same. The similarity between these institutions and individuals is that they are all working or interested in the field of international development co-operation.
Consequently, these handpicked websites stored in the social bookmark accounts are being indexed by our Google CSE search engine.
Thus, this initiative stores knowledge of valuable websites in the field of international development co-operation, shares this knowledge through the specific search engine and consequently let people create new knowledge within this specific field by letting them search through the knowledge of their peers.
If you require extra information about this Initiative, please do not hesitate to contact me.
Richard
Sarah, I was going to blog this but now I think I will comment instead.
Why would you willingly turn all your own bookmarks over to Google, when you've handpicked them and Yahoo bought them from you? Sounds a bit strange to me.
Is there a reason why Google doesn't already search delicious, or Furl for that matter, anyway? I'm afraid I'm with the commenter who noted that there is enhanced search in delicious now.
It also ought to be possible to build customised SEs for this purpose with other tools? just wondering...
Deligoo is great for this.
http://www.deligoo.com/en/
Congrats on the marriage proposal from Jonny btw.
Very interesting idea. Google CSE is a great tool and free to all. If you haven't you should look into it.
I have about 5000 delicious bookmarks, but when I sent a Gmail to Posterous, my first blog post has not even a 1000
http://john_7rrds.posterous.com/johnts-bookmarks
Also what about new bookmarks I add to delicious, will they also be included auto-magically
Really good idea indeed!
Great! But what if you want to export only selected bookmarks? For example, you want to use some bookmarks for one CSE, some â for another, and some you don't want to add to search at all.
I haven't found how to work with selections of bookmarks at Delicious. Though, you can make this via Diigo. This service has great Send to Blog feature (under More Actions... menu), which allows us to get HTML code of the list with selected bookmarks. Then you can post this code to Posterous and add links to CSE, as described in the article above.
By the way, if you're Delicious user, don't worry, Diigo can import your Delicious bookmarks.
Love the concept of searching among the nearly 400 bookmarks I ve saved.
http://www.delizzy.com is quick and easy to use for searching through your delicious bookmarks.
We are professional web development company providing customized solution in php, dot net, oracle and java. We have qualified professional team for development and marketing. You can allow us to link to your website. Please refer we are following theory of no spammin. Thanks
Hi I would like to mention another search engine for dot net developers which cover all good sites for dot net and related technologies as of August, 2008.
http://tips.developersvoice.com/devsearch
This developers search engine is inspired by searchdotnet.com and I try my best achieve even batter results. Both these developers search engines would greatly help the developer’s community.
DO test it yourself and let me know your comments.
This is my implementation CSE for del.icio.us