Can Google and Microsoft Be Challenged with Open Source Search?

Written by Alex Williams / November 16, 2009 12:56 PM / 9 Comments

Open-source search has some major advantages compared to its competitors.

First of all, it's free. Second, it stands up in comparison to the largest, proprietary search vendors. Third, there is a growing ecosystem around open-source search that makes it far easier to implement than ever before.

The combination makes open-source search a potent alternative to Google and Microsoft, arguably two of the biggest players in the enterprise search market.

Lucid Imagination

Lucid Imagination considers itself a bit like Red Hat. The company provides services for Lucene and Apache Solr, open-source search technologies. This week, Lucid Imagination is releasing a certified version of Lucene 2.9. What this means is that Lucid has tested and debugged Lucene to make it palatable for organizations to implement.

As a Lucid executive said today, it's Lucene with a "shampoo and a blow dry." It can be integrated quickly into an enterprise search environment. For example, a major online retailer downloaded the certified version and had it running within a few days across its Canadian, German and United Kingdom sites.

Lucene is downloaded several thousand times a day. It is used by more than 4,000 organizations. Many organizations have switched to Lucene to replace proprietary search software products. Beyond the issue of cost, organizations are using Lucene's flexible and scalable architecture for developing highly sophisticated full-text search applications.

Compared to Lucene, Google Search Appliance prices according to the number of searches performed. This can get pretty costly for larger scale search efforts. The API can be customized to some extent but Google protects its core technology, which requires the customer to do some work arounds. Google is targeting enterprise customers to do search within Sharepoint and in a number of other ways in the Enterprise.

Earlier this month, Google announced Commerce Search, a service designed for e-tailers to customize searches for their products.

Microsoft is banking on Sharepoint to position its search functionality within the enterprise. In particular, with Sharepoint 2010, Microsoft will launch FAST Search Server 2010 for SharePoint. Based on the FAST search technology, it combines the FAST's high-end search capabilities with SharePoint.

But the issue here is again how much customization the customer can actually do with the search technology. Due to its proprietary nature, the customer has little control over how it can be customized. Customers are forced to wait to see what features Microsoft develops.

No doubt, Lucene is a super-hot player in the enterprise search market. And who's to gain? Enterprise customers who want world-class search at cost you just can't beat. Even better is the fact that one company, Lucid Imagination, is dedicated to supporting Lucene. The company has the chance to score big in the market, especially with certified offerings such as what they are providing for Lucene 2.9.

Open-source search is here to stay. The proprietary players in the market will continue to keep significant market share but open-source search has to be considered in the mix as more companies seek to take control of its search environments.

Comments

Subscribe to comments for this post OR Subscribe to comments for all ReadWriteEnterprise posts

Note that vanilla Lucene can be "installed" in a few minutes, too. As can Solr. What takes a bit of time is integration and, in Solr's case, a bit of configuration. All that can be very, very quick if you know what you are doing or have somebody like Sematext help out.

The main benefits of any open-source (search) software are:

1) no licensing fees, no subscription fees, no per-document, no per-seat, no per-query, no per-server type costs

2) with commercial enterprise search solution the cost doesn't end at things listed under 1). You still tend to have to hire their high cost professional services people.

3) customize it to your heart's content - you can hire Lucene/Solr
experts for the *fraction* of the cost of any of the commercial enterprise search solutions (FAST, GSA, Vivisimo, Endeca, Coveo, ISYS, Attivio, etc.)

4) Support and community. Can any of the commercial enterprise search vendors beat this: http://www.jroller.com/otis/entry/lucene_solr_nutch_amazing_tech ?

5) You don't need to buy (read: pay) a solution that contains everything and the kitchen sink. You can get core search from Lucene/Solr, and get additional components that can improve your search from, say, http://www.sematext.com/products/index.html

Posted by: Otis Gospodnetic | November 16, 2009 8:01 PM

Interesting growth with open source search

Posted by: Ray Velez | November 17, 2009 8:34 AM

"Google Search Appliance prices according to the number of searches performed" this is not true, license is based on # of documents that will be indexed.

Posted by: Floris Weegink | November 17, 2009 3:57 PM

I can confirm what Floris said. I believe the intro prices (i.e. for the small number of docs) are low (think bait), and then go up more than linearly.

Posted by: Otis Gospodnetic | November 17, 2009 6:48 PM

I must say that you either confuse "Enterprise Search" with "Web Search" or over-simplify the subject of "Enterprise search". Enterprise search is all about making all content in a enterprise accessible and searchable, regardless of the source of the content (fileshares, CMS's, DMS's, databases, websites etc.)

The cost and effort to implement an enterprise search environment consist partly of license fees etc.

Search technology is becoming a commodity, as far as it concerns core search functionality: indexering textbased documents and performing full-text searches. So to some level you can switch technologies from Apache, Google and Microsoft. But search software vendors offer more than only "core search technology".

Enterprise search is all about giving insight and helping users to find relevant information and not just a list of documents. That's why the big players (Autonomy, Endeca, Vivisimo, Attivio, Exalead) put so much effort in the extra functiononalities like clustering, query expansion, topic clouds, entity extraction etc.

So for small implementations and web search you have a point. For serious search based applications and enterprise search you are off the point.

Posted by: Edwin Stauthamer | November 18, 2009 5:36 AM

Hi,

This artical is very useful for me. I am a Share Point developer and always looking to learn something new. I would like to introduce another good SharePoint blog, Have a look.

http://SharePointBank.com

lazy

Posted by: lazy | January 7, 2010 7:28 AM

There is more from Lucid Imagination in this new year. The release of much awaited Certified Distribution for Solr1.4 is finally available.

You can also download it :

http://www.lucidimagination.com/Downloads/LucidWorks-for-Solr

Posted by: Richa | January 18, 2010 9:59 PM

I suggest you have a look at the integration of the eZ Publish CMS and Solr using eZ Find. From what I've tried this is the most impressive integration yet. eZ Find keeps the CMS DB and a Solr taxonomy 1:1. And it's no beta: Production ready and installation is a snap.

Posted by: Jani Tarvainen | February 7, 2010 9:32 PM

It's interesting how proprietary vendors get bad reputations from free software advocates for being locked in or hindering innovation, but in my experience the free software is really just always catching up to copying what the paid software's R&D develops. I would be curious to see how much real "innovation" we saw in software if there was nobody making money off of it to fund research to advance it. It's easy to rally a bunch of developers across the world when you are basically cloning something that already exists. It's when you have to create a product that free software's premise starts to struggle a little bit.

Posted by: Jason | February 7, 2010 9:40 PM