Everytime I read the word Examine or Lucene it is always combined with doing some crazy data extravaganza that sounds magical but requires 2 strong men, a Tesla Roadster, some squirrels(N amount) and 400 man-hours to get done.

So once upon a time I got a call from a customer, "We want some simple search thingy on our page", and I was like "sure thing we got packages for that". I installed a brilliant package and they were happy, until they figured out they wanted handling for typos, when their customers searched for products and the speed was also so-so because of the node amount. So I set out to find another way around, and I found it by doing something quite simple with examine.

I'm no examine/lucene ninja, but I hope my 2 cents here can get people started on playing around on their own because with examine there is so many possibilities to use your content (sorting tons of content based on different parameters, searching etc.).

So the idea with this is giving you a little intro to what is needed to get started with examine with a very hands-on aproach :)

Examine (based on Lucene) is an indexing/search engine that takes our content/data and puts it into a "phonebook" of sorts(index), so that we can search/lookup through it with blazing speed even on large amount of data/content.

So we want to do 2 things, first we are gonna have a look on the configuration (what sort of stuff do we want in our index?), secondly we want to make it searchable.

Part 1 : The configuration 

So in every Umbraco installation there is a folder called "Config". Its filled with (you guessed it) configuration files that does tons of different stuff to your solution, and don't worry it's not dangerous to play around in here.

Two of these files we want to pop open and poke around inside.

/Config/ExamineIndex.config

/Config/ExamineSettings.config

ExamineIndex.config

In this file we want to define a new indexset, and the indexset contains the info on which doctypes and fields we want to index. For the example it could look something like this:

  <IndexSet SetName="MySearch" IndexPath="~/App_Data/ExamineIndexes/MySearch/">
    <IndexAttributeFields>
      <add Name="id" />
      <add Name="nodeName"/>
      <add Name="updateDate" />
      <add Name="writerName" />
      <add Name="nodeTypeAlias" />
    </IndexAttributeFields>
    <IndexUserFields>
      <add Name="bodyText"/>
      <add Name="siteName"/>
    </IndexUserFields>
    <IncludeNodeTypes>
      <add Name="umbHomePage" />
      <add Name="umbNewsItem" />
      <add Name="umbTextPage" />
    </IncludeNodeTypes>
  </IndexSet>

This block is just placed under the other <indexset>.

SetName is the reference, or the alias if you like that we want to remember when were gonna call the index from our providers.

IndexAttributeFields defines all the default Umbraco fields that a node contains such as name, nodetype and more.

IndexUserFields is the alias of the custom fields you have added to your doctypes.

IncludeNodeTypes is the alias of the doctypes you want to search through.

So now we have defined an indexset that takes 3 doctypes and looks for 2 properties. This is what we can search later on.

That was one file down and one to go.

ExamineSettings.config

So inside the examinesettings.config file we want to do 2 things, and that's adding a few providers (Index and search provider). These two handles, you guessed it, indexing our data/content and giving us the option to search it.

Index provider 

  <add name="MySearchIndexer" type="UmbracoExamine.UmbracoContentIndexer, UmbracoExamine"
      supportUnpublished="false"
      supportProtected="true"
      interval="10"
      analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net"
      indexSet="MySearch"/>

Paste it in just before

</providers>

</ExamineIndexProviders>

We have different settings here, should it index unpublished or protected nodes? and how often. One thing that is important is the IndexSet equals the Index SetName that we defined inside examineindex.config right before.

Search provider

The next thing we need to config is the search provider.

   <add name="MySearchSearcher" type="UmbracoExamine.UmbracoExamineSearcher, UmbracoExamine"
       analyzer="Lucene.Net.Analysis.Standard.StandardAnalyzer, Lucene.Net" indexSet="MySearch" enableLeadingWildcards="true"/>

This should just be added right before

</providers>
</ExamineSearchProviders>

Again some settings on how the search works, though not something we will talk about since we just run with defaults, but again we want to reference the IndexSet Name and enable leading wildcards.

If you want to dive deeper into the different params and get a better understanding for all this crazyness I recommend that you read this one :

http://umbraco.com/follow-us/blog-archive/2011/9/16/examining-examine.aspx

 Part 2 : Let the search begin

So now we have configured all the stuff that we need and can start by cooking up a new razor macro called something as magical as "Search" which we can embed into our awesome search template.

So now we want to create a SIMPLE search that can handle some spelling mistakes and does the "basic" job we want.

First solution
So lets look at how this can be done in a simple example first:

@inherits umbraco.MacroEngines.DynamicNodeContext
@using Examine.LuceneEngine.SearchCriteria
@{
    if (!string.IsNullOrEmpty(Request.QueryString["search"]))
    {

        //Fetching what eva searchterm some bloke is throwin' our way
        var q = Request.QueryString["search"];

        //Fetching our SearchProvider by giving it the name of our searchprovider 
        var Searcher = Examine.ExamineManager.Instance.SearchProviderCollection["MySearchSearcher"];
        
        //Searching and ordering the result by score, and we only want to get the results that has a minimum of 0.05(scale is up to 1.)
        var searchResults = Searcher.Search(q, true).OrderByDescending(x => x.Score).TakeWhile(x => x.Score > 0.05f);
           
        //Printing the results   
    <ul>
        @foreach (var item in searchResults)
        {
            var node = Model.NodeById(item.Fields["id"]);
            <li>
                <a href="@node.NiceUrl">
                    @node.Name 
                </a>
            </li>
        }
    </ul>
    }
}

 Just to do a quick run down the code we 3 things.

We fetch the term some dude just searched for from our query. We select which searchprovider to use (the one we setup in the config files remember), and last we search, oh yeah and print out (4 things sorry).

When we do the "search" we ask for results ordered by "score". Everytime we do something with Lucene, it's being returned with a score on how close it was to our search. I'm also asking it that we only want the items above a specific threshold so we ensure some kind of quality to our results.

So now when the customer searches for "geting started" instead of "getting started" it still finds the results.

Quite simple.

Little bonus: Also notice that we have something called "fields" where I'm fetching the ID of the node. But inside fields we can add all sorts of properties, so if you just needed to display the name of a page and the URL, we don't need to look up the node to fetch it, we could save some machine powa' by just adding the properties to the list of fields. That is what we did inside examineindex.config.

So this one would probably do it for most simple sites but let's say you wanna dive a bit deeper, you want to control which fields it should search through, and if some fields are more important than others. 

Second solution

Let's go a bit deeper down the rabbit hole with this one, so let's do something similar just where we have a few more options. 

In Lucene we can build our own queries for content. This can be done in 2 ways, and I will show the "Fluent" (chaining) way, while the other one is writing raw Lucene queries (you can look into this through some of the links at the bottom).

So this next example is quite similar to the first one but instead we are controlling which fields we wanna look into and if a field is more important.

@inherits umbraco.MacroEngines.DynamicNodeContext
@using Examine.LuceneEngine.SearchCriteria
@{
    if (!string.IsNullOrEmpty(Request.QueryString["search"]))
    {

        //Fetching what eva searchterm some bloke is throwin' our way
        var q = Request.QueryString["search"].Trim();

        //Fetching our SearchProvider by giving it the name of our searchprovider 
        var Searcher = Examine.ExamineManager.Instance.SearchProviderCollection["MySearchSearcher"];

        var searchCriteria = Searcher.CreateSearchCriteria(Examine.SearchCriteria.BooleanOperation.Or);
        var query = searchCriteria.Field("nodeName", q.Boost(3)).Or().Field("bodyText", q.Fuzzy());
      
        
        //Searching and ordering the result by score, and we only want to get the results that has a minimum of 0.05(scale is up to 1.)
        var searchResults = Searcher.Search(query.Compile()).OrderByDescending(x => x.Score).TakeWhile(x => x.Score > 0.05f);
           
        //Printing the results   
    <ul>
        @foreach (var item in searchResults)
        {
            var node = Model.NodeById(item.Fields["id"]);
            <li>
                <a href="@node.NiceUrl">
                    @node.Name 
                </a>
            </li>
        }
    </ul>
    }
}

So the difference here is that we can now control which fields we look at.

We do this trough something we call a SearchCriteria. Inside the SearchCriteria we tell which fields we want to look at like "nodeName" and bodyText.

Also we are telling the search that if it finds something in nodeName it's more important than bodyText since we are giving it a "boost". To a boost we add a value to buff the "score" of a resultitem.

We are also saying that we want to look at the bodyText, and we are adding a "fuzzy" option to it. This tells the search it should match on items that "looks like it". This is where we get some sort of spelling help.

The whole idea here is that we can keep on adding fields, and also if we want something specific to be true (could be inside a certain date range, only look at specific nodetypes etc.).

Wanna know more ?

 You should really take a look at these pages then :

http://umbraco.com/follow-us/blog-archive/2011/9/16/examining-examine.aspx

https://github.com/Shandem/Examine/wiki

Also ive created a small demo solution you can download and run, it's of course based on Umbraco 7 and one of the new brilliant starter kits :) If you download it you can just run the solution and open up /search

https://bitbucket.org/Rasmusfjord/24days.in-umbraco-lucene-demo/wiki/Home

Hope this will get you going on your way with Examine, now go drink some danish "gløgg" and start toying with the awesomeness that is Examine.

Merry Christmas