Stuck Index for FAST Search in SharePoint 2010

The Problem

Here was the situation: my fast search crawl seemed to be running fine. There were no crawl errors reported in Central Admin under the crawl content sources listed under the FAST Connector. However, I was not getting any results dated after Jan 12, 2013 (at the time of this post is it August 2013). Obviously something was wrong. I uploaded a new document within a document library. I ran an incremental crawl. I could see in CA that the document was crawled, but again when I searched for that document on the site, my new document did not return in the results.

I reset the index and still no luck. I’ve been working on this off and on for a while now. Finally tonight, I was successful in discovering the answer.

My index was stuck! Here’s how to find out if your index is stuck.

Determine If Your Index is Stuck

On your FAST server, open a CMD prompt, Within the directory for your FAST install, there are some tools to help you. In CMD, type c:\FASTSearch\bin\indexerinfo status (this is the default install location, yours may be different.  You will get some xml output. The important line is near the top and will say something like

 documents indexed="0" not_indexed="692711" size="721755503316.000000" total="692711"

Notice the part in that line about “not_indexed.”  So even though my crawl was good, my index was stuck.

Resume and Reset Indexing in CMD

Next step is to run the following in CMD (use the location of your FAST install, default is shown below):

C:\FASTSearch\bin\indexeradmin resumeindexing

This works for some people to get the index unstuck. But that didn’t work for me. Next step, in CMD:

C:\FASTSearch\bin\indexeradmin resetindex

Running this line made it so that all of a sudden I had NO results coming back for searches. Yikes!!

Reset and Clear the Index in Central Admin

So I cleared out the index with the following steps:

1.       Make sure the account performing this is a service application administrator for the FAST Content SSA

2.       Go to Central Admin à Application Management à Manage Service Apps

3.       Click the FAST Content SSA

4.       On the Search Administration Page under Crawling, click Index Reset

5.       Click OK

Before any new crawls are started you must manually clear the content from the content collection.

1.       Log onto your FAST server

2.       Open the Microsoft FAST Search Server 2010 for SharePoint shell as administrator

3.       Type the following command

Clear-FASTSearchContentCollection –Name sp*

4.       Wait for it to finish, it will most likely take quite a bit of time. 

*The name of the content collection could vary. To get the name of your FAST search content collection, in FAST powershell type

Get-FASTSearchContentCollection

Voila! Success! I have results and they are current now. Yay!

Resources

http://social.technet.microsoft.com/Forums/en-US/37b6c736-5340-46fb-8852-456f46d5ba8e/fast-sharepoint-content-crawled-verified-on-server-cant-get-results

http://technet.microsoft.com/en-us/library/ee943511(v=office.14).aspx

http://technet.microsoft.com/en-us/library/ff381262(v=office.14).aspx

Disable All SharePoint Timer Jobs Console Application

I found I needed to disable all the timer jobs in SharePoint 2010. Since there are over 100 jobs, I wrote a quick console application to disable them all for me. Below is the code. You can alter line 34 to job.IsDisabled = false; to make this application enable all the SharePoint timer jobs instead.

Note: Be sure to change the Target Framework of the project to .NET Framework 3.5.

using System;
using System.Linq;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;

namespace DisableTimerJobs
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {         
                SPFarm farm = SPFarm.Local; 
                //Get all SharePoint Web services 
                SPWebService service = farm.Services.GetValue<SPWebService>("");

                int Count = 0;

                foreach (SPWebApplication webapp in service.WebApplications)
                {

                    foreach (SPJobDefinition job in webapp.JobDefinitions)
                    {

                        //this example is to run on one service only
                        //if (job.Name == "SchedulingNotification")
                        //{
                            Count++;

                            try
                            {
                                //disable all jobs
                                job.IsDisabled = true;
                                job.Update();                               

                                Console.WriteLine(Count + ") " + job.Name + ": ");

                                if (job.IsDisabled == true)
                                {
                                    Console.ForegroundColor = ConsoleColor.Yellow;
                                    Console.WriteLine("Disabled");
                                    Console.ResetColor();
                                }
                                else
                                {
                                    Console.ForegroundColor = ConsoleColor.Green;
                                    Console.WriteLine("Enabled");
                                    Console.ResetColor();
                                }                                
                               
                            }
                            catch (Exception ex)
                            {                                
                                Console.WriteLine(Count + ") " + job.Name + ": ");
                                Console.ForegroundColor = ConsoleColor.Red;
                                Console.WriteLine("ERROR: ");
                                Console.WriteLine(ex.ToString());
                                Console.ResetColor();
                                Console.ReadLine();
                            }                            
                            
                        //}
                    }                   
                    
                }
            }

            catch (Exception ex)
            {
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine("ERROR: ");
                Console.WriteLine("An error has occured: ");
                Console.WriteLine(ex.ToString());
                Console.ResetColor();
                Console.ReadLine();
            }

            Console.WriteLine("Done. Press enter to exit.");
            Console.ReadLine();
        }
    }
}

Add Metadata Tags in SharePoint 2010 to Improve SEO

SharePoint 2010 doesn’t have a built in mechanism to handle metadata tags for improved Search Engine Optimization (SEO). My solution for adding metadata tags uses a combination of out-of-the-box (OOB) functionality combined with code to allow dynamic addition of a meta title, meta description, and meta keywords on a per page basis for content editors. I tried to keep it very simple, easy, and extendable.  There are four major components to this solution:

1.       Set up the pages library

2.       Create a SEO user control

3.       Add the reference to the web control on the master page

4.       Fill in the content

Pages Library Set Up

Enterprise Keywords
Enable Enterprise Keywords – In the pages library, go to Library Settings > Enterprise Metadata and Keyword Settings. Check the top box Add an Enterprise Keywords column to this list and enable Keyword synchronization. This adds a column called Enterprise Keywords to the list which is tied to the Metadata Term Store.

Out-of-the-Box Columns
The title and comments field are also used for this solution, but they are already in the list for you. The comments column on the pages library is the same as the description for the page. I don’t know why Microsoft called it comments. This confuses many content editors. But they did and there’s nothing I can do about that...

SEO User Control

Create a new user control. I called my control SEOPageMetaTags.ascx. In addition to the standard namespaces at the top of the SharePoint control, you also need to add the web control namespace at the top.

<%@ Register TagPrefix="SharePointWebControls" Namespace="Microsoft.SharePoint.WebControls" 
Assembly="Microsoft.SharePoint, Version=14.0.0.0, Culture=neutral, 
PublicKeyToken=71e9bce111e9429c" %>

Then add this code to SEOPageMetaTags.ascx:

<asp:PlaceHolder ID="MetaDescriptionHolder" runat="server">
<asp:Literal ID="metadescstart" runat="server"></asp:Literal><SharePointWebControls:FieldValue id="PageDescription" FieldName="Comments" runat="server"/><asp:Literal ID="metadescend" runat="server"></asp:Literal>
</asp:PlaceHolder>
<asp:PlaceHolder ID="MetaKeywordsHolder" runat="server">
<asp:Literal ID="metakeystart" runat="server"></asp:Literal><SharePointWebControls:FieldValue id="PageKeywords" FieldName="23f27201-bee3-471e-b2e7-b64fd8b7ca38" runat="server"/><asp:Literal ID="metakeyend" runat="server"></asp:Literal>
</asp:PlaceHolder>

It’s important to note that the metadata tags cannot be dynamically generated on the page, like with JavaScript, as this isn’t always able to be seen by crawlers. So, on the code behind (SEOPageMetaTags.ascx.cs), add the following code:

/// 

/// This control writes out the meta description and meta keywords in the page head used for SEO.
/// 
    public partial class SEOPageMetaTags  : UserControl
    {
        protected void  Page_Load(object sender, EventArgs e)
     {
        this.metadescstart.Text = "<meta  name=\"description\" content=\"";
        this.metadescend.Text = "\"  />";
        this.metakeystart.Text = "<meta  name=\"keywords\" content=\"";
        this.metakeyend.Text = "\"  />";
        }
     }

Ensure the SEOPageMetaTags.ascx page is uploaded/deployed to the location:

C:\Program Files\Common Files\Microsoft Shared\Web Server Extensions\14\TEMPLATE\CONTROLTEMPLATES

Master Page

Add this line at the top of the page so the master page knows what control to reference:

<%@ Register TagPrefix="MyCustomStuff" 
TagName="SEOPageMetaTags" 
Src="~/_controltemplates/SEOPageMetaTags.ascx" %>

Add this code in the <head> under the title tag:

<!-- for SEO --> 
<asp:ContentPlaceHolder id="SEOMetaTags" runat="server">
<MyCustomStuff:SEOPageMetaTags runat="server" />
</asp:ContentPlaceHolder>

Edit Page Properties

From the content editor perspective, the last step is to fill in the information on the page properties. Just edit the properties of the page and fill in the Title, Comments (same as page description), and the Enterprise Keywords field.

Benefits

There are three main benefits to using the Enterprise Keywords field in this situation.

1.       Keywords help a site’s SEO (albeit very minimally, it still helps).

2.       Keywords are standardized since they come from the Term Store.

3.       Enterprise Keywords show as refiners on your search results page so this enhances your page’s findability.

You could also use the terms entered here as a means to organize your content on your site, but that’s outside the scope of this article. It just goes to show that using Enterprise Keywords can do a lot of work for you that goes beyond just SEO and can help a ton especially if you have many content editors.

Finally, this solution is easy to extend. The SEO tags covered in this article included the Title tag, meta description, and meta keywords. You could add other tags as well. For example, an index tag to include the option for content editors to choose not to have a particular page indexed or to add a page canonical url. Using the separate web control for SEO allows you to quickly add whatever additional metadata you need.

It's been a while...

It's been a while since I've posted anything to my web design blog. I have spent the last year working on an amazing project building a global Internet site on the SharePoint 2010 platform. It consumed nearly every minute of my waking hours so I didn't have any time to add posts here.

All that has changed now and I have some ideas for some great posts particularly in the SharePoint world. I am currently working on a post on using a .Net control to add SEO meta tags to master pages in SharePoint 2010 since this feature was not included out-of-the-box. I hope to post that finished article this week. Other posts I have in mind include creating a secure store and using that secure store to access a SQL database to build external content types using BCS.

So stay tuned... Good stuff is on the way!