Well, today, I had a surprise. One of my colleagues asked me to test something for him, and update my Group Policy - so I did:

Then he asked me to search for something in the Vista start menu - and look what I found:

That’s right, a ‘Search SharePoint’ option. Naturally, I tried it - and it searched our internal system:

Great! Not being an admin bod, I don’t entirely know how this was set up, so I asked our admin. He told me that he’d used the Window 2008 Admin Tools Pack. This comes with an applet for Group Policy Management, and he then went into User Configuration > Administrative Templates > Windows components > Instant Search and that in there you can set up a custom search provider.

The custom search provider is just a URL with %w to place the search term. Thus, the setting here is something like:

http://deltamoss01/searchcenter/Pages/Results.aspx?k=%w

(Assuming, of course, that you’re using a search center)

I’m sure it’s been documented before, but hey, it’s neat!

• For Team Sites, the sites Regional Settings
• For Collaboration Portals, the Search Center’s Regional Settings

Now, if only I can remember that this time. Often I see systems showing the dates in American date format (don’t get me started on that one!) and this is how you change the displayed format.

Of course, this raises the question of, if you have offices in, say, London and Paris, how do you support the different time zones? Hmm. I’ll have to puzzle over that.

It turns out that there is a node in the results xml file that shows this - ContentClass :

As you can see, the first highlighted result is STS_ListItem_DocumentLibrary. Pretty clear what that is. Not all results actually have a content class - the second highlighted result is from a document on a file share, and it doesn’t have a ContentClass.

So what content classes are there? Well, Dan Attis has a good list, along with a caveat about Welcome pages in the results. There is a similar list here. (Nobody mentions not having a ContentClass though). But unfortunately, I don’t know how to add new content classes

So I opened up my search page in SharePoint Designer. This always looks a bit cluttered, but I went to my Core Search Results Web Part to modify it’s ShowActionLinks property.

I’ve scribbled on this to highlight the obvious web parts on the page. The Core Results web part is highlighted in cyan. Anyway, if you look at the code in the top of the screenshot you can see that the ShowActionLinks setting is already False. Eh?

Obviously, I decided to set this ShowActionLinks setting to true to see what happened:

Now there are two action links shown. (Please note, I made some other changes while figuring this out, which is why the results themselves look a bit different). Okay, so clearly, there must be another web part. Rather than look at the pretty pictures in SharePoint Designer, I looked at the rather more useful code and found:

That’s right, there is a second Core Results web part, which is nearly invisible in the Design view of SharePoint Designer. This web part’s XSL doesn’t actually process the Results nodes of the search results - so it doesn’t show any results (which kind of belies it’s name as the ‘Core Results’ web part). Instead, it just shows the action links.

Frustratingly, the second Core Results web part is very obvious if you edit the page in the browser:

I deleted that web part from my page, and the action links vanished. I guess the question that this leaves is why are the action links not in their own web part? After all, it’s not called the ‘Core Results and Action Links’ web part is it? It rather appears that they would nicely sit as their own web part.

The important thing to remember is that a simple search is just a URL, and specifies the search term and scope in the k and s parameters respectively. E.g. This search url:

http://deltamoss01/searchcenter/Pages/Results.aspx?k=Badgers&s=All%20Sites

would run a search for badgers in the All Sites scope.

It is worth noting, however, that this is saving the search, not a set of search results. As the index is updated, the results on this page may vary. Also, advanced searches don’t pass their parameters as GET parameters (i.e. within the URL), so you can’t save these the same way. I’ve not yet found a way to save them.

As another note, the standard Search results page shows the ‘Alert Me’ link:

This lets you set up email alerts about new items that match a search:

Really, that’s pretty neat.

Anyway, assuming that you’re not going to use Alerts, then you’re looking at storing URLs. You could use a List, such as a links list, but my colleague made the interesting suggestion of using the ‘My Links’ functionality of an account:

And then you can add it to your My Links list:

Which are shown on your My Links menu:

So there we’ve got one way of saving a search in SharePoint. In reality, there are a lot of alternative - Links lists being an obvious choice, but there are things like summary web parts too. Really, it’s anywhere you can store a URL.

When I perform a standard search in SharePoint, it shows me what I’ve been searching for:

That’s good - it’s useful to know what you’ve searched for. If I then click on ‘Advanced’ the advanced search page opens showing my search term, and a number of other fields I can fill in for a more specific search.

If I then run that search, though, I get the following results page:

Okay, now that is a little odd looking. There is no search term in the search input box. This is perhaps not surprising - the advanced search screen can express more complex queries than the text input box of the simple search control. However, there is nothing to tell you what the results are of. You have no idea what the search was that gave these results!

Worse happens if you try to go back to your advanced search page. Unless you use the browser’s Back button, you will lose your current query. There is no obvious way of ‘refining’ an advanced query.

It looks like this is happening because the simple Search box passes it’s parameters in the URL as GET parameters, but the Advance Search page uses a postback-then-forward approach.

Is it too much to ask for to have a web part that describes what the conditions of an advanced search are, and to store that query to pass back to the Advanced Search page, should the user wish to refine their query.

This isn’t news - the normally evangalistic Patrick Tisseghem posted about this a year and a half ago, and plenty of folks have expressed their surprise at this. I can’t say I blame them. There are third party products that given general wildcard searching, but I’ve not see or used any of them, and some of them get mixed reviews.

You might have noticed that I spoke specifically about general wildcard searching. Well, you can do wildcard searching when you’re search for a property of a particular value. For example, a search for Title:Water would return results about water, waterloo, watergate, watering, Muddy Waters, etc.. Perhaps a comparison would be useful to make things clearer.

A normal unstemmed search for Apple:

A Stemmed Search for Apple:

Notice how it returns both Apple and Apples as matching results

A Wildcard search:

Well, that’s just an example really; it doesn’t work. The results are for a normal search for Apple

A Wildcard Property Search:

Here we’re searching for things with a Title that includes Apple and we get a result for an item called Apple.jpg, and also we get a hit on Applegate too (and who wouldn’t like to hit on Christina Applegate?) Clearly ‘Applegate’ is not a stemmed form of ‘Apple’, so even if stemming were turned on (it wasn’t) you wouldn’t expect a match.

So there sort of is some wildcard searching. However, it’s completely unreasonable to expect your typical user to be able to use that - heck, I get confused about some of the fields and field names, so a user doesn’t stand much of a change. Plus, you have to know what field you’re searching over - my example wouldn’t have matched items which had a Keyword column containing Apple, and you can’t really search fields until you stumble on the right one for the results you’re after!

I guess I’m with Patrick on this one - Microsoft, sort it out! Give us general wildcard searching!

And if I turn on stemming and search right away I get:

I was curious, so I looked up about SharePoint search stemming. I found some posts about it from Mike Taghizadeh (parts one and two) which were interesting, but didn’t tell me much about how it applies to SharePoint. I found another article I could understand better from Joining Dots. According to them, search stemming can increase the size of your index - although I’m not clear how the Core Results Web Part can affect how the indexing works, so I’m not entirely convinced. Certainly I can turn on stemming and get stemmed results without recrawling, which suggests that the index isn’t the issue.

Personally, I suspect that stemming is turned off by default because it doesn’t work so well in some languages (and, as Mike points out, for some languages can’t really).

Anyway, if you want to turn it on, edit your search results page, then ‘modify’ the Core Results Web Part:

Check the ‘Enable Search Term Stemming’ box, press OK and save your page.

Search Scopes are found through the Shared Service Provider (SSP) Search settings page:

If we View the Scopes, we get a list of the scopes and which Web Apps they’re available on:

Click on New Scope to create a new scope (obviously!) At this stage you only give it a name; rules are defined later:

Hit OK and create the scope. The Scopes list view now prompts you to ‘Add Rules’ to the scope:

Click on Add Rules to do so:

Rules can be based on a number of things - Content Source, Web Address, or Property. Properties are just Managed Properties that are configured to allow themselves to be used in scopes. You can add to those Managed Properties and make more available in this list.

For our scope, I’m using the ContentClass property. I know from my previous article the value that this is for Picture items, and I’m going to make this rule Required - I want all items in this scope to have the content class of STS_ListItem_PictureLibrary (that just trips off the tongue).

Anyway, once those fields have been filled in, press OK. Go back to the Search Settings page and go to the Scope section:

The scopes periodically update themselves; this section will tell you when the next update is. If you’re impatient, you can kick off an update immediately.

So, once the scope is created and updated, we want to use it. First, we have to configure it in our Site Collection. This does mean that different site collections can share (or not) different scopes. Go to the Site Collection Settings:

Click on Search Scopes:

This shows us the search scopes available in the site collection and the ‘Display Groups’ those scopes are in. ‘Display Groups’ controls where the scopes are available. For example, the above screenshot shows the scopes available for the small search box and the advanced search.

We’ll add our scope to the small search box. Click on Search Dropdown.

Now we can edit the display group. Check the ‘Display’ checkbox next to our Images scope. Hit OK.

Next we need to go to a page which uses the small search box and shows the scope drop down list. Your system might not always show that list. I configured my search page to display this drop down list (by editing the Search Box web part), and ran a search for ’snake’ using the default ‘All sites’ scope:

This returned me document and images about snakes. I then used the drop down box to select my ‘Image’ scope, and ran the search again:

Hurray! Search results about snakes that only include Picture items in Picture Libraries. Obviously, you may want some sort of more complex Scope Rules to accept pictures from other sources.

Be careful about using the FileExtension managed property though (as suggested here) as the FileExtension property for images comes through in the results XML as ASPX?ID=x . I don’t get why, but I don’t seem to be able to get a url such as http://server/documents/myimage.jpg - instead they all come through as http://server/documents/Forms/DispForm.aspx?ID=x . I’ll investigate that sometime; this doesn’t seem to be a factor of the XSL though.

Anyway, using Picture Libraries is good if you can get away with it.

I have a Picture library with some images in it (they’re pinched from Flickr, so sorry for the copyright thing to their respective owners - but they’re lovely looking test data). You might notice that the images in the pictures Library are already thumbnailed:

Firstly, I took a look at all the xml I was getting back from the search results. (You can add to that.)

Great! So I’m getting the thumbnail for the picture through in the search results, and I can also identify what pictures are by the ContentClass. Nice one!

The first thing I noticed on opening up the XSL for the normal results page was that there is a section about showing a thumbnail. Normal search results don’t show a thumbnail though - so what gives?

Well, there are a few conditions before it’ll show the thumbnails - the ContentClass of the result (which is okay), the presence of a thumbnail URL, and the variable $IsThisListScope being true. I’m not sure what the $IsThisListScope variable is for, but I decided to delete it from the condition. As the other parts are met the results should show thumbnails for items which have them (i.e. Pictures).

And if I perform a search for, say, balloons, then I get some results with thumbnails:

Now, it’s worth noting, this page will still show non-pictures results - just my results didn’t have any. The next step would be to create a scope to only include images.

Our client noticed that some dates appeared on the search results page as a day earlier than the value in the lists themselves. For example, the list would show a document date of the 8th of May, but the search results would show the 7th of May. Curious, and the start of a bit of a detective story…

Tracking through the erroneous dates it became clear that these incorrect dates only occurred during British Summer Time (BST). For 2008, the change to BST happened in the last weekend of March (30-31st). The list view and search results for doucments with dates spanning that time show…

This was puzzling, so I took a look inside SQL server to see what was being stored there. You shouldn’t ever work with the database directly - and shouldn’t need to - but I figured there was no harm in looking:

This is correct. SharePoint actually holds dates as DateTime columns, with the time portion set to zero hundred hours (0:00:00). During the summer time, the UK is on GMT+1, whereas SharePoint always just uses UTC (which for all practical purposes in SharePoint is the same as GMT). Thus, to correct for summer time, our date ( 2008-05-08 0:00:00 ) gets clocked back an hour to UTC time ( 2008-05-07 23:00 ). You can see this happening for the first day of British Summer Time on the line highlighted in yellow.

To reiterate, this is entirely correct. All times in SharePoint are stored as UTC, and then adjusted to the local time when it comes to presentation. Thus, offices in London, Tokyo and New York all use the same ‘clock’ when storing things (the UTC time), but will adjust the displayed time appropriately.

However, looking at our search results, we can see that the date we’re getting is now wrong, due to this time zone shift. The subtraction of an hour moves us into the day before, so our date is wrong! We really don’t want this time shift - heck, we don’t even want the time part, and our customer is entirely within one time-zone.

Therefore, the problem seems to be that the Shared Service Provider isn’t using the same timezone for crawling the content as is used by the users.

Now, I would suggest that maybe search doesn’t adjust the timezone - after all, a single site collection could contain sites from multiple timezones. However, when I tried to replicate the customer’s problem on my system I found that:

• Summer-time dates went in as 2300 hours the day before, but…
• … the document date value shown in the search results was correct!

So, it should work okay, but there is something different between my system and theirs. So, what’s different?

• I examined the site collection’s regional settings - these were the same between my system and the customer’s:
  
• I examined the regional settings of the SSP - these were the same:
  
• I checked the user’s regional settings - these were the same:
  

Next, I checked the server’s own time settings. Their server was set to GMT, but not adjusting for daylight savings time. I set mine the same, rebooted and reindexed - and started to get the same problem!

A bit more testing showed that only the changing the ‘Automatically adjust clock for daylight saving changes‘ and the reboot are actually required. This is pretty interesting as it does mean that the search index and query server must work in UTC time too. I guess that makes sense.

Anyway, if you have this problem, the short answer is enable adjusting the servers clock for daylight saving, and reboot.

The observant amongst you will notice that the titles all contain the word ‘Barnacles’, but only some are highlighted - specifically where the word barnacles is not the last word before the dot-extension of the file name. I did some further testing and found that the search does recognise the word ‘Barnacles’ in the file name, but the hit highlighting doesn’t seem to.

I wondered if this was a fault in the XSL I’m using, so I looked at that:

This code can be found here.

As you can see, all it does is test to see if there is a hithighlightedproperties/HHTitle element. If so, display that, otherwise use the normal title.

Therefore, I opened up a search and looked at the raw XML I got back:

You can see that in this case it has picked up the terms to be highlighted in the title, and has given it a surrounding <c0>..</c0> tag. Digging on through this I found that filenames that had the search term immediately before their ‘dot-extension’ would not highlight correctly. If there was a space before the dot-extension, though, they would.

Looks like a minor buglet in SharePoint.

So, what we want to do is for each search result:

• see if it is a PDF
• if it is, pass the search phrase in the url

Um, I guess I’ll skip to the code:

You can get this as a text file here.

So, what does this show? Firstly, you can see that the query string is being passed into our Search Core Results web part, as described previously. We’re also selecting the FileExtension column, also as described previously. Our XSL looks to see if the extension is PDF, and if so it emits the url to the item plus:

#search='query‘

These bits are highlighted in yellow. Otherwise, we just output the link. (Note: this page doesn’t really use a lot of the neat features of your search results XML, like the hit-highlighted title).

So what happens then if a user clicks on this link? I searched for ‘Stupid Ferret’ in the example below…

Tada - our search terms are searched for automatically when we open the PDF!

Don’t believe me? Open SharePoint Designer, and go to the properties of the Search Core Results Web Part:

Look - it has parameters:

And you can see these in the code - in just the same structure as the Dataview WebPart:

Then you can use the query string value (which I’ve called query) like so:

<xsl:value-of select="&query" />

And I’ll explain why I’ve been going through all this in my next post…

Well, these columns are Managed Properties, and they’re configured on the Shared Service Provider (SSP)…

And the Managed Properties page shows a large list of the properties available:

So, what is a managed property? Well, the page describes it pretty well. The search crawler gathers a bunch of crawled properties. Users search against the managed properties, and these are a way of mapping the crawled properties to the properties that people can search on. That’s what that page above is for; defining the mapping.

So where do the Crawled Properties come from? Well, they’re created by the crawler - and can be based upon metadata you’ve created in SharePoint, specifically from Site Columns. For example, I’ve set up a Site Column called Animal Name, which is just text, and added it to a document content type which I’ve used:

After a full crawl, I went and created a new Managed Property:

And I could now select my new crawled property:

And so we’ve created a new property:

I ran a full crawl again, and now I could go and add my ‘AnimalName’ property as a column into the Search Core Results Web part (as I did with FileExtension previously), and now I can get it in my search results!

Please note though - you will need to have the content with the new Site Column crawled twice - once to make it available as a crawled property, and then after you’ve created your Managed property you have to crawl again to actually populate that Managed Property.

Edit: Damn, having just written this, I found a better description from Jonathon Frost. It’s well worth a read too.

…you may notice that the final 3 letters are ocx - because it’s a .DOCX file. Worse, these can be of different cases, and that is what has me looking at all of this in the first place.

Well, I simply couldn’t believe that file extension wasn’t available in the XML - but from what I’d seen so far, it wasn’t. However, looking at the Search Core Results web part, I noticed it has a ‘Selected columns’ setting in the “Results Query Options” section:

And opening this for editing shows a bunch of columns being selected:

These seemed to match my results fairly closely. Is there a FileExtension column?

Well, yes, as it turns out. I found this in the SharePoint Search XSL Samples on Codeplex. The installation instructions for the File Extension Pivot sample mention adding a column…

<Column Name="FileExtension" />

So I added this to my selected columns:

And the results now showed:

Great! And the values are consistently capitalised! Hurray! Now I can use it in my styling…

Well, from what I can fathom, SharePoint search results actually come from the Search.asmx webservice in the _vti_bin of your site. The results are actually returned as XML, and in the web parts (not sure which ones, but certainly the Search Core Results web part) you then render this XML to HTML via XSLT. (If that means nothing to you, I’m afraid you’ll have to look it up. That concept is a whole can of worms. Maybe even a crate of worms. Anyway…)

An obvious question is, what does that source XML look like. Well, it’s whatever you get back from the web service, and there are various ways of find that, but I found a neat little solution on a post from Tobias Zimmergren’s blog (in fact, it’s an interesting looking series for a search rookie like myself - I think I’ll have to read that). I’ll blatantly plagiarise that in a bit. Basically, his idea was to define some XSLT that just emits the XML unaltered, and then look at what you’re getting back from the search service on the search results page.

So, to test, I copied the normal search results page (mine was in a search center) to give myself somewhere isolated to test in. I went to it - and it was all empty. Not surprising - there were no web parts. I added a Search Core Results web part:

You might want to add a Search Box web part to give yourself somewhere to enter your search terms, though I’m a bit of a geek and was typing directly into the URL of the browser. For example,
?k=barnacles
will search for documents about, um, barnacles.

Fine. Now we need to apply the XSLT. Edit the Search Core Results Web Part:

and edit the XSL:

Blatant Plagiarism alert - Tobias’ XSL was just:

<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform”>
<xsl:output method=”xml” version=”1.0″ encoding=”UTF-8″ indent=”yes”/>
<xsl:template match=”/”>
<xmp><xsl:copy-of select=”*”/></xmp>
</xsl:template>
</xsl:stylesheet>

Note: Don’t just copy the above text - use this link. Wordpress has changed the speechmarks in the code above, and it won’t work.

Apply it and voila - XML:

Now I can get on with Styling it.

This control is a ‘Delegate control’ - that is, you can create features to override the currently used control. What my colleague wanted to do was not display the ‘Scope’ drop down list, the Advanced Search link, and to include prompt text (something like ‘Enter Search…’). A quick dig into the FEATURES folder in 12 Hive showed that the control had a number of properties.

(The features that this information applies to are the OSearchBasicFeature and OSearchEnhancedFeature. Both contain files called ‘SearchArea.xml’, and that contains the code below. I found the folders with this in:

%12 Hive%\Template\Features\OSearchBasicFeature

%12 Hive%\Template\Features\OSearchEnhancedFeature )

<?xml version="1.0" encoding="utf-8" ?>
<Elements xmlns="http://schemas.microsoft.com/sharepoint/">
<Control
Id="SmallSearchInputBox"
Sequence="25"
ControlClass="Microsoft.SharePoint.Portal.WebControls.SearchBoxEx" ControlAssembly="Microsoft.SharePoint.Portal, Version=12.0.0.0, Culture=neutral, PublicKeyToken=71e9bce111e9429c">
<Property Name="GoImageUrl">/_layouts/images/gosearch.gif</Property>
<Property Name="GoImageUrlRTL">/_layouts/images/goRTL.gif</Property>
<Property Name="GoImageActiveUrl">/_layouts/images/gosearch.gif</Property>
<Property Name="GoImageActiveUrlRTL">/_layouts/images/goRTL.gif</Property>
<Property Name="UseSiteDefaults">true</Property>
<Property Name="FrameType">None</Property>
<Property Name="ShowAdvancedSearch">true</Property>
</Control>
</Elements>

This shows a property ShowAdvancedSearch which sounded pretty promising for turning off the Advanced Search link. We decided to see what other properties were available, and found a good article by Clint Cherry about the SmallSearchInputBox control, and the MSDN docs. The Property tags in the XML for the delegate control set the properties of the web control class - e.g. GoImageUrl matches the GoImageUrl property on the class. Much to our pleasure, we found the QueryPromptString displays text in the search control which vanishes when it receives focus, and the DropDownMode property allows us to turn off the scope dropdown list. Hurrah!

For the values that you can set the drop down mode to, see the MSDN docs again

I was installing a MOSS system yesterday, and starting the WSS search service was causing me problems. I kept getting errors of the form

SearchServiceInstance.Provision (server ‘VM-AWB-MOSS01′) failed. Setting back to previous status ‘Disabled’.

Well, a google trawl turned up a fairly obvious solution - which just didn’t occur to me. Make sure the account name includes the domain!

Thanks Bergen!

I’m really not sure why, to be honest. I haven’t checked the alternative access mappings, but I don’t see why the search didn’t work without the fully qualified server name. The crawling of the My Sites and Central Admin site collections worked fine without being fully qualified.

I’m sure it’s something simple - any guesses anyone?