Fast Search - PHP Fulltext Search Plugin (MT Hacks)

Fast Search - PHP Fulltext Search Plugin

Update, Oct. 9, 2006 - A newer version of Fast Search has been released. Click here for details.

The built-in search tool in Movable Type can be slow and can take a lot of processing power, especially if you have blogs with many entries. When I recently upgraded to version 3.31 of MT, I was dismayed to learn that searches became slower and more processor-intensive (at least on my server). I was forced to disable the mt-search.cgi script, as it was overloading my dedicated server. So I went looking for an alternative. I found a PHP mySQL Fulltext method that worked very fast, and without the CPU problems.

So contacted Iñaki, the creator of this PHP solution, who granted me permission modify the solution and package it up as a Movable Type plugin. I have added a number of features, and I have made significant efforts to make the plugin easy to install and use.

Fast Search is a PHP plugin for Movable Type. This means that Fast Search uses Movable Type's dynamic PHP-based publishing system to display search results. However, you can continue to use static publishing for all of your indexes and archives. Fast Search is fast in part because it uses mySQL fulltext indexing -- basically this means that a search index will be created for the entries in your database, making searches much faster.

Template Tags

Version 1.0 of FastSearch is designed to easily replace the existing search functionality in Movable Type. For example, if you are currently using customized search templates, you can use them with Fast Search, with very few changes in most cases. Many Fast Search template tags have the same names as their CGI counterparts:

<MTSearchString> - The search string (query) being searched.
<MTSearchResults> - A container tag that displays search results. Arguments include (all are optional):

lastn - restricts the search to n results. (example <MTSearchResults lastn="10"> ) The lastn argument, like all arguments for MTSearchResults, can also be specified in the URL (...&lastn=10) or POSTed from a search form)
offset - used in coordination with lastn, starts M entries from the start of the list. (example <MTSearchResults lastn="10" offset="10"> will display results 11-20)
order - must be either 'date' or 'relevance', the default is 'relevance'. (example $lt;MTSearchResults order="date"> will provide similar results as the CGI search)
category - must be a valid category ID for the current blog - restricts the search results to entries from this category.
google_highlight - if specified, search words will be highlighted. (example <MTSearchResults google_highlight="1"> )
query - the search string (text) to be searched. In normal use, you would NOT include this argument as part of the <MTSearchResults> tag, rather, it would be specified as a URL paramter (example ...query=search+phrase) or POSTed from a search form.

<MTBlogResultHeader> - this is container tag that displays content before the search results. (Note: in version 1.0 of Fast Search, you cannot search multiple blogs in the same search)
<MTBlogResultFooter> - this is container tag that displays content below the search results.
<MTNoSearchResult> - A container tag that displays its content only if zero results are found. Note that this tag must be used after a <MTSearchResults> container.
<MTNoSearch> - A container tag that displays its content only if no search phrase was specified.
<MTSearchCount> - The number of search results displayed. This tag should be used with a <MTBlogResultHeader> or <MTBlogResultFooter> container.
<MTSearchFirst> - The number of the first search result on the current page. This tag should be used with a <MTBlogResultHeader> or <MTBlogResultFooter>container.
<MTSearchLast> - The number of the last search result on the current page. This tag should be used with a <MTBlogResultHeader> or <MTBlogResultFooter>container.
<MTSearchIfNextPage> - A conditional tag that displays its content if there is a subsequent page of search results. This tag should be used with a <MTBlogResultHeader> or <MTBlogResultFooter>container.
<MTSearchNextPageLink> - Output a URL to the next page of search results. This tag should be used within <MTSearchIfNextPage> tags.

As you may have noticed, most of the tags above are identical to the ones used for the existing CGI-based search results. This is by design, so you will easily be able to cut-and-paste your existing search template code into the Fast Search template, though some minor modifications may be required.

Working Examples:

Here are some working examples you can try. Feel free to try additional search queries and note the speed that searches are performed.

Mars Rover Blog has over 5,000 entries. Example search: "water on mars". Note that this example shows the default FastSearch template, which is very similar to the default search results template in MT 3.2.

Seinfeld Blog has over 8,000 entries, 180 of which consist of the episode descriptions and scripts of all 180 episodes. Fast Search powers the Seinfeld Script Search, which is restricted to a single category containing the episodes. Example search: 'bubble boy'

Requirements:

Movable Type 3.2 or 3.3
mySQL 4.0.1 or higher

Instructions

Download the plugin and expand the zip file.
Upload the contents of the 'plugins' folder to the 'plugins' directory of your MT installation (usually something like /cgi-bin/mt/plugins).
Upload the contents of the 'php' folder to the 'php' directory (usually something like /cgi-bin/mt/php)
Login to the MT admin interface. If you already logged in, choose 'Main Menu'
If your are running MT 3.3, you should be redirected to an 'Upgrade' screen at this point. The Fast Search upgrade process will update your database to add a fulltext index to the table in your database where entries are stored. Note that this may take some time if you have many entries, and this will add to the overall size of your database.
If you are running MT 3.2, you will not see the upgrade screen (as it is a 3.3 feature), so you need to create the index by browsing to the 'Settings' of a blog where you want to use Fats Search. Choose the 'Plugins' tab then find 'Fast Search' and choose 'Settings'. Now click the link in bold that says "Click here to create or rebuild the FastSearch fulltext index". Again, this may take some time, please be patient. After the index has been created, hit the Back button.
Now browse to the 'Publishing' settings tab for the blog, and scroll down to the Setting called "Dynamic Publishing". If the first option, "Build all templates statically", is selected, you should change this setting to "Set each template's Build Options separately" and click "Save Changes" at the bottom of the screen. Otherwise, you don't need to make any changes here.(Note that change this setting will create an .htaccess file in the root path of the blog. Normally you won't notice this and everything will work fine, but if you have an existing.htacces file, you may want to take a look at it to make sure everything looks okay)
If you just changed your Dynamic Publishing settings in the previous step, you should now rebuild all files for this blog by choosing "Rebuild Site" from the left menu, then "Rebuild all Files". You should also ensure that you have a "Dynamic Site Bootstrapper" (mtview.php) index template and make sure that it gets built, if it hasn't been built previously.
Now return to the Fast Search settings ('Plugins' tab within 'Settings') and check the box to enable Fast Search. You may also want to adjust the settings for Maximum Search Results and logging, then click "Save Changes".
Open the Fast Search settings once again and choose "Click here to install the default Fast Search Template." The template will be created and then displayed. This is an dynamic Index template (which can be customized) used to display the search results. (If desired you can also skip this step and create the index template manually, use the template tags as desired, or by cutting and pasting your exisiting search template code)

The Search Form

If you already have search forms on your blog pages, you need to update them to point to the new Fast Search template. For each template that contains a search form, look for the following (or similar):

<form method="post" action="<$MTCGIPath$><$MTSearchScript$>"

...and change it to:

<form method="post" action="<$MTBlogURL>fastsearch"

...save the template and repeat this step for every template that has a search form.

If you want to insert a new search form on your site, or completely replace existing forms, you can use the following:

<form method="get" action="<$MTBlogURL$>fastsearch">
    <label for="query" accesskey="4">Search this blog:</label><br />
    <input id="query" name="query" size="20" />
    <input type="submit" value="Search" />
</form>

Of course you can also add optional fields to the form such as:

<input type="hidden" name="order" value="date" />

....to sort the results by date, or....

<input type="hidden" name="category" value="5" />

...to restrict the search to category_id = 5.

Once you have the search form(s) on you site, rebuild those pages and start searching. You should notice significant speed and resource improvement, especially on blogs with many entries.

Download Fast Search

Non-commercial use - FREE ( although you must include a

Movable Type search results powered by 

<a href="http://mt-hacks.com/fastsearch.html">Fast Search</a>

link on your search results pages.

Also, donations are appreciated:

Commercial use - $97.00

Download Now

Membership required, please sign-in or register:

As always, suggestions and feedback are appreciated. Please reply to this entry.

Note: FastSearch v1.0 is a suitable replacement for most implementations of the built-in search function in Movable Type. However, there are a few features that Fast Search v1.0 does not include. Fast Search v1.0 does not support comment searches, multi-blog searches (searching multiple blogs at the same time), regexp search, case-senstive search, the use of AND and OR operators, and Edit Entry Links). Also, Fast Search v1.0 does not inlcude MT 3.3 search features such as tag search. Some of these features may be supported in future versions -- please let me know what features you would like to see.

Updated: 8/9/06 07:39:00 - added some additonal info to the instructions based on feedback from Patrick.

TrackBack

TrackBack URL for this entry:

Listed below are links to weblogs that reference Fast Search - PHP Fulltext Search Plugin:

» "Solid connection. Good price on from camera digital low price
Right now, Cox promotional pricing on a Digital Telephone and HSI package (Preferred, not the fastest, Premium tier) is about [Read More]

Comments (107)

Jesse Gardner

Oh man, this is going to make my life so much easier, thank you! I redesign often, and it's such an awful pain to have to go back to the search templates for these changes. Right on!

Posted by Jesse Gardner | August 8, 2006 1:25 PM

Posted on August 8, 2006 13:25

IÃ±aki

Great job, Mark! You've done a great job packaging it and making it easy to install for everyone, very good job indeed.

I highly recommend switching from MT's search to this method, we've been using it in our 16 blogs for months and works really well: searches are much faster for users, and the load on the system is much lower.

Posted by IÃ±aki | August 8, 2006 2:18 PM

Posted on August 8, 2006 14:18

Patrick

I was truly waiting for something like this, so thank you very much!! I'd like to share a few problems I ran into while installing this.

1. I have an all-static blog and the mtview.php file failed to be created when switching to "Set each template's Build Options separately". I had to create a static template for "myview.php" containing the following (found here: http://forums.sixapart.com/lofiversion/index.php/t49502.html).

<?php
include('<$MTCGIServerPath$>/php/mt.php');
$mt = new MT(<$MTBlogID$>, '<$MTCGIServerPath$>/mt.cfg');
$mt->view();
?>

2. Setting "Set each template's Build Options separately" caused MT to create an .htaccess file in my blog's directory, and since I use a global .htaccess at my site's root (which does all kinds of redirects under my blog's path), I had to delete that and add the following to my own .htaccess.

RewriteRule ^mt/fastsearch$ /mt/mtview.php [L,QSA]

3. <MTElse> doesn't seem to be supported, so I had to rewrite some things differently.

4. <MTNoSearchResults> must be used after <MTSearchResults>, not before.

5. Really DO rebuild all files, because that will create [redundant] entry data in the mt_fileinfo table, necessary for the PHP dynamic templates. If you have trouble with <$MTEntryPermalink$> or if getting unexpected "Page not found" errors, rebuild.

It took me a while to figure out that last one especially, but otherwise it went ok and my search now works faster than ever!

Posted by Patrick | August 9, 2006 5:19 AM

Posted on August 9, 2006 05:19

Mark Carey

Thanks for your detailed comments, Patrick.

1. Regarding the mt-view.php file, it seems like you have have deleted this template in the past (because you don't need it for an all static blog). When I installed Fast Search for MT Hacks, I had a similar problem, in which the mt-view.php template was previously set by me to NOt rebuild with indexes, and had never been built. So I had to build manually. Perhaps the plugin could check for these things -- at a minimum, a note about it in the Instructions may be helpful.

2. Yes, setting build options seperately does create that .htaccess file. In most cases, it plays nice with existing .htaccess files, but I have run into cases where I have needed to make modifications to get everything working. I am not sure if this is avoidable, but do like your one-line rewrite rule, which is great for people who only have Fast Search as a dynamic page/template.

3. Regarding <MTElse>, I assume you are using this in conjunction with the <MTSearchIfNextPage> tag? I will look into this, and may be able to add support for this in future.

4. True, <MTNoSearchResults> does need to come after <MTSearchResults>. This seems reasonable to me, as it is mutually exclusive to <MTSearchResults> -- are there cases where there would be advantages to reversing the order? I should at least add a note about this in the instructions.

5. Yes, I had the same confusion and problem intially. It was only after digging in the PHP code for <MTEntryPermalink> that I relaized that it was looking for the link data from the FileInfo table, and that MT creates FileInfo records for every page in your blog, even if only one of them is set to dynamic publishing. Seems a little odd, but the implication is that is important to do a full rebuild of all files after changing your dynamic publishing settings.

Patrick, your implementation looks good. It seems like you customized things to provide multiple-blog search results. I would be interested to hear how you set this up. ;)

Posted by Mark Carey | August 9, 2006 6:32 AM

Posted on August 9, 2006 06:32

Patrick

Hey Mark, thanks for the reply!

1. You must certainly be right, I often delete stuff that I don't need (or think I need!). :)

3 and 4. I was using <MTElse> with <MTNoSearchResults>, so that I could include header and footer text to the results (other than the <MTBlogResultHeader> which I used as a real per-blog header). My previous template was actually sort of like this (simplified):

<MTNoSearchResults>
Sorry, no matches for "<MTSearchString>".
<MTElse>
</h1>Search results for "<MTSearchString>".</h1>
<MTSearchResults>
<MTBlogResultHeader><h2><$MTEntryBlog$></h2></MTBlogResultHeader>
...
</MTSearchResults>
</MTNoSearchResults>

> Patrick, your implementation looks good. It seems like you customized things to provide multiple-blog search results. I would be interested to hear how you set this up. Wink

Actually I'm doing that with a rather patchy technique. That search.php script does HTTP requests to /mt/fastsearch, /mt/agenda/fastsearch (etc. for each of my blogs) and then outputs the results together. Each blog's fastsearch template is very minimal. You can see one's output here:
http://www.chipple.net/mt/fastsearch?search=test

Posted by Patrick | August 9, 2006 9:04 PM

Posted on August 9, 2006 21:04

Mark Carey

Regarding number 3 and 4, your example seems a little confusing, having a <MTSearchResults> inside a <MTNoSearchResults> tag. However, I do see your challenge, regarding the multi-blog search hack you are doing. it seems like you could achieve this by having the FastSearch template for the first blog to be like:

<MTSearchResults>
<MTBlogResultHeader>
<h1>Search results for "<MTSearchString>".</h1>
<h2><$MTEntryBlog$></h2></MTBlogResultHeader>
...
</MTSearchResults>
<MTNoSearchResults>
Sorry, no matches for "<MTSearchString>".
</MTNoSearchResults>

... and the subsequent blogs would omit the <h1> part:

<MTSearchResults>
<MTBlogResultHeader>
<h2><$MTEntryBlog$></h2></MTBlogResultHeader>
...
</MTSearchResults>
<MTNoSearchResults>
Sorry, no matches for "<MTSearchString>".
</MTNoSearchResults>

I was wondering if that was your approach to multi-blog search. Perhaps the next version of Fast Search can incoporate multi-blog searching in one form or another...

Posted by Mark Carey | August 10, 2006 7:41 AM

Posted on August 10, 2006 07:41

Jake

I am not a developer, but just an FYI: At least in what I just downloaded, the directory structure is in correct. Shouldn't it be:

plugins
--FastSearch
php
--Plugins

Right now the PHP files are in the php folder, and I know I couldn't get the forms to even load and got a bunch of smarty errors until I moved those files into the plugins folder.

Still working on getting it to work properly on my site, but after having trouble with mt-search for a long time, this will make me stupidly happy if I can get it to work :-)

Posted by Jake | August 14, 2006 7:22 PM

Posted on August 14, 2006 19:22

Mark Carey

Thanks for reporting that issue with the directory structure in the zip file. I was not aware that I didn't have the php files in a 'plugins' folder. I have updated the zip file to fix this.

Please let me know if you need any help or advice with getting it working on your server.

Posted by Mark Carey | August 14, 2006 7:54 PM

Posted on August 14, 2006 19:54

Jake Ortman

I actually figured out my problem, Mark, after digging through my ProNet email list archives. I had caching enabled in my mtview.php file, so it wasn't loading any search results. After I disabled caching line in there, it worked fine.

Posted by Jake Ortman | August 14, 2006 8:02 PM

Posted on August 14, 2006 20:02

Jake Ortman

OK, one feature suggestion: provide some .htaccess examples on how we can make this even more of a drop-in replacement for mt-search.cgi so we can redirect requests to that file to fastsearch and get rid of that cgi resource hog.

Posted by Jake Ortman | August 14, 2006 8:22 PM

Posted on August 14, 2006 20:22

Mark Carey

Good to hear, Jake. If you are interested, you will also find my solution for getting MT caching to play nice with Fast Search (click for ProNet message - requires ProNet login).

I will play around with .htaccess to see if I can come up with some rewrite rules. Intially I was hoping to just override the <MTSearchScript> tag and point it to Fast Search, but the default templates preface this with the <MTCGIPath> tag, which complicates things. That said, there may be some rewrite solutions.

Posted by Mark Carey | August 15, 2006 7:50 AM

Posted on August 15, 2006 07:50

Dascalargo

Mark-

I'm trying to isntall FastSearch. Got to step 6 (Yes, I am on 3.2) and when I click the link I get the following:

Can't call method "do" on an undefined value at /home/u2/dascalargo/html/mt/plugins/FastSearch/FastSearch.pl line 56.

Any ideas?

Posted by Dascalargo | August 21, 2006 2:19 PM

Posted on August 21, 2006 14:19

Mark Carey

Dascalargo, that section of the code relates to loading the database driver and executing some SQL code. I am not sure exactly why you are getting this error.

Please forgive the obvious question: can you confirm that your installation of MT is using mySQL as a data source? If you are not sure, look for an "ObjectDriver" line in your config file (either named mt.cfg or mt-config.cgi).

Posted by Mark Carey | August 21, 2006 2:42 PM

Posted on August 21, 2006 14:42

Dascalargo

Mark, no need to apologize for the obvious question because it did need to be asked. I glossed right over the MySQL requirement, but I figured that much out before you had the chance to respond.

Yesterday afternoon I decided to try to migrate over to MySQL. A lot of this stuff is groping in the dark, learning as I go along, and I'd done pretty well... until now.

I got MySQL set up, no problem. Started the MT upgrade which transferred everything over, but it bombed out in the middle for some reason. I got some totally non-descript error message -- something akin to, "I'm afraid I can't do that, Dave," with no further explanation. Long story short, the MySQL database was corrupted and my MT password ceased working.

So... I'm starting from scratch, installing MT 3.3. I had been considering doing that anyway -- I originally installed MT a few years ago, when even more of this was trial-and-error for me, then upgraded to 3.2, and then moved the entire install to a new server, so the set-up was in desperate need of a clean-up anyway -- but I always talked myself out of it because of the work involved. Looks like I convinced myself. :D

Bottom line: Once I get MT installed (this time using MySQL), I'm going to try FastSearch again. I look forward to using it.

Sorry to have unnecessarily bothered you with this.

Posted by Dascalargo | August 22, 2006 9:08 AM

Posted on August 22, 2006 09:08

Dascalargo

Mark-

Finally got Fast Search to work with my new install of MT... sort of.

It seems to not see certain words. Bat, Fox, Dog, between, able, two, first, something, and sent are all words that appear in the main body of an entry, but which Fast Search doesn't see. I have rebuilt the site as well as building, rebuilding, and re-rebuilding the fulltext index, and it's always the same words that go unnoticed.

Here's something else I noticed in experimenting. I have the word voice in two entries, both of which Fast Search returns when I search on that word. Fast search finds neither that nor to, but it CAN differentiate between and find the correct entry if I search for the phrases "voice that" and "voice to".

The page can be viewed at http://www.triviot.com/index2.php. It's not live and the subsequent archives and search results pages have no styles applied to them yet.

I generally tend to think I did something wrong in these cases, but this time I'm stumped.

Posted by Dascalargo | August 24, 2006 12:51 PM

Posted on August 24, 2006 12:51

Dascalargo

Oh, and yes, I'm on MySQL this time. :D

Posted by Dascalargo | August 24, 2006 12:52 PM

Posted on August 24, 2006 12:52

Dascalargo

Damn. One more thing I meant to tell you...

On your default search results page, the courtesy link back to you is broken. Looks like you left off the dot-com.

Sorry to be flooding the comments.

Posted by Dascalargo | August 24, 2006 1:03 PM

Posted on August 24, 2006 13:03

Mark Carey

Thanks for the note regarding the error in that link -- I also received an email about that and it has now been fixed.

You description above about the search results is normal and to be expected. mySQL fulltext search excludes words less than 4 characters by default -- this is why searches for words like bat, fox, and dog don't work. Furthermore, fulltext also excludes a list of "stop words", which likely include between, able, something, etc. The goal in both cases if to avoid really short or commmon words in order to return the most relevant results.

So you didn't do anything wrong. Some people don't like the fact that you can't search for 3 letter words -- this can be changed in the mySQL config file, but unless you have a dedicated server, this may not be possible.

Posted by Mark Carey | August 24, 2006 2:59 PM

Posted on August 24, 2006 14:59

Dascalargo

Thanks, Mark. I'm totally new to MySQL, so was ignorant about stopwords and word length. I've since done a bit of research.

Yeah, I'd probably want to be allowed to search three-letter words if I had the option -- my database should be small enough that I don't think it would make a big difference in building the file -- but that option isn't available to me. As for the stopwords, I have no problem with the concept that it doesn't include certain words. I mean, in reality I never would search on the words I tested anyway. I did find a copy of the built-in stopwords list (which my server is using), and it helps to see what isn't on the list.

Thanks again. I'm up and running, and fully understanding now. I think I'm finally out of your hair. :wink:

Posted by Dascalargo | August 24, 2006 4:18 PM

Posted on August 24, 2006 16:18

Arvind Satyanarayan

Quick question, I was using Joe D'Andrea's Seeker plugin which is quite similar to FastSearch. The only qualm I had with Seeker was that it couldn't log searches to the activity log, hence my question, does FastSearch like MT::App::Search log searches to the Activity Log?

Posted by Arvind Satyanarayan | August 26, 2006 11:37 PM

Posted on August 26, 2006 23:37

Mark Carey

Hi Arvind,

Yes, Fast Search does log searches to the Activity Log. Both 3.2 and 3.3 log formats are supported. Logging is off by default but can be enabled on a per-blog basis (blog-level settings).

Thanks for bringing to my attention the fact that I was not explicit about this above.

Posted by Mark Carey | August 27, 2006 6:59 AM

Posted on August 27, 2006 06:59

Anish:

Hi,

Can you provide installation instructions for lighty+fastcgi. We are currently getting 404 errors. Lighty will not recognise .htaccess files

Anish

Posted by Anish | August 29, 2006 12:01 AM

Posted on August 29, 2006 00:01

Ryan J. Bonnell

I was able to follow all of the instructions on setting up FastSearch for my Movable Type installation, but after completing step 10 and doing a test search the page returns an error message of "Page not found - /fastsearch".

I've added the Rewrite ruled in my .htaccess file as mentioned by Patrick and I even see the query show up in my the Movable Type activity log, but FastSearch doesn't seem to work for me (even after rebuilding several times to update the fulltext index).

Any thoughts on what I might be doing wrong? The server is Red Hat Linux Enterprise running Apache 2.0 on one of Media Temple's dedicated virtual servers.

.htaccess:
RewriteEngine on
RewriteRule ^/fastsearch$ /mtview.php [L,QSA]

Posted by Ryan J. Bonnell | September 14, 2006 4:42 PM

Posted on September 14, 2006 16:42

Mark Carey

Ryan,

Given your description, you can try two things:

1) Try rebuilding all indexes. However, it sounds like you have done this.

Is the "not found" page a standard apache 404 page, or does it look different? Specifically, I am wondering whether this page is being generated by mtview.php. If it is not, try the following:

2) Check to make sure that Apache is configured to allow the use of .htaccess files in the root directory of the web site. Since this is a dedicated server, you should be able to check and change this, if necessary. I recently came across this issue with a Fast search install. The solution was to add the following to the apache config:

Options Indexes FollowSymLinks
AllowOverride All
Order allow,deny
Allow from all

"AllowOverride All" is the key line above.

If you can provide a link to your search form, I can confirm whether the "not found" page is being generated by mtview.php, or not. If it is, it means that you .htacess is working fine, but the MT "FileInfo" record is not there. In this case, make sure that you have completed step 7, check that your Dynamic publishing settings are set to "Set each template's Build Options separately". As well, make sure that the Fast Search index template is present under Index Templates. If it is not there, revist step 10: Open the Fast Search settings once again and choose "Click here to install the default Fast Search Template." In the Fast Search index template is there, make sure it is set to build dynamically. Then rebuild indexes.

Please let me know if any of the above fixes the problem.

Posted by Mark Carey | September 14, 2006 8:02 PM

Posted on September 14, 2006 20:02

James Bow

I must have missed something. I followed your instructions and yet I'm getting 404 Not Found errors whenever I try a Fast Search. Have a look at http://www.bowjamesbow.ca/blog.shtml

Any suggestions appreciated.

Posted by James Bow | September 17, 2006 12:04 PM

Posted on September 17, 2006 12:04

Jim

Finally!! A search for Movable Type that WORKS!!

Thanks - I'll be installing it on all of my blogs

Posted by Jim | September 17, 2006 6:33 PM

Posted on September 17, 2006 18:33

Mark Carey

James,

It seems like you may be missing the "Dynamic Site Bootstrapper" (mtview.php) index template -- or, this index template has not been built. If you see this under index templates, rebuild it. If not, you need to create it. Name it "Dynamic Site Bootstrapper" and use output file "mtview.php". You can grab the default template from here. This template does not need to be "rebuilt with indexes", but be sure to build it at least once.

Let me know if this doesn't work...

Posted by Mark Carey | September 17, 2006 7:52 PM

Posted on September 17, 2006 19:52

James Bow

Does the Dynamic Site Bootstrapper have to be published dynamically, or should it be static?

Posted by James Bow | September 17, 2006 10:21 PM

Posted on September 17, 2006 22:21

James Bow

Okay, I don't know what I did (constantly switching mtview.php and fastsearch between static and dynamic and rebuilding), but it works now.

A relief too. Much better search method.

Posted by James Bow | September 17, 2006 11:14 PM

Posted on September 17, 2006 23:14

James Bow

I thought I should let you know that the plugin doesn't play nice with markdown and smartypants and any posts which are formatted using these plugins chokes the search. But this is easily fixed by adding the variable convert_breaks="1" in the EntryExcerpt field.

Posted by James Bow | September 17, 2006 11:33 PM

Posted on September 17, 2006 23:33

Aziz Ashofte

Mark,You've done a great job in Fast Search but the fact is that it is only working in Latin alphabets. I'm not very professional in php but it seems that you've used some functions that do not support 2-byte characters in UTF-8 format. Most of non-Latin languages are using this type of 2-byte chars and your search plugin is not working in these languages.

I want to know if you can do me and a lot of others, who are using MT to publish weblogs in non-Latin alphabets, a favour by solving this issue. I'm eager to donate some if you can solve this problem. besides I know a lot of other who are capable of donating more if you end this search nightmare for them.

Waiting to hear good news from you.

Posted by Aziz Ashofte | September 18, 2006 1:51 AM

Posted on September 18, 2006 01:51

George

I'm getting this error:

SQL/DB Error -- [You have an error in your SQL syntax near 'BOOLEAN MODE) AS score FROM mt_entry, mt_author WHERE entry_blog_id IN (1) AND e' at line 1]

I'm using the MT 3.33 on a MySQL database. Can't figure out what is wrong. I do see the fastsearch page, I do see a search box and I also see the error mentioned above. Is it a MT 3.33 problem? I have rebuilded the fulltextsearchindexthingie and my whole blog.

Posted by George | September 27, 2006 4:18 AM

Posted on September 27, 2006 04:18

Mark Carey

George, what version of mySQL are you running? This error sounds like you are using an old version of mySQL, that does not support fulltext searching. As stated above, mySQL 4.0.1 or higher is required for the current version of Fast Search. However, an upcoming release of Fast Search may support older version of mySQL. Stay tuned.

Posted by Mark Carey | September 27, 2006 7:05 AM

Posted on September 27, 2006 07:05

George

Hi Mark,

Oops. I'm using the 3.23.58 version. Going to kick my provider :-)

Thanks!

Posted by George | September 28, 2006 4:21 AM

Posted on September 28, 2006 04:21

Diego

Hi Mark,

I noticed there is a next page tag but I was wodering if in the future you plan to include a previous tag or why not some pagination numbers ala Google.

This is such a huge improvement from the search.cgi that I hope Six Apart implements this in their next major release!

Thanks.

Posted by Diego | September 29, 2006 11:59 AM

Posted on September 29, 2006 11:59

Diego:

Hi Mark,

Weird question: is there any restricted English words that won't appear in the search results page?

I'm having a problem with the word "associated". I've created an entry with the text below:

"an association of associates. associate. associater. associated. asociated."

and all the words can be found except "associated". I'm quite confused about this.

Do you have any suggestions or has anyone had a similar issue?

Posted by Diego | September 29, 2006 1:26 PM

Posted on September 29, 2006 13:26

Diego

I just realised that the issue with some words has to do with mySQL stop words (As somebody pointed out previously).

For those interested, the link to the full list is below:
http://dev.mysql.com/doc/refman/5.0/en/fulltext-stopwords.html

Posted by Diego | September 29, 2006 1:52 PM

Posted on September 29, 2006 13:52

Mark Carey

Hi Diego,

With respect to a "previous" tag, yes, I think that is something that I could add in future.

Regarding the pagination numbers, this is possible, but I decided against it, as it could affect perfomance. The challenge pagination numbers is that you need to know the total number of search results -- and this would require another "count" query. I'm not a mySQL expert, so I am not sure how much this would impact performance, but with blogs with many entries, it might.

Posted by Mark Carey | September 29, 2006 5:26 PM

Posted on September 29, 2006 17:26

Rich

Hi Mark,

Thanks for coming out with such a useful plugin. I was dissapointed at the speed of search.cgi in MT 3.3 too.

I managed to install the plugin but the fastsearch template breaks with some MT tags. I can get a full list of search results if I only list the <MTEntryTitle> or <MTEntryDate> tag in the template but the results break on the first one if I want to include <MTEntryExcerpt> or <MTEntryBody>. My entries are stored in a MySQL database.

Any thoughts? Thanks a lot!

Posted by Rich | October 1, 2006 5:44 AM

Posted on October 1, 2006 05:44

Mark Carey

Rich,

That's a strange one. Being a php plugin, Fast Search won't support many tags from other non-php plugins, BUT it does support core MT tags such as MTEntryExcerpt and MTEntryBody. Both of these have been tested and work fine in the search results. Can you provide an example of an error message? Or perhaps a link to your search form?

Posted by Mark Carey | October 1, 2006 8:22 AM

Posted on October 1, 2006 08:22

Patrick

> Regarding the pagination numbers, this is possible, but I decided against it, as it could affect perfomance.

MySQL has a SQL_CALC_FOUND_ROWS keyword that seems to makes this doable in a more efficient way than doing a separate COUNT().

http://dev.mysql.com/doc/refman/4.1/en/information-functions.html#id2826088
Some PHP related comments are here:
http://www.php.net/manual/en/function.mysql-num-rows.php

Posted by Patrick | October 1, 2006 9:32 PM

Posted on October 1, 2006 21:32

Mark Carey

Patrick, that sounds interesting, thanks for the pointer. I will try this out for the next version.

Posted by Mark Carey | October 2, 2006 1:37 PM

Posted on October 2, 2006 13:37

Diego

Hi Mark,

When I was validating the fast search results page with the html validator at http://validator.w3.org/ I got some errors due to encoding (My page's xhtml).

By encoding the ampersands in block.MTSearchResults.php line 100 as below the validation passes.

$next =
$uri[0]."?query=".urlencode($query)."&category=".$category."&order=".$order."&lastn=".$lastn."&offset=".$offset;

Hope it helps.

Posted by Diego | October 5, 2006 10:48 AM

Posted on October 5, 2006 10:48

Mark Carey

Thanks for all the suggestions in this discussion. Many of them have lead to new features, which have just been released in version 1.5.

Click here for deatils about Fast Search v1.5.

Posted by Mark Carey | October 9, 2006 11:04 AM

Posted on October 9, 2006 11:04

Mark Carey

James, I have move your comment over to the v1.5 discussion - see my response there.

Posted by Mark Carey | October 12, 2006 10:14 AM

Posted on October 12, 2006 10:14

Anonymous:

Warning: main(): open_basedir restriction in effect. File(/var/www/vhosts/tracli.net/cgi-bin/mt8/php/mt.php) is not within the allowed path(s): (/var/www/vhosts/tracli.net/httpdocs:/tmp) in /var/www/vhosts/tracli.net/httpdocs/test8/mtview.php on line 2

Warning: main(/var/www/vhosts/tracli.net/cgi-bin/mt8/php/mt.php): failed to open stream: Operation not permitted in /var/www/vhosts/tracli.net/httpdocs/test8/mtview.php on line 2

Warning: main(): Failed opening '/var/www/vhosts/tracli.net/cgi-bin/mt8/php/mt.php' for inclusion (include_path='.:/usr/share/pear') in /var/www/vhosts/tracli.net/httpdocs/test8/mtview.php on line 2

Fatal error: Cannot instantiate non-existent class: mt in /var/www/vhosts/tracli.net/httpdocs/test8/mtview.php on line 3

error -_-" why not

Posted by Anonymous | November 17, 2006 3:05 AM

Posted on November 17, 2006 03:05

Kyle

I'm testing fastsearch for a non-profit site I help out. I'm getting this error from the default template:

SQL/DB Error -- [You have an error in your SQL syntax near ' MATCH (entry_title, entry_text, entry_text_more, entry_keywords ) A' at line 1]

SQL/DB Error -- [You have an error in your SQL syntax near '()' at line 1]

Any idea what might cause this?

Posted by Kyle | December 11, 2006 6:43 PM

Posted on December 11, 2006 18:43

Mark Carey

Kyle, you will need to provide more details. Whate version of mySQL are you using? What FastSearch mode? My guess is that you are using an old version of mySQL (older than 4.0.1) and that you are using the Fulltext method. If both are true, try switching to the "Non-fulltext" method. Older versions of mySQL don't support Fulltext.

Movable Type Hacks, Plugins, and Tips