Search surgery in progress


PDA






Derek Zeanah
July 29, 2009, 10:30 AM
Working to make search a bit better, which will require breaking search while rebuilding indexes. We may even need to do this twice, actually, so please be patient.

If you enjoyed reading about "Search surgery in progress" here in TheHighRoad.org archive, you'll LOVE our community. Come join TheHighRoad.org today for the full version!
Derek Zeanah
July 29, 2009, 12:44 PM
OK folks: play with search.

How does it perform compared to the way it worked a month ago? Better? Worse? If worse, then how is it worse?

Thanks.

Derek Zeanah
July 30, 2009, 08:40 PM
Rebuilding the search index. Is gonna take many, many hours. Search will be broken tonight as a result.

Edited 8am on Friday: rebuild of the search index is still going just as it should, but it's something like 25% complete.

Thanks for your patience.

Derek Zeanah
August 1, 2009, 09:42 AM
We have 5.7 million posts in the database, and 4.2 million have been indexed so far. So we're processing 113,500 posts per hour on average, and with 1.5 million posts to go this suggests the process should be complete before midnight tonight EST.

So yeah, rebuilding indexes is a fast process.

Again, thanks for your patience.

If you notice any search issues on Sunday or later, please let me know via a post in tech support.

Thanks.

Derek Zeanah
August 1, 2009, 10:20 PM
And we're done.

Searching should work normally again.

Mal H
August 2, 2009, 10:44 AM
Thank you, Derek!

All of my test cases worked perfectly - at last! :D

Mags
August 2, 2009, 04:57 PM
Just tried searching for "357 Magnum" in the advanced seach within the Buy and Sell Handguns forum and I constantly recieve "database error" messages.

rbernie
August 2, 2009, 06:08 PM
I just tried that search and it worked fine. How about now, for you?

Nematocyst
August 2, 2009, 06:22 PM
It's nice to get results now simply for a caliber, like "357" or "44".

Thanks.

biblefreak
August 2, 2009, 06:42 PM
It's nice to get results now simply for a caliber, like "357" or "44".

Thanks.
Indeed!! One of the most frustrating aspects to most firearms related forums is the fact that you generally cannot search by calibers or model #'s while doing research on a particular hunt.

Thanks a million for fixing this!!

Mags
August 2, 2009, 07:50 PM
I selected sort by "relevancy" and typed in "357 Magnum", I then selected the "buy and sell handguns" forum here is a link of what I came up with "the database error".
http://www.thehighroad.org/search.php?do=process

rbernie
August 2, 2009, 08:20 PM
Hmm. Try using the default 'sort by date' and see what happens.

Mags
August 2, 2009, 08:24 PM
No problem with the default sort settings.

Mal H
August 2, 2009, 08:32 PM
I got the same db error using the same search arguments and sort by relevancy.

Here is the error message returned:

Database error in vBulletin 3.8.3:

Invalid SQL:

SELECT postid, post.dateline
, IF(thread.views=0, thread.replycount+1, thread.views) as views, thread.replycount, thread.votenum, thread.votetotal

FROM post AS post
INNER JOIN thread AS thread ON(thread.threadid = post.threadid)

WHERE (postid IN(1535, 1535, 2360, 2360, 4732, 4732, 4805, 4805, 4989, 4989, 5184, 5184, 5241, 5241, 5538, 5538, 5630, 5630, 5653, 5653, 5705, 5705, 6138, 6138, 6222, 6222, 6782,

... [almost every postid in the world included here] ...

, 5803153, 5803153, 5803200, 5803200)) AND post.dateline > 1246667103 AND thread.forumid IN(22);

MySQL Error : Got a packet bigger than 'max_allowed_packet' bytes
Error Number : 1153
Request Date : Sunday, August 2nd 2009 @ 08:25:03 PM
Error Date : Sunday, August 2nd 2009 @ 08:25:05 PM
Script : http://www.thehighroad.org/search.php?do=process
Referrer : http://www.thehighroad.org/search.php
IP Address :
Username : Mal H
Classname : vB_Database
MySQL Version : 5.0.45

The search worked fine using sort by last date.

Derek Zeanah
August 2, 2009, 11:05 PM
OK, try it now. I tweaked a little bit and tested that search and it seemed to work fine.

Mags
August 2, 2009, 11:11 PM
Works great now. Thanks!

Mal H
August 2, 2009, 11:23 PM
Yep, at least it doesn't go casters up now. ;)

However, using the same search keywords ( 357 magnum ), I see that the list of returned posts has this at the top:
"Search: Keyword(s): 357 OR magnum"

I thought AND was always the default Boolean operator. Even if OR is the default, how do we get AND to work?

I tried all of the following, but OR is always listed as the operator used:
357 + magnum
357 AND magnum
"357 magnum"

Any ideas?

Derek Zeanah
August 2, 2009, 11:40 PM
No, no idea. But it's something we can look at tomorrow. :)

Nematocyst
August 3, 2009, 12:06 AM
Since OR expands a search whereas AND narrows it, I can see that OR could be default.

It seems to be in some Apache products (http://lucene.apache.org/java/2_3_2/queryparsersyntax.html#Boolean%20operators), anyway.

Mal H
August 3, 2009, 10:35 AM
Looks like you've done something to fix it, Derek. Now the "AND" operator works properly and you have to explicitly use "OR" for that operator to take affect.

tydephan
August 12, 2009, 08:46 PM
I'm trying to use the search function and it's not working.

I'm using three operators. The keyword, the member name, and searching in a particular board as well.

I submit the search. It attempts to reload the page but loads nothing. No 404 message. No nothing. Just shows "done" in the status bar of the browser and never loads anything.

ETA: When I remove the keyword and just search a particular board by membername, it works fine.

I'm using IE7 in XP Pro.

Derek Zeanah
August 12, 2009, 10:10 PM
I just did a search on all forums using "tydephan" and it came up after about 4 seconds.

What were the search terms that failed? (You can PM me if you'd rather.)

PBinWA
August 12, 2009, 10:31 PM
Yay! I can search the Trading Post by state code again!

Thank you!!!!

Upriver
August 13, 2009, 03:14 PM
I keep getting repeated "do=process" errors using search the last few days, when using multiple search keywords.

Regards,

U

greyeyezz
August 24, 2009, 01:03 PM
I just get a blank page.

If you enjoyed reading about "Search surgery in progress" here in TheHighRoad.org archive, you'll LOVE our community. Come join TheHighRoad.org today for the full version!