Tech: Searching

Subject: Tech: Searching
From: Mrs_Annie
Date: 21 May 08 - 10:53 AM

When I do a search, it never seems to return any recent posts. I know there was a thread about The Lion who ate Albert earlier this year, but when I searched it only came back with threads from 2005 and earlier. What am I doing wrong?

I thought I'd be clever and find it by viewing my own posts, but I can't see how to do that either.

Please help a thickie :)

Subject: RE: Tech: Searching
From: Stilly River Sage
Date: 21 May 08 - 11:13 AM

If you click on the link to "Personal Page" it will show you all matters relating to your membership, as long as you're logged on. That's next to the "Lyrics and Knowledge" link in the top left corner under the Mudcat logo.

On that personal page, find "Click for a list of your posted thread messages" right there at the top.

Happy hunting! The best trick is to narrow the search words as much as possible. A phrase is less likely to get hits but one or two words should bring a lot more to work with.

SRS

Subject: RE: Tech: Searching
From: Amos
Date: 21 May 08 - 11:24 AM

The search box routine does seem to ignore the last year or two, and I do not know why. The listing of personal posts described by Stilly above does not, however. The filter box also does not clip the recent past -- you can set it for time frames of choice from one day to all the years of Mudcat.


A

Subject: RE: Tech: Searching
From: katlaughing
Date: 21 May 08 - 11:26 AM

To see all of your postings, you can also just click on your name in any heading of any of your postings. I did that to your posting in this thread and saw that you posting in March to THIS THREAD. Perhaps that is the one you are looking for?

Sometimes most recent things don't get indexed by the system right away, if I remember correctly, so if you can search a different way, such as looking at your postings, chances are better you will find what you are looking for. I am sure if I don't have that indexing bit right, someone will come in and say so.:-)

kat

Subject: RE: Tech: Searching
From: Mick Pearce (MCP)
Date: 21 May 08 - 11:26 AM

There was a major crash a few years ago and since then the search index has not been updated. Search will only find the earlier posts.

There are several alternatives you can use. The one I find most useful is to use the Filter box with the Age drop-down set to All and search for parts of a thread title. For songs in particular this works well.

You can also use Google and limit it to search www.mudcat.org which will find things as long as Google has indexed them (however things may float in and out of Google's index).

If you know you posted to a thread you're looking for you can do as SRS says above to get your own posts (OK as long as you don't have thousands!).

There are threads about this elsewhere. (If you use the filter as I mentioned above, using search as the thing filtered, you will find amongst other thread this one: Tech: Forum search after 2005?


Mick

Subject: RE: Tech: Searching
From: manitas_at_work
Date: 21 May 08 - 11:33 AM

Or try using Old Adv. Forum Search in Quick Links

Subject: RE: Tech: Searching
From: Mrs_Annie
Date: 21 May 08 - 03:09 PM

Thanks for all the tips, they were very helpful. Glad to see it wasn't just me with the searching. Now I know some other ways to find things - cheers. :)

Subject: RE: Tech: Searching
From: JohnInKansas
Date: 22 May 08 - 12:25 AM

A short (as always) bit on Refresh and Filter:

For recent threads, using the "Refresh" button with a "Filter" is probably the quickest way to find something recent, but even old-timers here may not be aware that the list of threads returned now is limited to 1,000 threads - regardless of how you filter it.

Setting the search with no filter word(s) hits 1,000 threads at 04 May 08 - 06:03 AM - i.e. there have been posts to 1,000 threads in only the past 2.5 weeks.

Putting the word "Lion" in the filter box, and selecting "all" for how far to go back, gets 145 threads, back to 23 Nov 99, but isn't too helpful since "million, rebellion, cotillion, stallion, etc all contain "lion" and the search can't be limited to "whole words only."

The thread found by kat is about the fourth one on the list, so you might have found it fairly easily that way; but for something a little further down it could be tough to search it out.

A method I have used when really desparate is to put in a filter word or two that I'm pretty sure will catch the thread I'm looking for even if it gives far too many results to look through quickly (but not over 1,000)

On the page of results, Ctl-A (Select All) to select the whole page, then in Word: Alt-E, E (Edit|Paste Special - Unformatted text) and paste as unformatted text into Word. The Word document can then be searched with a narrower and more specific filter or two. The search in Word can be specified as "whole words only" and with a few other little trickies that sometimes can find the specific thread wanted fairly quickly - if it's among the ones found by the filter.

The filter only looks at thread titles, and it can be difficult to remember exactly what a title was. Putting in anything that's not in the title you're looking for will of course mean that the one you want won't be in the list that comes back, so using only one or two words you're sure of is more likely to find what you want.

John

Subject: RE: Tech: Searching
From: JohnInKansas
Date: 22 May 08 - 12:29 AM

Additonal bits on searching for your (or others') posts:

If you click on your name at any post you've made, you'll get a list of all your own posts – 200 posts per page. Your browser should have an Edit|Find (Ctl-F) function that will let you search these return pages one at a time for a word or two in a thread title you're looking for. Mrs_Annie is still under 200 posts so that should be easy; but in a few weeks she'll probably join those of us with umpty-thousand posts and it will get a little tougher. Stilly has a little over 16,000 posts which means she'd have to search each of 82 pages of returns this way if she wanted to go all the way back in her history. I only have about 8,400, but several people who Bullsh*t even more than Stilly ;>) are more communicative have numbers around 35,000 or so.

If you want to find a thread that you didn't post to but can remember another member who did, you can do the same search for all posts by anyone (any name that's been used here).

John

Subject: RE: Tech: Searching
From: JohnInKansas
Date: 22 May 08 - 12:33 AM

All of the above (2 posts preceding) only allows you to find thread titles, or words in a thread title. To find anything in the content within threads the only tools built in at mudcat use indexed searches.

Someone has to go through the whole mess and pick out "things someone might want to look up," make a list of them, with a record of where they appear. You can then search the list of lookupables but there is no convenient way to just crawl through the website looking for a few words that someone might have used somewhere sometime. The indexes are quite good, for what's been indexed; but they are not up to date for recent stuff, so finding specific content can be difficult – and requires some trickery, treachery, and arcane cleverness, or a whole lot of persistence.

Interestingly, if you post something here and then go immediately to Google and search for "words in your post" you quite likely will get a hit on your own post. Unfortunately, if you go back in a week it will no longer be at the top of the list, and although Google will know about it you can't see more than a few hundred "most popular" returns so it will be unretrievable from a Google search.

The reason for using "indexed search" as is done by mudcat and by Google is that it's much quicker to look it up in the index. A compromise is required, however, since indexing everything would make the index as big as the source, and it would no longer be any quicker than making a new word-by-word search of the whole database.

I thought I remembered seeing a report on the size of the threads database, and making notes, but can't seem to find my notes now. Way back a long time ago, it was reported:

SuperSearch is Back on May-03-2000
That's right folks, we did it. The fund raising was effective and we got our new technology and I found the time to pull it off. If you've noticed the Mudcat has been a little squirly lately, that is why. There are over 218,000 messages in the forum now, which made up a 600+ MegaByte index file that the SuperSearch gets its answers from.

While the numbers given back then "don't quite compute" with my estimates of the size of mudcat then, that was quite a long time ago. Current new threads are getting thread numbers near 120,000 and threads probably average (ignoring a couple of stupid ones) at least a dozen or two posts per thread. (There are lots of really short threads to pull the average down, so I'm just making a SWAG at average size.) The current Forum at mudcat almost certainly includes more than a million individual posts and the database is well over a gigabyte – quite possibly several GB.(?)

On Lin's new computer, a search of her documents folder and subfolders for "files containing a word" runs through about 10 GB, 33,000 files, and takes about 16 minutes. My somewhat older machine takes about an hour an a half to run a similar search of my "documents" folder (only about 190,000 files, 65 GB, but I keep important stuff on another drive). Processor load hovers at very nearly 100% during such a search. Obviously, this kind of search is NOT ACCEPTABLE on a site that has to respond to many users simultaneously, several of whom might want to search at the same time. Hence "Indexed Search."

John

Subject: RE: Tech: Searching
From: JohnInKansas
Date: 22 May 08 - 12:38 AM

A side note on searching in the brave new world:

Proud owners of the new Vista operating system may not have noticed that the primary search method, the little search box that's on all the toolbars, does an indexed search only. It does search, automatically, the "entire internet," and "all of Micorosoft," and returns hits instantly of what's in the index. Unfortunately, with default settings, on your own machine it searches only an index of filenames in the My Documents folder, so searching for anything NOT IN THE FILENAME or for anything NOT IN THAT SPECIFIC FOLDER is a waste of the instant.

You can change the search parameters, and probably should. How to is another discussion.

There also are four other documented ways of searching in Vista; but finding them and figuring out which one to use is also another topic.

John