Tech: Forum search after 2005?

Subject: Tech: Forum search after 2005?
From: GUEST, Mr Grumpy
Date: 09 Apr 08 - 04:03 AM

Is it still the case that the forum search will only searchposts before a given date?

Subject: RE: Tech: Forum search after 2005?
From: Sandra in Sydney
Date: 09 Apr 08 - 09:16 AM

Quick links (top of every page) select 'Old Adv. Forum Search' from drop-dowm menu, then GO (this is the bit I usually forget till I realise I've been waiting forever & nuffin' has happened)

Subject: RE: Tech: Forum search after 2005?
From: JohnInKansas
Date: 09 Apr 08 - 12:10 PM

Click the FAQ button in the header, do a search on the FAQ page for "search" using your browser's "find on this page" (or Edit|Find if your browser has a toolbar) and the third of fourth "found" in FAQ will be a link to Searching for Mudcat Information where three or four different search methods you can use here are described.

None of the methods "searches everything" and all of them rely on "indexes" which may not be up to date, with the possible exception of the threads filter and "Refresh" - but it only finds thread titles that match the filter term - not "things in a thread."

If you have a Google toolbar, there is an option to "search on this site" that you can also try. It is also an "indexed search" but frequently the Google index hits on items here within a few minutes of the first post in a new thread. New stuff seems to disappear from the Google index though, unless the term found gets "enough hits" to keep it at the top of the Google index.

Any search method that actually "looks at the files" to find what's in them would be far too unwieldy to be of much use, for a database the size of mudcat. A search for "files containing a word" just of the threads to which I've posted (a mere 3300 threads - 500 MB) on my own hard drive takes more than a minute of drive thrashing (78 seconds for a "quick test" just now) using the old-fashioned "look at it all" method.

John

Subject: RE: Tech: Forum search after 2005?
From: BK Lick
Date: 24 Apr 08 - 04:14 PM

Hmm...don't think I understand John's last two paragraphs.
Visiting Google Advanced Search and putting mudcat.org into the "Search within a site or domain:" box quickly finds any Mudcat page containing a target word or phrase.

—BK

Subject: RE: Tech: Forum search after 2005?
From: Joe Offer
Date: 24 Apr 08 - 04:42 PM

So, if you use this Google search link and just add what you're looking for, you'll be able to search Mudcat. A search for twa corbies would look like this: (click).

Is there somebody who can set up a more "elegant" Google search link specifically for Mudcat?

-Joe-

Subject: RE: Tech: Forum search after 2005?
From: BK Lick
Date: 24 Apr 08 - 05:05 PM

How 'bout this: Mudcat advanced Google search

Subject: RE: Tech: Forum search after 2005?
From: open mike
Date: 24 Apr 08 - 05:13 PM

i often go to the refresh button and call up all entries for as long in the past as is available as a choice and then click my edit drop down menu for "search this page" the "built-in" search box does not seem to
find recent posts. Perhaps with the upcoming move this can be addressed.

Subject: RE: Tech: Forum search after 2005?
From: JohnInKansas
Date: 25 Apr 08 - 01:25 PM

Assuming that you have the Google toolbar, you can click on "Settings" at the far right where you'll find an "Options" button. The Options button will show you how to add a "Search only on current site" button to your toolbar. On mine, it looks sort of like a "magnifying glass with a handle" with a pair of "eyeballs" in the glass, and it's right next to the Google "Go" button.

Note that I'm looking at a "standard" Google toolbar in IE7. Your layout may vary some if you have a different browser or a different toolbar version.

There are "help" buttons splattered around in Google toolbar places, with one at the "Settings" button. If you poke around there, you should eventually find a way to "add search keys" and otherwise format the terms you type in the Google search box. SOME Boolian search forms can be used, and may be helpful sometimes. Just appending a +word or a -word will either find only results that include the word with the plus at the front and will EXCLUDE any results containing the word with a - in front. (Note that there IS NO SPACE between the + or - and the word followiing.)

The "form" for a search on a specific site is something like "searchterm site:mudcat.org." I'm not sure this is exactly the way you should do it, but it should be close enough for you to recognize it if you can find it in Google help. I dont' use the "manual form" much so I'd have to dig into stuff to find an exact script for you. (I've already got the button up.)

I've found the Google toolbar quite helpful, but would caution against going overboard with adding "lots of extra buttons." Try to figure out what will be helpful for your methods of searching and ignore some of the others.

Google also has quite a few "other accessories." Mostly I find them "underwhelming" and don't use many of them except for special situations.

John

Subject: RE: Tech: Forum search after 2005?
From: JohnInKansas
Date: 25 Apr 08 - 04:22 PM

I suppose that a couple of hours of "no comment" means everyone digested the bit about the easy way to do site searches using Google?

Anyone who really wants to see some success with searching on the internet MUST UNDERSTAND what is meant by an "indexed search."

Google does NOT go to the internet and look for the term you enter that you want to find.

Google "crawls" the internet and makes LISTS OF WORDS that people might look for, and LISTS THE WEBSITES where each word appears.

When you type a word and hit Go, Google looks at it's LISTS OF WORDS (Indexes) and shows you the sites where it already knows that "your word" appears.

Someone must decide in advance what words you "might look for" and ONLY THE WORDS that were chosen in advance to be added to the lists will appear in any search result.

If you are the first person to look for "a word" you might not find it using Google; but Google is pretty good at adding a "new word" if enough people look for it, so that people within the first dozen or so searchers may find a result.

(If someone posted here "lets all go look up amblyopliatious" - assuming it's NOT already in a Google index, it is very likely that the Google result would point back to the post here that started it off within about a dozen queries - maybe. Or maybe not. Google knows that one way to appear infallible is to be inscrutible.)

In a couple of actual cases, I've posted to a thread here, and gone immediately to a Google search for something else that was needed in the same thread, and found my post in the result on the first page - so it CAN BE really quick, or sometimes it can be NEVER.

If your search term is already indexed it may instantly appear in the search result; but if the same term appears on a few thousand "more popular" sites, you'll never see the result you want because although Google may fascetiously report 28,395,253,422 results (found in 0.0027 seconds), it will only allow you to look at the first hundred or so (in order of "popularity") of the ones it finds.

The actual number of "hits" you can look at seems to depend on "how popular" the search term is(?). I've seen very few of my searches that actually will display as many as 500 hits, although it's hard to come up with a term that gets "found" less then a million times.

A sort of "key" for using Google might be to look for a less popular term that might be in the result you want, and/or exclude as many extremely popular search terms as you can think of, in order to see a bit more of the otherwise "we have them but you can't see them" results. Penetrating much beyond the top layer can require exceedingly difficult/complex(?) "query construction," and still is unlikely to get very far.

Most searches at mudcat are index searches. Perhaps someone else can explain, in simple terms:

1. How the basic Digitrad Search is indexed.
       I don't think that index has been updated recently(?)
2. How the Advanced Search Index differs from the simple search box on the Discussion and Threads pages.
       I believe there was an update of the index for Advanced search since "the big crash." Does someone know for sure?
3. How the "Refresh" with filter word(s) really works.
       Refresh appears to list only a certain maximum number of returns, regardless of the time span you enter.
4. How to use the search by thread number.
       Discussed recently, and returns any thread, but only one at a time - unless someone's hiding a new trick that the rest of us don't know about.

(I've already strained my brain here, so I think I'll take a brief break. I think I've found an "almost textbook" example of a search system - not exactly at mudcat - that illustrates everything that can possible cause a search engine to fail but it will take some more study to put it into coherent form.)

John

Subject: RE: Tech: Forum search after 2005?
From: BK Lick
Date: 25 Apr 08 - 06:17 PM

All of this that John has just posted is very informative, but I think it doesn't answer Joe's request for a more elegant Google search link specifically for Mudcat.

If you just follow the link I gave above, Mudcat advanced Google search, and enter "I've already strained my brain here" into the "this exact wording or phrase:" text box, Google will point you to John's last message posted about 90 minutes ago. (Should work equally well with any browser and no toolbar is needed.)

—BK

Subject: RE: Tech: Forum search after 2005?
From: JohnInKansas
Date: 26 Apr 08 - 12:34 AM

BK Lick -

An advantage of the Google toolbar...

To get to the "Advanced Search" you have to open a Google Search Page, and then put your search string in and tell it what advanced features to use.

With the toolbar in place, you always have a search box where you can enter what you want to search for, and if you've added that one tiny button you can immediately search on any site where you have an open page. You can do a site search on any site you can open a page on, and you don't have to spell the name of the site carefully if you're already at the site.

Not a big-deal difference, but I find the Google toolbar fairly unobtrusive and quite convenient. Suit yourself.

If you want to look it up at Google, you can teach yourself to write the search string in the form that includes a site to be searched, and there are numerous other "specifications" that can be included. Without the toolbar, you'll still have to open a "Google" page to have a place to type in the correct search terms.

From a link in a recent thread:
For example, typing tips site:pcmag.com will limit your search to tips stories on PCMag.com

Typing:

"anything you want to search for" site:mudcat.org

should search only at mudcat.org.

The simple Search box at mudcat searches using only pre-set index pages. The index pages have not been updated, so far as I've heard, since sometime before the big crash.

The "Advanced Search" also searches using only pre-set index pages. I believe that an update to the Advanced Search threads and DT index was done sometime since the crash, but no subsequent update has been made so far as I know. Unfortunately "index" wasn't one of the keywords added to the index, so I'd have to go through all of the posts by Max (most likely) followed by all by Joe O (who might remember if he'd reported it) followed by all by Dick G or a few others who might have made the report in order to find when the last update was done.

The "keywords" that are indexed include some "main title words," some "first lines," a few "repetitive lines" or "characteristic terms" along with specific keys descriptive of genre or perhaps linking "trad" titles, and a few other kinds of "keys" that I'll leave as "miscellaneous." NOTHING ELSE can be reliably searched for and found using any built-in mudcat search utilties.

The thread "Refresh" button, using a filter word/phrase, can find words used in thread titles, but cannot search content within threads, and at present can't conveniently go "back to the beginnings" due to a limit on the number of items that can be returned.

Google actually can look within threads, but will return results only if it's new¹, becomes and remains popular as a search item outside mudcat, or is linked to by lots of external sites.

¹ "It's new" probably explains why you found "strained brain" here. Quite likely, a week or two from now, you'll get NO GOOGLE RETURN on that same search phrase; although there's no way to predict accurately what Google will do.

Some months ago, or maybe some years ago, I believe that Max reported that the mudcat threads database had "passed 600 MB." While not huge by current standards, allowing each individual who was curious to launch separate word-by-word searches of the entire database could potentially eat a very large bit of bandwidth. It likely is not in the best interests of mudcat to provide that kind of a search capability.

Should it be considered useful, perhaps an archive "threads" CD set or DVD could be made available to those interested, and they would be able to use their own computers offline for such searches; but I've heard of no plans for any such issue.

If you have the downloaded DT, you should be able to figure out how to do a word-by-word search on the data files, on your own machine and on your own time, although I frankly haven't tried it to see how well it works.

John