The Mudcat Café TM
Thread #78100   Message #1400465
Posted By: JohnInKansas
06-Feb-05 - 03:28 AM
Thread Name: BS: Again?
Subject: RE: BS: Again?
Not getting in doesn't necessarily mean you are the mudcatter who has the bug, but it appears that someone who comes here often keeps bringing it back. Unfortunately, we all have to do the checks because it's too easy to pick up this crud without even knowing it.

The problem seems to be a "search engine" that unloads it's bots to whatever sites are visited by someone with their toolbar (or some other crud gimme junk) installed. The bots do a sort of legitimate(?) search for links to send home for indexing. Unfortunately, mudcat is "link-intensive" with lots of links.

The "on-schedule" blocking of the 'cat on an almost daily basis suggests that the bots do their searches and "send home" their stuff on some kind of a schedule. (The other possibility is that the visitor with the crud visits on a regular schedule?) When they start opening "every link on the mudcat site" it simply swamps the connection. I generally get a reply that the 'cat is up, but then the download never completes - apparently due to "traffic."

Mudcat isn't the only one affected. I've been seeing quite a few sites - including the IRS (a mirror), a couple of state legislative sites, and one or two DoD (Dept of Defense) pages with big bold notices that "bot searches are PROHIBITED." Doesn't seem to do them much good.

It might be considered a "legitimate search" function if these guys didn't write such cruddy code...

John