Well, here's the scoop. The Mudcat is all hosted here, in Mudcattville (AKA West Chester, PA) on our local network at Onstage which is made up of about 25 computers on a T1 (1.54Mbps) line. Some machines of ours have 25 to 50 Web sites on ONE SINGLE MACHINE. No more California Bill D, that was years ago.

The Mudcat has a machine of its own. In fact it uses 4 computers. Shorty is the primary Web server, it holds all the Web pages. As mentioned above, almost all the pages on the Mudcat are dynamic which means the actually content of the pages comes out of a database.

The database server is Charlie. Charlie is a state-of-the-art monster and is not the problem.

As we got more popular late last year and earlier this year, I had to remove the supersearch because it was taking too long to get results and causing cascading failure. So I moved it to a 3rd machine named Jah.

I added a 4th machine recently named Loki which you can hit at http://loki.mudcat.org. It is an alternate version of the Mudcat that accesses the same database. This is in case something goes wrong with Shorty which is the usual cause of most of our problems. Loki is also a powerhouse, and is responsible for the media encoding that goes on here, such as Mudcat Radio.

Late last year we were averaging about 12,000 messages posted per month. That is about when I had to start spending a lot of time, energy and money dealing with popularity. There are 267,450 messages in the database right now, so we are dealing with a HUGE amount of data. 20,000 messages posted per month is now the average. See how many get posted? How many do you think get viewed? In technical terms… A lot. Both are intense computational processes.

OK, enough stats, lets get down to what's happening. The weak link is Shorty. Now don't get mad at Shorty, Shorty is a fine machine that works very hard. Here's what happens. When, for instance, you load the threads page, the queries in the page ask for data from the database, in this case the list of 1 day old threads. This takes about a 10th of a second (100ms). Now if in that 10th of a second, a 2nd person also asks for that data, the 10th becomes a 5th, 3 people - half a second, and so on. The longer it takes, the more likely it becomes for more people to ask for data at the same time. You see, its a double whammy. That is cascading failure. We are averaging about 25 requests per second which usually works out to be 5 simultaneous connections for nearly the whole day. And the worst day is Wednesday, it always has been.

Now that I've completely bored some of you and confused the rest of you, let me tell you the plan. Round Robin. Soon, Loki will share the www.mudcat.org name with Shorty. When someone types a URL into their browser, a request goes out to turn that name into a number, an IP address. To do this, a DNS server acts as the operator and finds the number of the Web site you are trying to call. I can program my DNS server to change its mind every couple of minutes, sometimes telling you its Shorty, others is Loki. So randomly splitting the requests in two. This is the poor man's solution because the two machines do not communicate so Shorty has no idea how busy Loki is and therefore cannot take over if needed. The drawback to you is… well nothing. The drawback to me is that now I have two copies of the Web site, so if I need to make a change, I have to do it twice. Fun huh?

So that's it for now. Do trust that my desire has always been the same, to make the Mudcat the best damn site on the Web. So time, money, energy willing… it will be perfect.