The Mudcat Café TM
Thread #61712   Message #993643
Posted By: GUEST,.gargoyle
30-Jul-03 - 03:21 PM
Thread Name: Tech: How do I download the whole shebang?
Subject: RE: Tech: How do I download the whole shebang?
Joe - some of the spiders/bots (spyders)(they've been around for 10 years an early one was "fly-cast") are so sophisticated they will copy the entire site - and then spin off and copy the sites of those linking TO the site you copied. The early ones were like having your own SakSam search-engine to scour for information http://www.datashaping.com/

Checkout this Wired article from 2001 - of a "Project Guttenberg" type approach with independent people working together to copy the net. http://www.wired.com/news/print/0%2C1294%2C47184%2C00.html

The BotSpot provides you with a wide selection of software posibilities and some are freeware:

http://www.botspot.com/BOTSPOT/Windows/Download_Bots/Off-line_Browsing_Bots/index.html

Details on some of them are below:

WebZip 5.0
http://www.spidersoft.com/webzip/tour/

Grab-A-Site
http://www.bluesquirrel.com/products/grabasite/index.html?ASCID=357

DISCo Pump
http://www.manac.com.au/Prod/PowerIntSuite.html

HAVE FUN!
Sincerely,
Gargoyle