Tech: How many files???

Subject: Tech: How many files???
From: Newport Boy
Date: 11 Oct 08 - 01:29 PM

My antivirus program has just finished a full scan, and reports 592,915 files - Huh?

I'll accept responsibility for about 10,000. Can Windows and programs really need 580,000 files?

Oh, the good news is only 1 infection - in a Microsoft update file!!

Phil

Subject: RE: Tech: How many files???
From: Bert
Date: 11 Oct 08 - 01:34 PM

Why did you think it was so slow?

Subject: RE: Tech: How many files???
From: Joe Offer
Date: 11 Oct 08 - 01:38 PM

My computer has well over a million files - articles in the Encyclopedia Britannica, tune files for the Digital Tradition, photos, and whatnot. Some programs make use of lots of little files.
But I haved to run antivirus checks overnight, or bother with being slowed down by the antivirus scan.
-Joe-

Subject: RE: Tech: How many files???
From: GEST
Date: 11 Oct 08 - 04:34 PM

How about defragging?

Subject: RE: Tech: How many files???
From: GUEST,goodlife
Date: 11 Oct 08 - 05:17 PM

If you think your computer is full of junk files why not do a search on the internet for cnet then find a programme called advanced windows care download and run its amazing what difference it will make it's shareware and free

Subject: RE: Tech: How many files???
From: Stilly River Sage
Date: 11 Oct 08 - 05:27 PM

You're probably seeing a lot of tiny little files that are part of your operating system and of your large integrated program suites (if you use MS Office, for example, all of those programs have many small programs and files within them).

SRS

Subject: RE: Tech: How many files???
From: Susan of DT
Date: 12 Oct 08 - 06:35 AM

I have around 600,000 also. I just checked how many files I had in major components of the system (Windows Explorer, right click on a directory and choose Properties). Roughly:
    My Documents:   27,000
    Windows:       33,000
    Program Files: 67,000
I have not gone hunting the rest. I had expected more files in the windows directory.

Subject: RE: Tech: How many files???
From: Mick Pearce (MCP)
Date: 12 Oct 08 - 08:30 AM

I have the same problem as Joe - my av scan reports about 1.9M files and I do my full scan overnight. I rarely bother chasing up the numbers - most of them are things I want, and unless disc space is desperate I don't think it's a problem. The av is the only thing it really affect.

Mick

Subject: RE: Tech: How many files???
From: Nick
Date: 12 Oct 08 - 11:47 AM

Treesize
is handy to tell you where your space and files are and better still it's free! Various different options from file size to file count etc

Subject: RE: Tech: How many files???
From: JohnInKansas
Date: 12 Oct 08 - 12:17 PM

Windows Explorer will show you how many files are in a folder but only gives "space values" if you click on an entire drive.

You can open a command window (DOS prompt), go to the drive (C:\ and Enter) type:

DIR *.* /s

which will list all of the files on a drive, and at the end of the listing will give you the total number of files and number of folders.

I don't run the above very often, since scrolling through all the files listed takes a "very long time."

You can get a quicker (somewhat) result by sending the result to a text file. Writing the result to your hard drive is very much faster than displaying the result on your screen. You can then open the file in Word (or sometimes in Notepad if it's not too large) and just Ctl-End to get to the final summary result.

In the "DOS" window, go to the drive or folder you want to know about (C:\ and Enter takes you to the root of the C:\ drive, usually just the drive name will go to root of another drive - e.g. G: and enter). At that location, type:

DIR *.* /s >filelist.txt

The file "filelist.txt" will be created in the location where you ran the command.

Creating the text file may still take a several minutes, but then you'll have a listing of everything that you can browse through at your leisure.

My AV (Norton) does slow things down if it's running "up front," but if I minimize its window it asks if I'd like to have it "run in background." With it running in background it still has some effect on use of the computer to do other things; but I don't find it really objectionable.

If I let it have the machine, a full system scan takes about 7 hours, and running it in background while I use the machine may add about a half hour. It reports scanning of a little over 1.8 million files.

I keep all my "working files" in a single folder on my C:\ drive. These are the files I've created and that I use - and change - fairly regularly. This folder has only about 144,000 files (a little over 6,00 folders. The text file listing for this folder is 3,465 pages in Word.)

I keep "reference files" - files I don't expect to change such as news items, downloaded reference manuals, etc., on a separate hard drive that contains only "data." This drive shows about 59,000 files in 9,000+ folders - 1,617 pages listed in Word.

I use a third drive for "collecting" backups from our four computers, so it has all the "documents" on my C:\ drive, Lin's C:\ and D:\ drives, our "community laptop" C:\ drive and my son's C:\ drive, essentially duplicating all the docs on my own machine and adding all of them from three other machines. I could probably speed up the AV scan by disconnecting this drive during scans, and everything has been scanned elsewhere before getting there; but I don't find it a problem to let Norton re-scan it all weekly. (417,00 files, 61,000 folders, 12,201 pages when listed in Word)

(Permanent/Archive backups are made on two separate hard drives that are disconnected except during backup or when something needs to be retrieved. These backup hard drives contain some additional "archive" material that's been removed from active machines but might be wanted later.)

With Vista - at least in Home Premium and Business versions - manual defrag is optional, since Vista defaults to scheduled weekly defrags. It allows a certain percentage of fragmented files, and you can run a manual defrag if you're fussy; but it really isn't necessary with either of these Vista versions. (I don't know whether the Vista Home Basic schedules defrag, but if you click on defrag it should tell you if it does.)

John

Subject: RE: Tech: How many files???
From: dick greenhaus
Date: 12 Oct 08 - 01:45 PM

Ah, for the days when an operating system used 2K, and you could know just what files were on your computer.

Subject: RE: Tech: How many files???
From: JohnInKansas
Date: 12 Oct 08 - 04:52 PM

The proliferation of files is one reason why Windows Explorer in Vista, as a default, does only an "indexed search." Windows has to compile indexes of what files are on your computer, and then when you search it does not look at your drives to see what's there. It ONLY looks at the index.

Indexing is done as a "background process" and can only be done when the machine isn't busy. It may thus take several weeks for Vista/WinExplorer to finish indexing what you have on your computer - especially what you copy from an older machine or new stuff that you copy from another drive. Until the index is completed, Vista search will tell you - often erroneously - that perfectly good files (or nasty bad ones) do not exist on your computer.

By default, Vista only indexes "My Crap" folders, so if you put files anywhere else, the don't exist unless/until you change the defaults to force it to index the other folders. Also by default, it tends to not index hidden, system, and/or program files. This gives the lovely result that if you search for a file that might be part of some malware, Vista Search may inform you that it is NOT ON YOUR computer, when in fact it is there plugging away at destroying you.

Even if you change defaults to tell it to search "everywhere" and for "everything," the inability to find files known to be present persists.

While it is claimed that you can use "Advanced Search" to find things in the content of files, the most favorable comment I can make is "it doesn't work." After manually finding files I've searched for unsuccessfully, copying short text strings directly from the file and pasting them in the Advanced Search box, Vista still tells me the files can't be found.

Usually a search for file content on my computer should take several hours. Vista frequently, using Advanced Search to tell it to look everywhere, reports "no files found" within three or four seconds, and REFUSES to make a new char-by-char search - for any search string that contains ANY WORD that it claims already to have indexed.

Example: Vista refuses to find any file containing the word "Loser" - although I know I've used that word in at least a dozen files on my computer where I've tracked and documented unsuccessful searches for stuff I know is there.

Workaround?: DOS command FIND can look in files for specific text strings. The problem is that it shows/lists every file it looks at whether or not the search string is found. If the search text is found, it's "quoted" with the rest of the line where it appears.

One can port the result to a text file (FIND "textstring" > search.txt - althugh the "switches" require something a bit more complex) and then use Word to search the 12,000 page "search.txt" result for the original search string in order to find the files that contained it. (It only takes a few hours; but at least it usually actually gives you a result.)

Google does a better job of indexing stuff, but who cares if they find 23 million results when they'll only show you the "most popular" couple of hundred. If you're not into sports, politics, soap operas, or Hollywood fandom, you can't SEE any of the interesting (to you) results in what they've indexed. The ad sites along usually wipe out 50% of the reportable hits that might otherwise be useful.

John

Subject: RE: Tech: How many files???
From: pavane
Date: 13 Oct 08 - 02:18 AM

There used to be a wonderful little program in DOS called XTREE, which could shown your files in various ways. One very useful display was all files on the machine, sorted by name and date, which often turned up several copies of the same file, mostly unwanted, and told you exactly where they were

Is there a WINDOWS equivalent of XTREE?

Subject: RE: Tech: How many files???
From: JohnInKansas
Date: 13 Oct 08 - 03:37 AM

XTREE is not a recognized command in Vista or WinXP.

Any "possibly DOS" command can be easily checked out using the Command (DOS) window. Just type the name of something you might think is a DOS command, followed by "/?" (without the quotes) and if it is a valid command you'll get a description of the form(s) in which you can use it, with all the switches and options. If it's not a valid command, you'll be told that Windows doesn't recognize it, with only a slight smirk about what an idiot you are for having asked.

The same /? tag will get similar descriptions for quite a number of not-DOS windows command line functions, although there are still some for which it doesn't work.

Theoretically, you could get an alphabetical list of all the files on your computer using the DIR command, piping it through the SORT filter:

DIR *.* /s | SORT

To avoid waiting for all the trash to flash by on your monitor, it's quicker to redirect the result to a file, so:

DIR *.* /s | SORT > SortedFiles.txt

The | key, called the "pipe," is the shifted backslash (Shift-\) key on US keyboards. It sends the output of one process (Dir *.* /S) to another process (Sort), and the > is here called a "redirect" which in this case sends the final result to a file with filename "whatever you type after the >"

For my C:\Documents folder, I get 3,463 pages of "result" but in Word, global replacements can be "easily" (a euphemism for "you do need to know what you're replacing") used to "simplify" it down to a slightly more manageable size.

In the latest Windows versions, the DIR command itself allows a switch /O (for Order) that you can use to list and sort the filenames in a single command:

DIR *.* /S /O:N

to sort alphabetically by filename.

The sorted list of filenames can be used easily to identify duplications, but unfortunately the sorting loses any association for each file with the folder(s) where it appears. More complex sort/replace edits can theoretically overcome this difficulty; but as a practical matter it's easier just to identify a filename that's present several times and then do a "DIR filename /S" to see where it appears, if you want to select which duplicates to remove.

John

Subject: RE: Tech: How many files???
From: pavane
Date: 13 Oct 08 - 04:59 AM

No, XTREE was a commercial program, not part of DOS, but unfortunately, could only handle the original file system, so it no longer works even in a DOS box. You could easily sort by size, and see all the duplicates of the largest files, which was the best strategy for reclaiming space.

Subject: RE: Tech: How many files???
From: JohnInKansas
Date: 13 Oct 08 - 06:57 AM

In WinXP or Vista, you can easily sort by size using the "DOS" command:

DIR *.* /S /O:S

This won't necessarily show duplicates of the same file adjacent to each other, but will show all files of exactly the same size in a single clump.

The simple fact that the number of files has bloomed to "unthinkable" numbers, and the very common use of "common files" in multiple necessary places makes the kind of manual detection of duplicates done with XTREE results rather impractical. There are just too many files on "modern" machines to make more than superficial "cleaning" practical. You can certainly remove obvious large dupes, but finding all the redundancies is too time consuming to be worthwhile (and a bigger hard drive is so cheap).

As an example, I have more than 70 "duplicates" of one file that is necessary in each place where it appears, because one of the users whose stuff is in my backups downloaded .htm files of a bunch of manuals and instructions from Gateway. Each of these .htm files has an "associated folder" containing the graphics and other "style lumps" that Gateway uses on all their web pages, so this particular .gif (along with 17 or 18 others) is in the linked folder for each of the Gateway .htm files. Deleting any one of them makes the associated .htm file "unloadable."

Had he saved as .mht (web archive) files those same .gif etc files would have been embedded in each of the .mht files, but wouldn't show as a separate file in each place.

Some Vista versions have a fairly sophisticated "synchronize" function that lets you, for example, keep a matched set of files on your desktop and on your laptop. Theoretically, according to Microsoft, the "synch" utility has the ability to remove duplicate files, although the exact way in which it does this is not too clear. Mickey give lots of "assurances" that it's wonderful, but not much in the way of useful information about what it actually does.

I don't believe this function is present in either Vista Home Basic or Home Premium. You have to move up to Vista Business or Vista Ultimate to get it. For my purposes, I consider it too dangerous to attempt to use, based on Microsoft's sparse description of how it works; although I do have it available on my desktop machine.

John

Subject: RE: Tech: How many files???
From: Newport Boy
Date: 13 Oct 08 - 02:56 PM

Thanks for all the suggestions, comments and info. Thanks particularly for TreeSize - I used to have this years ago and forgot how simple it was.

I've done a bit of digging using TreeSize, PerfectDisk and CA Antivirus.

TreeSize reports 83,652 files and PerfectDisk 92,335 files, which seems sensible. The difference is mainly system files, some of which are not reported by TreeSize. Breakdown of the PD figs as follows:

C: (Windows) 72,976
E: (Photo)    3,919
F: (Data)    5,222
M: (Music)    2,017
Q: (Backup)   3,854
R: (Backup)   4,347

So where are my missing 500,000 files? Well, a good chunk are in 24 .CAB installation files for Open Office. CA Antivirus counts all the files hidden in these, and also all the separate files in zip archives.

I feel lots happier now!

Phil