A decent Infrared illuminator should not be visible, and certainly shouldn't be seen as a "bright light." It's possible this person (not necessarily a guy?) has got him/herself an old surplus IR vision device that's not quite up to snuff in the performance department.

The second or so that it takes to build up to full brightness most likely is because his "lens" is leaking out the sides. When he points slightly away from you, it looks dimmer because all you're seeing is the "leakage." As he swings the beam in your direction it appears to get brighter.

Infrared filters are available for cameras, and it's possible he's just looking for wildlife to photograph. If he has really shitty collimation of his beam, and lots of leakage, the "shining in the windows" would be unavoidable if he's scanning the hedgerows for skunks - in any reasonably dense housing area. He/she may not be deliberately shining the light into peoples windows. A "real Klutz" might even really believe it's "invisible" to everyone else, and might just be really careless.

Infrared photography is a sort of fun thing, but the most common use of infrared photo filters now is for the night scenes in the movies. You can't expect actors to emote in the dark, and most of them are so clumsy they'd risk hurting themselves; so the scenes are filmed in broad daylight, through an IR filter, to make it look like it was dark. Normally, for this usage, no "extra" IR illumination is needed, since normal daylight contains enough IR for pretty decent photography, and the ordinary movie lights are hot enough to produce quite a lot - invisibly.

UV observation is a lot more popular in my area, 'cause they've found that all scorpions fluoresce under UV light. They used to be almost impossible to find, but UV makes it easy to spot them so the "scientists" can watch their "domestic activities."

John