Sure, an atheist can believe in ghosts. He just can't believe that a ghost is the manifestation of a soul. There are plenty of other possible explanations for ghostish phenomena. Just because something can't be explained using known scientific methods doesn't automatically require that "God" has to figure into any possible explanation. If I believe a ghost may be "a manifestation of residual energy of some unknown form" or "an anomaly in the space-time continuum", where've I mentioned God?

And as for UFOs, why would an atheist think this planet is such a special place that it's the only one with intelligent life on it? That's what the theists believe! Seems to me that an atheistic worldview would make one more prone to believing in a universe full of life forms, some of which may be sufficiently advanced to be capable of traveling large distances at very high speeds.

Denial of the existence of a supreme being does not equate to denial of the potential for the existence of unexplained or unexplanable phenomenon. It just denies the possibility of a supreme being as an explanation for them. To state categorically that such phenomena don't exist seems to me to be a remarkably egotistical attitude.