You will need to actually see (or encounter) a ghost. Then you will probably tend to believe in them. Whether you're an atheist or not doesn't necessarily have much bearing on the matter, though I suppose the average atheist is more likely to search around for any way he can convince himself it wasn't a ghost.

I bet there are some religious people who don't believe in ghosts too.

There are many ways of explaining a ghostly apparition, and people will usually latch onto whichever explanation suits their previous views on the matter...if those views still seem tenable.

The real question is: Do ghosts believe in atheists? Or are they in a state of denial? Or do they even care?