The beauty of PayPal is you don't have to have it on your website as a separate button, you just have to offer the possibility of it and people can do it all via email. I'm speaking from experience here - I haven't got around to putting the PayPal "button" on my website yet, but I've sold a number of CDs from my website using PayPal. It just means instead of completing an order form and putting it in the post with a cheque people send you an email, verify your PayPal email address and bob's your uncle.
As cheques are being phased out, it's probably a good way to go if operating a credit card system is too expensive.

Anne