The point of sending the pdf to amazon is to allow the converter to reflow the text (in a text only pdf) for better reading.
If you copy the pdf directly to the kindle (which, btw, is all I've done so far), the kindle will try to show a pdf page on the screen, often resulting in very small type; I've been rotating the screen orientation and reading in landscape, which is fine for the files I've got. (Using the zoom feature is unwieldy - you have to scroll horizontally to read a line).
The converter (and you don't need to go via amazon - there are free converters that will do the same job, convert to mobi, which is what amazon's azw format essentially is. Calibre mentioned above will do this) will allow the text to flow better in larger type on the screen. There do seem to be problems with pdf conversion however - header, footers and page numbers may end up in the text. If you can do without them, there are some recommendations to crop the pdf to the text are before conversion.
For image-based pdfs the problems are different. At least one site I looked at recommended using a utility which splits out the pages and alters the contrast for better reading on the kindle, then reassembling the pdf and converting to mobi. I haven't looked at this in detail (maybe tomorrow - or later today as it is now) so I can't say whether it's worth the trouble or not.