I found out that "pompey" is a term for someone from Portsmouth, England......and many British ships called at Portsmouth on their way to Australia; for a very long time the only way to get to Australia from anywhere else in the world was by sea. It's not too far a stretch to turn "pompey" into "pommy", then into "pom".

That's my theory, anyway.