My mate Kevin, from Adelaide, told me pohm (his spelling) comes from Prisoner Of Her Majesty

Actually, "pom" is an abbreviation - it's short for "pommie bastard".

Anyway... According to the OED, "pommie" is short for "pomegranate". In the 19th century, recent settlers were nicknamed "Jimmy Grants" (= "immigrants"). "Pomegranate" - or "pommy grant" - first appears around 1910, either as an alternative to "Jimmy Grant" or possibly to distinguish a new wave of English migrants from the established "Jimmy Grants".

It sounds far-fetched, but if you think about the way slang develops in families & workplaces you can see it's got the ring of truth. That's one way language develops - people make up stuff.