There actually is a proper definition which distinguishes between paedos, and pervs who lust after post pubescent underage kids ?

I think the word is "ephebophile". Not that it makes much difference, as you say - it's differently wrong, but 99% of the time it's no less wrong.