Beat me to it - is it a valve or a transistor, or even a more modern 'Integrated circuit based' type?
The gain is highest for the phono input, so any stray electric fields are likely to be magnified the most there.
Any faulty signal earth could be responsible.