Jim
If you are not willing or able to back up your outrageous allegations, it might be wise not to make them in the first place.
If your response to being challenged is then to suggest the discussion is moved elsewhere, I suggest you know you have no basis for those allegations.
Put up or shut up.