Instead of banning him, or withdrawing the invitation, let him make his lecture, listen to what he has to say, and then aftewards, refute him point by point: which, I am sure, would not be too hard to do.