It probably felt far worse for you than for the audience - after all, I assume many of them could see what was happening.
If you got no complaints, why worry? I do understand what you mean - but a few moments pause is far better than a few moments unnecessary waffle any day! And I like waffles!!