Have you tried something along the line of

(when they are very noisy again) I hope we do not disturb your conversation if we work on silently?

MudGuard