I once worked with a Scot who came from Arbroath. When he felt strongly, we had to have a Glaswegian translate! This is in an English-speaking country.

I have noticed that it is only on the telephone that I have trouble. Face to face I can understand pretty much any accent. There may be a lip-reading element.
I have discarded two cell-phones because of deteriorating earpiece speakers. Different brands. Land-lines seem to last better.