In classic Japanese culture, an underling who spends much time with the boss will sit at right angles to him so the boss can read his expression but not the reverse. In some primitive groups, the tribe's mana is thought to reside in the chief, and having the hoi polloi gaze too closely at him will damage or dilute it.

In the movie, the other characters regarded not looking at the king as normal. Except for his caretaker, who insisted on looking him in the eye to keep tabs on his condition. I posted the question because the movie made it seem an ordinary part of court life. Had it been an American movie, I wouldn't have asked. The Brits are usually pretty accurate, though.