"If you really watch him you could figure out what he's doing" - nothing wrong with that.

It isn't about it having to be difficult, it's about it sounding right. Sometimes sounding right means difficult, sometimes it means easy, but it's the sound that matters, never the difficulty. And a good player will sometimes make impossible stuff look easy anyway.

In any case, whether it's easy to see the changes doesn't have too much to do with the difficulty of making them. It depends on the chords, some look very similar, some look very different; and the size of the hands comes in too, it's just easier to see what's happening with thin fingers than with podgy ones. And so on.