In both terms one thing is most important.
"A coloured person" - it's "coloured".
"A person of color" - it's "person".

It is acceptable to talk of "black people" which, according to the rule above, would emphasise their race rather than them being people. African Americans are, first and foremost, Americans. Their African heritage, while important, is secondary.

The use of "a person of colour" in place of "a coloured person" is clumsy. I drive a red car, not a car of redness. I suspect it was developed to demonstrate the speaker's "wokeness" rather than there being a genuine need.

DC