True enough. There can be an assumption that, if it can be shown that a political movements is not "extreme" in left-right terms, then it isn't really dangerous.

The BNP is a case in point. It's evil and needs to be defeated not because it's particularly right-wing, but because it's racist and authoritarian.