while we both have the right to use the right of way, neither of us has any specific right to do it at any particular speed

I'd say the people who have set the speed limits have it right. They (whether by accident or by design... I suspect it's by design) have set the speed limits right in the middle of the range of driving abilities that we see in drivers on the road today. And that's exactly where it should be, because that's what serves the most number of people.

So enjoy your fantasies about driving fast. I'll just continue to be thankful there are some logical, practical, and level-headed people making the traffic laws.