The situations I try hardest to avoid are in heavy, rush-hour traffic. Too many people in a hurry who are tired, hungry and angry lead to stupid behavior. Fortunately, I am not forced to do this often any more. I am not a pokey driver, but neither do I assume that the speed limit is "whatever you can get away with". I HATE belligerent tailgaters, and will find almost any way to get them past & gone.

I simply do not understand the problem stated above by several above about parking. Perhaps it is a real problem with spatial relationships and/or 3D vision. I can park anywhere there is legal room, and negotiate small spaces... according to the size of the vehicle I'm driving. I can parallel park if there's enough room, and sometimes when there's not. (I once parked an old VW bus in a space 2" shorter than itself...I wish I'd had a camera -1965 or so)
I am fortunate, at my age, to have good night vision, no serious health problems and good reflexes.... but I do not use this good fortune to take silly chances. I even brake for Red Lights! (and in this area, that's a sign you're not a native.)

   I know that modern life in a big city often makes driving 'seem' necessary, but if I had a vote, a driver's license would be a lot harder to get & keep. People CAN adapt.