So why don't public loos have doors which open outwards to avoid the handling the handle problem?

And indeed, others. Heston service station on the M4 (close to Heathrow, so visitors might come across it) has won an award for its loo design. This included doors to the cubicles in the ladies which opened inwards through an arc leaving a small triangular space and perhaps two inches at the closest approach to the loo seat. It is not big enough for average sized women to fit in that space and then swing to door back to the closed position without squeezing up against the wall. In other toilets with similar designs I have seen that the cleaners are unable to deal with anything on the walls behind the door. Doors opening outwards would avoid those problems. They would also make it easier to deal with anyone falling ill inside. (Though difficult to keep shut if the fastening didn't work.)