I know a few former no 10 staff - they work incredibly long hours when required

As do nurses, doctors and many other other professions. Even in IT I occasionally worked weeks in excess of 100 hours.   

That does not justify clocking off every Friday at 4pm to drink, sometimes until the early hours of the morning. Which nurses etc did not.