From the Full Fact website:

Start with the law

The referendum was not legally binding. There’s no one source that can prove this statement true... That follows from the fact that the European Union Referendum Act 2015 didn’t say anything about implementing the result of the vote. It just provided that there should be one.

In other countries, referendums are often legally binding—for example, because the vote is on whether to amend the constitution. The UK, famously, doesn’t have a codified constitution.

A UK referendum will only have the force of law if the Act setting it up says so. In practical terms this would mean someone would be able to go to court to make the government implement the result. The Alternative Vote referendum in 2011, for example, was legally binding in this way.

Otherwise, as the High Court put it on 3 November:

“a referendum on any topic can only be advisory for the lawmakers in Parliament”.
So, purely as a matter of law, neither the government nor Parliament has to do anything about the referendum.

Brexiteers are clinging on to "it wasn't just advisory" because Cameron, then May, declared that they would be bound by it. They were both speaking way beyond their remit in saying that, tantamount to declaring a sort of edict. That was way out of order and was done as a political stunt in order to save face when they were confronted by opposition to brexit. They were going over the heads of our democratically-elected representatives, who overwhelmingly oppose brexit, to make a populist appeal directly to the electorate so that democratic opposition to brexit would be thwarted. To put it simply, the referendum was not legally binding, and it couldn't be made so simply via the say-so of two politicos who both seemed to forget what democracy is all about.