All post offices should have a rigid template for checking thickness and, in order to avoid any item attracting a surcharge further down the line, should use it where any item might be borderline. Make sure that they do, or you may pay too little (and risk the recipient having to pay extra) or too much.

We don't know exactly how heavy the packet was; assuming that it was between 2000 and 2250 grammes, then first class rate ought to be be £7.25; additional increments of 250 grammes being charged at 75p. Recorded delivery costs an additional 70p per item; offhand I don't know about insurance or 'compensation' rates for first class post, though the cost (as opposed to resale price) of modest items is usually refunded without question in the event of loss, provided you can prove it was actually posted.

Standard Parcel (usually considerably slower) has wider weight bands, so anything between 2kg and 4kg would cost £8.24; insurance up to £250 is a further £2.25, giving a total price of £10.49. Since the introduction of pricing in proportion ten months ago, there are a fair few cases where it's actually cheaper to send items as first class packets than as standard parcels. Frequently, people don't appreciate this anomaly and wind up paying more than they need to. Some Post Office Counters staff may point out the various options to you, while others may only answer the question you actually ask them.

'Is there a cheaper better company?' No, not really. The private companies are mainly interested in the lucrative business mailing contracts, while the Royal Mail has a statutory 'social' obligation that the competition doesn't. Ordinary 'domestic' mail, delivered to any address in the UK no matter how remote, costs, on average, more to process and deliver than the rates actually charged. The 'social' service is subsidised from the profitable bulk contracts. As private competitors (which are allowed to undercut, while the Mail is not) cream off more and more of these, it will become progressively harder to sustain the traditional universal service.

The situation in the USA is completely different, so not really relevant to this discussion.