This is definitely set in England. Casey is likely to be an Irishman as Casey is an Irish name, and there is a massive Irish community living in North London, although he could have lived in the US. Many Irish work in building and road construction here. The reference to the Golden Crown pub, Underground and to Bitter are a dead give away. American Emmy Lou Harris is so unfamiliar with the concept of bitter beer she sings it as "bitters". The use of subway is correct for the UK as the Underground is the generic name for the railway, but the passageways under the roads that lead to the stations are called subways here.
The last ride refers to the girl Casey is thinking about. He remembers how she welcomed him and still wanted him. He is regretting choosing somebody else instead of her and is now locked into a sour, loveless, non physical marriage but his children prevent him from leaving. Where the girl is located could be Ireland, the US or even in London.
He is thinking about the last time he spent with her and has not had a "ride" since. The ride he is thinking about has a double meaning - the trip to visit her and what happened when he did!
I think Emmy Lou understands this and she identifies very much with the woman in her version, which is probably why she covered the song and added the extra "I suppose you seldom think about me" and the "it's so good to feel your body" lines to express the woman's feelings. Sadly, Casey is trapped and can only reminisce over his beer.