The two phrases resonate around the same notion but express slightly different ideas.

Sydney or bust could well be equivalent to Hollywood or bust, in the sense of 'go for broke', 'give it your best shot', 'pull out all the stops' etc.

Sydney or the bush is usually carrying the message that Sydney is the only place worth the candle; everywhere else (whether a capital city or an isolated country pub) is "nowhere, man!"
It's mostly used by Sydneysiders. And, while all the interstaters give each other a fair amount of stick, there are those in rural NSW who can be quite disparaging about "Sydney".

Such as, "NSW really stands for Newcastle, Sydney, Wollongong."
Or "Outside the sandstone curtain ...." if you're desribing "the real world".
Sydneysiders have routinely been accused of thinking that "the west" starts at Parramatta (about 40 clicks west of the CBD) and "the Far West" starts at Katoomba (at the top of the Blue Mountains and where lots of city workers commute from, by train, daily in an hour or so's trip.

Being exMelbourne, I reckon its because of their penal colony background.

Cheers, Rowan