Here are a couple of variants:
Ride a cock-horse
To Banbury Cross,
To see what Tommy can buy;
A penny white loaf,
A penny white cake,
And a two-penny apple-pie.

Ride a cock-horse to Banbury Cross,To buy little Johnny a galloping horse
To buy little Johnny a galloping horse;
It trots behind and it ambles before,
And Johnny shall ride till he can ride no more

From a website about Banbury:
"It seems likely that the rhyme originated at the latest in The 16th Century, and very probably much earlier. The original cross at Banbury stood in the Market Place, and was built of stone with steps surrounding it. The tall shaft carried carvings and a crucifix. It was destroyed by Puritans in July 1660, because they thought that it attracted superstitious veneration from the people. A Jesuit priest in 1601 wrote, "The inhabitants of Banbury being far gone in Puritanism, in a furious zeal tumultuously assailed the cross that stood in the Market Place, and so defaced it, that they scarcely left one stone upon the other.

The term 'a cock-horse', has been variously used to describe a proud, high-spirited horse, the additional coach horse used when going up a hill, and the name for toy, or hobby-horse.

Finally, we come to the 'fine lady' finely dressed and parading about on a white horse. W.Potts in his book "A History of Banbury", thinks that the rhyme may be echoing back to pagan times. In pre-Christian times a young girl, representing the Earth goddess would ride around the fields hoping to increase their fertility. A sacred tree would be set up in the village and the people would sing and dance around it. Once Christianity had arrived, the tree was replaced with the May-Pole, the Earth goddess with the May Queen, and the festival now took place on 1st May."

And another:
Desert Orchid for hobby horse festival?
Britain's first-ever festival of hobby horses is being planned for the Oxfordshire town of Banbury - and there'll be no shortage of troughs in the town. But they'll be filled with flowers.

The festival theme is being picked up by the award-winning organisers of Banbury in Bloom, who plan to have horse-shaped planters and appropriate floral displays in the streets.

Banbury was the national runner-up in the large towns section of the 1999 Britain in Bloom contest, having won the South East title the year before.

Toni Magean, who's co-ordinating the floral fiesta, is hoping the horse theme will help win the first prize in 2000.

He thinks judges will appreciate the link to the town's famous "Ride a cock horse" nursery rhyme - which was the inspiration for the new event.

The Beasts Around Banbury Cross festival is due to take place on the weekend of July 1 and 2, 2000.

Members of Adderbury Morris Men and Bloxham Morris are helping to organise the event, and a number of morris creatures are expected to attend. The Outside Capering Crew is bringing four tourney-style horses.

Sandy Glover of the United Fools' Union, which also includes beasts of disguise in its membership, is hosting one of two web sites for the event.

Funding is being sought for a professional sculptor to work with schoolchildren to create a giant processional beast.

And Toni Magean has begun devising a floral hobby horse, to be shown off around north Oxfordshire.
Festival organisers are delighted with the Britain in Bloom link. "Perhaps it's too much to hope that we'll be able to feature Desert Orchid - either the flower or the racehorse," said a spokesman, "but we note that Jackstraws Morris has a beast called Daisy....."

And from a Coventry site:

clicky
Coventry's famed cross found its way into the nursery rhyme books, it is believed before the Banbury Cross rhyme:
Ride a cock horse to Coventry Cross,
To see what Emma can buy,
A penny white cake,
I'll buy for her sake,
And a two penny tart or a pie.

I always understood it to mean a stallion - a cock-horse, as in "cock-sparrow" - not always used for male birds; male lobsters and salmon are also cocks...

Jo