Could it perhaps be Issac Watts' "We're Marching to Zion"?

Come, we that love the Lord,
And let our joys be known
Join in a song with sweet accord,
Join in a song with sweet accord,
And thus surround the throne,
And thus surrond the throne.

REFRAIN
We're marching to Zion,
Beautiful, beautiful Zion,
We're marching upward to Zion,
The beautiful city of God.

Let those refuse to sing
Who never knew our God,
But children of the hev'n-ly King,
But children of the hev'n-ly King,
May speak their joys abroad
May speak their joys abroad.

REFRAIN

The hill of Zion yields
A thousand sacred sweets
Before we reach the heav'n-ly fields
Before we reach the heav'n-ly fields
Or walk the gold-en streets
Or walk the gold-en streets.

REFRAIN

Then let our songs a-bound
And every tear be dry,
We're march-ing thro' Imannanuel's ground,
We're march-ing thro' Immanuel's ground
To fairer wolrds on high
To fairer worlds on high.

REFRAIN