If Saint Stephen is good enough for Wikipedia, then it is good enough for me to use on Mainly Norfolk.
To be serious: The name is Saint Stephen, but St. Stephen is a common abbreviation. Note that some places, e.g. St Stephen in Hertfordshire, use the abbreviation in their official name.