yes, the people of Hiort/Hirta/St Kilda depended mainly on seabirds, not fish for nutrition. I read that the religious people who came to the island made life harder in a different way - banning past-times such as music which would have relieved some of the hardship of life on a remote island and making people spend long hours at dreary services. If the government wanted to encourage people to stay, they could have invested in services such as a regular mail boat and an air ambulance; but of course the authorities preferred to resettle islanders to less inaccessible places. Then the population dwindled to an untenable size and even people who wished to stay couldnt.

oh look at this from Wikipedia article "History of St. Kilda":

"One factor in the decline was the influence of religion. A missionary called Alexander Buchan came to St Kilda in 1705, but despite a lengthy stay there the idea of organised religion did not seem to take hold. This changed when Rev John MacDonald, the 'Apostle of the North' arrived in 1822. He set about his mission with zeal, preaching thirteen lengthy sermons during his first eleven days there. He returned regularly and fund-raised on behalf of the St Kildans, although privately he was appalled by their lack of religious knowledge. The islanders took to him with enthusiasm and wept when he left for the last time eight years later.

"His successor, who arrived on 3 July 1830 was Rev Neil Mackenzie, a resident Church of Scotland minister who greatly improved the conditions of the inhabitants. He re-organised island agriculture, was instrumental in the rebuilding of the village (see below) and supervised the building of a new church and manse. With help from the Gaelic School Society, MacKenzie and his wife introduced formal education to Hirta, beginning a daily school to teach reading, writing and arithmetic and a Sunday school for religious education.[9]

"Mackenzie left in 1844 and although he had clearly achieved a great deal, the weakness of the St Kildan's dependence on an external authority was exposed in 1865 with the arrival of Rev John Mackay, a minister in the new Free Church of Scotland. Mackay was a religious zealot who may have done more than any single individual to destroy the St Kildan way of life. He introduced a routine of three two to three-hour services on Sunday at which attendance was effectively compulsory. One visitor noted in 1875 that:

"   The Sabbath was a day of intolerable gloom. At the clink of the bell the whole flock hurry to Church with sorrowful looks and eyes bent upon the ground. It is considered sinful to look to the right or to the left.[10]

"The excessive time spent in religious gatherings began to interfere seriously with the practical routines of running the island. Old ladies and children who made a noise in church were lectured at length and warned of the dire punishments they could expect in the afterworld. During a period of food shortages on the island a relief vessel arrived on a Saturday only to be informed by the minister that the islanders had to spend the day preparing for church on the Sabbath and it was Monday before any supplies were landed. Children were forbidden to play games and required to carry a bible wherever they went. The St Kildans endured Mackay for twenty four years."
-----
regarding fish, the seabirds of today are having a hard time due to shortage of fish (which they need more than us humans do!):https://www.theguardian.com/environment/2015/dec/04/climate-change-threatening-puffins-kittiwakes-seabirds-st-kilda-scotland