There's an excerpt of a different version of the song in a book called Bed-Stuy in Da Brownstone House: Brooklyn Topographies as an Urban Metaphor, By Irene Fowlkes. The Fowkes book includes the chorus and the "Bajans" (pronounced BAY-juns) verse (shown below). The song below is my transcription of a recording by The Calypso Carnival, Irene Lusan, and Lord Zebedee on an album titled Calypso Carnival.

SMALL ISLAND

CHORUS
Small Island, go back where you come from
Small Island, go back where you come from
You come from Trinidad in a fishing boat
And now you wearing a great big overcoat!
Small Island, go back where you really come from.

You see them Bajans, they're the worse of them all!
You hear them say, 'I ain't gwine back at all.'
They come by the one and they come by the two,
And now you see them all over Lenox Avenue.
Small Island, go back where you really come from.

Back home she drink rum and coconut water,
Standin in the alley, tellin' you what you oughta;
They come over here, drinkin' scotch and soda,
And they she walks around with a perfume odor.
Small Island, go back where you really come from.

CHORUS

The other (?) West Indians, see them all around,
With American dollars, no more shilling and pound
They have Cadillac and they have diamond rings
And some of the men, they are great big number kings,
Small Island, go back where you really come from.

CHORUS

Down home, she wear a house frock all the time
She look so sad, you wonder what's on she mind
She come over here wearin' silk and laces,
And now she get drunk in exclusive places
Small Island, go back where you really come from.

CHORUS (TWICE)