Belafonte collaborator Irving Burgie seems to suggest Norman Luboff was inspired by... Irving Burgie:

"The second Belafonte album of my songs titled Belafonte Sings of the Caribbean was in the works and included "Island in the Sun" and "Lead Man Holler" from the film. "Angelina" was also among the ten songs set for the new LP. The album was released in 1958 around the time of the release of the film Island in the Sun.

Also in the same Belafonte album was a song entitled "Don't Ever Love Me." The chorus melody was my reworking of a song originally from Haiti called "Choucounne," which I had traced back to 1875. My version was the first time that the melody was used in a pop song. Seven months later, the melody was picked up by Norman Luboff, the choral director, who had worked on the Belafonte album. He called his version "Yellow Bird" and it was first sung by The Mills Brothers and became immensely popular. I also later recorded a new lyric for the song for a children's play for McGraw Hill Publishing Company."

Burgie, Irving, Day-O!!!, (Irving Burgie: Caribe, 2006, pp.208-209)