This is what MacColl says on the subject in his autobiography, followed by a few interesting snippets from Robin Denselow:

[1990:] At different times I've thought of myself as a dramatist, a slater's labourer, teaboy, errand lad, ballad-singer, apprentice motor mechanic, unemployed ... but never as a songwriter. [...] If the success [of The First Time Ever] had followed hard on the heels of the song's composition, it might have been a different story. I mean, I might have noticed what was happening. But it took more than fifteen years to take off and hit the jackpot. Peggy gave it its first performance at a solo concert in Los Angeles. It was part of an eight weeks' tour of the USA, for which, at the last moment, the State Department had refused to grant me a visa. [A] lot of people heard The First Time Ever sung in the way it was meant to be sung. Twelve months later the Kingston Trio recorded it in a sanitised version - they changed 'The first time ever I lay with you' to 'The first time ever I danced with you'. Several more months passed and it was recorded again, this time in a more apt arrangement by Peter, Paul and Mary. It was slowly catching on and, in the months that followed, it appeared on albums made by the Brothers Four, the Smothers Brothers, Harry Belafonte, Elvis Presley and others. [...] It made singularly little impression on me. I was scarcely conscious of it. I had lots of other things on my mind, not least of which was the need to earn a living. [...] It wasn't until 1971 when Roberta Flack recorded her soul version of the song that I became aware of having written a commercially successful 'hit'. At first I didn't realise just how successful. A friend happened to mention that he'd heard it sung in a pop-music programme on the radio. I was unimpressed and continued to be unimpressed. I was in my mid-fifties and had lived hand to mouth for almost all of my life. [...] What I do remember about the early days of the success of First Time Ever is a curious phone call. In 1971, an American voice at the other end introduced itself as belonging to the London representative of an American magazine. 'I'd like to ask you a few questions,' said the voice. 'Go ahead,' says I. 'How does it feel to have written the number one hit in the States?' I had just started to reply when the voice cut in again, the smooth interviewer's manner collapsing before the sudden flood of hate and anger. 'How does it feel to be a dirty fucking cocksucking red bastard ...' At this point he became inarticulate with rage and began to shriek obscenities. I was shaking when I put the phone down. Completely unnerved. [...] The success of The First Time Ever was a kind of watershed in our lives. Both Peggy and I were now in a position to spend a lot more time on political songwriting. (MacColl, Journeyman 362ff.)

---

[1989:] It is ironic, considering MacColl's history, that the one song for which he is best known is not political at all, but a love song, and that it became a hit not for him, or Peggy Seeger, but for Roberta Flack. [It] was written "in eight or nine minutes" in 1956 while MacColl, in London, was phoning Peggy Seeger in Los Angeles. Peggy sang it in public for the first time that same evening. In 1972 that same song caused a sensation when Flack's version was used in the soundtrack of the Clint Eastwood film 'Play Misty For Me'. It was a million-selling number one hit in the USA, and won Grammys for both her and MacColl, and has earned far more than anything else he has written. When I met him at his house in a quiet, tree-lined street in the south London suburb of Beckenham, I noticed that there were builders at work redecorating. [...] He could afford this, he said, because Oil of Ulay had used one line of The First Time Ever in a beauty products television commercial. It was all the more ironic considering the history of MacColl's relationship with the USA. In Britain, in the fifties, MacColl's views were profoundly opposed to those of the Conservative Government in power, but that didn't stop him working [...]. It was when MacColl tried to sing in the USA in 1960 that [...] the American Embassy in London refused to grant him a visa, and when he asked why, he was shown a dossier and a photograph of himself that had been taken at an anti-Fascist meeting outside the Free Trade Hall, Manchester, twenty-seven years earlier. Like so many in the USA, he was told he had committed that quite extraordinary sin of being a 'premature anti-Fascist'. (Denselow, Music 28)