"Congé means "leave" or "holiday"; "la belle" is a term much used in French poetry and song, and doesn't translate comfortably into English.  It's used both in the second and third person; "fair maid" is as close a match as I can think of, so "Taking leave of my fair maid", here, I'd say.  The last two lines both seem to be spoken by her: "mes belles amours" in indeed plural, but refers here to Love itself rather than the object of it; the word is usually masculine, but is sometimes given a feminine gender and plural form in poetry.

Malcolm