A person could read a whole novel in the time it would take to go through all those threads.

The line about sorrow means that the narrator always regretted that he was not permitted to marry the woman who moved through the fair. He married someone else and never said anything about his regret to his actual wife.

It is amazing that the author of the words managed to convey so much in so few words and with an admirable absence of self-pity.