The piano score is on IMSLP and makes it rather clearer what's going on. It begins with two descending lines in A misheberach backed by A minor harmony - scale a ^f e ^d c B A - then a line in D misheberach cadencing into A minor. At "Seul, pendant un instant" we move into E misheberach, again resolving to F sharp minor. Same scale for "De maniere a obtenit un creux", but this time it resolves into B minor. At "Tres perdu" we appear to be in E misheberach ending up in B minor again. "Portez cela plus loin" just cadences from A minor into F minor, except at "Ouvrez a tete" it's clear we aren't in F minor but F misheberach instead. End that phrase in D minor, reprise the opening and repeat it as a coda with an A minor finish.

That is, what he seems to be doing is using modal material in a tonal way - modulations to the subdominant, dominant and dominant-of-the-dominant but no change in the scale (whereas in the traditional uses of this scale you would probably move to a different mode on the same pitch set instead).