I think SRS has cut to the point. Letters are not words so you can't use the normal rules of grammar to make their plurals. You have already broken the rules so what you do after that doesn't really matter. The same applies to numbers but, fortunately, there are words for the numbers so you can write "all at sixes and sevens" instead of "all at 6s and 7s". I don't think there are any agreed words for the letters - Aye, Bee, See, Dee... How would you spell the word for the letter W?