To summarize, there seem to be three kinds of opposition to the Olympics intertwining in this discussion:

1. People who are against spectator sports in general, and either are indifferent or want to deny them to everybody else.

2. People who like sports, but not the Olympic variety, due to hype, hypocrisy, corruption, cost or whatever.

3. People who feel that the money could be better sent elsewhere.

On the latter point, it's not necessary that the Olympics lose money. The Winter Games in Calgary made a profit (and are reputed to have been the best games ever). (I don't take personal credit -- after the fact I regretted not having been a volunteer; however, I did give directions to the Rumanian bobsled team once...) The atmosphere in the city was exhilarating, especially in the evening in Olympic Plaza, where most of the medal ceremonies took place. Even my wife, who's generally indifferent to sports, got into the spirit.

I think if the games divert money away from social purposes, it's due to mismanagement or improper organization, not the fundamental nature of the Olympics themselves. If the Olympics are from a sporting perspective, this needs to be fixed, and it can be fixed if the people in charge want it badly enough.