Assuming that your two 8 ohm speakers are wired in parallel, and the speaker pair input impedance is nominally 4 ohms, running on the amp's 8 ohm (or 16 ohm) output impedance probably will introduce so much distortion due to the impedance mismatch that it won't make much difference what gain (volume) setting you crank the amp to.

Whether using the wrong amp output impedance will damage the speakers or the amp depends somewhat on the details of the components, but damage to either or both is fairly likely. (The cheaper component will be protected by the failure of whichever parts are most expensive to replace.)

This is especially true if you're attempting to crank the amp to higher power levels based on the mostly mythical notion that the amp has to run at high power to reduce distortion.

The lack of "tonal purity" you hear most likely is because you've been sitting in front of (or too close behind) too big an amp with too powerful speakers, for too many years, you're getting old, and the best cure for the "bad sound" would be a trip to a good audiologist. In this context, "old" means two or three years - or less - in the presence of high volume sounds, and some estimates are that more than half of people have "old ears" before they get out of high school in the US (about age 18). (That's why screaming has replaced singing in much of the "entertainment" business(?).)

On the possibility that there is a real problem with your sound kit, the only realistic fix is to get a smaller kit (especially the amp) for the times when you don't really need 50 watts. You could consider a pair of "low efficiency speakers" but speakers of the kind you'd need would likely cost as much or more than a decent 10W or 20W amp.

Note that the power rating of the amp does have some relationship to the level at which it's intended to operate. The "power rating" on the box for speakers usually is only the maximum power that you can put in without destroying them, and generally has little to do with what amount of power you need to put in for them to sound good.

You haven't indicated whether your problem is in a performance situation, large or small venue, or home practice etc., and the situation might affect recommendations; but a 50W amp is generally more than needed for a small indoor venue, and in a place big enough to justify it you'd often be better off with a preamp to plug into the house system where the "sound engineer" can take care of the amount of your noise that competes with the crowd.

John