I'm pretty sure AbcExplorer relies on ghostscript and abcm2ps to generate the notation images, which will make it pretty damn difficult to make portable in the way you suggest. My understanding is that apps have to be pretty much designed from the ground up to be executable from detachable media, it's not something that can happen purely by chance or desire.

Grishka's solution is a good one; the only other thought I have is that many people have like me moved on from AbcExplorer to EasyAbc, which, although still needing an installation, has an even lighter footprint on the host machine and will uninstall completely if you don't want to leave it in place.