Sound-proof the mutual wall from your side?
High density injectable foam, heavy sheetrock?
Think of it as constructing an isolated sound studio in reverse.
Good luck.