If you can't get the physical checks, you should at least be able to get digital images of those checks, front and back. You might even be able to get them online, if your bank provides "online banking." I mean, view them on your home computer screen, and print them out yourself, if you like.

I wouldn't change banks just yet. If you close your account, you will likely lose the ability to access that information (without a lot of trouble, anyway), and the bank will have even less incentive than they do now to help you.

Of course, you could decide to simply "let go" of the past problems and concentrate on not letting the same problems happen in the future. In that case, paying the tax in person is probably a good idea.