We poisoned the rats, patched up the holes where the birds and squirrels got in, turned the dog on the possums, and, for a while, we were literally shooting raccoons off the side of the house with the sprayer jet on the garden hose (they still come back to visit but they don't rip our shingles off any more).

Maybe you could stop giving him apples for awhile and then bait a live trap with one (if it's your groundhog, although I suspect a lot of other varmints will go for apple).