I think he's just playing with the mouse. He's still pretty young. He'll catch on. Your job is to grab the mouse after he releases it and either kill it or deposit it down the street.

Somehow that never works for us, the dumb things come right back and our cat, FANG then eviscerates them. She has a taste for mouse brains. She sucks out the grey matter, slits the belly open, drapes the guts all over the bathtub (her fave place to "play" with mice) and then walks away. Cat the Ripper.