That's because he knows you. I imagine it would be a different scenario if someone tried to sneak in through a window, or picked the lock on the door at 2:15 in the morning.