Jake, you aren't a 4 year old whose little brother is annoying you...   If it were your child or your head it was being held against, you might feel a little differently.

Sure it gives you all those feelings, but then so does holding a new born baby... much as I hate babies... I know which I'd prefer to hold.

LTS