Rag, the Irish were lampooned in Victorian Punch cartoons, as were the aristocracy, royalty, politicians and, especially, the English proletariat.

Can you find any examples of "Irish famine refugees depicted as apes?"
I do not believe that there ever were any.