I was taught that an English word gets the -es, but a foreign word gets the -s.

potatoes, but
scherzos

However, we borrow so many foreign words and get accustomed to them that after a while it's hard to tell what's foreign.

No doubt 'potato' itself came from some foreign tongue.

(I would use hobos because it's simpler.)