Jeremy, for a 'generic' cafe type place that sounds like a good set list - maybe it's the place, and not you. Venues have their own 'ambience' and sometimes a surly owner is a reflection of the general atmosphere of the place. Keep working on the eye contact, talking to the audience, etc. It's very important. Keep in mind that any kind of venue where there's eating, talking and especially drinking is probably going to be difficult. Don't get discouraged.

I used to perform solo a lot, but now for pubs and restaurants I have a banjo player and a fiddler that I work with. It is MUCH easier to keep the audience's attention with a couple of other people up on stage. The music is usually fuller, and you can banter with each other and bounce jokes off one another - and that sometimes helps to get the audience interested. Solo work in pubs is hard!