I was in Baku in 2006 so I know about the water... :P
I remember the open mic sessions near the British Embassy. The Irish pub advertised singers' nights but I only ever saw local jazz and rock groups actually play there.
I also got a gold tooth for $100 which I still have!