I can identify with a sense of discomfort about concerts in church due to my own teenaged experiences within the fundamentalist wing of UK Christianity. During that part of my life I would have disapproved of a church building being used for secular purposes. Having moved away from any version of Christianity, I am left with a tendency to associate certain kinds of church building ("Non-conformist") with a sense of feeling inhibited, which in turn (when I have attended concerts) affects my enjoyment. However it does not affect me in older more traditional church buildings whose architecture, history (and acoustics) I very much enjoy.

So the OP has a valid point that a (possibly small) number of punters may not attend such venues; I would be one such person. Its purely subjective, shaped by my personal past and the sense I make of that now.

(Would now be a good time to share the ironic flipside of this? I still love singing gospel songs, almost as a kind of cultural artefact in my personal heritage, but you will usually find me doing so in
The decidedly secular settings of folk clubs in pubs, or at festivals!)