I think authors under-estimate what a difficult task this is for a publisher. I want to believe in every manuscript, every opportunity, that crosses my desk. But I know that there's no market for some, and some (a surprisingly few) just aren't very good, many have the germ of a good idea, and some are good but just not good enough to run the gauntlet of publication. Then I have to believe in the author. Will they put in the hard yards of not only writing but of rewriting and drafting and rewriting again. And then of embracing the critical process of editing. Then of backing the book when it is published and then of doing it again for the next book and the next. Are they hungry enough? Reading between the lines of the covering letter of many of the manuscripts we receive, I sense that the author is in love with the idea of writing. It is a romance not a commitment.
Rejecting manuscripts is my least favourite task. When an unsolicited manuscript lands on my desk there's a reluctance to open it as I'm most likely to have to say no.