I'm surprised this hasn't been said yet... but what I hate most about Signal is its requirement for a phone number. I don't want to be identified, and I want to be able to create multiple separate accounts with different identities if I want to.
I also hate the fact that it's a mobile-first service. Yes, there is a desktop application (and just one really crappy one at that), but it's clearly designed to revolve first and foremost around your phone and be virtually impossible to use without one. As someone who hates writing on a 3-inch screen, this is a also non-starter for me.
I understand the arguments about perfectionism, but this is too much. I'll stick with XMPP, Matrix and IRC, thanks.