XMPP most definitely! Especially if you want to have connectivity to other servers at all (like simplex). It's much simpler, more well-known, battle hardened, and still supports E2EE and video calling very well.
I recommend prosody. I recently went through the process of setting up a server and have a draft blog on it half way finished if you want an account of the experience.
EDIT: Blog post is live at https://roguesecurity.dev/blog/xmpp