What I have seen people do in the past is use ansible secrets to secure the env file.
So only when the playbook is running does the env get decrypted.
Digital Ocean has an extensive how to on it.
https://www.digitalocean.com/community/tutorials/how-to-use-vault-to-protect-sensitive-ansible-data