Cheers for tagging me, I'll have to go through my stuff from when we did it and see if I have any feedback.
Fewer random comments from me when reporting the results. I looked back at them afterwards and thought they might be a bit much.
The random comments were very helpful and influenced the questions we used on lemmy.nz. I also think it's good to understand how the data has been analysed to highlight potential biases. i.e. don't feel bad about your rambling or I'll feel bad about mine ๐
I saw another comment suggesting various things that it would be fun to know about the community. These surveys aren't just a view into an instance, but the lemmy.ca and lemmy.nz surveys have been cited many times as they are some of the only info we have about Lemmy as a whole.
I was thinking we should have base questions, ones that we want to watch for changes over time. And then others that could be a snapshot insight that give us a glimpse into Lemmy with an understanding that people don't want to spend an hour filling in the survey so we can't ask everything.
I think the question list you have is a good set of base questions, and I don't think there are any you've suggested that shouldn't be asked each year. So I propose we pick a couple of extra questions. I think it would be nice for everyone to use the same questions and see differences across instances, but also using different ones per instance gives us wider insights. So I could be persuaded either way. The kind of things that @hendrik@palaver.p3x.de suggested, like whether people mostly use Subscribed, Local, or All feeds (mindful these might have different names in different apps/frontends). Or OS they use, etc. Just making sure it's only one or two questions so it's not putting people off doing the survey by making it too long.
Now to your question:
For self-identification, free text means people are more likely to write what they actually want instead of trying to push themselves into the box of listed options, even if there is an Other option. However, it's also a lot of work to group things, and things need to be grouped to make any decent result visualisation. Plus people should be allowed to group themselves instead of me doing it. So I suggest a predefined list with an Other free-text option.
I think this is the case for the other similar things you list as well. For ethnicity, for our survey we used the actual list from the NZ Stats department. It has been carefully refined over years, there's no reason to think we could do any better. But of course our list would not be very helpful for other countries, so for region-based instances, maybe something similar can be found from that region.
The disability question was quite tricky to work out how to put into a chart. People can have multiple, but then you don't really want them showing 10 times. I think I'd like to have a pre-defined list next time, with an Other free text option as well. Let people put themselves into their own categories instead of me trying to push them into groups.
For the fun question, what I like about Favourite Dessert is how groups of people are likely to tend towards certain answers, but different groups of people can tend towards quite different things (especially when it's regional). Favourite Comfort Food could be a good one, but it might be nice not to do food again. I'll let others suggest some things. I will note that I think this Fun question is different from the suggestion above to have snapshot questions to get insights into Lemmy. Having "What OS do you use" is a census question, "What is your favourite dessert" is a fun question. I think we should have both.