this post was submitted on 02 Feb 2024
132 points (83.0% liked)
Technology
59534 readers
3168 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
At least what I see with this experiment/article is that is overly verbose, he takes a long time to get to the point. And then when he does his methodology shows an experiment that cannot be verified. Even when something is "subjective" we can still draw conclusions from it if we set up proper non-subjective ways of evaluating the results we see (ie. Rubrics). The fact that he doesn't really say what leads him to say in detail what is a "terrible/v. bad/bad/good result" is a massive red flag in his method.
After seeing that, I no longer read the rest of it. Any conclusions drawn from a flawed methodology are inherently fallacies or hearsay.
If in any case it is further explained in the article and that somehow refutes what I've postulated later on, then I would have to say that the article is poorly written.
All this to say... I agree with you, not worth the read.
The entire post is exact details for why he decides each rating for each query
No it isn’t. He for example evaluate that Kagi and Marginalia get the same score if you have to read as far down as to the 10th result for Kagi, while Marginalia has no answer. How is that the same score? There is no explanation. There is a lot of text, and then in the end he has made some subjective choices.
Oh, then that's a bit inconsistent