this post was submitted on 28 Oct 2024
1535 points (98.8% liked)
Technology
59495 readers
3114 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
They're all built on top of OpenAI which is very unprofitable at the moment. Feels like the whole industry is built on a shaky foundation.
Putting the entire fate of your company in a different company (OpenAI) is not a great business move. I guess the successful AI startups will eventually transition to self-hosted models like Llama, if they survive that long.
Most projects I've been in contact with are very aware of that fact. That's why telemetry is so big right now. Everybody is building datasets in the hopes of fine tuning smaller, cheaper models once they have enough good quality data.
My company is realizing that hosting a model which will be private, cost-effective, and performing better than traditional algorithms is like finding a unicorn. Few months back, the top execs were jumping around GenAI like a bunch of kids. Fortunately, the Sr. research head beat some sense into them.
What kind of use-cases was it, where you didn't find suitable local models to work with ? I've found that general "chatbot" things are hit and miss but more domain-constrained tasks (such as extracting structured entities from unstructured text) are pretty reliable even on smaller models. I'm not counting my chickens yet as my dataset is still somewhat small but preliminary testing has been very promising in that regard.
Any time you ask very domain specific questions; eg "i have collected some soil samples from the mesolithic age near the Amazon basin which have high sulfur and phosphorus content compared to my other samples. What factors could contribute to this distribution?", both of-the-shelf local models & OpenAI fail.
The main reason is because these models are not trained on highly-specialized domains of text. Sometimes the models start hallucinating and which reduces our trust upon them.
Haha yeah the top execs were tripping balls if they thought some off-the-shelf product would be able to answer this kind of expert questions. That's like trying to replace an expert craftsman with a 3D printer.