227
Apple Intelligence summary botches a headline, causing jitters in BBC newsroom
(www.theregister.com)
this post was submitted on 17 Dec 2024
227 points (99.1% liked)
Technology
59963 readers
3471 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related content.
- Be excellent to each another!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, to ask if your bot can be added please contact us.
- Check for duplicates before posting, duplicates may be removed
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
In addition to pushing something before it’s ready and where it’s not welcome, Apple’s own stinginess completely screwed them over.
What do LLMs need to be smart? RAM, both for their weights and holding real data to reference. What has apple relentlessly price gouged and skimped on for years? Yeah, I’ll give you one guess…
Emphasis on the first L in LLM. Apple's model is specifically designed to be small to work on phones with 8 gigs of ram (the requirement to run this)
The price gouging for RAM was only ever on computers. With phones you got what got, and you couldn't pay for more.
Yeah... and it kinda sucks because it's small.
If Apple shipped with 16GB/24GB like some Android phones did well before the iPhone 16, it would be far more useful. 16-24GB (aka 14B-32B class models) are the current threshold where quantized LLMs really start to feel 'smart,' and they could've continue trained a great Apache 2.0 model instead of a tiny, meager one from scratch.
I don’t know how much RAM is in my iPhone 14 Pro, but I’ve never thought ooh this is slow I need more RAM.
Perhaps, it’ll be an issue with this stupid Apple Intelligence, but I don’t care about using that on my next upgrade cycle.
My old Razer Phone 2 (circa 2019) shipped with 8GB RAM, and that (and the 120hz display) made it feel lighting fast until I replaced it last week, and only because the microphone got gunked up with dust.
Your iPhone 14 Pro has 6GB of RAM. Its a great phone (I just got a 16 plus on a deal), but that will significantly shorten its longevity.
I wonder how much more efficient the RAM can be when the manufacturer makes the software and the hardware? It has to help right, I don’t know what a 16 Pro feels like compared to this, but doubt I would notice.
Your OS uses it efficiently, but fundamentally it also limits what app developers can do. They have to make apps with 2-6GB in mind.
Not everything needs a lot of RAM, but LLMs are absolutely an edge case where "more is better, and there's no way around it," and they aren't the only one.