366
this post was submitted on 03 Aug 2025
366 points (97.2% liked)
Technology
73734 readers
3896 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
If you don't want corporations to use you chats as data, don't use corporate hosted language models.
Even non-public chats are archived by OpenAI, and the terms of service of ChatGPT essentially give OpenAI the right to use your conversations in any way that they choose.
You can bet they'll eventually find ways to monetize your data at some point in the future. If you think GoogleAds is powerful, wait until people's assistants are trained with every manipulative technique we've ever invented and are trying to sell you breakfast cereals or boner pills...
You can't uncheck that box except by not using it in the first place. But people will sell their soul to a company in order to not have to learn a little bit about self-hosting
This is basically a "if you don’t want your data to be used, run your own internet" comment
It’s just not doable for pretty much everyone
Modern LLMs can serve you for most tasks while running locally on your machine.
Something like GPT4ALL will do the trick on any platform of your choosing if you have at least 8gb of RAM (and for most people nowadays it's true).
It has a simple, idiot-proof GUI and doesn't collect data if you don't allow it to. It's also open source, and, being local, it does not need Internet connection once you downloaded a model you need (which normally takes a single-digit number of gigabytes).
If you want actual good features like deep research or chain of thought, eh, not sure it’s a good choice
The models will also not be very powerful
And you don't need any of that. You don't even need a local LLM.
So if you decide you want it, then that's on you, and you have made the choice to give up your data.
and you don't need a computer, and you don't need to eat good food
It's just that you lose so much productivity, comfort and so on
When such a tool is a difference between 30 mins and 5 hours of work, then you simply use it. You either move with the masses to compete, or you don't, but you'll pay the price anyways.
If you think LLMs are as fundamental as having a computer or internet access, then I really just don't know what to say.
You have clearly never been in that situation then. It is obviously not like this for many people, but for students for example, it often means a lot more
While I don't fully share the notion and tone of other commenter, I gotta say LLMs have absolutely tanked education and science, as noted by many and as I witnessed firsthand.
I'm a young scientist on my way to PhD, and I get to assist in a microbiology course for undergraduates.
The amount of AI slop coming from student assignments is astounding, and worse of all - they don't see it themselves. When it comes to me checking their actual knowledge, it's devastating.
And it's not just undergrads - many scientific articles also now have signs of AI slop, which messes up with research to a concerning degree.
Personally, I tried using more specialized tools like Perplexity in Research mode to look for sources, but it royally messed up listing the sources - it took actual info from scientific articles, but then referenced entirely different articles that hold no relation to it.
So, in my experience LLMs can be useful to generate a simple text or help you tie known facts together. But as a learning tool...be careful, or rather just don't use them for that. Classical education exists for a good reason, and it is that you learn to get factually correct and relevant information, analyze it and keep it in your head for future reference. It takes more time, but is ultimately much worth it.
Sure, many don’t care and I have also experienced this, but it’s a fabulous way to quickly get a glimpse at a subject, or to get started, or to learn more. It’s not always correct, but for known subjects it’s pretty good
Anything related to law or really specific subjects will be horrible though
Sure, but not everyone teaches well enough, and LLMs are one of the ways to balance this, kinda
And if you don’t understand then… yea it’s still useful as a way to avoid failing a year which is morally questionable but hey, another topic
Alright, we generally seem to be on the same page :)
(Except numerous great books and helpful short materials exist for virtually any popular major, and, while they take longer to study, they provide order of magnitude better knowledge)
Yeah, this thing that's notorious for hallucinating and has only recently become even somewhat reliable is essential.
How did all those students from 2021 even survive??
Jesus, we're absolutely fucked.
Surviving is different than living in good conditions
As with every tool, it has downsides. Learn to use it or continue to whine
That is... literally my point.
You're comparing very disparate things. And I already pointed out the downside. No one here is whining about anything.
And I already pointed the upside. You don't have to use the default UI though, custom UI exists, APIs exists, and you don't have to enter personal infos in prompts, as well as use your residential IP... this pretty much makes it unlinkable to you
To use your analogy, it's like someone said, "Well, if you want a Michelin 5-star steak au jus, then your wallet is gonna take a hit". And you replied, "That's like saying in order to eat dinner you need to raise your own livestock and train for years as a professional chef".
I'm not saying corporate LLMs are bad, or that they have no upside. I'm saying your scale for what's essential and what's a luxury is alarming.
I wish they were always a luxury, but in some situations they’re just too important to me
Hi there, I’m thinking about getting into self-hosting. I already have a Jellyfin server set up at home but nothing beyond that really. If you have a few minutes, how can self-hosting help in the context of OPs post? Do you mean hosting LLMs on Ollama?
Yes, Ollama or a range of other backends (Ooba, Kobold, etc.) can run LLMs locally. Huggingface has a huge number of models suited to different tasks like coding, storywriting, general purpose, and so on. If you run both the backend and frontend locally, then no one monetizes your data.
The part I'd argue that the previous poster is glazing over a little bit is performance. Unless you have an enterprise-grade GPU cluster sitting in your basement, you're going to make compromises on speed and/or quality relative to the giant models that run on commercial services.
It's also going to cost more, because you almost certainly are only going to be using your hardware a tiny fraction of the time.
Possibly, yes. There are models that will run on consumer-grade GPUs that you might already have or might have purchased anyway, where you might say there's no incremental cost. But the issue is that the performance will be limited. The models are forgetful and prone to getting stuck in loops of repeated phrases.
So if instead you custom-build a workstation with two 5090s or a Pro 6000 or something that pushes you up to the 100 GB VRAM tier, then absolutely, just as you said, you'll be spending thousands of dollars that probably won't pay back relative to renting cloud GPU time.
Thanks for the info. Yeah, I was wondering what kind of hardware you’d need to host LLMs locally with decent performance and your post clarifies that. I doubt many people would have the kind of hardware required.