Selfhosted

59897 readers

751 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Another reason to self host your own AI (aussie.zone)

submitted 2 weeks ago* (last edited 2 weeks ago) by SuspiciousCarrot78@aussie.zone to c/selfhosted@lemmy.world

50 comments fedilink hide all child comments

Recent post re: AI as utility

https://www.tomsguide.com/ai/people-will-buy-intelligence-from-us-on-a-meter-chatgpts-ceo-sam-altman-has-critics-worried-with-his-ai-vision

Myself, I'm a fan of local LLM / self hosted ML.... but if you ever needed a clarion call that a hard pivot is coming (soon) for online/ cloud based AI...Altman et al are making some concerning mouth noises (to say nothing of broader concerns with OAI, Anthropic etc).

Right now, I'm sketching out a plan where my Raspberry Pi (always on, 2-3w) uses a magic packet to wake up my modest AI server (Lenovo P330 with Tesla P4) if/when needed (Qwen 3.6-35B-A3B); no point in chugging down 80-100w, 24/7 for no good reason.

If the trend continues the direction it appears to be (increasing costs, environmental impacts etc) then I'd feel a lot better hosting my own as port of first call and replacing simpler tasks with more traditional programs. YMMV.

you are viewing a single comment's thread
view the rest of the comments

[–] SuspiciousCarrot78@aussie.zone 3 points 2 weeks ago* (last edited 1 week ago)

Respectfully, that's not really how local LLMs work.

A GGUF model sitting on my hard drive has no ability to "send content back home" any more than a PDF or a JPEG does. If you're running something like llama.cpp or Ollama entirely locally, the model weights are just data files.

The real privacy concerns are cloud APIs, telemetry in front-ends, browser extensions, analytics, update services, or accidentally exposing a service to the public internet.

"Self-hosted AI" isn't one thing. There's a huge difference between:

Running ChatGPT through an API
Running a commercial AI appliance
Running a local Qwen/Mistral/Llama model on your own hardware

Firewalling internet-facing services is good advice. Assuming every local model is secretly uploading prompts is not.

EDIT: for the record, I didn't down vote you - that was someone else.