Selfhosted

59897 readers

524 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

Rules:

Be civil: we're here to support and learn from one another. Insults won't be tolerated. Flame wars are frowned upon.
No spam.
Posts here are to be centered around self-hosting. Please ensure it is clear in your post how it relates to self-hosting.
Don't duplicate the full text of your blog or git here. Just post the link for folks to click.
Submission headline should match the article title.
No trolling.

Resources:

selfh.st Newsletter and index of selfhosted software and apps
awesome-selfhosted software
awesome-sysadmin resources
Self-Hosted Podcast from Jupiter Broadcasting

Any issues on the community? Report it using the report flag.

Questions? DM the mods!

founded 3 years ago

MODERATORS

curbstickle@anarchist.nexus

curbstickle_lw@lemmy.world

Another reason to self host your own AI (aussie.zone)

submitted 2 weeks ago* (last edited 2 weeks ago) by SuspiciousCarrot78@aussie.zone to c/selfhosted@lemmy.world

50 comments fedilink hide all child comments

Recent post re: AI as utility

https://www.tomsguide.com/ai/people-will-buy-intelligence-from-us-on-a-meter-chatgpts-ceo-sam-altman-has-critics-worried-with-his-ai-vision

Myself, I'm a fan of local LLM / self hosted ML.... but if you ever needed a clarion call that a hard pivot is coming (soon) for online/ cloud based AI...Altman et al are making some concerning mouth noises (to say nothing of broader concerns with OAI, Anthropic etc).

Right now, I'm sketching out a plan where my Raspberry Pi (always on, 2-3w) uses a magic packet to wake up my modest AI server (Lenovo P330 with Tesla P4) if/when needed (Qwen 3.6-35B-A3B); no point in chugging down 80-100w, 24/7 for no good reason.

If the trend continues the direction it appears to be (increasing costs, environmental impacts etc) then I'd feel a lot better hosting my own as port of first call and replacing simpler tasks with more traditional programs. YMMV.

you are viewing a single comment's thread
view the rest of the comments

[–] sturmblast@lemmy.world 3 points 2 weeks ago (2 children)

P100s are dirt cheap on ebay fyi

[–] brucethemoose@lemmy.world 3 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

In practice, they’re not very good because of broken FP16, broken kernels, high idle usage and a bunch of other things.

Same with the AMD MI50 and MI100. Looks great on paper, not practical IRL, unless you want to pay a whole team of software devs to fix them for you.

Better to just save up for a 2080 TI or 3090, sadly.

[–] sturmblast@lemmy.world 1 points 2 weeks ago

Not having issues

[–] SuspiciousCarrot78@aussie.zone 2 points 2 weeks ago* (last edited 2 weeks ago) (1 children)

Huh - cheaper than the P40s (though less VRAM) but larger bandwidth due to HBM2. Good looking out

[–] sturmblast@lemmy.world 1 points 2 weeks ago (1 children)

They rip

[–] surewhynotlem@lemmy.world 1 points 2 weeks ago (1 children)

I was looking at that. Does it end up faster than something like a 1080?

[–] SuspiciousCarrot78@aussie.zone 2 points 2 weeks ago* (last edited 2 weeks ago)

Numbers about 3-4x. The P100 is near 800 GB/s. The 1080 is what... 192GB/s? Hell, even if it were double that, HBM2 simply has larger bandwidth. The 1080 was a gaming card; the P100 is a server / number cruncher.