My rack is finished for now (because I'm out of money).
Last time I posted I had some jank cables going through the rack and now we're using patch panels with color coordinated cables!
But as is tradition, I'm thinking about upgrades and I'm looking at that 1U filler panel. A mini PC with a 5060ti 16gb or maybe a 5070 12gb would be pretty sick to move my AI slop generating into my tiny rack.
I'm also thinking about the PI cluster at the top. Currently that's running a Kubernetes cluster that I'm trying to learn on. They're all PI4 4GB, so I was going to start replacing them with PI5 8/16GB. Would those be better price/performance for mostly coding tasks? Or maybe a discord bot for shitposting.
Thoughts? MiniPC recs? Wanna bully me for using AI? Please do!
Well, I always advocate for using the stuff you have. I don't think a Discord bot needs four new RasPi 5. That's likely to run on a single RasPi3. And as long as they're sitting idle, it doesn't really matter which model number they have... So go ahead and put something on your hardware, and buy new one once you've maxed out your current setup.
I'm not educated on Bazzite. Maybe tools like Distrobox or other container solutions can help running AI workloads on the gaming rig. It's likely easier to run a dedicated AI server, but I started learning about quantization, tested some models on my main computer with the help of ollama, KoboldCPP and some random Docker/Podman containers. I'm not saying this is the preferrable solution. But definitely enough to get started with AI. And you can always connect the computers within your local network, write some server applications and have them hook into ollama's API and it doesn't really matter whether that runs on your gaming pc or a server (as long as the computer in question is turned on...)
Ollama and all that runs on it its just the firewall rules and opening it up to my network that's the issue.
I cannot get ufw, iptables, or anything like that running on it. So I usually just ssh into the PC and do a CLI only interaction. Which is mostly fine.
I want to use OpenWebUI so I can feed it notes and books as context, but I need the API which isn't open on my network.
Thanks for the info. Some day I'll try the shiny modern distros and learn the little peculiarities. I use a weird mix of Debian, NixOS and LMDE and it's relatively straightforward to add firewall rules to those, both dynamically to nftables and to the persistent config... And I believe Debian didn't even come with firewalling out of the box... But I understand Debian might not be the best choice for gaming and there is for example some extra work involved to get the latest Nvidia drivers. Neither is it an atomic distro.
Honestly if you're not gaming or playing with new hardware, there is absolutely no point.
I've considered swapping this computer over to Fedora for a hot minute, but it really is a gaming PC and I should stop trying to break it.
You could probably run several discord bots on a Raspberry Pi 3, provided they aren't public and popular