Hi all! I had posted a similar post in selfhosted, but it was deleted because it was only hardware-related. Therefore a new attempt here in the correct sub.
I want to reorganize my home server landscape a bit. A Proxmox server is to receive an LXC, with ollama and open webui. This is to be used for other containers that categorize via AI (paperless-ai, hoarder, actualbudget-ai, maybe home assistant speech) and also for one or the other chat. Speed is not so important to me, focus is on low idle and models up to 100 GB. It's ok to wait several minutes for answers. I don't want a GPU.
(Currently I successfully use models up to 32b on a Lenovo M920x with i7-8700 and 64 GB RAM. These models are supposed to run faster or slow up to 100b)
I want to spend 2000, if necessary 3000€ (mainboard, CPU, RAM).
My research showed that the bottleneck is always the memory bandwidth of the CPU. I would take 128GB RAM and want to use all channels.
Current variants: Intel i9-10940X, 14C/28T, 3.30-4.80GHz Quad Channel DDR4 93.9GB/s
Intel i9-14900K, 8C+16c/32T, 3.20-6.00GHz Dual Channel, DDR5, 89.6GB/s
Intel Core Ultra 9 285K, 8C+16c/24T, 3.70-5.70GHz Dual Channel, DDR5, 89.6GB/s (CU-DIMM: 102.4GB/s)
Intel Xeon Silver 4510, 12C/24T, 2.40-4.10GHz 8-Channel, DDR5, 281.6GB/s (means you need 8 sticks!)
Intel Ryzen 9 9950X, 16C/32T, 4.30-5.70GHz Dual Channel, DDR5, 89.6GB/s
These would all be in the €1500-2500 range.
Do you have any concrete experience? At the moment I prefer the Ultra 9, it's the latest platform and expandable. I prefer Intel because it is more energy efficient in idle mode.
The Xeons have the disadvantage that they have many channels, which then have to be filled with memory sticks to achieve the full rate =more energy, more cost).