I consolidated my setup a bit. This is my local LLM hosting server. I took a gamble on a Chinese NVLink SXM2 mezzanine board from ebay, it was surprisingly plug and play for my dual 16GB v100s ๐. I'm also running a Tesla P40 that I repasted with liquid metal that sits at 24C idle ๐ฅถ.
It's been a fascinating experience so far and I feel like I'm only touching the surface. I'm exploring some of the various memory harnesses to hook into Hermes Agent and see how it learns with extended use.