The performance target of "linux desktop running lxd" and the price tag "not crazy expensive" is not very clear...
I would personally look for a fanless barebone pc and equip it with as much RAM and storage as you like. One example for this could be the Zotac ZBOX CI629, which you can get for around 400 Euros and has a 13th gen Intel i3 built in.
Is this within your budget?
Even a raspberry pi can run linux with an lxd desktop, and this i3 is a few times faster than a raspberry pi. But it depends on what you want to with it...