Technology

What is an offline GPU?

Use it to run !localllama@sh.itjust.works stuff and Stable Diffusion

You may have already considered this, but if you can do so, I would not use a laptop for this. Laptops are built to be power-efficient, space-efficient, and generate and deal with a limited amount of heat. Hardware that is good at running latent diffusion software is bad at these.

What you might give serious consideration to doing, if you have not already considered doing so and just specifically want to have this running on hardware you control, is building a desktop and then simply accessing it remotely from a laptop, especially since you say that you already intend this only to be a secondary laptop. I have a beefy desktop that I have used Stable Diffusion and KoboldAI on remotely, either via WiFi or cell phone data link. The software is designed to run in that kind of environment, run by a remote Web browser. You don't need a whole ton of bandwidth, particularly for KoboldAI. It has some other benefits:

• You won't be sitting at a laptop with the fans spooled up and sounding like a jet engine.

• Power use is minimal. Gaming laptops have very limited battery life when actually using their GPUs (and even those don't compete with desktop GPUs on performance). If were going to be doing the computations on a laptop, I'd either need it to be plugged in, have only very short sessions, or have some kind of large powerstation. Even a 100Wh laptop battery, the most you're going to be able to get, won't last long on a powerful laptop GPU running all-out. Here's a comparison between the desktop and laptop RTX 4090. If your GPU is drawing 80W to 150W -- what the laptop variants there are set up for -- even disregarding the rest of your laptop's hardware use, it's going to go through a laptop battery that's 100Wh at most very rapidly.

• You'll have better performance on a desktop.

• Your laptop won't be hot.

Now, that won't give you a gaming laptop for running games on the go, and I don't know if that means that it won't cut the mustard for you. My personal take is that for really fancy 3D games, I'd rather game on a desktop for similar reasons, though 3D games are less-problematic in my book than latent diffusion software. There are plenty of games that one can perfectly reasonably use on a laptop.

It may be that you've considered that and are fine with that. Maybe you are always going to use the thing for latent diffusion and maybe gaming where you can plug it in. Maybe the heat doesn't bother you (external keyboard/mouse?) and the fan noise doesn't bother you (headphones with good sound isolation?).

But if you haven't, I'd consider whether using a remotely accessible desktop might fit your use case better.

EDIT: I'd also throw out one other possibility. If you want local use and are confident that you are always going to have an external source of power when doing this (or are willing to lug around a power station or 12V/24V/48V solar power system or something), you might consider a small, portable desktop-style system. I've been considering one of those recently myself (for a different reason, just because I'm so annoyed with how dramatically fewer options one has for building a laptop to one's desired configuration than one does a desktop). That won't be as power-efficient or as portable as a gaming laptop, but it'll cost less, have better performance, and provide a lot more options as to build.

Whatever laptop you do choose, make sure it has a fully featured USBC port. So you have the option to use an eGPU.

If you want something that lets you physically remove a WiFi card check out the frameworks laptop. Other than them there is no such thing as a “WiFi card” in laptops, it’s all fully integrated now and I’d be surprised to see a new laptop from factory without some version of WiFi built in.

Gaming laptops (especially 17 inch devices) tend to have removable wireless networking cards.