this post was submitted on 25 Apr 2024
1103 points (99.1% liked)

Memes

45704 readers
1308 users here now

Rules:

  1. Be civil and nice.
  2. Try not to excessively repost, as a rule of thumb, wait at least 2 months to do it if you have to.

founded 5 years ago
MODERATORS
gary_host_laptop@lemmy.ml
sexy_peach@feddit.de
cyclohexane@lemmy.ml
cypherpunks@lemmy.ml
1103
Meta AI (image.nostr.build)
submitted 6 months ago by n7gifmdn@lemmy.ca to c/memes@lemmy.ml
23 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] mp3@lemmy.ca 51 points 6 months ago* (last edited 6 months ago) (2 children)

The quantized model you can run locally works decently and they can't read any of it, which is nice.

I use that one specifically https://huggingface.co/lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF/blob/main/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf

If you're looking for a relatively user-friendly software to use it, you can look at GPT4All (open source) or LM Studio.

[–] passepartout@feddit.de 16 points 6 months ago

If you're ready to tinker a bit i can recommend Ollama for the backend and Open web UI for the frontend. They can also both run on the same machine.

The advantage is that you can use your GPU to compute, which is a lot faster.

[–] nialv7@lemmy.world 12 points 6 months ago (1 children)

Pretty sure LM studio is not open source

[–] mp3@lemmy.ca 7 points 6 months ago

You're right, I thought they were but I checked their GitHub and LM Studio itself isn't.