Forum Discussion

vel0c1ty's avatar
vel0c1ty
Puritan
15 hours ago

Curious about home lab or on-prem models

A friend told me to stop trying to squeeze local models onto my gaming PC and just buy the right box. So I did, and hoping not to regret it. Ordered a BOSGAME M5. Ryzen AI Max+ 395, 128GB unified memory, up to 96GB of it assignable to the GPU. Size of a hardcover book.

I've been running LM Studio with models off Hugging Face. Fine for small stuff. So, people who've actually done this at home... What are you running besides LM Studio? What hardware did you buy and regret? Where does unified memory fall apart in practice?

One ask. Human answers only. I can already ask a model what a model thinks. I want to know what broke in your house at 11pm on a Tuesday.

ai
AMD
genai
llm

1 Reply

  • vel0c1ty's avatar
    vel0c1ty
    Puritan
    14 hours ago

    I guess I could just edit too, but curious about what anyone does homelab rack wise? I'm looking at maybe doing a 10" rack for this since it is small stuff like Lab Rax or something similar.