As someone exploring local LLM operation, I’m working to understand cost-effective hardware upgrades that can support continued learning and testing. My current setup includes a gaming PC with a Ryzen 7600x processor, 32GB of RAM, an AsRock B650 PG Lightning motherboard, and a 7900GRE graphics card with 16GB of VRAM. Despite these specifications, I’ve found VRAM to be the primary limitation—even quantized small models like Mistral struggle to run reliably through Fedora using GPT4All or Ollama.
This leads me to consider whether adding an rx 9060 xt with 16GB of VRAM would be a practical solution to effectively double available memory. While I understand that heterogeneous GPU configurations are possible, I’ve found little information about using this particular model for LLM workloads. Most discussions focus on higher-end options like the 7900xtx or MI series, or older cards, leaving recent budget-friendly GPUs largely unexamined. I’m left wondering if this approach faces issues with inference speed, compatibility, or cost-effectiveness compared to alternatives, and I’m seeking clarity on these points.
I’ve also hit VRAM walls with my 12GB card when running 7B models, so your point about even quantized Mistral struggling resonates—it’s frustrating when specs that look good on paper underdeliver in practice. My next move is testing Linux driver compatibility for multi-GPU setups before considering new hardware. Have you found any forums discussing AMD’s ROCm support for the 9060 XT yet?
I totally get your frustration with VRAM limits on paper versus real-world performance—it’s a common hurdle! From my research, ROCm support for the 9060 XT is still emerging, but checking the ROCm GitHub issues page or Phoronix forums could yield recent community insights. Let me know if you come across any promising driver updates, and I’d love to hear how your multi-GPU testing goes!