• 0 Posts
  • 4 Comments
Joined 3 years ago
cake
Cake day: December 15th, 2023

help-circle


  • I don’t know how YAMS is set up, but I keep my arr stack and my torrent docker setup completely separate, on separate computers. I use the qbittorrent PIA docker image (since my VPN is PIA) on a separate, dedicated mini PC so it only has Internet if it’s connected to the VPN. I just use a static IP for both PCs on my internal network and point the arr stack at the torrent client to handle everything.

    Not sure if separating them is the right solution for you, but if all else fails, it might be with a try. I prefer it that way so I can literally just unplug or turn off the torrent box if I need to without disrupting my actual media server.


  • I can second Ornith! I have an AMD 6650xt 8gb (it’s rdna2 so I can’t take advantage of rocm and have to use vulkan).

    Ornith is absolutely the best code assist model I’ve found in the small local space so far. It provides usable output and is really good with tool calling.

    The key for getting a model running well locally is not just the model, but figuring out exactly what arguments you should use with llamacpp to run it well on your machine. Since I’m on a dedicated GPU, I need to make sure the full context and kv cache fit fully in VRAM and morning gets offloaded. Offloading to CPU and RAM is the biggest hindrance to token throughput that I’ve found.