Shatur@discuss.tchncs.de to LocalLLaMA@sh.itjust.worksEnglish · 9 days agoQwalmost there!discuss.tchncs.deimagemessage-square12linkfedilinkarrow-up148arrow-down115file-text
arrow-up133arrow-down1imageQwalmost there!discuss.tchncs.deShatur@discuss.tchncs.de to LocalLLaMA@sh.itjust.worksEnglish · 9 days agomessage-square12linkfedilinkfile-text
minus-squareAlex@lemmy.mllinkfedilinkEnglisharrow-up5·9 days ago24gb, luxury 😉 I have to make do with 8gb if vram but the previous 3.5-9b-artimus worked ok. I wonder how much of the 27B is used for the multi model support?
minus-squareShatur@discuss.tchncs.deOPlinkfedilinkEnglisharrow-up2·9 days agoI offload mmproj to the CPU to keep as much model weights in VRAM as possible 🙂
24gb, luxury 😉 I have to make do with 8gb if vram but the previous 3.5-9b-artimus worked ok. I wonder how much of the 27B is used for the multi model support?
I offload mmproj to the CPU to keep as much model weights in VRAM as possible 🙂