lilbee / will it fit your GPU
Will this model fit your GPU? See before you download. lilbee reads the model file and your hardware, then tells you whether it fits, and splits big models across GPUs.
No more downloading a 40 GB model only to watch it fail to load. lilbee estimates VRAM up front from the real GGUF file and the devices it finds on your machine.
the problem with guesswork
Picking a local model is usually trial and error: download something, try to load it, watch it run out of memory, pick a smaller quant, try again. lilbee does the arithmetic before the download. It reads the GGUF headers, checks the free and total VRAM on each device, and tells you whether the model and its context window will fit.
a rough VRAM guide
A starting point for a common 4-bit quant. Your exact numbers depend on the quant, the context length, and the runtime, which is why lilbee estimates from the actual file rather than a table.
7B
about 5 GB at a 4-bit quant. Comfortable on an 8 GB card.
13B
about 9 GB at a 4-bit quant. Fits a 12 GB card with room for context.
34B
about 22 GB at a 4-bit quant. Wants a 24 GB card, or two smaller ones.
70B
about 42 GB at a 4-bit quant. Needs multiple GPUs, where the tensor split matters.
multiple GPUs, sized to each card
When a model is too big for one card, lilbee spreads it across the GPUs you have, with a tensor split sized to each device's free VRAM rather than an even cut that overflows the smallest card. It runs on Metal, Vulkan, and CUDA, and can offload part of a model to system RAM when you want to stretch a tight setup. This is part of lilbee's model manager.
questions
How much VRAM do I need to run a model?
It depends on the parameter count and the quantization. As a rough guide at a 4-bit quant: 7B is around 5 GB, 13B around 9 GB, 34B around 22 GB, and 70B around 42 GB, plus a little for the context window. lilbee reads the actual file to estimate your case.
How does lilbee know if a model fits?
It reads the GGUF file's headers with a gguf parser and probes your devices for free and total VRAM, then estimates whether the model plus its context will fit. If it will not, it tells you before you download.
Can it use multiple GPUs?
Yes. lilbee places a large model across multiple GPUs with a tensor split sized to each device's VRAM, so a model that does not fit on one card can still run.
What about Apple Silicon and Metal?
On Apple Silicon the GPU shares system memory, so the limit is your unified memory rather than a separate VRAM pool. lilbee runs on Metal and sizes the model to what is available.
What if a model is too big?
lilbee flags it before download. You can pick a smaller quant, offload part of the model to system RAM, or spread it across more GPUs.