01What each method actually changes

The three methods are not points on a quality spectrum. They change different things, and that is the distinction to reason from.

Full fine-tuning updates every weight in the network. It can teach genuinely new capabilities, shift the model’s latent representations, and change behaviour that is baked deep into the layers. It can also destroy capabilities the base model had, which is why catastrophic forgetting is a full fine-tuning problem rather than a LoRA one.

LoRA freezes the base weights and inserts small trainable matrices alongside chosen layers. The base model is unchanged; you are learning a low-rank correction applied on top. That constrains what you can teach, and the constraint is usually a feature: you get the new behaviour without disturbing the old.

QLoRA is LoRA with the frozen base held in 4-bit rather than 16-bit. The adapters still train in higher precision. Quantising the base introduces some error, but because those weights are frozen the error does not compound through training the way quantisation error would in a full fine-tune.

02The memory difference is the first filter

Before considering quality, check what fits. Our companion guide on sizing VRAM works through the arithmetic; the summary that matters here:

On a 16 GB card, QLoRA on a 7B model is comfortable and LoRA is not. On a 40 GB card, LoRA on 7B is comfortable and full fine-tuning is impossible for anything above roughly 2B parameters. That single table eliminates most of the decision space before you have thought about quality at all.

03When full fine-tuning earns its cost

Full fine-tuning is the right choice in a narrow set of cases, and the narrowness is the point.

• A genuinely new domain the base model has not seen. Adapting a general model to a specialised notation, a low-resource language, or a scientific representation it never encountered can require moving the representations themselves.
• Small models. Under about 2B parameters the memory cost is manageable on a 40 GB card and the argument for adapters weakens.
• You need to remove behaviour, not add it. Suppressing something the base model does reliably is often beyond a low-rank correction.

Against that: it needs multi-GPU capacity at any interesting model size, it produces a full-size checkpoint per experiment rather than a small adapter, and it can degrade general capability. If you are unsure, the honest answer is that you probably do not need it yet.

04When LoRA is the right default

Most adaptation work is style, format, tone, domain vocabulary or task framing. All of that is well within reach of a low-rank correction, and LoRA is the default we would suggest starting from.

The practical advantages compound. Adapters are tens of megabytes, so you can keep dozens of experiments rather than choosing which checkpoints to delete. You can serve one base model with several adapters swapped per request. And because the base is frozen, a bad run is recoverable by discarding the adapter rather than reverting a whole model.

On a Professional plan, LoRA on a 7B model leaves substantial headroom for activations, which in turn lets you use a longer sequence length or a larger batch without fighting for memory.

05When to reach for QLoRA

Two situations, both about constraints rather than preference.

The model does not fit otherwise. QLoRA is what lets a 16 GB card fine-tune a 7B model at all, and what lets a 40 GB card reach into the 30B range. If LoRA does not fit, this is the lever before adding hardware.

You want more concurrent experiments. Because the footprint is roughly a quarter of LoRA, you can run several jobs on one card, or use the freed memory for a much longer context.

The cost is real but usually small: some quality loss from the quantised base, and slower steps because of dequantisation overhead during the forward pass. For most adaptation tasks the difference is smaller than the variance between two runs with different seeds — but measure it on your own evaluation rather than taking that on trust.

06Choosing rank and target modules

Once you have picked LoRA or QLoRA, two settings do most of the work.

Rank controls the capacity of the correction. Low ranks learn style and format cheaply. Higher ranks can absorb more genuinely new information but move toward full fine-tuning in both memory and overfitting risk. Start low, raise it only if your evaluation says the adapter has run out of capacity rather than out of data.

Target modules decide where the adapters go. Attention projections are the conventional choice and usually sufficient. Extending to the feed-forward layers increases capacity and cost. Changing this is a bigger lever than people expect, and worth a controlled comparison rather than a guess.

07A decision procedure

In order. Stop at the first line that applies.

1. Compute the fixed memory cost for full fine-tuning at your model size. If it exceeds your card, full fine-tuning is out.
2. If you need to change what the model fundamentally is, and the model is small enough to fit, full fine-tune.
3. If LoRA fits with room for your batch size and sequence length, use LoRA.
4. If LoRA does not fit, or you want several concurrent jobs, use QLoRA.
5. If QLoRA does not fit either, the honest answer is a bigger card or more than one.

If you would like the arithmetic checked against your actual model and dataset before committing to a plan, describe the workload: contact@vijaycloud.com.