LoRA vs QLoRA: which should you use to fine-tune an LLM?

Use LoRA when the 16-bit model fits on a GPU you can afford, and QLoRA when it does not. On Nodus, both run on the nodus/sft training runtime with one parameter, method: LoRA or method: QLoRA, and the platform picks a GPU that fits. Up to about 8B parameters the choice moves your bill by cents; at 70B it is the difference between a GPU listed at $1.09/hr and one at $5.50/hr.

What is the difference between LoRA and QLoRA?

Both train the same small adapters; they differ only in how the frozen base model is stored. LoRA freezes the pretrained weights and trains low-rank matrices added to each layer. The LoRA paper reports 10,000 times fewer trainable parameters and 3 times less GPU memory than full fine-tuning of GPT-3 175B with Adam. Quality was on par or better on the models it tested.

QLoRA keeps those adapters but stores the frozen base in a 4-bit format called NormalFloat (NF4). Gradients flow through the quantized base into the 16-bit adapters. The QLoRA paper also quantizes the scaling constants a second time and pages optimizer state to absorb memory spikes. With that setup it fine-tuned a 65B model on a single 48 GB GPU.

In code, the difference is one quantization config. This is the 4-bit setup from the Hugging Face PEFT quantization guide, passed to the model loader before you attach a normal LoRA config:

import torch from transformers import BitsAndBytesConfig bnb_config = BitsAndBytesConfig( load_in_4bit=True, bnb_4bit_quant_type="nf4", bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.bfloat16, ) 

Leave it out and you are running plain LoRA on a 16-bit base.

How much memory does QLoRA save?

QLoRA cuts the memory of the frozen base weights by about three quarters. A 16-bit weight takes 2 bytes and a 4-bit weight takes half a byte, so the math is short:

Parameter counts come from the Qwen3-4B and Qwen3-8B model cards and our 70B VRAM breakdown. This is weights-only parameter math. Real 4-bit checkpoints come out a few GB higher because the embedding and output layers usually stay at 16-bit.

The 4x cut saves 6 to 12 GB at 4B and 8B, but over 100 GB at 70B.

The adapters, their optimizer state and the activations come on top, and they are the same size for both methods. For Llama 3.3 70B with one 4,096-token sequence, our full estimate is about 50 GB for QLoRA and about 152 GB for LoRA. That gap is what moves a 70B fine-tune from several GPUs, or one B200, to a single 80 GB card.

Does QLoRA hurt model quality?

The QLoRA paper says its 4-bit setup preserved full 16-bit fine-tuning task performance, and it trained more than 1,000 models to check. That is strong evidence for the models and datasets in the paper. It does not prove the same for your data, so test it.

The cheap way to test is a held-out set you never train or tune on. On Nodus, set data.validationFraction and parameters.validation in a TrainingJob and the run keeps the checkpoint with the lowest validation loss. By default it fails evaluation if that checkpoint does not beat the starting model, and the outputs stay downloadable either way.

Validation loss picks checkpoints; it does not prove task quality. The Nodus training guide says to compare the chosen recipe against the base model on the same held-out tasks, including formatting and failure rates. If you are torn between LoRA and QLoRA, run both through that same comparison.

Is QLoRA slower than LoRA?

Usually somewhat, because the base is stored in 4-bit but computed in 16-bit. The bnb_4bit_compute_dtype=torch.bfloat16 line above means every 4-bit block is converted back to bfloat16 for its matrix multiply, in the forward and the backward pass. Plain LoRA skips that work.

How much it costs per step depends on the model, sequence length and GPU. We have not benchmarked it for this post, so we will not quote a percentage. Run 50 to 100 steps of each on your own data and compare seconds per step before you commit to a long run.

What matters for the bill is cost per finished job, not seconds per step. That leads to a simple rule.

When does QLoRA actually save money?

QLoRA saves money when it lets you drop to a cheaper GPU setup and its slowdown does not eat the price gap. Divide the LoRA setup's hourly price by the QLoRA setup's. QLoRA is cheaper as long as its job takes fewer than that many times the LoRA job's hours.

At 70B the gap is large. LoRA needs about 152 GB, so its smallest single-GPU option is one B200, listed at $5.50/hr on the Nodus pricing page. QLoRA fits on one A100 80 GB at $1.09/hr or one H100 at $2.60/hr.

• QLoRA on an A100 80 GB is cheaper unless it takes more than 5.50 / 1.09 = 5.05 times as long.
• QLoRA on an H100 is cheaper unless it takes more than 5.50 / 2.60 = 2.12 times as long.
• If LoRA on the B200 finishes in 10 hours ($55.00), QLoRA breaks even at 50.46 hours on the A100 and 21.15 hours on the H100.

At 70B, QLoRA can run 5 times longer on an A100 and still cost less than LoRA on a B200.

At 4B the gap nearly disappears. The nodus/sft runtime's presets run Qwen3-4B LoRA on one L40S, RTX A6000 or A100 80 GB, and Qwen3-4B QLoRA on one RTX 4090, L40S or RTX A6000. The runtime also publishes the seconds per step its dry-run estimate uses for each GPU. Here is a 2,000-step run priced with those figures:

Prices checked November 3, 2026 on the Nodus pricing page; step times from the nodus/sft runtime reference.

For a 4B model, the GPU that only QLoRA can use saves 5 cents over the cheapest LoRA option.

Those step times are one figure per GPU, not separate LoRA and QLoRA measurements, so treat the table as an estimate. The point survives the caveat: for small models, the cheapest LoRA option and the cheapest QLoRA option differ by cents. Pick on quality and simplicity there, and save QLoRA for the models that do not fit.

When is plain LoRA the better choice?

Plain LoRA is the better choice whenever the 16-bit model already fits. It avoids the dequantization work, so each step does less, and there is one less variable to evaluate. Nodus's own presets follow this line. Qwen3-4B gets LoRA on a card with 48 GB or more, while Qwen3-8B switches to QLoRA to stay on that class of card.

LoRA also wins when the deadline matters more than the hourly price. A faster step on a bigger GPU finishes sooner, and the break-even rule above tells you what that speed costs.

It is also the choice when you will serve the merged 16-bit model. A QLoRA adapter learned its corrections against a 4-bit copy of the base, not the exact weights you will serve. Evaluate the merged model, not just the adapter during training.

Finally, QLoRA is GPU only on Nodus. If you want a quick CPU smoke test of your data and config, the runtime's tiny test model runs LoRA on CPU.

How do you run LoRA or QLoRA on Nodus?

Change one parameter. This TrainingJob is the LoRA example from the training guide with Qwen3-8B and method: QLoRA, which matches the runtime preset of one L40S, RTX A6000 or A100 80 GB:

apiVersion: nodus.dev/v1beta1 kind: TrainingJob metadata: name: support-qlora spec: runtime: nodus/sft model: uri: hf://Qwen/Qwen3-8B@b968826d9c46dd6066d109eabc6255188de91218 data: volume: support-chats path: chats.jsonl format: JSONL columns: {prompt: question, completion: answer} parameters: method: QLoRA steps: 2000 lora: {r: 16, alpha: 32} maxCostUSD: "20.00" 

Price it before anything runs, then start it and watch it:

$ pip install nodus-compute $ nodus login $ nodus apply -f sft.yaml --dry-run -o yaml $ nodus apply -f sft.yaml $ nodus get tj -w 

The dry run shows the GPUs it picked and the estimated cost to completion. maxCostUSD caps the spend. The runtime saves a recovery checkpoint every 200 steps by default, so a run whose capacity is reclaimed resumes from the last one instead of starting over. For plain LoRA, set method: LoRA or leave it out, since LoRA is the default.

For the full 70B memory picture, read how much VRAM a 70B model needs. For a single-GPU recipe with another trainer, see fine-tuning with Unsloth on a cloud GPU, and for H100 cost math, the cheapest way to fine-tune Llama on an H100.

FAQ

Can you fine-tune a 70B model on one GPU?

Yes, with QLoRA on an 80 GB GPU such as an A100 80 GB or H100. A 48 GB card such as the L40S is tight: Llama 3.3 70B needs about 43 GB before activations, which leaves room for short sequences only.

What LoRA rank should I start with?

Start with the nodus/sft defaults: rank 16, alpha 32, dropout 0.05, applied to all linear layers. Raise the rank only if validation loss stalls well above where you need it.

Is QLoRA the same as 8-bit LoRA?

No. QLoRA specifically means a 4-bit NF4 base with LoRA adapters. The bitsandbytes library behind it can also load a model in 8-bit, which saves less memory than 4-bit.

What is the cheapest GPU for QLoRA on Nodus?

For Qwen3-4B, the runtime preset includes the RTX 4090, listed at $0.29/hr on the Nodus pricing page. For a 70B model, the cheapest single GPU with enough memory is the A100 80 GB at $1.09/hr.

Originally published on the Nodus blog.