Is an RTX 5090 good for AI compared with an A100 or H100?

Yes: for AI work that fits in 32 GB, such as quantized inference or a LoRA fine-tune of an 8B model, the NVIDIA GeForce RTX 5090 is usually the cheapest GPU per finished job. Nodus lists the RTX 5090 at $0.48/hr, the A100 80GB at $1.09/hr and the H100 SXM at $2.60/hr, and picks the GPU with the lowest expected cost to finish each job. Once the job needs more than 32 GB on one card, fast links between GPUs or FP64 math, the datacenter cards earn their price.

We have not benchmarked the 5090 against either card. Every comparison below comes from NVIDIA's published specs and the Nodus pricing page, with the arithmetic shown.

How does the RTX 5090 compare with an A100 and H100 on paper?

The 5090 is close on memory bandwidth and far behind on memory capacity and multi-GPU links. These are NVIDIA's own figures for the RTX 5090 (bandwidth from NVIDIA's GeForce compare page), the A100 80GB SXM and the H100 SXM:

Prices checked November 8, 2026 on the Nodus pricing page.

Tensor Core throughput is missing from the table on purpose. NVIDIA rates the 5090 at 3,352 "AI TOPS" without a matching dense BF16 figure on the same page, while the A100 is rated at 312 dense BF16 TFLOPS. Those numbers measure different things, so we do not divide them.

Is the RTX 5090 faster than an A100?

For memory-bound inference, no: on paper the A100 80GB moves bytes 1.14 times faster (2,039 / 1,792), and the H100 SXM 1.87 times faster (3,350 / 1,792). Generating tokens one at a time reads every weight once per token, so bandwidth sets the ceiling.

For a model whose weights take 16.4 GB, like Qwen3-8B at 16-bit, that ceiling is 1,792 / 16.4 = 109 tokens per second on a 5090. The A100 80GB tops out at 124 and the H100 at 204. Real servers land below these ceilings, and batching many requests shifts the limit toward compute, where we have no like-for-like spec to compare.

The price gap is much wider than the speed gap. The A100 80GB costs 2.27 times as much per hour as a 5090 on Nodus ($1.09 / $0.48), and the H100 costs 5.42 times as much ($2.60 / $0.48).

On bandwidth alone, neither datacenter card comes close to earning back its higher hourly rate.

What does a finished job cost on each GPU?

On a bandwidth-bound job that fits in 32 GB, the 5090 costs about half as much as the A100. It costs about a third as much as the H100. Take a batch generation job that takes 10 hours on one RTX 5090, and assume each card's speed scales with its bandwidth:

The H100 finishes 4.65 hours sooner and costs $9.11 more.

That gives a one-line decision rule: divide the bigger GPU's hourly rate by the 5090's. On Nodus that is 1.88x for an A100 40GB ($0.904/hr), 2.27x for an A100 80GB, 2.81x for an RTX PRO 6000 96GB ($1.35/hr) and 5.42x for an H100 SXM. The bigger GPU is cheaper per finished job only if it runs your job more than that many times faster. If you need the result sooner, pay for the H100 and know what the hours cost you.

Which AI workloads fit in 32 GB?

Most single-GPU work on models up to about 30B parameters fits, once quantized. Weights take roughly parameters times bytes per parameter, and you need room on top for the KV cache, activations and the CUDA runtime. We use the measured figures from our posts on LoRA vs QLoRA and VRAM for a 70B model:

• Qwen3-8B weights are 16.4 GB at 16-bit and 4.1 GB at 4-bit. Both serve on a 5090, and a QLoRA fine-tune of an 8B model has plenty of headroom.
• A plain LoRA fine-tune at 16-bit fits on a 5090 for an 8B model with modest sequence lengths. Long contexts or big batches push activations past 32 GB.
• Llama 3.3 70B takes 39.8 GB at 4-bit and 72.7 GB at FP8. Neither fits on one 5090.

A 70B model misses the 5090 even at 4-bit, so it goes to an 80 GB card or an RTX PRO 6000 96GB.

Nodus applies a similar rule when you name a model but no GPU. Its Jobs guide sets a memory floor of 24Gi for 7B and 8B models, 32Gi for 13B and 14B, and 80Gi for 70B and 72B. If 24 GB is enough, the RTX 4090 at $0.29/hr is cheaper still.

When do you need an A100 or H100 instead?

Pick a datacenter GPU when one of these limits applies:

• The model, optimizer state and activations need more than 32 GB on one card. A full fine-tune with Adam in mixed precision needs about 16 bytes per parameter, per the ZeRO paper, so roughly 112 GB for a 7B model.
• The job spans several GPUs that talk constantly. NVIDIA lists NVLink as "No" for the 5090, while the A100 and H100 move 600 GB/s and 900 GB/s between GPUs. Without NVLink, that traffic goes over PCIe Gen 5, which NVIDIA's H100 page rates at 128 GB/s.
• You need double precision. NVIDIA's 5090 page lists no FP64 figure; the H100 SXM is rated at 34 TFLOPS.
• You want to split one card into isolated slices. The A100 supports up to 7 Multi-Instance GPU partitions of 10 GB each.
• A compute-bound training run is large enough that H100 speed beats its price ratio. Our A100 vs H100 and H100 vs H200 posts work through that trade.

Is a 5090 cheaper on other providers too?

The hourly ratios differ by provider, so the speedup a bigger card needs changes too. Runpod's pricing page lists the RTX 5090 at $1.19/hr, the A100 SXM at $1.79/hr and the H100 SXM at $3.99/hr on Secure Cloud. There an A100 needs to be 1.50 times faster than a 5090 to cost less per job, and an H100 3.35 times faster. Both bars are lower than on Nodus, but bandwidth alone still does not clear them. On Runpod's Community Cloud the 5090 is $0.69/hr, the A100 SXM $1.39/hr and the H100 SXM $2.69/hr, which puts the bars at 2.01 and 3.90.

When is a 5090 on your own desk the better choice?

When the card would be busy most days or the data must stay on your machine. Owning also wins for offline work and the quick edit-run loop of local development. Our post on running an LLM locally vs in the cloud works out the RTX 5090 buy-or-pay-by-the-hour break-even.

How do you run a job on an RTX 5090 on Nodus?

Name the GPU, or name the memory you need and let Nodus choose. The Jobs guide says to name a GPU the way you would say it, so RTX 4090 and rtx-4090 mean the same card. A name Nodus does not offer is refused before anything runs. Check availability and rates first:

$ pip install nodus-compute $ nodus login $ nodus get gpus --gpu rtx-5090 -o wide 

Then run your script with a spending cap and a time limit:

$ nodus run --name qlora-8b --gpu rtx-5090 --image nodus/pytorch --max-cost 5 --timeout 6h -- python train.py 

If you do not care which card runs it, ask by memory instead. The GPU guide says minMemory with no type accepts any accelerator with at least that much memory per GPU, and the scheduler picks by expected cost to finish:

apiVersion: nodus.dev/v1 kind: Job metadata: name: qlora-8b spec: image: nodus/pytorch command: [python, train.py] resources: gpu: {minMemory: 24Gi} timeout: 6h maxCostUSD: "5.00" 

nodus apply -f job.yaml --dry-run=server -o estimate shows which offering Nodus would use and the expected cost before anything starts.

FAQ

Is the RTX 5090 good for training LLMs from scratch?

Only small ones. Pretraining anything beyond a few billion parameters needs many GPUs exchanging gradients, and the 5090 has no NVLink. For multi-GPU training an A100 or H100 node is the practical choice.

Can an RTX 5090 run a 70B model?

Not on one card. Llama 3.3 70B needs 39.8 GB at 4-bit before the KV cache. Two 5090s can split it over PCIe, but one RTX PRO 6000 96GB, listed at $1.35/hr on the Nodus pricing page, holds it on a single GPU.

Is the RTX 5090 better than an RTX 4090 for AI?

It has 32 GB instead of 24 GB, which matches the floor Nodus sets for 13B and 14B models and leaves room for longer contexts. If your job fits in 24 GB, the 4090 is listed at $0.29/hr on the Nodus pricing page against $0.48/hr for the 5090.

Does the 5090's 3,352 AI TOPS mean it beats an H100 at training?

No. NVIDIA ties the 5090's AI performance to FP4, while the H100's training numbers are quoted per format such as BF16 and FP8. Compare like-for-like formats, or run a short probe of your own script on both cards.

Originally published on the Nodus blog.