RentServersBy the RentServers team at Wells Software Consulting LLC · Updated
A100 specifications
| Architecture | Ampere |
|---|---|
| Memory | 40 GB or 80 GB HBM2e |
| Memory bandwidth | ~2.0 TB/s (80 GB SXM) |
| BF16 Tensor (dense) | ~312 TFLOPS |
| FP8 | Not supported |
| NVLink | 600 GB/s |
| Max power | 400 W (SXM) |
Figures for the SXM/HGX version unless noted, from NVIDIA. Dense Tensor Core throughput; NVIDIA's headline numbers with sparsity are double.
40 GB and 80 GB are not interchangeable
Many of the cheapest listings are 40 GB cards. If your model or batch size needs 80 GB, filter for it explicitly, since the 80 GB SXM version is the one that's comparable to an H100 setup.
Where the A100 still wins
- LoRA and QLoRA fine-tuning of 7B–34B models.
- Batch and offline inference where latency doesn't matter.
- Teaching, research and CI workloads that run for days on a budget.
Without FP8 support, it falls well behind the H100 on modern LLM training throughput. Run the numbers per job, not per hour.
A100 questions
How much does it cost to rent an A100?
GetDeploying's tracker (October 16, 2026) shows a median of $1.79 per GPU-hour on demand and $1.60 reserved across A100 variants. In-stock 80 GB on-demand offers started at $1.09.