RentServersBy the RentServers team at Wells Software Consulting LLC · Updated
H200 specifications
| Architecture | Hopper |
|---|---|
| Memory | 141 GB HBM3e |
| Memory bandwidth | 4.8 TB/s |
| FP8 Tensor (dense) | ~1,979 TFLOPS (same as H100) |
| BF16 Tensor (dense) | ~989 TFLOPS |
| NVLink | 900 GB/s |
| Max power | Up to 700 W (SXM) |
Figures for the SXM/HGX version unless noted, from NVIDIA. Dense Tensor Core throughput; NVIDIA's headline numbers with sparsity are double.
Same compute, more memory
Raw FLOPS are identical to the H100 SXM. The difference is 76% more memory and about 43% more bandwidth. Compute-bound training jobs that already fit on H100s won't run much faster.
Workloads that are limited by memory capacity or bandwidth do improve: serving 70B+ models, long context windows with large KV caches, and big batch inference.
What fits on H200s
- 70B model in FP8 for inference: about 70 GB of weights leaves around 70 GB for KV cache on one GPU. That's the main reason teams pick the H200.
- 70B in BF16: about 140 GB, which is too tight for one card once you add KV cache. Use 2.
- Training: fewer GPUs are needed to hold the model, but for compute-bound jobs, H100s at a lower hourly price often finish cheaper.
Pricing
H200 supply is thinner than H100. GetDeploying's in-stock on-demand floor ($3.85) is more than double the H100's, but the reserved medians are close ($3.84 vs $3.34). That makes reserved H200 capacity attractive for inference fleets.
When to pick something else
If your job is compute-bound and fits in 80 GB, rent H100s. If you need FP4 or more than about 1.5x Hopper throughput, compare B200 quotes.
H200 questions
How much does it cost to rent an H200?
Median reserved pricing is $3.84 per GPU-hour and median on-demand is $4.94 on GetDeploying's tracker (October 16, 2026). In-stock on-demand offers started at $3.85.
Is the H200 faster than the H100?
For compute-bound work, not meaningfully. For memory-bound inference it can serve more tokens per GPU because the model and KV cache fit on fewer cards.