How to rent a GPU cluster: a buyer's checklist

By the RentServers team at Wells Software Consulting LLC · Updated October 16, 2026

1. Size the job before you size the cluster

Start from memory, not GPU count. For training with mixed-precision Adam, budget about 16 bytes per model parameter for weights, gradients and optimizer state, plus activations. A 70B model needs over 1.1 TB, which means at least 16 80 GB GPUs once you leave headroom. For inference, budget the weights at the precision you'll serve (2 bytes per parameter for BF16, 1 for FP8) plus the KV cache for your context length and batch size.

Then estimate time. If you know your model's training FLOPs and expect around 35–50% utilization, you can turn a deadline into a GPU-hours number and compare on-demand against a reservation.

2. Decide on-demand or reserved

On-demand suits experiments and bursts. Reservations suit anything running more than a few weeks. Across the market, reserved medians run below on-demand (H100: $3.34 vs $3.74 per GPU-hour on our price page), and large multi-month commitments usually go lower.

3. Match the network to the workload

Multi-node training needs a high-bandwidth, low-latency fabric, typically 400 Gb/s InfiniBand or RoCE per GPU. Single-node fine-tuning and most inference don't, and Ethernet-only nodes are cheaper. Ask for the fabric topology in writing.

4. Compare quotes line by line

• Form factor: SXM or PCIe, and the exact memory size.
• CPU, RAM and local NVMe per node.
• Shared storage: capacity, throughput and price per TB.
• Egress: some providers charge, some don't.
• Support and SLA: response times, and credits for downtime.
• Software image: drivers, CUDA, and whether Kubernetes or Slurm is managed.

5. Read the contract for these terms

• Delivery date and what happens if it slips.
• Failed-node replacement time, and whether you're billed while a node is down.
• Payment terms: monthly in arrears, prepayment, or a deposit.
• Early termination and whether you can resell or assign capacity.
• Data deletion at the end of the term.

6. Run an acceptance test

Before the clock starts on a reservation, run NCCL all-reduce tests across nodes and a short version of your real job. Confirm throughput matches what was promised.