GPUs for LLM inference, sized and priced by the hour

LLM inference is the workload this whole GPU market is being rebuilt around, and it prices differently from training: fit is VRAM, economics are batching, and the market split is token-priced APIs versus GPU-hours. 21 providers serve inference-capable capacity here; this page is the fit-and-economics map for readers who already know what a transformer is. The sections run in decision order - fit, pricing model, verification, traps - because that is the order in which a deployment actually gets decided.

The VRAM fit table

The arithmetic that sorts the market [5]:

• 7B-class at FP16 - ~14 GB weights plus KV cache: fits commodity cards (RTX 4090's 24 GB class) with room for batching.
• 13-30B-class - 26-60 GB: A100 40/80 GB territory, or quantized fits on smaller cards.
• 70B-class - ~140 GB unquantized: 80 GB cards with quantization, or multi-GPU with interconnect the h100 hub prices.
• Quantization changes the tier - INT8/INT4 halve or quarter the footprint at some quality cost; the trade is model-specific and testable on one card before renting a fleet.

The GPU workload tool computes the fit and picks a card; the feed prices it live - the current spread runs from $0.36/hr commodity hours to $2.56/hr datacenter ones.

Token economics versus GPU-hours

The market's central trade [1][2][3]:

• Per-token APIs price convenience - no infra, no cold starts, no interruption. The margin is the price of never thinking about GPUs.
• Dedicated GPU-hours price control - cheaper at steady batch loads, expensive at bursty ones, and interruption-tolerant only if your serving layer is stateless.
• The crossover is your utilization: above roughly 40-60% sustained utilization, dedicated capacity usually wins; below it, per-token pricing is cheaper than idle GPUs.

The honest method: replay a week of real inference traffic against both models and compare finished-request cost. The gpu-prices page supplies the GPU side; the compute hub supplies the interruption discipline.

Live from our database - caveats and sources in each linked review.

Verification, inference edition

Inference is the workload decentralized markets verify best, because outputs are sampleable [2][3]:

• Checkpoint sampling - the network re-runs slices of your inference traffic on other nodes; disagreement exposes GPU swapping.
• TEE attestation - hardware-rooted proof of what ran; the strongest answer, still rare.
• Bonded reputation - hosts stake against verification failures; common on the larger networks.

Match verification to exposure: chat workloads tolerate sampled verification; anything with compliance obligations should insist on attestation or dedicated infrastructure.

Egress and state: the inference-specific traps

The compute hub covers interruption and egress generally; inference has two specific versions of both problems:

• Model weights are the egress bill. Deploying to multiple platforms means moving multi-GB weights repeatedly - ask the transfer price before multi-cloud, per the storage hub exit-cost rules.
• Serving state is the interruption surface. KV caches and session state do not survive preemption; stateless serving with external KV storage is the design that survives marketplace tiers [2].

The deployment checklist

1. Fit the model - the workload tool with your real context length, not a demo prompt [5].
2. Price the crossover - replay traffic against per-token and GPU-hour costs before committing either way [1][2].
3. Design for preemption - stateless serving, external weights, health-checked autoscaling. The rtx-4090 hub covers the commodity tier where interruption is the discount.
4. Verify at the tier you will actually run - attestation on the cheapest tier is the question that sorts the compute population honestly.
5. Watch the feed weekly - the gpu-prices page tracks the spread; $2.56/hr median versus the floor is the interruption discount made visible.
6. Budget the crossover re-check monthly. The feed moves and your traffic moves - the crossover from the figure above is a monthly five-minute replay, not a one-time decision.

21 providers serve this population; 86% of all rows carry facts we verified with dates. Inference is the workload most tolerant of market economics - which is why it prices cheapest here, and why the rent-gpu guide is the practical starting point for a first deployment. The honest closing rule: the crossover, the fit, and the verification tier are three separate decisions - and replaying the traffic once at each decision point is what keeps an inference deployment priced on evidence instead of on the cheapest GPU-hour in the feed.