What Is HBM? HBM3E vs HBM4 for AI Accelerators

High Bandwidth Memory (HBM) is stacked DRAM bonded over through-silicon vias and placed millimeters from an AI accelerator on a silicon interposer, delivering terabytes-per-second bandwidth that ordinary DIMMs or GDDR cannot reach. That is why no frontier AI GPU runs its working set on standard memory. The two generations that matter now are HBM3E — 12-high 36GB stacks at about 1.2 TB/s over a 1024-bit interface — and HBM4, which roughly doubles bandwidth to 2 TB/s or more via a 2048-bit interface and scales to 64GB per stack. The point most buyers miss: HBM is almost never sold as a discrete component. SK Hynix, Samsung, and Micron sell stacks directly to TSMC and to GPU and ASIC makers under long-term agreements tied to specific customer slots. The stack is bonded onto an interposer with the processor and leaves the packaging line as one integrated module — an NVIDIA H200, B200, or Vera Rubin, an AMD MI300X, a Google TPU. The module, not the stack, is the unit of commerce. As of 2026-08, the year’s HBM output is effectively sold out. This guide explains the technology, the HBM3E-versus-HBM4 differences, and what that means for sourcing.

Why HBM exists

AI accelerators are bandwidth-starved. A large model’s throughput is often gated by how fast parameters and activations move between memory and compute — the “memory wall.” HBM answers it with width instead of clock speed: stacking DRAM dies vertically and wiring them to the processor across a very wide bus on the interposer. A single HBM3E stack moves data as fast as a whole bank of conventional modules, at far lower energy per bit and board area. That is why the technology commands its price.

HBM3E vs HBM4

The jump from HBM3E to HBM4 is primarily about interface width and capacity. HBM3E uses a 1024-bit interface; a 12-high 36GB stack lands around 1.2 TB/s at roughly 9.2–9.6 Gbps per pin. HBM4 widens the interface to 2048 bits with 32 channels, so even at similar or modestly higher per-pin rates it clears 2 TB/s per stack — vendor parts are quoted from about 2.0 up to 3.3 TB/s. Capacity climbs too: HBM4 targets 36GB at 12-high and 48–64GB at 16-high. Stack height itself is a gating engineering problem, since taller stacks mean thinner dies (around 50µm at 12-high, 30µm at 16-high) and unforgiving TSV yield — one bad via fails the whole stack.

Comparison table (as of 2026-08)

Per-stack figures are analyst estimates, not purchasable SKUs; HBM is sold inside accelerators, not on trays.

Why HBM ships integrated, not discrete

There is no side door for distributors. Because the stack is bonded onto the interposer with the SoC during TSMC CoWoS packaging, it cannot be resold as a loose component after the fact — and before packaging it is committed to a named customer. Integration examples make the scale concrete: an H200 carries six HBM3E stacks for about 141GB and 4.8 TB/s; a B200 uses eight stacks for roughly 192GB; NVIDIA’s Vera Rubin generation moves to HBM4 with about 288GB across eight stacks and roughly 22 TB/s. For a buyer, this means the sellable product is the accelerator. If you need HBM capacity, you are really sourcing AI GPUs with the memory already on them.

Allocation and supply

HBM is the tightest link in the AI hardware chain. The entire 2026 output is sold out, customers reserve years ahead, and the constraints are physical as much as commercial: tall-stack yield, die thinning, thermal dissipation, and TSMC CoWoS interposer capacity all gate output — not raw DRAM wafer starts alone. One HBM bit consumes roughly 300% more wafer capacity than a DDR5 bit, so HBM demand is simultaneously crowding out conventional DRAM and driving that separate shortage.

Pricing reflects the squeeze. Analyst per-stack anchors sit near $300 for HBM3E 12-high 36GB and about $550 for HBM4 12-high 36GB — HBM4 roughly doubling the per-gigabyte cost. TrendForce projects HBM contract prices could rise 70–140% in 2027; buyers without long-term agreements face 30–50% spot premiums where supply exists at all. Vendor share as of 2026 runs roughly SK Hynix 54%, Samsung 28%, Micron 18%.

For a broker, the realistic role is allocation access and HBM-bundled compute, not tray-level stacks. We use HBM specs to qualify a requirement, then source the integrated module through allocation-controlled OEM channels at 26–52+ week lead times. Talk to our HBM sourcing desk about which accelerator generation fits your target and what allocation is realistically available.

FAQ

What is HBM? High Bandwidth Memory is stacked DRAM — 4 to 16 dies bonded over through-silicon vias — mounted millimeters from an AI accelerator on a silicon interposer. The wide interface delivers terabytes per second of bandwidth that ordinary DDR or GDDR cannot match, which is why frontier AI GPUs use it for their working set.

What is the difference between HBM3E and HBM4? HBM4 roughly doubles per-stack bandwidth to 2 TB/s or more via a 2048-bit interface (versus 1024-bit on HBM3E), and lifts capacity to 36–64GB per stack at 12- to 16-high. HBM3E 12-high 36GB delivers about 1.2 TB/s. HBM4 also costs roughly twice as much per gigabyte.

Can I buy HBM stacks directly? Effectively no. Makers sell stacks to TSMC and GPU/ASIC makers under long-term agreements tied to named customer slots; the stack leaves packaging already bonded inside a finished accelerator. There is no tray-level distribution, so the real buy path is HBM-bundled compute.

Is HBM available in 2026? The entire 2026 HBM output is effectively sold out and allocation-controlled, with shortages expected into 2027. Buyers without long-term contracts face 30–50% spot premiums where any unallocated supply exists at all (as of 2026-08).

How much of an AI GPU’s cost is HBM? Roughly 30–40% of manufacturing cost. On a B200-class part the eight HBM3E stacks alone represent several thousand dollars of bill-of-materials, which is why HBM pricing drives accelerator pricing.

Bottom line

HBM3E and HBM4 set the bandwidth and capacity ceiling for AI accelerators, but you buy them bundled on the GPU, under allocation, not as discrete stacks. See our HBM sourcing and AI GPUs pages, then request a quote for allocation guidance and indicative pricing.