ByteDance MegaScale Ampere cluster

ByteDance’s engineers describe this production cluster and its use in large language model training.

ByteDance MegaScale Ampere cluster is an operational GPU cluster run by ByteDance. It has 12,288 NVIDIA Ampere GPUs installed.

Big tech

Verified 11 Nov 2026

Operator page Compare with others Request GPU capacity

Facts

Source per value

Owner / operator Organization that owns or operates the GPUs. Hosting and facility roles are listed under Partners.: ByteDance 1
GPU maker NVIDIA, AMD or Intel, taken from the GPU model.: NVIDIA 1
GPU model Accelerator model or family as disclosed. A GH200 is one GPU on a superchip; an MI300A is an APU that contains a GPU.: NVIDIA Ampere 1
GPUs installed Physical GPUs in the cluster, from the operator or a deterministic sum of disclosed quantities (nodes × GPUs per node). Planned counts, H100-equivalents and unaudited press totals are never counted here.: 12,288 1
GPU count notes What the count covers (one partition, one site, a design limit), qualifiers and competing figures.: Documented 175B-model experiment scale; not total ByteDance fleet. 1
Location City, region and country where the cluster sits. Empty when the operator does not name the site.: Not published
Live power MW Power in service, in megawatts of IT load where the source says so. Company figures are marked in the notes; Epoch AI satellite estimates are labelled as estimates.: Not published
Status operational = running and named as such by the operator or a primary source; partly operational = a phase is live and more is coming; under construction = building underway; announced = disclosed or contracted with no confirmed go-live.: operational 1
Status notes The source’s wording on status, with the date.: Production cluster as of September 2023. 1
Interconnect Cluster network fabric.: Ethernet network with Broadcom Tomahawk 4 switches 1

Sources

arxiv.org accessed 2026-11-11

Related clusters

All 87 clusters

Lists

Need GPU capacity at this scale? Request quotes from GPU cloud providers

Tell us what you need once. We send it to matching GPU cloud providers and they reply to you directly.