Is AI infrastructure a bubble or a revolution? With planned investments like the $500 billion Star Gate Project and North American tech giants' combined capex reaching $690 billion in 2026, the AI computing power gold rush shows no signs of slowing. Nvidia's revenue surged from 186 billion yuan in early 2023 to 1.49 trillion yuan by early 2026 — a nearly 8x increase. Let's dissect the full supply chain: chips, PCBs, optical modules, switches, cooling, and power systems.
The Core Components of AI Computing Power
1. Computing Hardware: Nvidia DGX H100 AI Server
A standard Nvidia DGX H100 AI server (worth over 2 million RMB) is a marvel of engineering:
- 8 Nvidia H100 GPU chips, each surrounded by 6 SK Hynix HBM high-bandwidth memory chips using TSMC's top-tier packaging
- 4 Nvidia NVSwitch chips enabling zero-latency communication between GPUs
- 2 Intel Xeon CPUs, 2TB of system memory, and PCIe data lines for data scheduling
- High-speed network cards with optical modules for data transmission
- All components mounted on a high-performance PCB
# Key Components of Nvidia DGX H100
- GPUs: 8 x Nvidia H100
- Memory: 6 x SK Hynix HBM per GPU
- Switches: 4 x Nvidia NVSwitch
- CPUs: 2 x Intel Xeon
- System Memory: 2TB
- Network: High-speed NICs with optical modules
- Carrier: High-performance PCB2. Network Equipment: Switches and Optical Modules
To ensure efficient collaboration among hundreds of thousands of GPUs, network equipment is crucial:
- Leaf Switches (Cabinet-Level): Distribution centers collecting data from the same or adjacent cabinets, using multi-mode (light blue) and single-mode (yellow) optical fibers via optical modules.
- Spine Switches (Independent-Level): Aggregate hundreds of leaf switches, connecting countless cabinets into a massive computing cluster.
3. Cooling and Space: Liquid Cooling
Traditional air cooling limits server density. Liquid cooling, as seen in Elon Musk's XAI data center, solves this: it removes air-cooling systems, reducing server thickness by half. A single cabinet can now hold 8 servers (64 H100 chips), doubling density and cutting costs for cloud vendors.
The Evolution to NVL72: A Paradigm Shift
Nvidia's latest GB300 NVL72 cabinet (mass-produced May 2025) represents a radical redesign:
- Compute Trays: Each contains Nvidia Grace CPUs, Blackwell GPUs, DPUs (security and scheduling), and ConnectX network cards — all Nvidia-owned hardware.
- NVLink Spine Backplane: A 70-pound copper interconnection system connecting 72 GPUs into a single virtual super GPU, eliminating optical modules inside the cabinet.
- Switch Trays: 9 trays with NVLink switch chips enable seamless communication.
This design pushes Scale-Up (physical memory sharing) to 72 GPUs, reducing latency and increasing bandwidth to 130TB/s.
Supply Chain Reconstruction and Competitive Landscape
Nvidia's Dominance: It has evolved from a GPU supplier to a system-level infrastructure provider. Cloud vendors must adopt its entire ecosystem — networks, CPUs, GPUs, and cabinet standards. Secondary suppliers risk obsolescence if they can't keep pace.
Practical Insights for Investors and Builders
Identify "shovel sellers" in this ecosystem:
- Chips and Servers: Nvidia (GPUs, DPUs), Intel/AMD (CPUs), TSMC (packaging)
- Network Equipment: Cisco, Broadcom (switches), Huawei, ZTE (optical modules)
- Cooling and Power: Companies specializing in liquid cooling and high-power supply systems
AI infrastructure is not a bubble but a necessary investment for the next tech revolution. Understanding each component's role and dynamics helps you navigate this landscape and seize opportunities.
常见问题
Is the AI infrastructure boom a bubble like the dot-com era?
The comparison is natural but the dynamics differ. Dot-com was fueled by speculation on future profits; AI infrastructure is backed by real, measurable demand — large models with trillion-level parameters need massive compute for both training and inference. Training one model can require 100,000 GPUs running for 3 months. Revenue at companies across the supply chain (switches, optical modules, storage chips, PCBs) has all grown significantly. The risk isn't a bubble popping — it's overcapacity if model efficiency improvements reduce compute demand faster than new applications increase it.
Why is Nvidia so dominant in AI infrastructure?
Nvidia's CUDA ecosystem created a massive software moat — most AI frameworks (PyTorch, TensorFlow) are optimized for CUDA first. Their NVLink and NVSwitch technologies enable GPU-to-GPU communication that competitors can't match. With NVL72, they've moved from selling GPUs to selling entire cabinets where every component (CPU, GPU, DPU, NIC, switch) is Nvidia-designed. Cloud providers effectively must adopt the full Nvidia stack. Competitors like AMD and Intel are closing the hardware gap but remain years behind on the software ecosystem.
What does this mean for AI developers and startups?
Compute costs will likely continue falling per unit of performance, even as total spending rises. The NVL72 architecture reduces per-GPU costs through density improvements. For developers, the key takeaway is that AI model training and inference will get cheaper over time — plan your product roadmaps accordingly. For startups building AI infrastructure tools, the opportunity is in the gaps Nvidia doesn't cover: specialized inference hardware, edge computing, and software that makes heterogeneous compute (mixing GPU brands) practical.