No products in the cart.
NVLink vs PCIe for AI: When the Interconnect Matters and When It Does Not
NVLink delivers 7 to 14 times the bandwidth of PCIe Gen 5, but most AI workloads do not need it. The interconnect matters specifically when activations or gradients move between GPUs on a tight latency budget, which narrows down to a defined set of workloads. This guide identifies which side of that line each workload sits on.
The bandwidth numbers
| Interconnect | Bandwidth (bidirectional) | Where it appears |
|---|---|---|
| PCIe Gen 4 x16 | 64 GB/s | Older platforms (Threadripper PRO 5000WX, EPYC 7003) |
| PCIe Gen 5 x16 | 128 GB/s | Threadripper PRO 9000WX, EPYC 9005, Xeon W-3500 |
| NVLink 4 (Hopper) | 900 GB/s | H100 SXM5, H200 SXM |
| NVLink 5 (Blackwell) | 1.8 TB/s | B200 SXM, GB200 |
| NVLink 5 + NVSwitch (rack) | 130 TB/s aggregate | GB200 NVL72 (72-GPU rack) |
The headline jump is from PCIe Gen 5 to NVLink: 7x with Hopper, 14x with Blackwell. That gap is real, but its impact depends entirely on whether the workload generates cross-GPU traffic in the first place.
What the interconnect actually carries
Inter-GPU traffic in AI workloads comes from a few specific sources:
- Tensor parallelism (forward and backward). Activations transfer between GPUs on every layer. This is the dominant cost in large-model tensor-parallel inference and training.
- All-reduce for gradient sync. In data-parallel training, gradients are averaged across GPUs at the end of each step. This is bandwidth-intensive on large models.
- Pipeline parallelism (activations between stages). Less frequent than tensor parallelism but still measurable on long sequences.
- KV cache movement in disaggregated serving. Some production setups split prefill and decode across different GPUs and transfer the KV cache between them.
If none of these are present in the workload, the interconnect bandwidth is irrelevant. Examples: single-GPU inference, single-GPU fine-tuning, data-parallel inference where each GPU runs a complete model copy on independent requests.
Where NVLink matters most
Tensor-parallel inference of large models
When a model is split across 4 or 8 GPUs (Llama 3.1 405B FP16, future trillion-parameter models), every forward pass shuffles activations across the GPU set. At NVLink 4 (900 GB/s), the transfer takes microseconds. At PCIe Gen 5 (128 GB/s), the same transfer takes roughly 7x longer and shows up as visible step-time overhead. For production serving where every millisecond of latency matters, NVLink is the difference between viable and not.
Full fine-tuning with gradient sync
An all-reduce across 8 H100 SXM5 GPUs via NVSwitch is dominated by bandwidth, not latency, and the per-collective overhead is negligible compared to PCIe. The same all-reduce across 8 PCIe-connected GPUs can consume 30 to 40% of a training step on large models. Over an entire fine-tuning run, that is a meaningful difference in wall-clock time and electricity cost.
Pre-training and large-scale training
Pre-training a foundation model from scratch is the most communication-intensive AI workload. NVLink and NVSwitch are not optional for this; PCIe-only training of frontier-scale models is not practical.
Trillion-parameter serving
Models in the 1T+ parameter range require multi-node serving with both intra-node NVLink and inter-node InfiniBand or equivalent. The GB200 NVL72 rack with 130 TB/s of aggregate NVLink bandwidth across 72 GPUs is designed specifically for this class of workload.
Where PCIe Gen 5 is the right choice
Single-GPU work
If the workload runs on one GPU, the interconnect to other GPUs is not used. A single RTX PRO 6000 Blackwell at PCIe Gen 5 x16 runs 70B at Q4, 32-34B at FP16, QLoRA fine-tuning of 70B, and many other production workloads with no NVLink anywhere in sight.
Data-parallel serving
Two RTX PRO 6000 Blackwell cards in a workstation running a 70B inference workload, with each card serving a separate stream of users, do not communicate beyond initial model loading. PCIe Gen 5 x16 per GPU is more than sufficient.
LoRA and QLoRA fine-tuning
Gradient traffic for LoRA is small because only adapter weights are updated (a few hundred MB versus the full model). All-reduce overhead on PCIe Gen 5 is modest. LoRA and QLoRA on multi-GPU PCIe workstations is a well-supported, production-ready configuration.
Tensor-parallel inference of mid-size models
A 70B model split across two PCIe Gen 5 GPUs sees more communication overhead than over NVLink, but the absolute volume of traffic is modest enough that the user-facing latency impact is small for typical context lengths.
Workload mapping
| Workload | Interconnect | Form factor |
|---|---|---|
| Single-user 7B-34B inference | None (single GPU) or PCIe | Workstation |
| Single-user 70B Q4 inference | None or PCIe Gen 5 | Workstation |
| Multi-user 13B-34B serving | PCIe Gen 5 (data parallel) | Workstation or 1U server |
| Multi-user 70B Q8/FP16 serving | PCIe Gen 5 or NVLink | Workstation or 4U server |
| LoRA / QLoRA fine-tuning 7B-70B | PCIe Gen 5 | Workstation |
| Full fine-tuning 7B-13B | PCIe Gen 5 | Workstation or 2U server |
| Full fine-tuning 70B | NVLink | 4-8x H100/H200 SXM server |
| Tensor-parallel 405B FP16 serving | NVLink (strongly preferred) | 4U server |
| Pre-training foundation models | NVLink + NVSwitch | Multi-node server cluster |
| Trillion-parameter serving | NVLink 5 + InfiniBand | GB200 NVL72 or equivalent |
SXM versus PCIe form factor
NVLink availability is determined by the GPU form factor. SXM (NVIDIA's proprietary socket and baseboard) GPUs include NVLink; PCIe-form-factor GPUs do not (with rare exceptions on older generations that supported bridge connectors). The decision is therefore not just "do I want NVLink" but "do I want SXM or PCIe":
| Form factor | GPUs | Chassis | Power per GPU |
|---|---|---|---|
| PCIe (workstation card) | RTX PRO 6000 Blackwell, RTX 6000 Ada, L40S, RTX PRO 4000 Blackwell | Tower workstation or 2U/4U PCIe server | 300-600W |
| SXM (datacenter) | H100 SXM5, H200 SXM, B200 SXM | 4U or 8U server with SXM baseboard | 700-1000W |
PCIe cards are flexible: same card runs in a workstation tower, a 1U server, or a 4U PCIe-GPU server. SXM cards are not: they require the SXM baseboard and the server chassis built around it. The form factor decision usually maps cleanly to the workload: development and small-team work goes PCIe workstation, production training and serving goes SXM server.
NVSwitch and rack-scale fabrics
NVLink alone connects GPUs in a point-to-point or limited topology. NVSwitch chips provide non-blocking all-to-all NVLink connectivity across larger GPU sets. In an 8-GPU H100 or H200 SXM server, NVSwitch enables every GPU to communicate with every other GPU at full NVLink bandwidth simultaneously.
The 4th-generation NVSwitch on Blackwell is used in the GB200 NVL72 rack, which connects 72 B200 GPUs with 130 TB/s of aggregate all-to-all bandwidth. Each GPU still has 1.8 TB/s of NVLink 5 bandwidth, and NVSwitch provides the fabric to use it simultaneously with every other GPU in the rack. This is rack-scale, not workstation-scale, and is the target architecture for trillion-parameter serving.
The buying decision in one paragraph
Start from the workload. If it is single-user or small-team inference, LoRA or QLoRA fine-tuning, or any case where the model fits in one or two GPUs, choose PCIe Gen 5 on an AMD Threadripper PRO Workstation. If the workload is multi-user production serving at scale, full fine-tuning of 70B-class models, or training of large models from scratch, choose an NVLink-connected SXM server (EPYC GPU server with H100, H200, or B200). The interconnect is rarely the limiting factor in a build decision; the workload chooses the form factor, and the form factor determines the interconnect.
Hardware FAQ
What is NVLink?
NVLink is NVIDIA's proprietary point-to-point GPU interconnect. It connects GPUs directly to each other (and in newer generations, to CPUs via NVLink C2C) at bandwidth far above PCIe. NVLink 4 on Hopper (H100, H200) provides 900 GB/s bidirectional per GPU across 18 links. NVLink 5 on Blackwell (B200, GB200) provides 1.8 TB/s bidirectional per GPU across 18 links at 100 GB/s each. NVLink is only available on SXM-form-factor GPUs and certain professional cards; standard PCIe GPUs do not have NVLink.
What is the bandwidth difference between NVLink and PCIe?
PCIe Gen 5 x16 provides 128 GB/s bidirectional per GPU. PCIe Gen 4 x16 provides 64 GB/s. NVLink 4 (H100, H200) provides 900 GB/s, roughly 7x PCIe Gen 5. NVLink 5 (B200) provides 1.8 TB/s, roughly 14x PCIe Gen 5. The bandwidth gap is largest at the top tier, but PCIe Gen 5 itself is a meaningful improvement over Gen 4 and is sufficient for many AI workloads that do not involve heavy cross-GPU traffic.
When does NVLink actually matter for AI?
NVLink matters most for tensor-parallel training and inference of large models, and for full fine-tuning with gradient synchronization. For tensor parallelism, activations transfer between GPUs on every forward pass; at 900 GB/s NVLink, the transfer takes microseconds, while at PCIe Gen 5 the same transfer takes roughly 7x longer and becomes a visible bottleneck. For all-reduce on training steps, PCIe-connected 8-GPU setups can consume 30-40% of step time on large models, versus negligible overhead on NVSwitch. NVLink matters less for LoRA, QLoRA, data-parallel inference, and any workload where the model fits in one GPU.
When is PCIe Gen 5 sufficient for AI?
PCIe Gen 5 is sufficient for single-GPU inference, single-GPU fine-tuning, multi-GPU data-parallel serving (independent jobs per GPU), LoRA and QLoRA fine-tuning on multi-GPU workstations, and tensor-parallel inference of smaller models or shorter contexts. The PCIe bottleneck shows up specifically when many activations transfer between GPUs per step on a tight latency budget. For most workstation-class workloads, PCIe Gen 5 x16 per GPU is the right choice and NVLink is not available anyway.
Does the RTX PRO 6000 Blackwell support NVLink?
No. The NVIDIA RTX PRO 6000 Blackwell (both Workstation and Server Edition) does not support NVLink. Multi-GPU configurations communicate over PCIe Gen 5 x16. For NVLink connectivity, the SXM-form-factor datacenter GPUs (H100 SXM, H200 SXM, B200 SXM) are the path, and they require server-class chassis with the SXM baseboard. The RTX PRO 6000 Blackwell at 96GB GDDR7 ECC is excellent for single-GPU and PCIe multi-GPU workstation builds; it is not the right card for NVLink-required workloads.
What is NVSwitch?
NVSwitch is a chip that provides non-blocking all-to-all NVLink connectivity between multiple GPUs. In an 8-GPU H100 or H200 SXM server, NVSwitch enables every GPU to communicate with every other GPU at full NVLink bandwidth simultaneously. The 4th-generation NVSwitch on Blackwell is used in the GB200 NVL72 rack, which connects 72 B200 GPUs with 130 TB/s of aggregate all-to-all bandwidth. NVSwitch is what makes large-scale training and serving practical at the rack and pod level.
Does NVLink matter for LLM inference?
It depends on whether the model fits in one GPU and whether the workload uses tensor parallelism. For models that fit in a single GPU's VRAM, NVLink is irrelevant (no cross-GPU traffic). For tensor-parallel inference of large models, NVLink reduces activation transfer time and lets the workload scale better across more GPUs. For data-parallel inference (one model copy per GPU serving independent users), NVLink is irrelevant because GPUs do not communicate. In practice: single-user 70B inference does not need NVLink; production serving of 405B at FP16 across 8 GPUs benefits significantly from it.
How does PCIe Gen 4 vs Gen 5 affect AI workloads?
PCIe Gen 5 doubles the per-lane bandwidth of Gen 4, taking x16 from 64 GB/s to 128 GB/s bidirectional. For single-GPU workloads, the difference is modest because model weights load to VRAM once and stay there. For multi-GPU PCIe workloads, Gen 5 helps materially: tensor-parallel inference, all-reduce, and data transfer between GPU and CPU all benefit. For workstation builds on AMD Threadripper PRO 9000WX and EPYC 9005 Turin, PCIe Gen 5 is standard and recommended over Gen 4 platforms.
Ready to buy?Does VRLA Tech build workstations with NVLink?
For workstation form factor, NVLink is not available on current professional GPUs (RTX PRO 6000 Blackwell, RTX 6000 Ada, L40S all communicate over PCIe). VRLA Tech builds Threadripper PRO Workstations with full PCIe Gen 5 x16 per GPU, which is the right interconnect for workstation multi-GPU work. For NVLink, VRLA Tech builds EPYC GPU servers with H100, H200, or B200 SXM GPUs. VRLA Tech is based in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Enterprise clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Does VRLA Tech build NVLink-connected GPU servers?
Yes. VRLA Tech AMD EPYC GPU servers with H100 SXM5, H200 SXM, or B200 SXM GPUs include NVSwitch fabric for full all-to-all NVLink bandwidth between every GPU pair. These configurations are the standard for full fine-tuning, large-model training, and production serving of 405B-class models. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Do I need NVLink for my AI workload?
VRLA Tech sales engineers help match the right interconnect to the workload. NVLink is required for full fine-tuning of 70B-class models, training of large models from scratch, and tensor-parallel production serving at scale. NVLink is not required for single-GPU inference, LoRA and QLoRA fine-tuning, data-parallel serving, or any workload where the model fits in one GPU. Submit a workload description at vrlatech.com/contact and a sales engineer will recommend the right form factor. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
What is the price difference between PCIe and NVLink builds from VRLA Tech?
NVLink-connected EPYC GPU servers with H100, H200, or B200 SXM GPUs are substantially more than PCIe workstation builds due to SXM baseboard, datacenter chassis, NVSwitch fabric, and the GPU cost. The interconnect choice should follow from the workload, not the budget: NVLink-required workloads (full fine-tuning of 70B+, large-model training, tensor-parallel production serving) justify the NVLink hardware; LoRA, QLoRA, and most inference workloads run well on PCIe workstations. Submit workload details at vrlatech.com/contact for a current quote. VRLA Tech is based in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Enterprise clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Does VRLA Tech build B200 GPU servers?
Yes. VRLA Tech builds EPYC GPU servers with NVIDIA B200 SXM GPUs (180-192GB HBM3e, 8 TB/s memory bandwidth, NVLink 5 at 1.8 TB/s). B200 configurations target trillion-parameter model serving, full fine-tuning of 70B and 405B, and the most demanding training workloads. Lead times depend on B200 supply. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Does VRLA Tech support NVLink server deployment in colocation facilities?
Yes. VRLA Tech builds rackmount EPYC GPU servers for direct deployment in customer colocation space or for VRLA Tech-managed deployment. NVLink-connected configurations are validated at full load before shipping and arrive ready for rack installation. For larger deployments, see the VRLA Tech data center deployment page. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Can VRLA Tech build NVLink servers for HPC research?
Yes. VRLA Tech builds HPC servers for research labs and academic institutions with H100, H200, or B200 SXM GPUs and full NVSwitch interconnect. Configurations are quoted to specific scientific computing workloads including molecular dynamics, computational chemistry, climate modeling, and AI training. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Enterprise and research clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
How long does VRLA Tech take to deliver an NVLink GPU server?
Most VRLA Tech builds take about 2 weeks for building and stress testing before shipping, with a 48-hour burn-in (including NVLink fabric validation for SXM builds). NVLink GPU servers depend on SXM GPU availability — for mission-critical timelines, mention the deadline early so VRLA Tech can plan around supply and any expedited handling. VRLA Tech is located in Los Angeles, has been building custom AI hardware since 2016, and ships with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University. Request a quote at vrlatech.com/contact.
Does VRLA Tech price-match NVLink server builders?
VRLA Tech price-matches comparable NVLink server configurations from other US-based AI server builders. Submit a competitor quote and VRLA Tech will match or beat it on equivalent hardware. VRLA Tech configurations include DDR5 ECC RDIMM, 48-hour burn-in including NVLink fabric validation, and a 3-year parts warranty plus lifetime US-based engineer support. Located in Los Angeles, building custom AI hardware since 2016. Enterprise clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Can VRLA Tech help me decide between PCIe workstation and NVLink server?
Yes. The decision usually comes down to workload, not budget. VRLA Tech workstations on PCIe Gen 5 suit single-developer fine-tuning, LoRA and QLoRA work, and inference of models that fit in one or two GPUs. VRLA Tech EPYC GPU servers with NVLink suit full fine-tuning, large-scale training, and production serving. Sales engineers walk through the decision case by case. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Does VRLA Tech offer financing for NVLink servers?
Yes. VRLA Tech accepts purchase orders from qualified enterprises, universities, and government entities, and works with PO financing partners for net-30, net-60, and longer terms on larger orders including NVLink server configurations. Standard payment methods include wire, ACH, credit card, and PO. Request financing options at vrlatech.com/contact. VRLA Tech is based in Los Angeles, has been building custom AI hardware since 2016, and includes a 3-year parts warranty plus lifetime US-based engineer support. Enterprise clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Does VRLA Tech help calculate ROI for NVLink servers versus cloud H100 rental?
Yes. The VRLA Tech AI ROI calculator compares the total cost of an on-premise NVLink GPU server against equivalent cloud H100, H200, or B200 rental over 12, 24, and 36 month horizons. For sustained training and fine-tuning workloads (over roughly 8 hours per day, every day), on-premise typically breaks even in 6 to 14 months. For sporadic workloads, cloud may be the right answer. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Does VRLA Tech build AI training clusters with NVLink and InfiniBand?
Yes. VRLA Tech AI training cluster builds combine NVLink and NVSwitch within nodes with InfiniBand or 400G Ethernet between nodes for multi-node distributed training. NDR and XDR InfiniBand options are supported for the lowest cross-node latency. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
How do I get an NVLink server quote from VRLA Tech?
Request a quote at vrlatech.com/contact with the target GPU (H100 SXM5, H200 SXM, B200 SXM), the number of GPUs (4, 8, or larger), the target workload (training, fine-tuning, serving), and any compliance requirements (HIPAA, ITAR, FedRAMP). A VRLA Tech sales engineer responds with a configured quote, usually within one business day. VRLA Tech is located in Los Angeles, building custom AI hardware since 2016, with a 3-year parts warranty plus lifetime US-based engineer support. Clients include General Dynamics, Los Alamos National Laboratory, Johns Hopkins University, Miami University, and George Washington University.
Configuring an NVLink server or PCIe workstation?
Tell VRLA Tech the workload at vrlatech.com/contact — sales engineers match the right interconnect, quote back within one business day.