Chinese Researchers Develop Optical
Chip Interconnect That Delivers 150× Faster AI Inference with One-Ninth the
Computing Power
Peking University researchers develop
new all-optical interconnect system linking standard electronic chips with
specific algorithms
·
Researchers
at Peking University have developed an all-optical interconnect
system that links conventional electronic chips, significantly improving AI
inference performance while drastically reducing computing requirements.
·
The new
architecture achieved over 100-fold faster distributed AI inference
while using only one-ninth of the computational resources typically
required by a commercial GPU.
·
The
research was published in the journal National Science Review.
·
The study
was led by Shu Haowen and Wang Xingjun.
·
The
system uses Field-Programmable Gate Arrays (FPGAs) as modular computing
units, which are widely deployed in high-parallel processing applications such
as autonomous driving, missile guidance and data centres.
·
The FPGA
chips are interconnected using two custom optical communication components:
o
a 400
Gbps silicon photonic transceiver that converts electrical signals into
optical signals and vice versa; and
o
a 16×16
optical switching chip that dynamically routes optical data between up to
16 input and 16 output ports.
·
Researchers
connected five FPGA chips in parallel to perform an AI-based
image-denoising task.
·
The
system processed 1,000 images (32,768 bits each) in just 105.16
microseconds.
·
In
comparison, a commercial 16.96-teraflop GPU completed the same task in 15.6
milliseconds, making the optical FPGA system nearly 150 times faster.
·
Despite
the higher performance, the five FPGA chips together provided only 1.969
teraflops of theoretical computing power—less than one-ninth of the
GPU's compute capability.
·
The
performance advantage stems from the characteristics of optical
communication, including:
o
optical
switching losses below 5 decibels, enabling high-speed, error-free data
transmission without external optical amplification; and
o
direct
optical data flow between processing stages.
·
Unlike
conventional GPU-based AI inference, where intermediate feature maps must
repeatedly be written to and read from memory, the optical architecture allows
data to flow directly between neural network layers, eliminating memory
bottlenecks.
·
This
pipeline-like processing keeps all processing units continuously active,
reducing latency and improving overall hardware utilization.
·
The
researchers conclude that future AI performance gains may depend more on intelligently
connecting chips than simply increasing the number of processors.
·
The
co-design of algorithms, processor architectures and chip-level optical
interconnections could enable high-performance AI with significantly lower
computational resources.
·
The
technology also has the potential to reduce energy consumption in data
centres and improve latency and power efficiency in edge-computing
applications, making AI infrastructure more sustainable.
Chinese
researchers have developed a new all-optical interconnect system linking standard
electronic chips, boosting AI distributed inference speeds by over 100 times while
using just one-ninth of the typical computational resources.
AI
models are permeating ever more applications, expanding the industry’s appetite
for computational power. The conventional response has been to pile on more GPUs
and build ever-larger data centres in a seemingly endless race for energy and brute
force.
But
a new study from Peking University suggests a radically different path: by optically
linking chips with specific algorithms, they boost inference speeds by a factor
of over 100 while slashing compute needs to just one-ninth.
The
work was published in the journal National Science Review, and its corresponding
authors include Shu Haowen and Wang Xingjun from Peking
University.
The
team’s “Lego” building blocks were field-programmable gate array (FPGA) chips: programmable
devices widely used in fields that demand high parallel-processing ability, such
as missile guidance, autonomous driving and data centres.
The
“joints” connecting these FPGAs are two custom-designed communication hardware components.
The first is a silicon photonic transceiver chip running at 400 gigabits per second,
responsible for converting electrical signals to optical and vice versa.
The
second is a 16×16 optical switching chip that acts as a router during optical transmission
and can establish arbitrary connections between up to 16 input ports and 16 output
ports.
By
wiring five FPGA chips in parallel, the researchers tackled an image-denoising task
on a dedicated dataset, processing 1,000 images of 32,768 bits each in just 105.16
microseconds.
For
comparison, the team ran the same image-denoising task on a commercial GPU rated
at 16.96 teraflops. The GPU completed the task in 15.6 milliseconds, making the
FPGA system nearly 150 times faster.
The
five FPGAs together offered a combined theoretical compute power of about 1.969
teraflops – less than one-ninth the GPU’s resources.
This
advantage stems from the inherent benefits of optical communication.
First,
the switching chip’s total loss is below 5 decibels, enabling high-speed, error-free
data transfer without any need for external optical gain compensation.
More
importantly, in conventional GPU-based neural network inference, a cascade of intermediate
results, namely feature maps, is generated between layers.
These
must be stored and retrieved repeatedly, and each layer must wait for the previous
one to finish before it can start, leaving compute units idle.
In
optical computing, by contrast, results from one layer can flow directly into the
next like an assembly line, bypassing memory altogether and keeping all chips continuously
busy.
The
research underscores a pivotal insight: in the relentless pursuit of AI compute
power, “stacking more chips” might not be as effective as “connecting them intelligently”.
“Specific
objectives can be realised under limited computational resources when algorithms,
processor micro‑architectures and chip-level interconnections are co‑designed,”
the authors wrote in the paper.
“This
fabric can also alleviate unsustainable energy usage in data centres and optimise
latency or consumption in edge-computing scenarios,” they added.