Photonic convolutional neural network with pre-trained in-situ training

Abstract

The authors propose a fully-optical convolutional neural network system for MNIST image classification that achieves 94% test accuracy while maintaining coherent processing throughout. Unlike traditional photonic ML systems that require frequent optical-to-electrical-to-optical (O/E/O) conversions, this architecture keeps computation entirely in the photonic domain using Mach-Zehnder interferometer (MZI) meshes, wavelength-division multiplexed (WDM) pooling, and microring resonator-based nonlinearities.

Key Contributions

• Fully photonic CNN — no O/E/O conversions in the forward path
• Silicon-photonics-based max pooling implemented without electrical conversion
• Hybrid training methodology combining digital-twin backpropagation with in-situ fine-tuning via Simultaneous Perturbation Stochastic Approximation (SPSA)
• 2,132 individually tunable on-chip parameters — reported as largest single integrated system to date
• Energy efficiency 100–242× better than state-of-the-art electronic GPUs

Methodology

The system maps standard CNN topology onto silicon photonic hardware:

• Convolution — MZI meshes execute matrix multiplications through optical interference
• Pooling — WDM-based passive delay networks
• Nonlinearity — microring resonator activation functions
• Training — Mathematically exact differentiable digital twin for ex-situ backpropagation, followed by in-situ fine-tuning via SPSA to correct for fabrication-induced deviations between the digital twin and the physical chip

Results

• 94% MNIST test accuracy
• Robust to thermal crosstalk — only 0.43% accuracy degradation under severe coupling conditions
• 100–242× energy-efficiency improvement vs. state-of-the-art GPUs for equivalent inference throughput

Limitations

Not explicitly detailed in the abstract. Open questions include scaling to larger benchmarks beyond MNIST, generalization to tasks requiring deeper networks or attention mechanisms, and yield/calibration cost at scale.

Source: Photonic convolutional neural network with pre-trained in-situ training — Saurabh Ranjan, Sonika Thakral, Amit Sehgal, April 2026