120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training

Zhongjin Lin; Bhavin J Shastri; Shangxuan Yu; Jingxiang Song; Yuntao Zhu; Arman Safarnejadian; Wangning Cai; Yanmei Lin; Wei Ke; Mustafa Hammood; Tianye Wang; Mengyue Xu; Zibo Zheng; Mohammed Al-Qadasi; Omid Esmaeeli; Mohamed Rahim; Grzegorz Pakulski; Jens Schmid; Pedro Barrios; Weihong Jiang; Hugh Morison; Matthew Mitchell; Xun Guan; Nicolas A F Jaeger; Leslie A Rusch; Sudip Shekhar; Wei Shi; Siyuan Yu; Xinlun Cai; Lukas Chrostowski

doi:10.1038/s41467-024-53261-x

120 GOPS Photonic tensor core in thin-film lithium niobate for inference and in situ training

Nat Commun. 2024 Oct 21;15(1):9081. doi: 10.1038/s41467-024-53261-x.

Authors

Zhongjin Lin^{1

2}, Bhavin J Shastri³, Shangxuan Yu¹, Jingxiang Song¹, Yuntao Zhu², Arman Safarnejadian⁴, Wangning Cai¹, Yanmei Lin², Wei Ke², Mustafa Hammood¹, Tianye Wang¹, Mengyue Xu², Zibo Zheng⁴, Mohammed Al-Qadasi¹, Omid Esmaeeli¹, Mohamed Rahim⁵, Grzegorz Pakulski⁵, Jens Schmid⁵, Pedro Barrios⁵, Weihong Jiang⁵, Hugh Morison³, Matthew Mitchell¹, Xun Guan⁶, Nicolas A F Jaeger¹, Leslie A Rusch⁴, Sudip Shekhar¹, Wei Shi⁴, Siyuan Yu², Xinlun Cai⁷, Lukas Chrostowski⁸

Affiliations

¹ Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, British Columbia, Canada.
² State Key Laboratory of Optoelectronic Materials and Technologies, School of Electronics and Information Technology, Sun Yat-sen University, Guangzhou, Guangdong, China.
³ Department of Physics, Engineering Physics and Astronomy, Queen's University, Kingston, Ontario, Canada.
⁴ Department of Electrical and Computer Engineering, Université Laval, Québec City, Québec, Canada.
⁵ Advanced Electronics and Photonics Research Centre, National Research Council, Ottawa, Ontario, Canada.
⁶ Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China.
⁷ State Key Laboratory of Optoelectronic Materials and Technologies, School of Electronics and Information Technology, Sun Yat-sen University, Guangzhou, Guangdong, China. [email protected].
⁸ Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, British Columbia, Canada. [email protected].

Abstract

Photonics offers a transformative approach to artificial intelligence (AI) and neuromorphic computing by enabling low-latency, high-speed, and energy-efficient computations. However, conventional photonic tensor cores face significant challenges in constructing large-scale photonic neuromorphic networks. Here, we propose a fully integrated photonic tensor core, consisting of only two thin-film lithium niobate (TFLN) modulators, a III-V laser, and a charge-integration photoreceiver. Despite its simple architecture, it is capable of implementing an entire layer of a neural network with a computational speed of 120 GOPS, while also allowing flexible adjustment of the number of inputs (fan-in) and outputs (fan-out). Our tensor core supports rapid in-situ training with a weight update speed of 60 GHz. Furthermore, it successfully classifies (supervised learning) and clusters (unsupervised learning) 112 × 112-pixel images through in-situ training. To enable in-situ training for clustering AI tasks, we offer a solution for performing multiplications between two negative numbers.

Abstract

Grants and funding