Various photonic neural network designs have been proposed and demonstrated to encode inputs and weights to the light magnitude/phase and circuit transmission, respectively, and perform ultra-fast matrix multiplication [1, 2, 3, 4, 5]. Typically, the photonic circuits are sensitive to thermal variation as temperature impacts the refractive index of the optical component. Especially for compact microring resonator (MRR)-based photonic accelerators, the weight $w$ is encoded by the differential transmission of the add-drop MRR as $w=g(2a-1);~{}a=\frac{\alpha^{2}-2r\alpha\cos{\phi}+r^{2}}{1-2r\alpha\cos{\phi}+r^{2}\alpha^{2}}\in(0,1)$, where $\alpha$ is the attenuation factor, $r$ is the coupling factor, $\phi$ is the round-trip phase shift, $a$ is the through-port transmission, and $g$ is the scaling factor. To accommodate multiple channels in a single free-spectral range (FSR) while minimizing spectral crosstalk, i.e., large finesse, MRR is designed to have a mid-to-high quality factor. This design choice makes the MRR highly sensitive to small disturbances, highlighting the urgency in addressing challenges related to thermal variation robustness.

Noise-aware optimization for photonic accelerators includes two categories, i.e., offline optimization and on-chip optimization. Prior offline methods inject noise into ONN model training to obtain smooth solution space for better noise tolerance [10]. However, such a method requires accurate noise modeling and can only handle static, known variations, which cannot be adapted to dynamically drifting, unknown variations. On-chip training, as a trend, has been demonstrated for online adaptability and efficiency in in-situ accuracy recovery. The pretrained optical NNs are fine-tuned on a target dataset with on-chip gradient calculation [19, 20, 21]. However, they rely on accurate gradient calculation and require costly forward and backward propagation on a given training set, which can significantly degrade the edge inference throughput and efficiency and bear data privacy issues.

Since MRR weight banks are typically considered to be one of the most thermal sensitive designs among different kinds of ONN designs [23, 24, 25], we focus on photonic accelerator architectures based on MRR weight banks as a challenging case study to showcase the effectiveness of our DOCTOR method, shown in Fig. 3. We assume a multi-core accelerator with $R$ tiles and $C$ photonic tensor cores (PTCs) per tile. Each PTC is a $k\times k$ add-drop MRR weight bank. The partial sums are reduced in each tile. Hence, it can finish an $Rk\times Ck$ matrix-vector multiplication (MVM) at each cycle. A large $M\times N$ matrix will be partitioned and simply mapped to this accelerator using $\lceil M/(Rk)\rceil\cdot\lceil N/(Ck)\rceil$ cycles. Alongside each PTC, we assume a dedicated local buffer, an electrical local control unit, and thermal sensors for temperature monitoring, control, and processing.

We first give a thorough noise modeling and sensitivity analysis for this photonic accelerator, including dynamic phase variations, drifting environmental temperature, and inter-device thermal crosstalk.

Given a pretrained NN, we map the ideal weights $W^{*}$ to the photonic accelerator. However, the actual weights $\widetilde{W}$ will have a deviation from the ideal ones. To quickly recover the inference accuracy on the fly without training or utilizing any labeled calibration datasets, we introduce salience-aware sparse calibration in Fig. 8, which is formulated as a batched block-wise regression problem,

By optimizing the latent weight $W$, we want to minimize the distance between the expected noisy weights $\mathbb{E}[\widetilde{W}]$ and the ideal weights $W^{*}$ on each $k\times k$ block. The loss degradation of the NN can be approximated by Taylor expansion

As an approximation, we can rewrite the objective $\mathcal{L}_{calib}$ in Eq. (3)

This fundamentally decouples the calibration procedure from the labeled dataset and task-specific loss function $\mathcal{L}$. By solving this regression problem concurrently on all matrix blocks ($\forall i,j$), we can efficiently resume the task performance. The expected noisy weights $\mathbb{E}[\widetilde{W}]$ can be probed by shining an identity matrix $I\in\mathbb{R}^{k\times k}$ through the MRR weight bank $m$ times and calculating the average,

Compared to on-chip training with backpropagation [22, 21], our calibration method has three major advantages:

(1) High efficiency – Our method does not require costly forward propagation or error feedback. The gradient of the calibration objective w.r.t. the latent weights $\nabla_{W}\mathcal{L}_{calib}$ can be efficiently approximated using a straight-through estimator. With MAE as the loss, the gradients are simply

(2) Accurate gradients – Unlike on-chip training, where gradient errors will exponentially accumulate through layers, our method only calculates gradients for each block without backpropagation. Hence, our method avoids slow convergence or divergence issues even under large noise.

(3) Task-agnostic & Data-free – We do not use any labeled dataset or task-specific loss function, which avoids dataset storage costs and data privacy issues on edge devices.

Motivated by the important observation of the nonuniform noise distribution across devices and cores in Fig. 1(c), we propose architectural variation-aware tile remapping to remedy the accuracy degradation. We try to find a matrix-to-tile index mapping better than the direct mapping ordering, shown in Fig. 9. Given a weight-stationary dataflow, we map a $Rk\times Ck$ weight matrix block onto the accelerator for MVMs and move to the next matrix block. A direct mapping will map $R\times C$ subblocks onto the $R\times C$ PTC arrays following their original order. This is suboptimal because weight blocks have different sensitivities, and PTCs show different error levels. It is natural to remap weight blocks onto PTCs to minimize errors.

However, to avoid complicated dataflow, we cannot arbitrarily remap $RC$ weight blocks to $RC$ PTCs, which will lead to nontrivial architectural overhead. First, as imposed by the dataflow in the accelerator topology, the same input will be broadcast via photonic waveguides to all cores in a column, e.g., PTC($r$,1) $\forall r\in[R]$. The partial sum from cores within one row (tile) will be accumulated via photocurrent summation. Therefore, we can only remap the workloads in the granularity of tiles. Formally, we denote the indices of tiles as $\mathcal{V}=[v_{1},v_{2},\cdots,v_{R}]$ and the indices of weight chunks as $\mathcal{U}=[u_{1},u_{2},\cdots,u_{R}]$, where $W_{u_{r}}\in\mathbb{R}^{k\times Ck}$. Tile remapping is formulated as a linear assignment problem (LAP),

Each entry in the cost matrix $\epsilon\in\mathbb{R}^{R\times R}$ represents the edge weight in the complete bipartite graph, shown in Fig. 9. The edge weight $\epsilon_{ij}$ is an indicator of errors when mapping $W_{i}$ to tile $j$. Similar to the salience scores in the calibration, we also use the first-order Taylor expansion to calculate sensitivity-aware error terms $\epsilon_{ij}=|\nabla_{W_{i}}\mathcal{L}^{T}\cdot(\mathbb{E}[\widetilde{W}_{i}^{j}]-W_{i}^{*})|$. This LAP can be optimally solved in polynomial time. The cycle cost of remapping one $Rk\times Ck$ weight matrix block is,

where the probing times $m=1$. We periodically solve this optimal remapping $f$ to save cost and apply it to all following inferences.

To dynamically determine when to trigger our remediation flow, we introduce an adaptive controller that periodically monitors cheap but informative statistics, i.e., temperature per PTC, and determine whether to trigger remediation based on a threshold, e.g., when average chip temperature drift from last remediation is above 0.01K, i.e., ($\frac{1}{RC}\sum T_{r,c}-T_{prev}>0.01$). If not, we will perform a more expensive probing, the normalized MAE (NMAE) $\|\widetilde{W}-W^{*}\|_{1}/\|W^{*}\|_{1}$, with a slight cycle overhead. If the NMAE is above 5%, we will trigger remediation. To avoid overly frequent remediation that keeps interrupting the online inference stream, we set a cooling time $\tau$ for our remediation procedure to control the maximum acceptable overhead. This can be reconfigured by users based on the preferences between accuracy and inference throughput. For example, with 10K total inferences, each remediation is as expensive as 10 inferences, and a cooling time of 200 inferences will lead to a max overhead of $\frac{10K\times 10}{200\times 10K}=5\%$.

(1) Phase Variation (PV). To simulate real-world chip noise scenarios, two cases are created: "Low Noise" and "High Noise," corresponding to low and high standard deviation (std.) in phase shift, modeled as $\Delta\phi\in\mathcal{N}(0,\sigma^{2})$. As explained in Section III-B, the noise std. $\sigma_{ij}^{t+1^{\prime}}$ for each MRR device is time-variant and sampled at each time step from a noise level map $\mathcal{N}\in(\mu_{s}(t),\sigma_{s}^{2}(t))$. The noise level distribution gradually changes its high-noise region from the chip’s left edge to the top-left corner ("Edge-to-Corner") to capture dynamic noise profile drifting. Specific noise level functions are chosen for the Low and High Noise cases:

• •

  Low Noise: $\mu_{s}(t)=0.0025t,\sigma_{s}(t)=0.004t+0.002$

• •

  High Noise: $\mu_{s}(t)=0.01t,\sigma_{s}(t)=0.005t+0.005$

The damping factor $\beta_{\sigma}$ is set to 0.9. The phase error intensities adopt the typical values in the literature [10, 26].

(2) Temperature Drift (TD): Two cases are designed to represent different types of temperature change with $t_{max}=20,000$:

• •

  Linear: $T(t)=300K+t/t_{max}$

• •

  Cosine: $T(t)=300.25K-0.25K\cos(10t/t_{max})$

The temperature drifts adopt typical values in the literature [23, 24]. We also consider two representative spatial distributions of temperature.

• •

  Uniform: All PTCs have the same temperature drift following the Linear/Cosine scheduling.

• •

  Corner Hotspot: the upper-left corner of the chip experiences a high temperature drift, exponentially decreasing with distance, i.e., $T(t)\leftarrow e^{-\sqrt{r^{2}+c^{2}}}(T(t)-T(0))+T(0)$, which represents a local hotspot scenario during the execution of the accelerator.

(3) Thermal Crosstalk (CT):, we use a default MRR spacing $l_{v}=200\mu m$, $l_{h}=60\mu m$, crosstalk coefficient $k_{1}=0.1$, and model crosstalk only among MRRs within the same PTC [28, 25, 30].

In Table III, we compare DOCTOR with three baselines: (1) BS1: deploy pre-trained models; (2) BS2 [10]: noise-aware training with (2%) weight error injected during pretraining; and (3) BS3 [21, 22]: on-chip training for 1 epoch on a calibration dataset (10% of training set). The parameters for our DOCTOR framework are: (1) Calibration: probing times $m=1$, salience score $s=|\nabla_{W}\mathcal{L}|$, calibration sparsity $\beta=0.2$, max calibration iteration $T_{calib}=20$ (50 for ResNet18); (2) Remapping: first-order Taylor expansion as $\epsilon$; (3) Controller: remediation cooling time $\tau$: 200 inferences (50 for ResNet18); trigger remapping $\rightarrow$ calibration.

Across all benchmarks, BS1 suffers from severe accuracy degradation (30%-60%). Noise-aware training [10], though it enhances the smoothness of the solution space of the model, shows limited effect on accuracy improvement, as the noise distribution used in pre-training is significantly different from the physical variations. Note that our DOCTOR is orthogonal to noise-aware training, allowing simultaneous application for a solution space that is both locally smooth and adaptive to the drifting noise distribution. On-chip training [21, 22] can boost the accuracy on small networks, e.g., 8% on CNN-FMNIST and VGG8-CIFAR10. However, it performs poorly on deep ResNet18 and is not stable on certain benchmarks, e.g., CT+PV.2+TD.1. The fundamental reason is that the gradient estimation error exponentially accumulates with backpropagation, which can lead to poor training performance and even divergence. Also, though it is only trained for 1 epoch on a 10% training set, it consumes 5-6$\times$ more runtime (cycles) in training, which is not practical to be deployed on throughput-restricted edge platforms. In contrast, our proposed DOCTOR method can stably maintain high accuracy even with rapidly drifting noise distributions and temperature with less than 1-2.5% accuracy drop at the cost of merely 0.1%-5.1% cycle overhead. On average, DOCTOR is +34% more accurate and 2-3 orders-of-magnitude more efficient than on-chip training. We visualize our DOCTOR flow in Fig. 13 to show the temperature drift is detected and the accuracy is rapidly resumed with only 3.6% cycle overhead.