Ph.D. Public Lecture

Speaker: Anif Shikder (Western)

"Opening the Black Box: An Exactly Solvable Neural Network That Remembers, Predicts, and Remains Mathematically Tractable"

Time: 13:00 - 14:00

Room: ZOOM

Two problems sit at the heart of machine learning and computational neuroscience: associative memory, how a system recalls patterns from partial input, and sequence modelling, the long-range processing behind modern language models. The dominant architectures, Hopfield networks for memory, transformers and state-space models for sequences, work well but resist analysis. Their computation lives in implicit recurrent updates or all-pairs attention, and we understand it mostly through post hoc interpretability rather than first principles. My thesis addresses that transparency gap by building models whose computation admits an exact, closed-form description. The unifying object is a complex-valued nonlinear oscillator network, a generalization of the Kuramoto model. A nonlinear coordinate transformation linearizes it and collapses the dynamics into a single closed-form propagator, the matrix exponential of K times t. One operator, used in three studies. First, memory. Patterns are stored as eigenmodes of a complex connectivity matrix, and recall becomes a complex-walk filtration: phase-coherent interference along walks through the network reconstructs the occluded input. There are no spurious attractors, and it outperforms modern Hopfield networks on occluded-MNIST, including unseen digits. Second, sequences. Here, I focus on S4, a leading alternative to the transformer for long sequences where attention becomes prohibitively expensive, which makes understanding it especially worthwhile. I establish a formal correspondence between diagonal state-space models and the oscillator network and derive an exact operator for the forward pass of S4. A low-order truncation recovers roughly 94% of full performance on a real-world task. Third, a new architecture, S1, whose entire forward pass is a closed-form operator parameterized by the eigenvalues and eigenvectors of K. It's competitive on several benchmarks, and because the spectrum is explicit, it supports causal control through targeted modal intervention. Together, these show that recall and sequence processing are two regimes of one operator. Memory is the truncated walk-order expansion of the propagator; sequence processing is the same propagator sampled in discrete time, distinguished by the spectrum of K. Mathematical transparency and competitive performance are compatible, with implications for trustworthy sequence models in high-stakes domains.