A statistical-field-theory account of what a deep neural network does before training — deriving, layer by layer, how a network's internal representations "flow" with depth.
MSc Mathematics thesis, School of Mathematical Sciences, National Institute of Science Education and Research (NISER), Bhubaneswar — May 2023.
Deep networks work spectacularly well but remain black boxes. This thesis treats a network not as an optimization problem but as a physical system: the pre-activations of a Multi-Layer Perceptron (MLP) at initialization are a field, the layer width plays the role of a large-$N$ limit, and depth is a flow. From that vantage point the statistics of an untrained network become analytically tractable, and you can derive — not just measure — how each layer reshapes the distribution of its representations. The organizing analogy is the renormalization group from physics, hence Representation Group Flow.
This is a self-contained, MSc-level re-derivation of the effective-field-theory picture of an MLP at initialization from Roberts, Yaida & Hanin's The Principles of Deep Learning Theory (arXiv:2106.10165) — worked out explicitly from first principles. Its contribution is pedagogical clarity, not new theorems: every Wick contraction and Schwinger–Dyson step is written out, as a gentle on-ramp to that 400-page book.
-
An effective field theory for the MLP at initialization. The collection of layer-wise pre-activations
${z^{(1)}, \dots, z^{(L)}}$ is treated as a field with an action$S_\text{full} = \sum_\ell S_M(z^{(\ell)}) + \sum_\ell S_I(z^{(\ell+1)}\mid z^{(\ell)})$ , and the output distribution is obtained by marginalizing layer by layer — exactly the scale-by-scale integration that produces beta functions in physics. -
A precise "when does Gaussianity break" result. Using
$M$ -point correlators, Wick's theorem and a Schwinger–Dyson derivation, the thesis shows the first layer is exactly Gaussian (zero mean, variance set by the first-layer metric, vanishing connected 4-point correlator), while the second and deeper layers are not: a four-point vertex$V$ appears — a quartic interaction between neurons. -
Non-Gaussianity is controlled by 1/width. The quartic coupling is suppressed by
$1/n$ (network width). Wide networks are nearly Gaussian; finite width is what makes them interesting. This gives a controlled perturbative expansion. -
Representations "run" with depth. The quadratic and quartic couplings obey recursion relations from one layer to the next — the network's own analogue of RG running couplings. Representation Group Flow is the resulting flow: repeatedly marginalizing fine-grained representations in shallow layers yields the coarse-grained representation read out at the end.
- It makes "depth" a first-class mathematical object, not a hyperparameter — a flow with its own recursion relations.
- It bridges two languages, reading deep learning through statistical field theory (the machinery physicists use for many-body systems).
- It is derivational. Everything about the untrained network follows from Gaussian initialization plus the width expansion — no training, no fitting.
Read this if you want Roberts–Yaida–Hanin's Chapter 4 unpacked slowly, with every contraction and Schwinger–Dyson step written out — a gentle on-ramp before the full book. Skip it if you already know the finite-width / NNGP literature; this is exposition, not new results.
| Chapter | Content |
|---|---|
| 1. Introduction | Deep networks as black boxes; the case for an effective theory |
| 2. Fundamental Concepts | Field theory and RG; the MLP (neuron → network → output distribution); the field-theory analogy |
| 3. Mathematical Concepts |
|
| 4. Derivation & Summary | First layer (Gaussian) → second layer (four-point vertex, Schwinger–Dyson) → deeper layers (recursions) → the RG-flow picture |
Built on the program of Roberts, Yaida & Hanin, The Principles of Deep Learning Theory (arXiv:2106.10165).
@mastersthesis{tata2023representation,
title = {Representation Group Flow},
author = {Tata, Satya Pratheek},
school = {National Institute of Science Education and Research (NISER), Bhubaneswar},
type = {MSc thesis, School of Mathematical Sciences},
year = {2023},
month = may
}Satya Pratheek Tata — satyapratheek2000@gmail.com · github.com/TataSatyaPratheek · LinkedIn
With thanks to Prof. Subhankar Mishra and Prof. Deepak Kumar Dalai (NISER).
© 2023 Satya Pratheek Tata. Text and figures shared for scholarly reading and citation; please cite rather than redistribute modified copies.