tensor programs framework

E102299

The tensor programs framework is a theoretical approach developed by Greg Yang that rigorously analyzes and characterizes the behavior and scaling limits of large neural networks using tools from probability and random matrix theory.

All labels observed (1)

Label Occurrences
tensor programs framework canonical 1

How this entity was disambiguated

Statements (42)

Predicate Object
instanceOf neural network theory framework
theoretical framework
analyzes behavior of wide neural networks
scaling behavior of deep neural networks
appliesTo convolutional neural networks
fully connected neural networks
transformer-style architectures
associatedWith Greg Yang's research on deep learning limits
Microsoft
surface form: Microsoft Research
assumes large-width asymptotics
random initialization of network parameters
basedOn Gaussian process limits
probabilistic limit theorems
random matrix theory techniques
characterizes distributional behavior of activations at initialization
gradient behavior in wide networks
scaling of parameters with width and depth
developer Greg Yang
enables rigorous proofs of convergence of network statistics
systematic study of architectural variations at infinite width
field deep learning theory
machine learning theory
probability theory
random matrix theory
hasConcept master theorem for tensor programs
program limit
tensor program
influencedBy Gaussian process theory
classical random matrix theory
influences design of scalable neural network architectures
theoretical understanding of large-scale deep learning
provides a language for describing tensor computations in neural nets
rigorous conditions for infinite-width limits
tools for analyzing signal propagation in deep networks
purpose characterization of scaling limits of neural networks
rigorous analysis of large neural networks
relatedTo infinite-width neural networks
neural tangent kernel
scaling laws in deep learning
usedFor deriving kernel limits of neural networks
designing scaling rules for neural network hyperparameters
understanding training dynamics in the infinite-width limit

How these facts were elicited

Referenced by (1)

Full triples — surface form annotated when it differs from this entity's canonical label.

Greg Yang notableConcept tensor programs framework