Instance Normalization

E701501

Instance Normalization is a neural network normalization technique that normalizes each individual sample and channel independently, commonly used in tasks like style transfer to stabilize training and control feature statistics.

All labels observed (4)

How this entity was disambiguated

Statements (47)

Predicate Object
instanceOf neural network normalization technique
normalization layer
advantage better control of style in style transfer
more stable behavior for small batch sizes
alsoKnownAs InstanceNorm
appliedAfter convolution layers
applies affine transformation
category feature-wise normalization
commonlyUsedFor image generation tasks
image-to-image translation
neural style transfer
computes per-instance mean
per-instance variance
computesStatisticsPer channel
sample
differsFrom Batch Normalization
Group Normalization
Layer Normalization
doesNotDependOn batch size
doesNotUse batch statistics at inference
domain computer vision
generative modeling
epsilonRole numerical stability in variance normalization
followedBy learnable affine transform y = gamma * x_hat + beta
goal control feature statistics
reduce style variance across spatial locations
stabilize training
hasParameter scale parameter gamma
shift parameter beta
implementedIn PyTorch as torch.nn.InstanceNorm1d
PyTorch as torch.nn.InstanceNorm2d
PyTorch as torch.nn.InstanceNorm3d
TensorFlow Addons as tfa.layers.InstanceNormalization
linked to: TensorFlow Addons
inspired use in fast neural style transfer networks
introducedBy Dmitry Ulyanov
introducedInPaper Instance Normalization: The Missing Ingredient for Fast Stylization
introducedInYear 2016
invariantTo global contrast of each instance
mathematicalOperation x_hat = (x - mu_{n,c}) / sqrt(sigma_{n,c}^2 + epsilon)
normalizes feature activations
normalizesAcross spatial dimensions
oftenReplaces Batch Normalization in style transfer networks
operatesOn individual channels
individual samples
relatedTo style normalization
usedIn convolutional neural networks
deep learning models

How these facts were elicited

Referenced by (5)

Full triples — surface form annotated when it differs from this entity's canonical label.

Layer Normalization relatedTo Instance Normalization
Instance Normalization introducedInPaper Instance Normalization: The Missing Ingredient for Fast Stylization
linked to: Instance Normalization
Instance Normalization implementedIn PyTorch as torch.nn.InstanceNorm1d
linked to: Instance Normalization
Instance Normalization implementedIn PyTorch as torch.nn.InstanceNorm2d
linked to: Instance Normalization
Group Normalization relatedConcept Instance Normalization