activity perturbation
A learning algorithm that perturbs the inputs to neurons and measures the effect on the objective function, using fewer parameters than weight perturbation and thus having lower variance
As defined in Geoffrey Hinton - Two Paths to Intelligence
Where it comes up · 1
Most important claims · 2 of 2Search all claims for “activity perturbation” →
- Backpropagation requires knowing the properties of the forward pass through layers. If you don't know the hardware properties, you need a different learning algorithm. Activity perturbation (perturbing neuron inputs and measuring the effect) works better than weight perturbation because there are fewer neurons than weights, so it has much less variance.Geoffrey Hinton - Two Paths to Intelligence
- Activity perturbation can learn MNIST-level problems reasonably fast, and can learn CIFAR-10 if you're very determined, but imagenet with millions of images is just too slow to scale activity perturbation learning to large networks.Geoffrey Hinton - Two Paths to Intelligence