Sheela
Sheela
Home
Add
Questions
Thinkers
Claims
People
Concepts
Jargon
Sources
Health
Predictions
Collections
Domain Landscape
Contribute
Review
Dedup
Topic requests
Settings
Help

← Jargon

Jargonmachine learning· 2 claims· 1 episode

activity perturbation

A learning algorithm that perturbs the inputs to neurons and measures the effect on the objective function, using fewer parameters than weight perturbation and thus having lower variance

As defined in Geoffrey Hinton - Two Paths to Intelligence


Where it comes up · 1
  • Geoffrey Hinton - Two Paths to Intelligence2 claims

Most important claims · 2 of 2Search all claims for “activity perturbation” →
  • Backpropagation requires knowing the properties of the forward pass through layers. If you don't know the hardware properties, you need a different learning algorithm. Activity perturbation (perturbing neuron inputs and measuring the effect) works better than weight perturbation because there are fewer neurons than weights, so it has much less variance.Geoffrey Hinton - Two Paths to Intelligence
  • Activity perturbation can learn MNIST-level problems reasonably fast, and can learn CIFAR-10 if you're very determined, but imagenet with millions of images is just too slow to scale activity perturbation learning to large networks.Geoffrey Hinton - Two Paths to Intelligence