Distributed Data Parallel
A PyTorch container (DDP) that automatically synchronizes gradients across multiple GPUs during backpropagation.
As defined in Andrej Karpathy — AGI is still a decade away
Where it comes up · 1
Most important claims · 1 of 1Search all claims for “Distributed Data Parallel” →
- Coding agents are good at boilerplate and code common on the internet, but they fail on unique, intellectually intense code: they over-rely on memorized common patterns, bloat code with defensive try-catch and production scaffolding, use deprecated APIs, and can't internalize custom non-standard implementations—so for novel work autocomplete is the sweet spot rather than full agents.Andrej Karpathy — AGI is still a decade away