What this covers

Jeff Dean and Noam Shazeer trace twenty-five years of machine learning progress at Google—from the early scaling experiments that proved neural networks could work at scale, through the Transformer era, to current thinking on the next architectural phase. The conversation moves between concrete technical details and larger strategic forecasts about how AI systems will evolve. Both speakers draw on direct experience building models that moved from research curiosities to production systems serving billions, and they use that history to argue for specific technical directions they believe are worth pursuing.

The technical core concerns hardware-algorithm co-design: because data movement is now far more expensive than arithmetic, neural network design has shifted to exploit this reality. They discuss Mixture of Experts not as a way to skip computation but as a structural principle for modular improvement, explain why inference-time compute scaling could unlock new capability tiers, and examine how quantization and other efficiency techniques compound. On the human side, they explore the research methodology of vetting ideas at thousand-times-smaller scale, why individual improvements often fail in combination, and the role of sample efficiency in learning from actions rather than passive observation. The discussion also covers the familiar trade-off between information stored in parameters versus context window, recent work on interpretability by Chris Olah at Anthropic, and a skeptical look at claims that text data is running out. They close with a measured view on AI safety—neither dismissing risk nor accepting inevitability, but treating it as an engineering problem requiring active technical and policy steering, using generation-verification asymmetry as one practical lever.

Sharpest takeaway

Dean and Shazeer argue that AI progress is increasingly driven by tight hardware-algorithm co-design and could enter a self-reinforcing feedback loop, with the next paradigm being a single, organically-grown modular Mixture-of-Experts model that all of Google flows through and that AI itself helps improve.

  • Algorithms now follow hardware because arithmetic is cheap and data movement is expensive, enabling low-precision deep learning
  • Automated exploration of architectures plus inference-time compute scaling could rapidly compound model capability
  • A modular, continually-learned MoE 'blob' would let teams independently improve pieces and distill them down for efficient serving

The claims · ranked46 claims · weighted by value

This asset isn't compiled yet

You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.

0.80

There is a misconception that Mixture of Experts means you avoid retrieving unused expert weights; in reality, because efficient serving uses very large batches of independent requests, all experts are looked at every step, with only a small fraction of the batch sent through each expert—so you still need the whole model in memory (enough HBM to hold it), and running an unused expert at batch size one would be massively inefficient given modern hardware's high operational intensity.

factualhigh valueestablishednovelty 4/4durability 3/4· Noam Shazeer

“there's kind of this misconception running around with Mixture of Experts that, okay, the benefit is that you don't even have to go through those weights in the model. If some expert is unused, it doesn't mean that you don't have to retrieve that memory because, really, in order to be efficient, you're serving at very large batch sizes.”

0.78

Interpretability work by Chris Olah and others at Anthropic trained a very sparse autoencoder that could deduce what characteristics a particular neuron has—finding, for example, a Golden Gate Bridge neuron activated when discussing the Golden Gate Bridge—and similar approaches could work at the expert level, though such deep interpretability may not be necessary if the model's collective output is good, which is one of the beauties of deep learning.

factualhigh valueestablishednovelty 3/4durability 3/4· Jeff Dean

“they trained a very sparse autoencoder and were able to deduce what characteristics some particular neuron in a large language model has, so they found a Golden Gate Bridge neuron that's activated when you're talking about the Golden Gate Bridge.”

0.78

Information memorized in model parameters is memory-efficient (roughly one fact per parameter) but squishy, leading to hallucination and confused facts because trillions of tokens are churned together; information in the context window is sharp and clear thanks to the attention mechanism, but each token costs far more memory (kilobytes to a megabyte of keys and values per token across layers), creating a fundamental trade-off.

factualhigh valueestablishednovelty 3/4durability 3/4· Jeff Dean

“information in the context window, in the input of the model, is really sharp and clear because we have this really nice attention mechanism in transformers.”

0.78

Neural nets are so tolerant of noise that bugs in the code often do nothing, sometimes make the model worse, and sometimes make it better—so misconfigurations slip through silently, and occasionally a bug never tried at scale leads to discovering something new because the model simply learns to work around problems.

factualhigh valueestablishednovelty 3/4durability 3/4· Jeff Dean & Noam Shazeer

“That's actually a bit of a problem with neural nets, is they're so tolerant of noise. You can have things set up kind of wrong in a lot of ways, and they just figure out ways to work around that or learn.”

0.76

Per Rich Sutton's Bitter Lesson, of all the approaches one can try, the two techniques that are incredibly effective are learning and search, because they can be scaled algorithmically or computationally and then outperform other approaches across a broad variety of problems—so search must be part of the solution to spending more inference-time compute.

normativehigh valueestablishednovelty 2/4durability 4/4· Jeff Dean

“the two techniques that are incredibly effective are learning and search. You can apply and scale those algorithmically or computationally, and you often will then get better results than any other kind of approach”

0.76

Google already trains models across multiple metro areas, as disclosed in the Gemini 1.5 tech report, using high-bandwidth but high-latency connections between data centers; this works because each training step takes seconds, so 50-millisecond latency is irrelevant—only bandwidth matters, as long as parameters can be synced and gradients accumulated across data centers within the step time.

factualhigh valueestablishednovelty 3/4durability 2/4· Jeff Dean

“in the Gemini 1.5 tech report, we said we used multiple metro areas and trained with some of the compute in each place. And then a pretty long latency but high bandwidth connection between those data centers, and that works fine.”

0.75

An effective research methodology is to have a roughly 1/1000th scale problem on which you vet 100,000 ideas, then scale up only those that seem promising; the bottleneck is that the largest-scale experiments cannot be sped up and remain N=1 experiments requiring brilliant people to stare at why something works or fails.

normativehigh valueestablishednovelty 2/4durability 3/4· Jeff Dean & Noam Shazeer

“It's very helpful to have a 1/1000th scale problem and then vet 100,000 ideas on that, and then scale up the ones that seem promising.”

0.75

Individually promising improvements often fail to combine well—roughly half the time when integrated—because of unanticipated interactions (for example, the way video data input is handled may interact with how model parameters are updated), so teams must periodically verify that things thought to be good are good together and keep the integrated system clean, since complexity slows things down and adds risk.

factualhigh valueestablishednovelty 2/4durability 3/4· Jeff Dean & Noam Shazeer

“It happens 50% of the time.”

0.74

A giant language model doing a trillion operations per token costs roughly 10^-18 dollars per operation, yielding about a million tokens per dollar; this is about 100 times cheaper than reading a paperback (10,000 tokens per dollar), 10,000 times cheaper than a customer support agent, and a million-plus times cheaper than a software engineer, doctor, or lawyer—so there is enormous headroom to spend far more compute at inference time to make models smarter.

factualhigh valuecontestednovelty 4/4durability 2/4· Noam Shazeer

“you're getting a million tokens to the dollar. I mean compare that to a relatively cheap pastime: you go out and buy a paper book and read it, you're paying 10,000 tokens to the dollar. Talking to a language model is like 100 times cheaper than reading a paperback.”

0.74

An algorithmic improvement for inference efficiency is speculative decoding with drafter models: a small language model predicts several tokens (e.g., four) one at a time, then the big model checks all of them at once and advances past those it agrees with, turning a one-token-wide computation into a four-token-wide parallel computation and avoiding the single-token decode bottleneck, with the big model acting as a verifier.

factualhigh valueestablishednovelty 3/4durability 3/4· Jeff Dean

“you have a really small language model that you do one token at a time when you're decoding, and it predicts four tokens. Then you give that to the big model and you say, "Okay, here are the four tokens the little model came up with. Check which ones you agree with."”

0.73

If machine learning researchers gently steer a more automated search process rather than hand-babysitting experiments, they could vet 100 times more ideas faster, winnowing promising ones at small scale and bringing them into production training—because a brilliant ML idea has only about a 2% chance of working, so trying hundreds, thousands, or a million ideas is how breakthroughs happen, and top labs already have roughly a million times the compute it took to train the original Transformer.

forecasthigh valuecontestednovelty 3/4durability 3/4· Jeff Dean & Noam Shazeer

“If we can do that 100 times faster through those machine learning researchers just gently steering a more automated search process, rather than hand-babysitting lots of experiments themselves, that's going to be really, really good.”

0.73

Humans are sample-efficient partly because they explore the world and take actions and observe results—infants learn gravity by dropping things, which is much harder to learn when not initiating the action—so a model that can take actions as part of its learning process and observe corresponding results would be far better than one passively observing a giant dataset, and much can even be learned from thought experiments and self-play without external input (as in chess from only the rules).

normativehigh valuecontestednovelty 3/4durability 3/4· Jeff Dean & Noam Shazeer

“one of the ways people are so sample efficient is they explore the world and take actions in the world and observe what happens... I think having a model that can take actions as part of its learning process would be just a lot better than just sort of passively observing a giant dataset.”

0.73

Current pre-training likely doesn't extract maximal value from every token, because predicting the next token from previous ones isn't really how people learn; the model should think much harder on some tokens than others (do a lot more work when reaching 'the answer is' than 'the'), learn data forwards and backwards every which way, and use techniques like dropout and many epochs—so the claim that we're running out of text data is doubtful since far more capability can be extracted from existing text.

normativehigh valuecontestednovelty 3/4durability 3/4· Jeff Dean

“I'm not sure we're extracting the maximal value from every token we look at with the current training objective. Maybe we should think a lot harder about some tokens. When you get to "the answer is," maybe the model should, at training time, do a lot more work than when it gets to "the".”

0.71

Rather than the two extreme views—that superhuman AI will overwhelm us or that AI poses no concern at all, both of which treat our role as laissez-faire—the right approach is to actively shape and steer how AI is deployed via policy and technical safeguards, maximizing benefits in areas like education and healthcare while steering away from harms like the system taking unlimited control, treating safety as an engineering problem analogous to rigorous airplane software development.

normativehigh valuecontestednovelty 2/4durability 3/4· Jeff Dean

“what we're going to do is try to shape and steer the way in which AI is deployed in the world so that it is, you know, maximally beneficial in the areas that we want to capture and benefit from”

0.70

Contrary to claims that it is hard to decode what each expert does, in the original Mixture of Experts work the experts were relatively easy to understand—one expert handled words referring to cylindrical objects, another was good at dates and times—though you don't need that human understanding to operate the model, since a learned router selects experts based on the example.

factualhigh valuecontestednovelty 3/4durability 3/4· Noam Shazeer

“in the past, I found experts to be relatively easy to understand. I mean, the first Mixture of Experts paper, you could just look at the experts.”

0.70

Deep learning took off largely because arithmetic became very cheap while moving data around became comparatively expensive; deep learning can be built from matrix multiplications that are N-cubed operations on only N-squared bytes of data communication, so algorithms have evolved to exploit this hardware economics.

causalhigh valuecontestednovelty 3/4durability 3/4· Jeff Dean & Noam Shazeer

“at this point, arithmetic is very, very cheap, and moving data around is comparatively much more expensive. So pretty much all of deep learning has taken off roughly because of that.”

0.70

Current models can break a complex problem into about five to ten steps and give a perfect answer maybe 80% of the time, but the major capability leap they are aiming for is reliably handling tasks decomposed into 100 to 1,000 sub-steps at 90% success—a jump that would dramatically expand what the models can do and does not necessarily require new hardware.

forecasthigh valuecontestednovelty 3/4durability 3/4· Jeff Dean

“if you could go from, yeah, 80% of the time it can give you a perfect answer to something that's ten steps long to something that 90% of the time can give you a perfect answer to something that's 100 to 1,000 steps of sub-problem long, that would be an amazing improvement”

0.69

The 2007 Google machine translation system won a DARPA contest by a large BLEU-score margin but took 12 hours per sentence because it did 100,000 disk seeks per word; by designing an in-memory compressed representation of two-trillion-word five-gram data across 200 machines with a batched lookup API, Jeff reduced translation time from roughly a night per sentence to about 100 milliseconds.

factualhigh valueestablishednovelty 3/4durability 2/4· Jeff Dean

“That enabled us to go from taking a night to translate a sentence to basically doing something in 100 milliseconds or something.”

0.68

The naive attention algorithm is quadratic, so it can barely be made to work for millions of tokens even on substantial hardware and has no hope of naively scaling to trillions of tokens; reaching trillion-token context (the entire internet, all a user's personal data, all of Google's source code) will require interesting algorithmic approximations that let the model conceptually attend to vastly more tokens.

factualhigh valueestablishednovelty 2/4durability 3/4· Jeff Dean

“the naive attention algorithm is quadratic. You can barely make it work on a fair bit of hardware for millions of tokens, but there's no hope of making that just naively go to trillions of tokens.”

0.68

Over the last decade, general-purpose CPU-based machine scaling has slowed dramatically: fabrication process improvements now take three years instead of two, and multi-core architectural improvements no longer give the boosts seen 20-to-10 years ago, while specialized ML accelerators like TPUs and ML-focused GPUs now deliver high performance and efficiency for modern computations.

factualhigh valueestablishednovelty 2/4durability 3/4· Jeff Dean

“the general-purpose CPU-based machine scaling has not been as good, like the fabrication process improvements are now taking three years instead of every two years.”

0.68

An early Google Brain unsupervised model trained on 10 million randomly selected YouTube frames across 2,000 computers using 16,000 cores spontaneously developed a high-level neuron that activated for cat faces—never having been told what a cat was—and the same model, about 50x bigger than prior models, improved the state of the art on the ImageNet 20,000-category challenge by 60% relative, demonstrating that scaling neural nets works.

factualhigh valueestablishednovelty 2/4durability 3/4· Jeff Dean

“that model was actually able to build a representation at the highest level where one neuron would get excited by images of cats. It had never been told what a cat was”

0.68

Neural nets were the right abstraction even in 1990—solving tiny toy problems no other approach could—but required roughly a million times more compute before they worked on real problems, which only became available in the 2008-2010 timeframe thanks to Moore's Law.

factualhigh valueestablishednovelty 2/4durability 3/4· Jeff Dean

“it turned out we needed about a million times more compute before they really started to work for real problems, but then starting in the late 2008, 2009, 2010 timeframe, we started to have enough compute”

0.68

Because analyzing text is easier than generating it—it is easier to recognize something than to produce it—language models' ability to analyze other models' output and flag what is problematic or dangerous will be a key part of solving AI control and safety problems, with models used to check themselves and each other.

factualhigh valuecontestednovelty 2/4durability 3/4· Noam Shazeer

“the good news is that analyzing text seems to be easier than generating text. So I believe that the ability of language models to actually analyze language model output and figure out what is problematic or dangerous will actually be the solution to a lot of these control issues.”

0.66

Language modeling—producing a probability distribution over the next word—is an ideal problem because it is simple to state, has essentially infinite self-supervised training data from the text of the web, and is AI-complete: doing it well means you can do essentially anything.

factualhigh valuecontestednovelty 3/4durability 3/4· Noam Shazeer

“And then it's AI complete. If you can do a great job of that, then you can pretty much do anything.”

0.66

Google moved from asynchronous CPU training to fully synchronous TPU training because synchronous runs are replicable—results no longer depend on incidental factors like a web crawler running on the same machine—making it an easier mental model where only your algorithm fights you rather than asynchrony plus the algorithm both battling you, though at greater scale some asynchrony may return.

factualhigh valueestablishednovelty 2/4durability 2/4· Noam Shazeer

“It was so pleasant to go from asynchronous to synchronous because your experiments are now replicable, rather than your results depend on whether there was a web crawler running on the same machine.”

0.65

Despite the appeal of gluing models together, in the current period of rapid progress there is a countervailing scientific interest: to run controlled experiments comparing one thing to another (which helps decide what to build), it is often best to start from scratch and compare complete training runs—though versioned modularity, freezing a model and swapping in a variant module, could capture much of that benefit while making research cheaper and faster.

normativehigh valuecontestednovelty 3/4durability 2/4· Noam Shazeer

“if you want to do sort of controlled experiments... it's often best to just start from scratch so you can compare one complete training run to another complete training run at the practical level because it helps us figure out what to build in the future.”

0.65

Google has done further training of a Gemini model on its internal code base, and according to Sundar Pichai roughly 25% of the characters being checked into Google's code base are now generated by AI-based coding models with human oversight.

factualhigh valueestablishednovelty 2/4durability 1/4· Jeff Dean

“Sundar, I think, has said that 25% of the characters that we're checking into our code base these days are generated by our AI-based coding models with kind of human oversight.”

0.64

Improvements from one generation of these models to the next are partially driven by hardware and larger scale, but equally and perhaps even more so by major algorithmic improvements and changes in model architecture and training data mix that make the model better per FLOP applied.

causalhigh valuecontestednovelty 2/4durability 3/4· Jeff Dean

“the improvements from generation to generation of these models often are partially driven by hardware and larger scale, but equally and perhaps even more so driven by major algorithmic improvements and major changes in the model architecture”

0.63

Future inference compute demand could grow by multiple orders of magnitude compounding together: inference-time scaling can make a single request 50-1000x more computationally intensive for the same output; adoption could rise from ~10-20% of computer users today to 100% with heavier use (another order of magnitude or two); and models will get bigger (another order or two)—so extremely efficient inference hardware is essential.

forecasthigh valuecontestednovelty 2/4durability 2/4· Jeff Dean

“what's currently like one request to generate some tokens now becomes 50 or 100 or 1000 times as computationally intensive... As that pushes towards 100%... that's going to be another order of magnitude or two of scaling.”

0.63

Google had capable language model and chatbot technology (Meena, used internally before ChatGPT) but delayed public release because from a search perspective these models hallucinate and need ~100% factuality, and had safety issues; Google underappreciated how useful they were for non-search tasks like drafting notes or summarizing, but taking time to release quite capable chatbots was not a bad path.

normativehigh valuecontestednovelty 2/4durability 2/4· Jeff Dean

“what we didn't quite appreciate was how useful they could be for things you wouldn't ask a search engine... I think we took our time and got to the point where we actually released quite capable chatbots... I think that's actually not a bad path to have taken.”

0.63

Model precision has dropped far beyond what supercomputing experts of 20 years ago would have believed possible: inference now uses INT4 or FP4, and some people quantize models down to two bits or even a single bit (with a sign bit per group), a trend enabled by hardware-algorithm co-design that improves throughput-to-cost ratios.

factualhigh valueestablishednovelty 2/4durability 2/4· Jeff Dean

“People are now using INT4 or FP4, which sounded like, if you said to someone like we're going to use FP4, like a supercomputing floating point person 20 years ago, they'd be like, "What? That's crazy."”

0.61

Future models should move from the current regular structure—where every expert is the same size and paths merge back together quickly—toward a more organic structure with specialized portions, like a brain has areas good at different things, where you add capacity to grow expertise where needed and let the model's connectivity adapt to the hardware's connectivity (dense within a chip, sparser across chips, minimal across metro areas, sending only the most salient representations over long-distance links).

normativehigh valuespeaker onlynovelty 4/4durability 3/4· Jeff Dean

“I feel like this kind of more organic growth of expertise, and when you want more expertise of that, you add some more capacity to the model there and let it learn a bit more on that kind of thing.”

0.61

A practical safeguard against runaway self-improvement is to keep a human in charge of the algorithmic research loop—the system explores the space and produces results, but a person decides whether to incorporate a given learning algorithm or change into the core code base—plus exposing capabilities only through APIs or interfaces that allow monitoring and boundaries, so you get the benefits of self-improvement without letting the system go fully autonomous.

normativehigh valuecontestednovelty 2/4durability 3/4· Jeff Dean

“you can put in safeguards like that that enable us to get the benefits of the system that can sort of improve or kind of self-improve with human oversight, uh, without necessarily letting the system go full-on self-improving”

0.61

Publishing the Transformer and other research that helped competitors was net positive because AI is far from a fixed-pie situation: the current state is about as far from fixed-pie as possible, with orders-of-magnitude improvements coming in GDP, health, and wealth, and seeing the scale of the opportunity—often reflected in what other companies do—was itself necessary to recognize it.

normativehigh valuecontestednovelty 2/4durability 3/4· Noam Shazeer

“it's not a fixed pie. The current state of the world is pretty much as far from fixed pie as you can get. I think we're going to see orders of magnitude of improvements in GDP, health, wealth, and anything else you can think of.”

0.58

You are starting to see many Google uses of Gemini models that aren't fine-tuned but just given instructions per use case, pointing toward a future where the underlying model is shared across more and more services—conceivably a base model with customized modules (some access-restricted, like an internal Google-only module or per-company modules served via cloud APIs) such that products like Search, Images, and Gmail all flow through one huge specialized mixture of experts.

forecasthigh valuespeaker onlynovelty 4/4durability 2/4· Jeff Dean

“You're starting to see some of this by having a lot of uses of Gemini models across Google that are not necessarily fine-tuned. They're just given instructions for this particular use case”

0.57

Reasoning from first principles, people will spend some fraction of world GDP on AI, and world GDP itself will almost certainly rise two orders of magnitude (100x) above today because of legions of artificial engineers improving things, alongside likely solutions to unlimited energy and carbon, so the compute dedicated to assisting each person will be astronomical.

forecasthigh valuefringenovelty 3/4durability 1/4· Noam Shazeer

“The world GDP is almost certainly going to go way, way up, two orders of magnitude higher than it is today, due to the fact that we have all of these artificial engineers working on improving things.”

0.57

If model pieces could be developed somewhat independently—small teams creating great training data and training a modular piece (e.g., for Southeast Asian languages or Haskell reasoning) that hooks onto a larger model—then 100 teams around Google or people worldwide could collectively and continually improve one model, decomposing the current monolithic pre-training process and giving a software-engineering benefit, which is itself a form of continual learning.

forecasthigh valuespeaker onlynovelty 3/4durability 3/4· Jeff Dean

“It would be really great if we could have a small set of people who care about a particular subset of languages go off and create really good training data, train a modular piece of a model that we can then hook up to a larger model”

0.57

Quantization only gets adopted through co-design where someone sees the whole picture: in isolation, algorithm designers reject low precision as risky and irritating, and chip designers defer to those algorithm designers, so without a holistic view of how throughput-to-cost can improve dramatically, the beneficial low-precision path never gets taken.

causalhigh valuespeaker onlynovelty 3/4durability 3/4· Noam Shazeer

“It really has to be a co-design thing because, if the algorithm designer doesn't realize that you can get greatly improved performance, throughput, with the lower precision, of course, the algorithm designer is going to say, "Of course, I don't want low precision. That introduces risk."”

0.57

Google Brain used a bottom-up 'UBI' compute allocation where everyone got one credit and could pool them, which incentivized people to drop failing projects; Gemini has been mostly top-down, yielding more collaboration and less duplicated work but creating an incentive for every group to claim success and request more chips rather than admit something isn't working—so the future needs a blend of both to incentivize collaboration and flexibility.

normativehigh valuespeaker onlynovelty 3/4durability 3/4· Noam Shazeer

“Gemini has been mostly top-down, which has been very good in some sense because it has led to a lot more collaboration... But on the other hand, it does lead to some incentive to say, "Hey, what I'm doing is working great."”

0.56

Distillation could run continuously at the module level: each module keeps a big version and a small version, the small one continually distills from the big one; once the small version is caught up you delete the big one, add more parameter capacity, and train it on more data to learn what the small distilled version doesn't know, repeating this process in thousands of places in a modular model in the background.

forecasthigh valuespeaker onlynovelty 4/4durability 3/4· Jeff Dean

“It has a really big one. It's got a much smaller one that is continually distilling into the small version. And then the small version, once that's finished, you sort of delete the big one and you add a bunch more parameter capacity.”

0.53

The chip design process currently takes about 12-18 months with around 150 people before handing to TSMC (which takes 3-5 months to fab), but if a much more automated search process explored the whole design space with feedback from all aspects of chip design, a few people could design chips in a much shorter timeframe, making fabrication the dominant portion, enabling more specialization and letting designers target only 6-9 months ahead instead of 2-2.5 years.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Jeff Dean

“instead of taking 12 to 18 months to design the chip with 150 people, you could shrink that to a few people with a much more automated search process, exploring the whole design space of chips”

0.52

Larry Page reportedly said Google's second biggest cost is taxes and its biggest cost is opportunity cost, framing innovation as identifying the opportunities you are missing—for example, realizing chip area filled with too few arithmetic units represents a massive missed opportunity to do orders of magnitude more arithmetic.

factualhigh valuespeaker onlynovelty 2/4durability 3/4· Noam Shazeer

“this is something like Larry Page, I think, used to always say: "Our second biggest cost is taxes, and our biggest cost is opportunity costs."”

0.48

If memory and data flow had become cheap while arithmetic stayed expensive—the inverse of what actually happened—AI today would rely on many lookups into very large memories rather than dense arithmetic, possibly resembling AI of 20 years ago but in the opposite direction.

forecasthigh valuespeaker onlynovelty 3/4durability 2/4· Jeff Dean

“You'd have a lot more lookups into very large memories.”

0.43

The trick to career longevity and breadth across many fields is to dive into new areas by paying attention to the evolving research landscape and working with five or six colleagues who have different expertise, enabling the group to collectively do what none could individually, while their expertise rubs off on you—expanding your toolset for the next problem.

normativeestablishednovelty 1/4durability 3/4· Jeff Dean

“often working with a set of five or six colleagues who have different expertise than you do. It enables you to collectively do something that none of you could do individually. Then some of their expertise rubs off on you”

0.35

Major research breakthroughs are best understood as partially in the air: several separate existing research ideas are squinted at when solving a new problem, drawn on for inspiration, and combined with some genuinely new unsolved aspect, so that morphing existing ideas plus new contributions yields a result that did not exist before.

definitionestablishednovelty 1/4durability 3/4· Jeff Dean

“I like to think of a lot of ideas as being partially in the air, where there are a few different, maybe separate research ideas that one is squinting at when you're trying to solve a new problem.”

0.26

Google's mission was always broader than information retrieval: 'organize the world's information and make it universally accessible and useful' is a broad mandate that now extends to creating new information from guidance (drafting a letter, summarizing a video), understanding all modalities (text, video, lidar, genomics, health data) including non-human-oriented ones, and synthesizing answers from many sources where no single source suffices.

normativespeaker onlynovelty 1/4durability 2/4· Jeff Dean

“I think we're an "organize the world's information" company, and that's broader than information retrieval. Maybe: "organizing and creating new information from some guidance you give it".”