Gemini was designed as multimodal from the beginning, ingesting text, images, video, and audio and converting them into sequences of tokens that are processed by a transformer-based model, with separate decoding paths for text generation and image pixel generation that can be initialized from the transformer's learned state.

factualpending

Speaker

Justin Basilico

Evidence Quote

“we turn that into a sequence of tokens that we then train a transformer-based model on... we have a couple of different decoding paths... initialize the decoder with the state uh that the transformer has learned, and then we can sort of go from that state to a full uh you know, set of pixels for an image”

Source

Jeff Dean (Google): Exciting Trends in Machine Learning— Rice Ken Kennedy Institute
Created: 8/11/2026, 1:28:51 AM

Your notes

Loading notes...