episode
In RL, a single sequence of interaction from start to terminal reward; here used to note imitation learning has a token-long episode.
As defined in Some thoughts on the Sutton interview
Where it comes up · 2
- Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI9 claims
- Some thoughts on the Sutton interview2 claims
Most important claims · 10 of 11Search all claims for “episode” →
- Imitation learning is not categorically different from RL; it is just short-horizon RL where the episode is a single token long: the LLM conjectures the next token from its world understanding and receives reward proportional to how well it predicted the true token.Some thoughts on the Sutton interview
- An LLM being RLed on outcome-based rewards learns only O(1) bits per episode even though the episode may be tens of thousands of tokens long, whereas animals extract far more information from interacting with their environment than just the end-of-episode reward signal.Some thoughts on the Sutton interview
- The Graffiti graph exposes which episodes have a particular metadata tag (e.g., 'verified'), but the agent must execute the business logic—applying which facts require ALL parents verified vs. ANY unverified parent blocking action—because this is situational and context-dependent.Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI
- Lineage mapping shows which facts are derived from source data to be deleted; if only some source data needs deletion (not all), a fact survives deletion if other parent episodes still support it, but a fact is deleted only if no remaining episodes support it.Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI
- Using graph metadata projection, Graffiti can model classifications that span many different episodes and facts derived from them (e.g., an 'EHR tag' inherited by all entities and facts derived from an EHR episode), allowing agents to filter facts by veracity source (e.g., retrieve only facts from verified clinical sources).Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI
- When new data contradicts existing data and mutates a fact, Graffiti captures this by adding an invalid date to the mutated edge and noting the source episodes that resulted in the edge mutating against the fact.Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI
- Provenance in a context store containing facts can be modeled on a graph as relationships: source data nodes become episodes, extracted entities become nodes, and edges between entities represent facts that can be hydrated (made readable) as statements (e.g., 'Patient has a penicillin allergy').Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI
- For a fact with multiple parents (sources), veracity policies differ based on context: for a penicillin allergy in a prescription scenario, the agent should block the prescription if any parent source is unverified (because missing one unverified source could be lethal), but for a consent-on-file fact, every parent source must be verified (because the entire consent chain must be verified or the consent is invalid).Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI
- Graffiti's temporal graph support is built in at the database level, enabling agents to understand provenance across facts spanning multiple episodes (50+ edges/relationships) using a separate data structure from the graph itself.Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI
- The add_episode method in Graffiti runs a complicated pipeline on episode ingestion with multiple high-level steps: structured extraction (entities and facts), deduplication and deconfliction of entities and facts, and mutation tracking (invalidating facts when contradicted by new data).Citation Needed: Provenance for LLM-Built Knowledge Graphs — Daniel Chalef, Zep AI