
Ep.4 - Chunking Strategies Explained: How to Structure Data for LLMs #RAG #llm #ai #aiapplications
What this covers
This eight-minute video from Farabi Labs, presented by a single speaker whose name isn't given in the material (the publisher credits Rabea Abdelwahab as author), is a walkthrough of how to split a corpus for RAG. It starts with the retrieval side of the pipeline, where data is pulled from a knowledge base or external store and how well that works depends on how the corpus was divided. It then goes through four named strategies in turn, and closes with a testing routine and a hybrid approach.
The four strategies are each tied to a use case. Fixed-size chunking, such as uniform 500-word pieces, is the simplest and suits FAQ chatbots or e-commerce catalogs with standardized attributes. Sliding window chunking keeps overlapping text between neighbors, which the speaker pairs with customer service bots where conversation history matters. Structured chunking follows headings, paragraphs and sections, and the example is an earnings report split into quarterly performance, revenue breakdown and future projections. Semantic chunking is the most involved: chunks are embedded, consecutive ones are compared, and a similarity threshold decides whether to merge or separate them.
The more contentious or practical turns come around the edges of those definitions. The speaker says semantic chunking costs extra computation on the LLM and vector database side, justified only when precision and contextual integrity are priorities. Bad chunking can fail in two directions: too much irrelevant detail dilutes the answer, while too little context pushes the model back onto its training data. The advice is a three-step loop of analyzing the data, choosing a strategy, and measuring the result. A hybrid approach is also described, applying structural chunking to well-defined sections and semantic chunking to unstructured community discussion within one document. The video ends by noting that newer chunking strategies are still emerging.
The speaker argues that effective RAG systems require careful chunking strategy selection matched to data structure and use case, with testing and iteration being essential to maximize LLM performance.
- Chunk size and structure directly impact whether LLMs use retrieved context or rely on training data
- Different data types and applications benefit from different chunking approaches (fixed, sliding window, structured, semantic)
- Hybrid chunking strategies can combine multiple approaches to handle heterogeneous content within single documents
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Semantic chunking uses vector similarity to identify context boundaries by embedding chunks, measuring similarity between consecutive chunks, and merging chunks that exceed a similarity threshold while separating chunks that fall below it, thereby creating chunks with genuine semantic coherence rather than arbitrary size or structure boundaries.
“semantic chunking which is another strategy and this one is a little bit Advanced with the idea that if we do the example where there is a book with 20 pages and those 20 pages are going to be uh split originally into uh a certain mixed sized length chunks in other words and then all these are uh put into a vector database with it their own embeddings and then we measure the similarity with a certain threshold between all these chunks till we find out that something is breaking that threshold and it's not as similar anymore we understand that the context is different and then whoever those similar chunks are going to be generated into and embedded by itself while the rest that is far away from it is going to be also embedded by itself”
Hybrid chunking approaches—combining multiple chunking strategies within a single application—can be used to handle documents with heterogeneous content, such as applying structural chunking to well-defined sections while using semantic chunking for unstructured community discussions, thereby optimizing context integrity across the entire document.
“one of the strategies could be to have an hybrid approach of Chunk um hybrid approach of chunking strategies that can be used together to help accomplish the best context for your llm application so let's say you have a document where this document has a lot of structure in place that is well known so a structural chunking mechanism or strategy can help a lot but at the same time there is a place where there is um Community discussions in that document and it will benefit a lot from semantic chunking where it will relate this the uh in places that the structural chunking is not going to help so in other words a hybrid approach could be also uh the way to go for your llm application”
Sliding window chunking preserves overlapping text between consecutive chunks, ensuring continuity of context across chunk boundaries, and is particularly useful for applications like customer service bots where understanding conversation history is critical to answering questions correctly.
“one of the other strategies of chunking is the sliding window chunking and the idea is it it's preserving some window of text that's going to overlap between one chunk and another and that assures that continuity of context between multiple chunks over the overlapping nature of these chunks makes sure that the context keeps going while you're going from One content piece to another one example for the sliding window chunking is a customer service bot that you could be helping the customer troubleshoot so the continuity of the context from the user is important for you to answer the questions properly so a sliding window chunking mechanism can help because it will always have an overlap of what happened before in the history of that chat or um the conversation”
To maximize the benefit of chunking in RAG systems, practitioners should follow a three-step process: first, analyze the data to understand its structure and characteristics; second, select a chunking strategy to implement; third, iteratively test and measure the impact of the chosen strategy on application performance.
“to benefit from chunking in your rag system it's best to do three things one is first to analyze your data and actually see if it has a certain structure in place or is it simple as pretty flat text that you can try to test out a certain chunking strategy with so the second step is going to be choosing one of the chunking strategy to actually start using and then the third just keep testing and iterating and see what kind of impact that chunking strategy is making to your application”
In RAG systems, the effectiveness of LLM responses depends heavily on the chunking strategy used to break down the corpus, because poorly chunked data can either provide too much irrelevant detail that waters down the model's response or too little context, forcing the LLM to rely on its training data rather than the retrieved information.
“we need to use the best way to split the content for our applications so we need to think of what our AI applications is doing or llm application is doing and understand that we maybe cut a chunk that is not too large because if it's too large it's going to hold a lot of details and that could also water down what the large language model can come back with or we don't want it to be very short that means it's not going to be enough context and the large language model has a higher probability of trying to use its own training data”
Semantic chunking requires additional computational overhead on the LLM and vector database side to generate embeddings and perform similarity comparisons, but this added work is justified when higher precision and contextual integrity are priorities for the application.
“so in other words it's more work on the llm and the vector database side to create those embeddings do the comparison and once the comparison is done done now you have really good semantic meaning captured that you understand how chunks differ from each other or are they in different context so they don't have a lot of similarity between each other and that is an extra step definitely but it's worth it if you need more precision and more contextual Integrity”
Structured chunking leverages the inherent document structure (headings, paragraphs, sections) to break content into chunks, preserves contextual integrity by respecting the author's organizational intent, and works best when documents follow repetitive, well-defined structural patterns, such as breaking down earnings reports into predictable sections like quarterly performance, revenue breakdown, and future projections.
“another strategy is the structured Chun in which also depends heavily on your content because you can Leverage The inherit nature of the document itself so its headings its paragraphs its sections so if you have documents that have a certain structure and it's a repetitive pattern in your content then structural chunking makes sense as a strategy which will preserve also the contextual Integrity of your content one example for structural chunking is if you have a an earnings report a public earnings report that you want to break down into your rack system now you have certain sections that are well known and you can break down the chunks into these whether it's whether it's quarterly performance the revenue breakdown the future projections so you have well-known sections that you can actually chunk uh each one by itself from that document”
Embeddings capture semantic meaning in the content being chunked, and selecting the right chunking strategy is necessary to ensure that embeddings preserve the most effective pieces of data for the LLM to answer questions.
“we talked before in a previous video about embeddings and how it catches the semantic meaning but for you to actually embed the best piece of data that your rack system is going to maximize on its potential we need to use the best way to split the content for our applications”
Fixed size chunking—dividing text into uniform chunks of a set word count (e.g., 500 words each)—is the simplest chunking strategy and is appropriate for applications where content naturally fits uniform sizes, such as FAQ chatbots with question-answer pairs or e-commerce product catalogs with standardized attribute structures.
“the first chunking strategy and the easiest one is the fixed size chunking strategy in other words you are saying um whatever body of text or content I have I'm going to split it into 500 Words each and each one is going to be a chunk that I'm going to be embedding into my Vector store which is can serve a purpose in some of the LM app applications out there when you think about say you are building a chat bot uh that is llm based for an FAQ so you can have the question answer combination as one chunk for each one and most probably they are less than 500 words or let's say if you're using it for um e-commerce uh product details so each product details most probably you have the common size of attributes etc for each product so that could be a fake size also”
Additional chunking strategies beyond fixed-size, sliding window, structured, and semantic chunking are being introduced and developed, suggesting the field is actively evolving, and practitioners should select the strategy that best aligns with their specific data structure and application goals.
“there's definitely a lot of more of the chunking strategies and uh there's also techniques that are being like introduced newly also so the types not I mentioned are the ones that the most common and definitely dig more into what makes sense for your llm application what helps you accomplish the best context Integrity for users and your customers”
In RAG systems, the retrieval component retrieves data from a knowledge base or external data store, and the effectiveness of this retrieval depends on how the corpus is split and structured.
“in rack systems we heavily depend on the idea that we are pulling data or retrieving data from a knowledge base or an external data store and we want to make sure that the large language model is getting enough and the most effective piece of information needed for it to answer the questions that you have”