Building, Evaluating, and Optimizing your RAG App for Production | Jerry Liu

Overview

In this H2O World session, Jerry Liu, co-founder and COO of LlamaIndex, walks through how to move a retrieval augmented generation (RAG) system from a five line prototype to a production ready application. He breaks down the two stage RAG stack, data ingestion and querying, the most common failure modes in naive RAG, bad retrieval and bad generation, and how to evaluate systems using both retrieval specific and end to end metrics. The session closes with a tour of optimization levers, from chunk sizing and prompt engineering to embedding model choice and small to big retrieval.