Lecture

Retrieval-Augmented Generation

18 Aug 2024 • Richard Kuo

Introduction to RAG, LlamaIndex, examples.

RAG (檢索增強生成)

RAG Survey

A Guide on 12 Tuning Strategies for Production-Ready RAG Applications

NLP • Retrieval Augmented Generation

	Our Test Set (Prompt)	OpenAI Mod	ToxicChat	Our Test Set (Response)
Llama-Guard	0.945	0.847	0.626	0.953
OpenAI API	0.764	0.856	0.588	0.769
Perspective API	0.728	0.787	0.532	0.699

Contextual Retrieval RAG

Anthropic Prompt Caching

Standard RAG

Contextual Retrieval RAG

Blog: Implementing Contextual Retrieval in RAG pipeline

Frameworks

LlamaIndex

Code: https://github.com/run-llama/llama_index
Docs:

LangChain

langchain-core: Base abstractions and LangChain Expression Language.
Integration packages (e.g. langchain-openai, langchain-anthropic, etc.): Important integrations have been split into lightweight packages that are co-maintained by the LangChain team and the integration developers.
langchain: Chains, agents, and retrieval strategies that make up an application’s cognitive architecture.
langchain-community: Third-party integrations that are community maintained.
LangGraph: Build robust and stateful multi-actor applications with LLMs by modeling steps as edges and nodes in a graph. Integrates smoothly with LangChain, but can be used without it. To learn more about LangGraph, check out our first LangChain Academy course, Introduction to LangGraph, available here.
LangGraph Platform: Deploy LLM applications built with LangGraph to production.
LangSmith: A developer platform that lets you debug, test, evaluate, and monitor LLM applications.

Kaggle: https://www.kaggle.com/code/rkuo2000/rag-with-langchain

Building RAG-based LLM Applications for Production

https://github.com/ray-project/llm-applications/blob/main/notebooks/rag.ipynb
(1)將外部文件做分塊(chunking)再分詞(tokenize)轉成token
(2)利用嵌入模型，將token做嵌入(embeds)運算，轉成向量，儲存至向量資料庫(Vector Database)並索引(Indexes)
(3)用戶提出問題，向量資料庫將問題字串轉成向量(利用前一個步驟的嵌入模型)，再透過餘弦(Cosine)相似度或歐氏距離演算法來搜尋資料庫裡的近似資料
(4)將用戶的問題、資料庫查詢結果一起放進Prompt(提示)，交由LLM推理出最終答案
以上是基本的RAG流程，利用Langchain或LlamaIndex或Haystack之類的應用程式開發框架，大概用不到一百行的程式碼就能做掉(含LLM的裝載)。

Anyscale剛剛發布的一篇精彩好文，裡頭介紹了很多提升RAG成效的高段技巧，內容包括：
🚀從頭開始建構基於RAG的LLM應用程式。
🚀 在具有不同運算資源的多個工作人員之間擴展主要工作負載（載入、分塊、嵌入、索引、服務等）。
🚀評估應用程式的不同配置，以最佳化每個元件（例如retrieval_score）和整體效能（quality_score）。
🚀 透過開源和閉源LLM實作混合代理路由方法，以建立效能最佳且最具成本效益的應用程式。
🚀以高擴展性與高可用性的方式為應用程式提供服務。
🚀了解微調、提示工程、詞彙搜尋(lexical search)、重新排名、資料飛輪(data flywheel)等方法如何影響應用程式的效能。