RAG Frameworks

Frameworks for building retrieval-augmented generation applications.

Comparison

Framework Best For Learning Curve Flexibility

LangChain Agents, chains, tools Steeper Highest

LlamaIndex Data indexing, simple RAG Gentle Medium

Sentence Transformers Custom embeddings Low High

LangChain

Orchestration framework for building complex LLM applications.

Core concepts:

• Chains: Sequential operations (retrieve → prompt → generate)

• Agents: LLM decides which tools to use

• LCEL: Declarative pipeline syntax with | operator

• Retrievers: Abstract interface to vector stores

Strengths: Rich ecosystem, many integrations, agent capabilities Limitations: Abstractions can be confusing, rapid API changes

Key concept: LCEL (LangChain Expression Language) for composable pipelines.

LlamaIndex

Data framework focused on connecting LLMs to external data.

Core concepts:

• Documents → Nodes: Automatic chunking and indexing

• Index types: Vector, keyword, tree, knowledge graph

• Query engines: Retrieve and synthesize answers

• Chat engines: Stateful conversation over data

Strengths: Simple API, great for document QA, data connectors Limitations: Less flexible for complex agent workflows

Key concept: "Load data, index it, query it" - simpler mental model than LangChain.

Sentence Transformers

Generate high-quality embeddings for semantic similarity.

Popular models:

Model Dimensions Quality Speed

all-MiniLM-L6-v2 384 Good Fast

all-mpnet-base-v2 768 Better Medium

e5-large-v2 1024 Best Slow

Key concept: Bi-encoder architecture - encode query and documents separately, compare with cosine similarity.

RAG Architecture Patterns

Pattern Description When to Use

Naive RAG Retrieve top-k, stuff in prompt Simple QA

Parent-Child Retrieve chunks, return parent docs Context preservation

Hybrid Search Vector + keyword search Better recall

Re-ranking Retrieve many, re-rank with cross-encoder Higher precision

Query Expansion Generate variations of query Ambiguous queries

Decision Guide

Scenario Recommendation

Simple document QA LlamaIndex

Complex agents/tools LangChain

Custom embedding pipeline Sentence Transformers

Production RAG LangChain or custom

Quick prototype LlamaIndex

Maximum control Build custom with Sentence Transformers

Resources

• LangChain: https://python.langchain.com

• LlamaIndex: https://docs.llamaindex.ai

• Sentence Transformers: https://sbert.net