Vector Databases Are Dead. Use This Instead

Summary

The video argues that vector databases are not dead, but have shifted from being the default center of AI retrieval to one tool among several. It compares three alternatives or complements: agentic search, live application connectors, and precompiled knowledge files.

1. How vector databases fit into retrieval

A vector database stores text as numerical embeddings and retrieves passages whose meaning is similar to a query. The video describes TurboPuffer’s newer design as giving documents regular IDs, with vectors used alongside keyword search and filters rather than acting as the core organizing structure. It notes that this version was not yet in production at the time discussed.

2. Agentic search: better on hard questions, slower and costlier

A traditional RAG pipeline retrieves a fixed set of passages once and inserts them into the prompt. An agentic system can search, inspect results, identify gaps, and search again.

• On Google’s FRAMES benchmark—824 questions across 12,000 Wikipedia articles—the agent scored 92.7%, compared with 78.9% for the best of 18 fixed pipelines. With the ideal articles supplied, it scored 93%.
• On questions requiring at least five articles, the agent scored 90%, versus 62% for the pipeline.
• The agent cost about three times as much per question. Its slowest response among 20 took 134 seconds, compared with 54 seconds for the pipeline; 7% of questions accounted for 37% of its budget.
• The video also cites an independent study that found agents used more than three times as many input tokens and took 50% longer, while a conventional reranker performed better at ranking.

Takeaway: Repeated search can improve answers to difficult, multi-step questions, but each search round adds cost and delay.

3. Keyword search can outperform vectors on company data

The video highlights BM25, a traditional keyword-ranking method that looks for exact word overlap.

• On a benchmark involving 500,000 business documents from Slack, Gmail, Jira, Confluence, and GitHub, BM25 scored 68.8%, versus 51.4% for vector search.
• The proposed reason is that general-purpose embedding models may not understand company-specific identifiers, abbreviations, and ticket numbers.
• The results were not uniform: another benchmark cited in the video had an agent using embedding search score 70.1%, compared with 55.9% using BM25.
• Cursor is also reported to have improved its coding agent’s accuracy by 12.5% after adding semantic search on top of grep.

Takeaway: Exact keyword search is a strong, inexpensive starting point for company information; semantic search can help with fuzzy or meaning-based queries. Combining them may be more useful than relying on either alone.

4. Live connectors: current data without building an index

MCP and similar federated connectors let an AI query applications such as Notion or Salesforce directly, using the user’s access rights instead of copying data into a central index. The video cites Microsoft Copilot’s federated connectors and Anthropic’s connector marketplace as examples.

The trade-offs are that responses depend on the speed and search quality of each connected application. The video cites Glean pricing live-connector tasks at $2.98, compared with $0.58 for its indexed approach, with the connector workflow using roughly three times as many tokens. It also notes that app-provider policies can limit how much data organizations are allowed to export or sync.

Takeaway: Live access can suit data that changes frequently or is difficult to index, but it may be slower, less consistent, and more expensive.

5. Compiled knowledge files: useful, but potentially costly

The video revisits Google’s Open Knowledge Format (OKF), which compiles information into Markdown files for an AI to consult.

• An August study found that creating the compiled wiki used 100 times more tokens than building a RAG index. It also reported 21 times more tokens per query, with no break-even point in its analysis.
• The wiki nevertheless performed better in that test: 40% of its statements were correct, compared with 19% for RAG.
• The video cautions that this was a small study by one author, with only 30 questions.
• It says the format’s development appeared less active, while noting related ideas in products such as LangChain’s Open Wiki and Pinecone’s Nexus.

Takeaway: Precompiled knowledge may improve answer quality for stable, valuable information, but the cited evidence suggests substantial token costs and is limited in scope.

Practical guidance from the video

• Use a single-pass RAG pipeline when queries are simple, frequent, and cost or speed matters most.
• For agents, begin with keyword search, then add vectors for fuzzy or semantic matches.
• For small coding-agent contexts or a limited number of documents, grep and a notes file may be sufficient.
• For company data spread across applications, consider live connectors first; add an index if latency or ranking quality becomes a problem.
• Compile knowledge into a wiki or similar format when the information is stable and valuable enough to justify its preparation cost.
• For fewer than 10 million vectors, the video says TurboPuffer’s CTO considered PostgreSQL with pgvector a cheaper option. It also recommends updating to version 0.8.7 to address a security vulnerability.
• Before choosing a retrieval system, consider how many search steps typical queries need and whether users will tolerate long response times.

Reviews, guides, and comparisons covered

• Comparison of agentic search with fixed RAG pipelines, including FRAMES benchmark results.
• Comparison of BM25 keyword search and vector search on enterprise and other benchmark data.
• Overview of live MCP/federated connectors and their cost and latency trade-offs.
• Reassessment of the Open Knowledge Format compiled-folder approach, with a caveat about the small study.
• Practical retrieval-system recommendations based on query difficulty, data location, cost, and speed.

Main speakers and sources

The video is presented by a Devsplainers narrator; no separate on-camera speakers are identified in the subtitles. Sources and organizations discussed include TurboPuffer and its CTO, the FRAMES benchmark and Pipes Hub (as rendered in the subtitles), an LREC study, Enterprise Rack Bench, Browse Comp Plus Research Benchmark, Cursor, Microsoft Copilot, Anthropic, OpenAI, Glean, and research and community discussion about Google’s Open Knowledge Format.