Retrieval Augmented Generation (RAG) with Google Cloud Gemini and MongoDB Vector Search

Summary

This hands-on livestream builds an insurance-policy chatbot using retrieval-augmented generation (RAG). The demo combines Google Gemini 1.5 Pro, Google Cloud’s Vertex AI text-embedding model, MongoDB Atlas Vector Search, and LangChain.

How the RAG application works

Without RAG, the chatbot can answer general questions but cannot access a customer’s private insurance documents. With RAG, it:

1. Loads policy PDFs and splits their text into overlapping chunks.
2. Converts each chunk into a vector embedding and stores it in MongoDB Atlas with the text and source metadata.
3. Searches the vector index for chunks relevant to a user’s question.
4. Adds the retrieved text to the prompt sent to Gemini, which generates a contextual answer.

In the demo, a question about mechanical-failure coverage received a generic answer without retrieval, but a specific answer grounded in the policy when RAG was enabled. The chatbot could also respond to a differently worded question about a car breaking down.

Implementation and product details

• The web interface is an Angular application, and the backend is a Node.js/Express server.
• LangChain provides integrations for Vertex AI, Google embeddings, and MongoDB, as well as utilities for loading PDFs and splitting text.
• The server uses a system prompt to guide the assistant to rely on supplied context, avoid irrelevant answers, and not invent information. Conversation history is passed to the language model to support follow-up questions; vector searches are performed independently.
• The example processes 13 PDFs into 79 text chunks. Each stored document includes its text, source, and a 768-dimensional embedding.
• MongoDB Atlas Vector Search requires a vector-search index with a matching embedding dimension and field path. Atlas supports three similarity measures. The presenters used Euclidean distance in the demo and noted that the best choice depends on the data and should be checked against the documentation.
• The presenters highlighted that keeping operational data and embeddings in Atlas avoids synchronizing a separate vector database. They also described using Atlas Triggers to create or update embeddings when source documents change.
• For images, they recommended embedding the image while storing the image file in object storage, such as Google Cloud Storage, and keeping a reference to it in the database.

Tutorial and guidance

The presenters walk through a Google Cloud Skills Boost quick lab, which provides a temporary Google Cloud account so participants do not need to set up personal Google Cloud credentials for the exercise. Participants create a free Atlas account and cluster, configure a connection string, run the starter application, embed the PDFs, create a vector index, and add retrieval logic to the server.

The demo uses a free Atlas M0 cluster and temporarily allows network access from anywhere for convenience. The presenter explicitly warns against using that configuration in production. The code is available in a public GitHub repository, and the lab is linked at mdb.link/gcraglab.

Discussion and caveats

• Chunk size, overlap, model settings such as temperature, and retrieval behavior are suitable areas for experimentation. The presenters did not offer a definitive fix for cases where larger chunks score better than more relevant smaller ones.
• Embedding long conversation histories was discussed, but the presenter did not recommend it based on her experience. Retrieval may be distracted by prior messages instead of focusing on the current question.
• The session is a tutorial rather than a comparative product review. In a Q&A answer, Shane referenced a Retool survey that he said rated MongoDB highly as a vector database; no survey details were presented.
• For further learning, the presenters pointed viewers to the MongoDB Developer Hub, MongoDB Community Forums, and MongoDB examples on Hugging Face.

Main speakers and sources

• Mera — MongoDB developer advocate; led the implementation and technical walkthrough.
• Shane McAllister — MongoDB developer relations; hosted the livestream and contributed explanations and Q&A.
• Supporting materials: Google Cloud Skills Boost quick lab, public GitHub project repository, MongoDB Developer Hub, Community Forums, and MongoDB’s Hugging Face examples.