Third episode of the series. After unpacking embeddings, today we tackle the most-sold acronym of 2026: RAG, Retrieval-Augmented Generation.
If in the past year you received a quote for "an AI solution on your corporate documentation", 80% of the time they're selling you a RAG. Understanding what it really is — and when you really need it — is the difference between spending 30,000 euros well and spending them just to tell your customers you've done AI.
The metaphor
A RAG is an assistant that, before answering you, leafs through your archive.
When you ask a question to generic ChatGPT, the model answers based on what it learned during training — a mix of Wikipedia, books, public articles, up to a certain date. It knows nothing about you. It doesn't know your contract with vendor X, your 2026 vacation policy, the installation manual for product Y.
A RAG fills exactly this gap. Here's how it's done:
- Take your documents (PDF, Word, web pages, databases).
- "Chunk" them into small pieces (chunking).
- Transform each piece into an embedding — a vector of coordinates in "meaning".
- Save them in a vector database.
- When the user asks a question, the system searches the database for the 5-10 pieces most similar to the question, and passes them to the AI model as context.
- The model generates the answer based on those pieces.
Hence the name: Retrieval-Augmented Generation. Retrieval (from your documents) that augments generation (by the AI model).
Why it exists
Three reasons, in order of importance.
1. Factuality. The AI model without RAG can invent ("hallucinate"). With RAG, answers are anchored to your documents — not to what the model learned from Reddit in 2024. If the source is your manual, the answer will be faithful to your manual.
2. Updatability. Retraining an AI model costs millions of dollars. Updating a RAG costs the time to re-index new documents — minutes, hours at most. When your price list changes, the RAG knows the new price. The "pure" model doesn't, until it's retrained.
3. Permissions. A RAG can respect access rights to documents. If Marco can see document X but Sara can't, the RAG can filter accordingly. A pure model can't: once the document is in its weights, it knows it for anyone who asks.
When you really need it
Three typical cases where RAG makes sense. More precisely: where RAG is the only sensible solution.
Case 1 — Customer support on complex product/service. You have 200 pages of technical documentation and your customer care agents spend time searching them. RAG. It works, reduces average response time, is easily measurable.
Case 2 — Internal help desk. Employees ask recurring questions about company policies (vacation, reimbursements, HR procedures). Many questions, few HR people. RAG. Time recovered, frustration lowered.
Case 3 — Pre-sales assistant. You have a complex offer, multiple configurations, different use cases. An assistant that guides the customer through questions, retrieving relevant information from your brochures, is real value.
In all three cases, the return on investment is clear: less human time spent searching for information, more time spent doing things that require judgment.
When you don't need it (and they're selling it to you anyway)
Three operational red flags.
They propose a RAG to "help you make strategic decisions". A RAG doesn't make decisions. It retrieves information. If your problem is decisional, RAG alone isn't the solution: a more complex system (agent, workflow) is needed, or maybe you don't need AI at all.
Your documentation is disorganized, incomplete, or contradictory. A RAG amplifies the quality of your documentation. If your documents are confused, the RAG will produce confused answers. It doesn't solve the problem; it automates it. Before a RAG, sometimes a content due diligence is needed.
You have fewer than 50 documents. Below a certain volume, a RAG is overkill. An AI assistant with documents loaded into context statically does the same job with a tenth of the complexity.
What to ask the vendor
Six questions, in decreasing order of importance. If the vendor can't answer one of these without hesitation, take time before signing.
- How does chunking work? (Do they split documents into pieces of how many characters? Do they maintain section context? Do they handle tables and images?)
- Which embedding model do you use? (And does it know Italian? See the embeddings piece.)
- How do you handle permissions? (If a document is confidential, does the RAG know certain users can't see it?)
- How do you handle updates? (When I add a new document, how quickly is it available? Is there version control?)
- What does the assistant do when it can't find the answer? (Says "I don't know" or tries to invent? The right answer is the first.)
- How do you test answer quality? (Do they have a battery of test questions? Do they run it on every update?)
Three well-formulated answers, and you can proceed. Three hesitations or "we evaluate case by case", and they're probably selling you an idea, not a system.
Summary in three lines
A RAG is an AI that, before answering, leafs through your documents.
Solves the problem of hallucinations and updatability — if your documents are of quality.
You really need one only if you have a lot of documentation and a lot of recurring questions. Otherwise they're selling you an expensive trend.
Monday, July 13 we change register: The founder's summer. 4 things to do in August that really weigh in September. No "recharging the batteries". Four low-volume, high-impact jobs.
Have a RAG in mind for your company and don't know if it makes sense? Under artificial intelligence for business we audit RAG-quotes in half a day. We explain what you're buying, before you buy it. Let's talk.