AI ON BUSINESS DATA
Three ways AI answers from your data: RAG, text-to-SQL and indexed retrieval
"Is this RAG?" usually means "are the numbers right and is my data safe?" Three approaches, three different answers to that.
RAG
Retrieval-augmented generation cuts documents into chunks, turns each chunk into an embedding, and stores them. A question is embedded the same way, the most similar chunks are fetched, and they go into the prompt so the model answers from them rather than from memory. It is the right approach whenever the answer is in a passage: what the return policy says, how the warranty process works, which clause covers early termination. The retrieval is by meaning, so wording does not have to match, and the model's answer can cite the chunk it came from.
It fails at the question business teams ask most. "Total revenue by region" is not a passage. There is no chunk that contains the answer, because the answer has to be computed from every order, and RAG does not compute; it retrieves. Asked anyway, it will find chunks that mention revenue and regions and produce a fluent paragraph with a plausible number in it, which is the worst possible failure mode: wrong, and confident.
Text-to-SQL
Here the model is given the database schema and writes SQL from the question. The database runs the query and the model explains the result. It is genuinely powerful: the model can express joins, subqueries and window functions that no fixed vocabulary would offer, and an engineer watching over its shoulder can get a lot of exploration done quickly.
The failure modes are the reason it rarely ships to business users. Ambiguous joins: three ways to join orders to customers, one of them right, and the model picks by the names. Silently wrong queries: SQL that runs, returns a number, and has the date filter on the wrong column. Load on production: every question is a query, possibly a scan, on the database that is taking orders. Hard to constrain: a model that can write SELECT can be talked into writing almost anything, and limiting it means parsing SQL. And the schema goes into every prompt, which is both a cost and a disclosure.
Indexed retrieval
The third approach gives the model a vocabulary instead of a language. The vocabulary is the indexed collections - orders, customers, products, invoices - with their fields, the filters that apply to them, and the facets that can be asked for. The model's job is to choose: this collection, these filters, this facet. It does not write a query; it fills in a request. The index then does the searching and the counting, and returns the records and the facets: North 395 · West 247 · East 168 · South 142, exactly, the same numbers a dashboard over the same index would show.
Production is never on the path, because the index is a copy in your own infrastructure. And the constraint is the guardrail: a request can only reference what exists, so there is no injection surface and no plausible-but-wrong query, only a request that maps or a question that the assistant has to say it cannot answer. How can AI answer questions about business data? follows one question through this approach step by step.
Side by side
| Criterion | RAG | Text-to-SQL | Indexed retrieval |
|---|---|---|---|
| Data type | Documents: policies, manuals, contracts | Structured tables | Structured records in collections |
| How numbers are produced | They are notThe model reads passages and may quote a figure | The database computes a model-written queryRight if the query is right | The index computes facetsCounts and totals, exact |
| Where it runs | A vector store beside the model | The production database | An indexed copy in your infrastructure |
| Load on production | None | Every question is a query on production | None |
| Exactness | Approximate by designSimilarity, not arithmetic | Exact when the SQL is rightSilently wrong when it is not | ExactSame numbers a dashboard shows |
| Typical failure | The right passage was not retrieved | An ambiguous join; a query that runs and is wrong | The question could not be mapped, and the assistant says so |
| Guardrails | Which documents are indexed | Hard to constrainFree SQL is the attack surface | A fixed vocabulary of collections, fields and facets |
| Freshness | As of the last document load | Live | As of the last importMinutes to hours, per collection |
| Best for | Questions answered by reading | Engineers exploring a schema with supervision | Business questions with numbers in the answer |
Combining them
They are not rivals either. RAG belongs alongside indexed retrieval in any organization that has both documents and records: the policy question goes to the documents, the revenue question goes to the index, and an assistant can route between them. Text-to-SQL keeps its place as an engineer's exploration tool, with a person reading the SQL. And the indexed tools - list collections, describe fields, search, facet - are exactly the kind of structured, bounded operations that MCP (Model Context Protocol) is designed to expose, so the same approach that powers an assistant can be offered to any AI client through an MCP server. MCP server explained covers what that server should and should not expose.
Decision guide
Three questions to ask about your use case.
- Is the answer a passage from a document?RAG
- Is the answer a number over many records?Indexed retrieval
- Must it be exact to the second, on live transactions?Text-to-SQL, carefully
Is the answer a passage or a number? If someone would answer it by reading, it is RAG. If they would answer it by counting, it is not.
Who is asking, and who is checking? An engineer who will read the query can use text-to-SQL. A business user who will act on the number needs the number to be exact without reading anything, which points to the index.
Can the production database carry it? If every question becomes a query on the system that runs the business, the answer is usually no, and the copy - the index - is where the questions should go.