SECURITY & DEPLOYMENT

What leaves your environment when AI reads your documents

The files stay. The index stays. What travels to the AI model is text, at two moments: once when a document is indexed, and again when a question is asked.

Leadership Engineering EXPLAINER 6 min read
Stays in your infrastructure
  • The original files
  • The extracted text and every passage
  • The index, by words and by meaning
  • Summaries, keywords and categories, once written
  • Accounts, workspaces and permissions
Sent to the AI model you chose
  • At indexing: the text of the document, to write its summary and keywords
  • At indexing: each passage, to index it by meaning
  • At question time: the question, and the passages retrieved for it
Host the model yourself and the right-hand column stays inside too
Files and index stay. Text travels to the model at two moments.

The question behind the question

"Is it safe to put our documents into an AI system?" is usually two questions. Where do the documents live once the system has them? And what does the AI model actually receive? They have different answers, and treating them as one is how a reasonable security review turns into a stalemate.

For a document search system that runs inside your own infrastructure, the first answer is short: the documents live where you put them. The second deserves a careful, specific answer, because something does go to the model, and the honest account is more useful than a slogan.

What stays put

Start with what does not move. The original files stay on your storage; a system can index files where they already sit without copying them anywhere. The text extracted from them, the passages that text is cut into, and the index built from those passages are all written inside your deployment. So are the summaries and keywords once they have been produced, and so are the accounts, workspaces and permissions that decide who can find what.

The index is the part people worry about least and should think about most, because it is a complete, searchable copy of everything the documents say. It belongs behind the same boundary as the files, and in a self-hosted system it is.

Moment one: when a document is indexed

The first time text goes to a model is when a document arrives, and it happens for two reasons.

One is to describe the document. The model is given the document's text and asked for a short summary, a set of keywords and a category. For a very long document, typically only the first part of the text is sent, up to a limit; the result is a description of the document, not a copy of it.

The other is to index the document by meaning. Each passage is sent to an embedding model, which returns a list of numbers standing for what the passage is about. Those numbers come back and are stored in your index; they are what search by meaning compares. Keyword vs semantic vs hybrid search explains what they are for.

This moment happens once per document, and again only if the file changes. It is also the moment that sends the most: over the two steps, essentially the full text of the document passes through the model.

  • A document arrivesmoment one
  • Your AI modelsummary · meaning
  • A question is askedmoment two
  • Your AI modelthe answer
Once per document, then once per question.

Moment two: when a question is asked

The second moment is every question put to the assistant. What is sent is the question itself, the handful of passages retrieved for it, and the recent turns of the same conversation so that a follow-up makes sense. How an AI answer gets its citations walks through that exchange.

The volume here is small and specific. A question about expense deadlines sends the passages about expense deadlines, a few paragraphs each, not the handbook and not the library. A plain search by meaning, with no assistant involved, sends even less: only the words typed into the search box, so they can be placed on the same map as the passages.

What is not sent

It helps to be as exact about the negative. The original files are not sent; a model receives text, not PDFs. Documents in workspaces you cannot access are not retrieved for your question, so their passages are not sent on your behalf. At question time, documents unrelated to the question are not sent, because retrieval did not select them. Accounts, passwords, access keys and the structure of your storage are not part of any request.

And a keyword search sends nothing at all. It is answered entirely by the index.

Three ways to run the model, and a fourth without one

Everything above describes text travelling "to the model". Where that is depends on a choice that is yours, and it changes the answer to the original question more than anything else.

SetupWhere the model runsWhat crosses your boundaryWhat works
A model provider's APIWith the provider, under your account and keyDocument text at indexing; question and passages at question timeEverything
A model in your own cloud accountIn the cloud account you already useThe same text, to a service inside that accountEverything
A model you host yourselfOn your own serversNothingEverything, at the speed of your hardware
No modelNowhereNothingKeyword search and filtersNo summaries, no search by meaning, no assistant

With a provider's API, the text leaves your network for the provider's service, under an account you hold and a key you issue, and the provider's terms govern what happens to it there: how long it is kept, and whether it is used for anything else. Those terms are a contract between you and the provider, and they are worth reading before the first document is indexed. Bring your own LLM covers what holding that relationship yourself gives you.

With a model served inside a cloud account you already operate, the text goes to a service within an environment you have already approved for company data, which for many organizations is the end of the discussion.

With a model you host yourself, the text goes from one of your servers to another. Nothing crosses the boundary, at indexing or at question time. The cost is hardware and a model that may be less capable than the largest hosted ones.

And with no model at all, a document search system should still work: every document read, cut into passages and searchable by its words, with filters and highlights. What is missing is the summary, the search by meaning and the assistant. That is a real option for the most sensitive collections, and a sensible first stage for a cautious rollout.

Mixing the options

The choice does not have to be one for the whole company. Because documents are grouped and access is granted per group, a deployment can be planned around sensitivity: a hosted model for the general library where capability matters most, and a separate deployment with a self-hosted model, or none, for board papers and personnel files. What matters is that the decision is made deliberately, per collection, by the people responsible for the documents.

Six questions to ask

Any vendor offering AI over documents should be able to answer these plainly.

Where do the files, the extracted text and the index live? At what moments is text sent to a model, and how much each time? Which model, run by whom, under whose account and key? What do that provider's terms say about retention and reuse? Can the model be hosted by us, and what stops working if we use none? And does a person's question ever retrieve passages from documents that person cannot open?

The last question is the one most often skipped. A system can keep every file inside your walls and still leak between departments, if access is checked after retrieval instead of during it. Data flow map: what leaves your environment, per product asks the same questions of dashboards, assistants over business data and MCP servers.

The short version

Files and index: inside. Text to the model: twice, once per document and once per question, to a model you chose. Change the model's location and you change what crosses the boundary, down to nothing. That is a more useful sentence in a security review than either "it is completely private" or "it sends your documents to an AI".

See it on real data.

The demo instance runs dashboards, data grids and the AI Assistant on real business data. No sign-up.