Bring your own LLM

The model is not the product

An assistant over business data is two things: the part that understands questions and writes answers, which is a language model, and the part that knows your data and produces the numbers, which is the index and the logic around it. The second is the product. The first is a commodity that improves every few months and that your organization may already have a policy about. Bundling the two means buying the vendor's model choice, at the vendor's price, under the vendor's terms. Bringing your own means the assistant uses the model you choose, under the account you hold.

The distinction is easy to test. Ask who would be called if the model's provider changed its prices or its terms tomorrow. With a bundled model the answer is the vendor, and the change reaches you as a contract conversation. With your own key the answer is you, directly, with the option of moving to another provider by the end of the week.

What stays with you

The account, the key, the bill and the terms. You open the provider account - OpenAI, Amazon Bedrock - under your organization's name, buy usage from them directly, and configure the key in the assistant. Every token the assistant uses is billed to that account by the provider, at the provider's rates, visible in the provider's console. What the provider may do with what it receives is governed by your agreement with them, which your legal and security teams have already read, rather than by a clause in a vendor contract.

Why it matters for cost

Usage is yours to see and to cap. A provider account shows exactly what the assistant spent, by day, and lets you set limits. A bundled model hides that inside a subscription, which is simpler until usage grows, at which point it is either marked up or rationed. Paying the provider directly also means paying the provider's price, which falls over time, rather than a price fixed at the vendor's contract date.

Why it matters for policy

Most organizations have already decided which AI providers they will use, through security review, data-processing agreements and regional requirements. Bring your own LLM lets the assistant fit that decision instead of forcing a new one. If the policy says Bedrock in a particular region, the assistant uses Bedrock in that region. If the policy changes, the key changes. The product never has to be re-approved because the model moved.

Switching, and why it is cheap

1. Choose a providerThe one your policy and budget prefer.OpenAI · Amazon Bedrock
2. Buy usage from themTokens are billed to your account, by them.your account
3. Configure the keyPaste it into the assistant. No usage is bundled into the product.your key
4. Change your mind laterA new key, a new model; the assistant, the collections and the dashboards stay.swap

Because the model only chooses fields and writes words, it is replaceable without touching anything else. The collections do not change. The dashboards do not change. The vocabulary the assistant gives the model is the same vocabulary. A new provider or a newer model is a configuration change and a short test, not a migration. That is the practical meaning of keeping the model out of the product: the part most likely to improve is the part easiest to swap.

The arrangement also keeps the product's price honest. A product that bundles model usage has to guess how much its customers will ask, and the guess is built into the price for everyone; heavy users are subsidised by light ones, or capped. Paying the provider for exactly what your team used, at the provider's rate, is the only arrangement where the number on the bill is the number of questions asked.

What the provider receives

Bringing your own key does not change what is sent; it changes who it is sent to and under whose terms. The assistant sends the information required for the request - the question, the field names it needs, the result it writes up - to the provider you chose, under your account. What the LLM sees is the exact list, and Data flow map places the flow beside the others.

What to check with a vendor

Whether the key is yours or theirs. Whether usage is billed by the provider to you or bundled into the product price. Which providers are supported today, by name, and what it takes to add one. And whether switching providers requires anything beyond a new key. A vendor who answers all four plainly has built the assistant the right way round.

Related

EXPLAINER What the LLM sees Not your database, not your index, not your collections. The question, the names it needs to map it, and the result it writes up. That is the list. PILLAR How can AI answer questions about business data? The model does not read your database. It turns your question into a structured query, runs it against indexed data, and writes up the result. The numbers come from the index; the words come from the model. EXPLAINER Data flow map: what leaves your environment, per product One boundary, four flows. What goes in, what stays, what goes out, to whom, and who controls it - for each product separately, because the honest answer differs.