AI terms for decision-makers: what you really need to know in 2026
The old arocom blog carried a primer on artificial intelligence from 2023 onwards. It explained neural networks and machine learning. Solid, but aimed at the wrong reader: in 2026, managing directors do not need to know how a neural network learns. They need to know what is meant when a vendor offers "RAG with fine-tuning", and whether the price for it is reasonable.
Here are the ten terms that cause the most confusion in our client conversations. Each comes with what it means in practice and the follow-up question you should ask.
The ten terms and the most important follow-up question for each
| Term | Means in practice | Your follow-up question |
|---|---|---|
| LLM (large language model) | The language model behind ChatGPT, Claude & co. The base technology, not the product. | Which model, and what happens if the vendor switches it? |
| Prompt | The working instruction given to the model. Good prompts are work products, not magic. | Do the prompts developed in the project belong to us in the end? |
| Context window | How much text the model can "see" at once. Limits what it can process in one step. | Is that enough for our longest documents? |
| RAG | The model answers questions using your own documents instead of just its training knowledge. | How current are the answers when our data changes? |
| Fine-tuning | Additional training of a model on your data. Expensive, often unnecessary when RAG is enough. | Why is RAG not sufficient here? |
| Agent | AI that carries out multi-step tasks on its own and operates tools, instead of just answering. | What may the agent do without human approval, and what not? |
| MCP | Open standard through which AI tools access systems such as CRM or CMS. | Does the vendor build on open standards or are they building us an island? |
| Hallucination | The model invents plausible-sounding false statements. Not a bug, but how it works. | Which review step catches this before it reaches customers? |
| Token | The billing unit of the models, roughly: word fragments. Determines the running costs. | What does a typical month cost at realistic usage? |
| Inference costs | The running costs per request in operation, as opposed to project costs. | How do the costs develop if usage grows tenfold? |
The table is deliberately a negotiation aid, not an encyclopedia. You will find the technical depth on each term in our knowledge section.
In our experience, four of the ten terms cause the most expensive misunderstandings in proposals. We dedicate one example from mid-sized business to each of them.
RAG by example: the product catalogue chatbot
A supplier with 8,000 articles wants customers to ask questions in the web chat: which variant fits which machine, which standard does the material meet? The answers live in the product catalogue and in data sheets, not in the training knowledge of a language model.
That is exactly the RAG case. The system retrieves the matching catalogue passages for the question and has the model formulate an answer from them, with a source reference. If a data sheet changes, it is re-indexed. No retraining, no waiting time, and the answer can be traced back to its source. For the follow-up question from the table, this means: ask the vendor how quickly a catalogue change shows up in the answers. Hours are a good sign, weeks are a warning signal.
The expensive misunderstanding: treating RAG as a small technical extra. The real work lies in data quality. A catalogue full of outdated PDFs delivers outdated answers, just phrased politely. Plan data maintenance as a fixed part of the project, not as a footnote.
Fine-tuning by example: when tone of voice really needs training
A publishing house produces hundreds of short texts a month in a very distinctive house style. Prompt guidelines with good examples delivered most of the desired tone; the last gap was only closed by additional training on thousands of edited texts. That is a legitimate fine-tuning case: high volume, tight style requirements, a stable task.
The cross-check for your project is: is this about knowledge or about style? Knowledge belongs in RAG, because it changes. Style can justify fine-tuning, but only once prompts with examples have demonstrably failed to suffice. Demand that evidence before you pay for retraining.
There is also a lock-in question: a fine-tuned model is tied to its base version. If the vendor discontinues the base model, you train again, at the same cost.
Agent by example: processing incoming invoices
A wholesaler has an agent process incoming invoices. The agent reads the invoice, finds the matching purchase order in the ERP, compares line items and amounts and creates a posting proposal. For deviations above a defined threshold, it stops and hands over to a human.
The limits matter more than the capabilities here. The agent may read, match and create proposals on its own. It may not release payments, change master data or communicate with suppliers on its own. These limits are written into the concept before the first line is built, and they are written into the proposal.
This is also where the circle closes back to MCP from the table: for the agent to look up purchase orders in the ERP, it needs an interface to it. If that runs over an open standard, you can switch vendors later without paying for the connection again.
The expensive misunderstanding: equating "agent" with "runs unsupervised". An agent without defined intervention points does not save work, it shifts it into troubleshooting.
Inference costs by example: what a chatbot costs per month
A simplified example calculation meant to show orders of magnitude, not a price list. A support chatbot answers 3,000 requests a month. Per request, the model processes around 4,000 tokens of input (the question plus the supplied knowledge excerpts) and produces 500 tokens of answer. With a mid-range model at roughly 1 euro per million input tokens and 4 euros per million output tokens, a single request costs well under a cent. The month comes to roughly 18 euros.
That sounds harmless, and that is exactly the trap. The bill grows tenfold with usage, it multiplies with a stronger model, and agent workflows with several steps per task multiply the token volume again. 18 euros can quickly turn into four-digit monthly costs.
So demand both numbers in the proposal: the project costs and an inference cost estimate for realistic usage as well as for ten times that usage. Serious vendors can run that calculation for you in half an hour.
Three patterns you should recognise
The relabelling. Products that were called "search" or "automation" two years ago are now called "AI agent". Ask what the system could do without the new label. If the answer is the same, you are paying for a word.
The fine-tuning upsell. Retraining sounds like a tailored suit and is priced accordingly. In our projects, RAG solves the task better in most cases: more current, cheaper, easier to trace. Fine-tuning has its place, but that place is narrow.
The unmentioned running costs. Proposals state project costs and stay silent on inference costs. A chatbot with a 30,000 euro project budget can generate additional five-digit yearly costs under intensive use. Demand both numbers.
Do we have to hand our data to the model vendor?
With API usage, the large vendors process your requests on their own servers, but according to their terms of service they do not use them for training by default. For sensitive data there are graduated options: EU hosting, contractual commitments or locally operated models. Which level your case needs depends on the type of data and your industry, and it belongs on the requirements list before proposals are requested.
What can we learn internally, and where do we need partners?
Writing prompts, evaluating tools and building small automations is something a motivated team can teach itself in weeks. Partners pay off where mistakes get expensive: integration with existing systems, permissions and data protection, agents with write access. Our rule of thumb from projects: learn the reading side internally, have the writing side accompanied.
How quickly does this knowledge become outdated?
Model names and prices change quarterly; the ten concepts on this list have been stable for around three years. RAG, agents and inference costs will keep appearing in the proposals of the coming years. If you can place the concepts, you do not have to chase every model release.
The next step
You do not have to become an AI expert. It is enough to ask the ten follow-up questions from the table and insist on clear answers. Vendors who cannot answer them sort themselves out.
If you have a concrete proposal on the table and want a second opinion: we will review it as part of the Future Check, including the cost side.
Go deeper in our knowledge base
Want to know what these topics mean for your company? The Future Check shows you the biggest levers within 2–4 weeks.
