---
title: "AI terms for decision-makers: what you need to know in 2026"
lang: en
type: blog
canonical: https://www.arocom.de/en/blog/ai-terms-for-decision-makers
alternate: https://www.arocom.de/blog/ki-begriffe-fuer-entscheider
author: "Axel Roth"
published: 2026-06-05
last_updated: 2026-07-29
---

# AI terms for decision-makers: what you need to know in 2026

Ten AI terms shape proposals and strategy discussions in 2026: LLM, prompt, context window, RAG, fine-tuning, agent, MCP, hallucination, token and inference costs. Decision-makers do not need to master them technically, but they need to place them correctly, because vendors hide very different services and prices behind the same words. This post explains each term in two sentences and names the follow-up question that makes the difference in negotiations. The four terms with the greatest potential for misunderstanding (RAG, fine-tuning, agent, inference costs) get a concrete example from mid-sized business each, and an FAQ answers the most common follow-up questions.

## AI terms for decision-makers: what you really need to know in 2026

The old arocom blog carried a primer on artificial intelligence from
2023 onwards. It explained neural networks and machine learning.
Solid, but aimed at the wrong reader: in 2026, managing directors do
not need to know how a neural network learns. They need to know what
is meant when a vendor offers "RAG with fine-tuning", and whether
the price for it is reasonable.

Here are the ten terms that cause the most confusion in our client
conversations. Each comes with what it means in practice and the
follow-up question you should ask.

## The ten terms and the most important follow-up question for each

| Term | Means in practice | Your follow-up question |
| --- | --- | --- |
| LLM (large language model) | The language model behind ChatGPT, Claude & co. The base technology, not the product. | Which model, and what happens if the vendor switches it? |
| Prompt | The working instruction given to the model. Good prompts are work products, not magic. | Do the prompts developed in the project belong to us in the end? |
| Context window | How much text the model can "see" at once. Limits what it can process in one step. | Is that enough for our longest documents? |
| RAG | The model answers questions using your own documents instead of just its training knowledge. | How current are the answers when our data changes? |
| Fine-tuning | Additional training of a model on your data. Expensive, often unnecessary when RAG is enough. | Why is RAG not sufficient here? |
| Agent | AI that carries out multi-step tasks on its own and operates tools, instead of just answering. | What may the agent do without human approval, and what not? |
| MCP | Open standard through which AI tools access systems such as CRM or CMS. | Does the vendor build on open standards or are they building us an island? |
| Hallucination | The model invents plausible-sounding false statements. Not a bug, but how it works. | Which review step catches this before it reaches customers? |
| Token | The billing unit of the models, roughly: word fragments. Determines the running costs. | What does a typical month cost at realistic usage? |
| Inference costs | The running costs per request in operation, as opposed to project costs. | How do the costs develop if usage grows tenfold? |

The table is deliberately a negotiation aid, not an encyclopedia.
You will find the technical depth on each term in our
[knowledge section](https://www.arocom.de/en/knowledge/ai/artificial-intelligence).

In our experience, four of the ten terms cause the most expensive
misunderstandings in proposals. We dedicate one example from
mid-sized business to each of them.

## RAG by example: the product catalogue chatbot

A supplier with 8,000 articles wants customers to ask questions in
the web chat: which variant fits which machine, which standard
does the material meet? The answers live in the product catalogue
and in data sheets, not in the training knowledge of a language
model.

That is exactly the RAG case. The system retrieves the matching
catalogue passages for the question and has the model formulate an
answer from them, with a source reference. If a data sheet
changes, it is re-indexed. No retraining, no waiting time, and the
answer can be traced back to its source. For the follow-up
question from the table, this means: ask the vendor how quickly a
catalogue change shows up in the answers. Hours are a good sign,
weeks are a warning signal.

The expensive misunderstanding: treating RAG as a small technical
extra. The real work lies in data quality. A catalogue full of
outdated PDFs delivers outdated answers, just phrased politely.
Plan data maintenance as a fixed part of the project, not as a
footnote.

## Fine-tuning by example: when tone of voice really needs training

A publishing house produces hundreds of short texts a month in a
very distinctive house style. Prompt guidelines with good examples
delivered most of the desired tone; the last gap was only closed
by additional training on thousands of edited texts. That is a
legitimate fine-tuning case: high volume, tight style
requirements, a stable task.

The cross-check for your project is: is this about knowledge or
about style? Knowledge belongs in RAG, because it changes. Style
can justify fine-tuning, but only once prompts with examples have
demonstrably failed to suffice. Demand that evidence before you
pay for retraining.

There is also a lock-in question: a fine-tuned model is tied to
its base version. If the vendor discontinues the base model, you
train again, at the same cost.

## Agent by example: processing incoming invoices

A wholesaler has an agent process incoming invoices. The agent
reads the invoice, finds the matching purchase order in the ERP,
compares line items and amounts and creates a posting proposal.
For deviations above a defined threshold, it stops and hands over
to a human.

The limits matter more than the capabilities here. The agent may
read, match and create proposals on its own. It may not release
payments, change master data or communicate with suppliers on its
own. These limits are written into the concept before the first
line is built, and they are written into the proposal.

This is also where the circle closes back to MCP from the table:
for the agent to look up purchase orders in the ERP, it needs an
interface to it. If that runs over an open standard, you can
switch vendors later without paying for the connection again.

The expensive misunderstanding: equating "agent" with "runs
unsupervised". An agent without defined intervention points does
not save work, it shifts it into troubleshooting.

## Inference costs by example: what a chatbot costs per month

A simplified example calculation meant to show orders of
magnitude, not a price list. A support chatbot answers 3,000
requests a month. Per request, the model processes around 4,000
tokens of input (the question plus the supplied knowledge
excerpts) and produces 500 tokens of answer. With a mid-range
model at roughly 1 euro per million input tokens and 4 euros per
million output tokens, a single request costs well under a cent.
The month comes to roughly 18 euros.

That sounds harmless, and that is exactly the trap. The bill grows
tenfold with usage, it multiplies with a stronger model, and agent
workflows with several steps per task multiply the token volume
again. 18 euros can quickly turn into four-digit monthly costs.

So demand both numbers in the proposal: the project costs and an
inference cost estimate for realistic usage as well as for ten
times that usage. Serious vendors can run that calculation for you
in half an hour.

## Three patterns you should recognise

**The relabelling.** Products that were called "search" or
"automation" two years ago are now called "AI agent". Ask what the
system could do without the new label. If the answer is the same,
you are paying for a word.

**The fine-tuning upsell.** Retraining sounds like a tailored suit
and is priced accordingly. In our projects, RAG solves the task
better in most cases: more current, cheaper, easier to trace.
Fine-tuning has its place, but that place is narrow.

**The unmentioned running costs.** Proposals state project costs
and stay silent on inference costs. A chatbot with a 30,000 euro
project budget can generate additional five-digit yearly costs
under intensive use. Demand both numbers.

### Do we have to hand our data to the model vendor?

With API usage, the large vendors process your requests on their own servers, but according to their terms of service they do not use them for training by default. For sensitive data there are graduated options: EU hosting, contractual commitments or locally operated models. Which level your case needs depends on the type of data and your industry, and it belongs on the requirements list before proposals are requested.

### What can we learn internally, and where do we need partners?

Writing prompts, evaluating tools and building small automations is something a motivated team can teach itself in weeks. Partners pay off where mistakes get expensive: integration with existing systems, permissions and data protection, agents with write access. Our rule of thumb from projects: learn the reading side internally, have the writing side accompanied.

### How quickly does this knowledge become outdated?

Model names and prices change quarterly; the ten concepts on this list have been stable for around three years. RAG, agents and inference costs will keep appearing in the proposals of the coming years. If you can place the concepts, you do not have to chase every model release.

## The next step

You do not have to become an AI expert. It is enough to ask the
ten follow-up questions from the table and insist on clear
answers. Vendors who cannot answer them sort themselves out.

If you have a concrete proposal on the table and want a second
opinion: we will review it as part of the
[Future Check](https://www.arocom.de/en/future-check), including the cost side.