AI agents, RAG, MCP: understanding AI solution architecture
By Jérôme Denis, AI consultant & solution architect · 7 min read · Updated 11 June 2026
Understanding the architecture of an AI solution is no longer a technician's luxury: it is what saves you from paying for an autonomous agent where three automation scenarios would do — or the reverse. The market is indeed shifting at speed: according to Gartner, fewer than 5% of enterprise applications embedded AI agents in 2025, a share expected to reach 40% by the end of 2026. Here is each building block, explained in plain English, with SME examples.
The LLM: the engine behind everything else
The LLM (Large Language Model) is the engine that understands and produces text: GPT (OpenAI/ChatGPT), Claude (Anthropic), Gemini (Google), Mistral, or open-source models distributed through Hugging Face. On its own, an LLM knows neither your customers nor your procedures: the whole point of an architecture is to plug it into your reality. Three selection criteria dominate: quality on your type of task, cost per million tokens, and where the data is processed — a decisive point for sensitive data, where self-hosted open-source models regain the advantage.
RAG: plugging the model into your documents
RAG (Retrieval-Augmented Generation) lets the model answer from your documents: procedures, contracts, catalogues, archives. At question time, the system retrieves the relevant passages and feeds them to the model, which answers while citing its internal sources.
SME example: an accounting firm builds a RAG assistant on its internal knowledge base; junior staff get sourced answers in seconds that previously required a senior partner. The key advantage over fine-tuning: knowledge stays up to date — add a document and it is included.
MCP: plugging the model into your tools
MCP (Model Context Protocol, the open protocol published by Anthropic in late 2024) standardises the connection between an AI model and your business tools: CRM, ERP, calendar, email, spreadsheets. Instead of building a custom integration for every tool-model pair, you install reusable connectors.
SME example: an assistant connected via MCP to the CRM and calendar prepares every morning's meeting briefs, with client history and open items. Through 2025-2026, MCP became the de facto industry standard, adopted well beyond Anthropic.
AI agents: autonomy over a complete process
An AI agent combines the previous blocks and adds the action loop: it perceives (an email arrives), decides (is this application complete?), acts (replies, follows up, updates the tracking sheet), and repeats — without being spoken to. It is the most powerful and the most demanding block: it requires clear rules, clean data and guardrails (human validation on sensitive actions).
A real example: for a Paris art fair, an agent handles the entire exhibitor application pipeline — file analysis, reply emails, follow-ups, contracts, invoicing — with human validation at the commitment steps. This kind of build is the core business of NOIA NOGAIN, the custom AI agent practice for SMEs that I co-founded.
Low-code: orchestration without development
Low-code platforms (such as Make) assemble automations from visual blocks, with AI model calls in the middle of the flow. It is the ideal entry point: an invoice arrives by email → AI extracts the data → entry in the accounting sheet → archiving. Implemented in days, not weeks. The limit: as soon as the logic gets complex or the volume grows, a custom agent becomes more reliable and cheaper to run.
How do you choose the right architecture?
The rule fits in one sentence: start from the process, never from the technology. Three questions are enough to narrow it down:
- Is the knowledge in documents? → RAG.
- Does the action happen in your tools? → MCP or low-code.
- Must the process run without human intervention? → an agent, with guardrails.
And before any commitment, demand the quantification of running costs (API consumption, hosting, maintenance) on top of the implementation cost — that is where the bad surprises hide. It is exactly what a prior AI diagnostic is for.
Conclusion
RAG for your documents, MCP for your tools, agents for autonomy, low-code to start fast: the right architecture is the one that fits your process, not the latest demo. A few hours of independent scoping saves months of technical debt.
A process to automate? Describe it — I'll tell you what it takes technically. Scoping is free.
Discover the AI architecture offerFAQ
What is an AI agent in a business context?
An AI agent is a system that runs a complete process autonomously: it perceives (emails, documents, data), decides based on rules and language models, and acts on your tools (CRM, invoicing, email). Unlike a chatbot, it works without being spoken to.
What is the difference between RAG and fine-tuning?
RAG gives the model access to your documents at question time: knowledge stays up to date and verifiable. Fine-tuning modifies the model itself by training it on your data: useful for a specific style or vocabulary, but more expensive and frozen. For most SMEs, RAG is enough.
Does MCP replace traditional APIs?
No, it standardises them. The Model Context Protocol is an open protocol that lets an AI model connect to any tool (CRM, ERP, calendar) through reusable connectors, instead of building a custom integration per tool and per model.