The same model, a different result
The same model can look brilliant in a demonstration and ordinary in a workflow.
Ask it to summarise a public document and it may respond fluently. Ask it to help an employee resolve a live customer issue and the real questions begin.
Which customer? Which product? What has already happened? Which policy applies today? What is the employee authorised to do? Which promise was made in the previous interaction? When should the system stop and involve a person?
The model may be capable. Without that context, it is still guessing at the work.
This is why enterprise AI advantage is unlikely to come from access to a model alone. Capable models are increasingly available to many organisations. The difference appears in how well an organisation can connect that capability to its own knowledge, responsibilities, workflows and evidence.
Context turns a general answer into a relevant one. Governance determines whether that answer can be trusted and used.
Context is more than company data
It is tempting to define enterprise context as a collection of documents: policies, manuals, product notes and process guides.
Approved knowledge is essential, but it is only one layer.
Useful context normally includes five things.
- PurposeThe task, decision or outcome required
- RoleWho is asking, their responsibility and authority
- KnowledgeApproved policies, products, processes and expertise
- SituationThe current customer, case, workflow and constraints
- HistoryRelevant prior actions, commitments and outcomes
Purpose, role, approved knowledge, current situation and relevant history form five distinct layers of enterprise context.
1. Purpose
What task, decision or outcome is required? “Help with this account” is vague. “Prepare the employee to explain the delay, confirm the unresolved need and agree the next authorised action” is much clearer.
2. Role
Who is asking? Their role affects the language, detail, responsibility and authority of the response. A frontline employee, manager, compliance reviewer and customer should not receive the same assistance.
3. Approved knowledge
Which policies, products, procedures and expert sources are authoritative? Are they current? Who owns them? Can the system show where its answer came from?
4. Situation
What is happening now? The customer, case, location, workflow stage, channel, available time and consequence of error all shape what a useful response looks like.
5. History
Which prior actions, commitments and outcomes are relevant? History prevents the system from treating every interaction as if it were the first.
Context is not the amount of information available. It is the relevance of the information assembled for the work.
Retrieval helps. It does not finish the job.
Retrieval-augmented generation, commonly called RAG, established an important technical pattern: combine a model’s learned parameters with information retrieved from an external source. In the original 2020 paper, the researchers evaluated this approach on knowledge-intensive language tasks and reported stronger results than the parametric-only baseline they tested. Read the NeurIPS paper.
For an enterprise, retrieval can help bring current, approved material into the response. It can also make source references possible.
But retrieval is not the same as context.
A system may retrieve an outdated document, a policy for the wrong jurisdiction or a paragraph the user is not permitted to see. It may retrieve something relevant to the topic but irrelevant to the decision. It may fail to retrieve the exception that changes the answer.
The work therefore includes source ownership, metadata, access control, freshness, ranking, evaluation and a clear response when evidence is insufficient.
The AI does not become enterprise-ready because it can search a folder.
More context is not automatically better
Another tempting response is to give the model everything: every document, message, meeting, transaction and conversation.
That creates noise, privacy risk and conflicting instructions. It may also fail technically.
In “Lost in the Middle”, researchers tested language models on multi-document question answering and key-value retrieval. Performance often changed depending on where the relevant information appeared, with weaker results when it sat in the middle of a long context. The finding concerns the models and tasks studied; it is not a universal limit. It does challenge the assumption that a larger context window automatically means robust use of everything inside it. Read the TACL paper.
Good context is selective.
It brings forward the information that matters, keeps provenance visible and leaves out material that is irrelevant or unauthorised.
Consider a service team handling a repeat request
Illustrative scenario
Imagine a customer contacts a service team because an issue marked as resolved has returned.
A generic assistant can produce a courteous apology and a list of troubleshooting steps.
A context-aware assistant could work from the customer’s authorised case history, the product involved, the last commitment made, the current service policy, the employee’s permission level and the next available workflow action.
It might help the employee recognise that the customer has already repeated the standard steps. It could surface the relevant exception, suggest a clarifying question and identify when escalation is required.
This is an illustrative scenario, not a reported client result.
The value is not simply a better paragraph. It is a better-supported moment of work.
The boundaries matter equally. The assistant should not reveal another customer’s information, infer an entitlement that policy does not support or make a decision outside the employee’s authority. If the evidence conflicts, the correct response may be to pause and request human review.
Make context operational
Context becomes an enterprise capability when it is designed as part of the workflow rather than added to a prompt at the end.
Start with a specific moment
Choose a recurring task or decision where relevant assistance could improve quality, speed or consistency. Avoid beginning with “give everyone an AI assistant”.
Identify the minimum useful context
Map what the system needs to know about purpose, role, knowledge, situation and history. Separate essential signals from information that is merely available.
Establish source ownership
Every important source needs an owner, a freshness expectation and a way to handle conflict. An elegant answer built on an obsolete rule remains wrong.
Enforce permissions before retrieval
Access should follow the person, purpose and task. The system should not retrieve sensitive material and hope the final response hides it.
Ground the response
Show the source or basis for consequential claims. Make uncertainty visible. Define what the system should do when evidence is missing, contradictory or below the required confidence.
Preserve human responsibility
Clarify which actions the AI may support, which require confirmation and which remain prohibited. NIST’s AI Risk Management Framework describes AI systems as socio-technical: their risks and benefits depend not only on technical components, but also on how they are used, who operates them and the context in which they are deployed. See the NIST AI RMF.
Evaluate the whole interaction
Test realistic cases, including exceptions and failure conditions. Review whether the right source was retrieved, whether the response was appropriate for the role and whether the resulting action was correct.
This creates a learning loop. Outcomes can improve source curation, retrieval rules, workflow design and evaluation. It should not mean automatically using every interaction to retrain a model.
- Select
- Retrieve
- Check permission
- Ground & cite
- Human judgement
- Act
- Learn
Selected information passes through retrieval, permission, grounding, human judgement and action before feedback informs future improvement.
Context needs boundaries
Enterprise context can include personal, confidential and commercially sensitive information. “The model needs context” is not permission to collect or expose everything.
Define acceptable use, data boundaries, retention, access, oversight and routes for correction. NIST’s Generative AI Profile similarly frames risk management around specific settings, requirements and organisational priorities, with attention to governance, provenance and testing. Read the NIST GenAI Profile.
Context also decays. Products change. Policies are superseded. Customers make new decisions. Roles acquire different permissions. A reliable context layer needs ownership and maintenance, not only an initial upload.
The advantage is organisational
The model matters. So do retrieval, integration and interface design.
But the durable enterprise capability sits around them: knowing which work matters, organising trusted knowledge, connecting authorised signals, defining decision rights and learning from outcomes.
That capability is difficult to copy because it reflects how the organisation actually operates.
The next enterprise AI conversation should therefore move beyond
Which model are we using?
And ask
- What work is it supporting?
- What relevant context can it use?
- What is it permitted to do?
- How will we know the resulting action was sound?
The model provides potential. Context makes that potential specific. Governance makes it usable.
Sources and reading
National Institute of Standards and Technology. “Artificial Intelligence Risk Management Framework (AI RMF 1.0).”
Use: the socio-technical framing of AI systems and the importance of intended use, operators, social context, governance and lifecycle risk management. Qualification: a voluntary, use-case-agnostic framework; it does not endorse this article’s specific enterprise-context architecture.
National Institute of Standards and Technology. “Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile.” NIST AI 600-1.
Use: the need to manage generative-AI risk in relation to specific settings, requirements, risk tolerance and organisational priorities, with attention to governance, content provenance and testing. Qualification: suggested risk-management actions, not evidence that a particular implementation is safe or effective.
Lewis, P., et al. (2020). “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” NeurIPS.
Use: the original RAG pattern combining a pretrained model’s parametric memory with retrieved non-parametric memory, and its reported results on evaluated knowledge-intensive tasks. Qualification: used Wikipedia-based retrieval and specific benchmarks; not evidence that every enterprise RAG deployment will be accurate, secure or effective.
Liu, N. F., et al. (2024). “Lost in the Middle: How Language Models Use Long Contexts.” TACL.
Use: the finding that model performance in the evaluated tasks changed with the position of relevant information in long input contexts. Qualification: tested particular models and tasks; use to challenge the assumption that more context is automatically better, not to claim every model always fails on long contexts.
Lurny Insights · Enterprise AI & Field Lessons