AI agent for internal documents: answer questions, cite sources, respect permissions

It is Thursday afternoon, and a sales employee wants to confirm to a customer by when a maintenance agreement can be terminated. She opens the shared drive and finds three folders with similar names and two PDFs called "final" and "final_new". Then she sends a message to the managing director, who is out of the office.

The scene is an everyday one, and in this scenario it is our starting point: a company with about twenty employees whose knowledge is stored in manuals, contracts, quotes and project files. None of it is poorly maintained; over the years it has simply ended up in different places. Add to that email attachments that were never filed and notes that only one person understands.

The temptation is great to simply put a chatbot over the entire file stock. We think that is the wrong first thought. The question is not whether a language model can read documents; it can. The question is whether the answer is substantiated, current and permissible for this person. That decides whether an AI agent that searches documents gains trust in everyday use or is ignored after three weeks.

Behind the question about the notice period lies a pattern that repeats in every small business. Knowledge migrates to the people who have often needed it, and those people become the house's search engine. That works as long as they are there. On vacation, during illness or after someone leaves, it breaks down, and nobody had noticed how much of the orientation depended on one head.

The most dangerous failure pattern is the smooth, wrong answer. A language model forms a clean sentence from text fragments, even if the fragments come from different versions or do not fit the question at all. The reader sees no break, and that is exactly what makes the error expensive. Anyone relying on the information only notices when the customer objects.

Then there are errors that arise from the state of the documents themselves. There are manuals dated four years ago that nobody deleted, and contract drafts lying next to the signed version. Scanned contracts without text recognition are invisible to an agent or are read incorrectly. Tables in PDFs lose their row structure when extracted, so a number gets assigned to the wrong column.

The third area concerns access. If the agent accesses everything with a service account and outputs the answers unfiltered, a trainee can indirectly ask about the salary scheme. This does not happen out of malice, but because nobody thought that a search that finds everything also reveals everything. You therefore have to check permissions before the answer, not after.

Before anything is indexed, two lists are needed. The first describes the stock: which document types exist, where are they, in which format, who maintains them, and which are considered authoritative? The second collects real questions from everyday work. These are not wish-list questions but ones that actually come up in emails and conversations, for example about deadlines, contacts, execution requirements or the status of a project.

Together, the two lists give a picture that is often more sober than expected. Some of the questions concern manuals and work instructions, where an answer with a source reference works well. Another part concerns contracts and figures, where a wrong answer has consequences. A third part concerns knowledge that is not written down in any document. No agent helps there, only writing it down.

A simple trial run by hand is helpful. Take ten of the collected questions and answer them the way a new colleague would have to: without asking anyone, using only the existing documents. The questions they fail on show whether the obstacle is the search or whether the answer is simply nowhere to be found. Only in the first case can an agent help.

We would also record who owns which documents. Personnel files, quotes with calculations and contracts with third parties are subject to different access rules, which are usually already reflected in the file system or the document management system. These existing permissions are the most reliable starting point, because they reflect what the company has actually decided, not what someone newly invents for the agent.

A common stumbling block is the assumption that folder permissions are clean. In small businesses they rarely are: a folder was opened once for a project and never closed again, a former intern is still in a group. The agent would not cause such legacy issues, but it would make them visible and amplify them. A short permissions review before launch is therefore part of the project, even if it is inconvenient.

There are three ways to approach the task. The first is classic full-text search, which delivers hits but does not formulate an answer. The second is a chatbot fed with all the documents that answers freely. The third is an agent that searches in a targeted way, checks the sources and answers only on that basis. We would choose the third way, but in a narrow form.

Narrow here means: the agent answers questions exclusively from approved sources, names the location and the date of the document for every statement, and says openly when it finds nothing. It changes nothing, sends nothing and makes no decisions. This restriction is a strength because it limits the error risk and makes checking simple. Anyone who doubts an answer opens the cited passage.

Against pure full-text search speaks the fact that many questions are not phrased in the wording of the documents. Anyone searching for "cancellation" will not find "termination of the contractual relationship". Against the free chatbot speaks the fact that it offers no guarantee of substantiated answers and makes it hard to enforce permissions. The agent combines both: it searches by meaning, filters by permissions and discloses the evidence.

A trade-off remains. The more narrowly the agent is guided, the more often it says "not found", and the more likely users are to find it curt. The more freely it answers, the smoother it seems and the harder it is to check whether it is right. For internal documents, we would choose the stricter side and consider loosening it only after weeks of spot checks, not the other way around.

A good answer has three parts: the statement in two or three sentences, the sources and a note on certainty. The source gives the document title, section or page and the date of the version, ideally with a link to the original file. The note says whether the sources were unambiguous or whether there were discrepancies. If the source is missing, the answer is not an answer.

The attitude toward gaps is especially important. If the agent finds nothing suitable, it should say so and offer to pass the question on to a person, rather than guess at an obvious answer. This willingness to say "I found nothing on that" is a quality feature, not a defect. You must define it explicitly during setup and check it in tests, because language models tend to answer.

Stricter rules apply to figures and legal matters. The agent reproduces amounts, deadlines, quantities and prices only by quoting them, with the original passage alongside, and does not calculate on its own. For contract clauses, it provides the wording and a brief classification, but no legal assessment. Whether a clause is valid or what it means in a dispute is a matter for a qualified person. The agent can prepare, not advise.

An example shows the difference. Asked about the warranty period in a particular project, the agent would name the contract and the associated annex, quote the wording and note that a later quote contains a different formulation. It would not state a figure it cannot substantiate verbatim, and would deliberately leave open the decision on which version applies.

Technically, the system breaks down into a few building blocks. An ingestion layer reads documents from defined sources, such as a file server or a cloud storage, converts them to text and splits them into meaningful sections. Scans go through text recognition, and its output is spot-checked. For each section, metadata is stored: title, path, date, version, document type and permitted groups.

The search combines semantic similarity with classic keyword search, because neither is reliable on its own. Keyword search is better at finding numbers, product names and contract numbers, semantic search at finding paraphrased questions. Before every search, results are filtered by the permissions of the person asking. This matters: the filter acts on the hit list, not only on the text of the answer, so content that has not been approved never reaches the model at all.

The language model then formulates the answer from the sections found and is instructed not to claim anything beyond them. A verification layer then compares whether each statement is covered by a cited section and discards answers without coverage. Question, sources used and result are logged so that errors can be traced later. The personal data in the logs must be handled sparingly and with retention periods.

The choice of location is a decision in its own right. Internal contracts and personnel documents should not go unchecked to just any service. Depending on sensitivity, a European provider with a data processing agreement, operation in your own environment or a restriction to less sensitive document classes come into question. This is orientation, not legal advice, and should be clarified with the data protection officer before launch.

The most common special case is the second version. The manual says a period of two weeks, a later circular email says four. The agent recognizes the difference only if the date and status of the documents are known. That is why every document gets a status such as "authoritative", "draft" or "archived", and only authoritative versions are searched by default. Archived ones can be explicitly queried when needed.

If a genuine contradiction remains, for instance between a contract and a quote, the agent should not resolve it but name it. It cites both sources with their dates and points out that a person must clarify which applies. At first this feels less helpful than a clear answer, but it prevents a wrong reading from becoming entrenched in the business. The note also often uncovers a gap in document maintenance.

There are further edge cases. Handwritten notes on scans are poorly recognized and should be flagged as not evaluable. Very long documents, such as framework agreements with annexes, require sections that keep their context, otherwise the link between clause and annex is lost. And questions that link several documents, such as "Which projects use this template?", are analyses, not searches, and belong more in a structured directory.

Finally, there is the question of what the agent should not answer. This includes information about individuals that only HR managers may know, legal assessments and statements about documents it is not allowed to read. In these cases it responds with a fixed, friendly notice and names the responsible contact, rather than explaining why something remains hidden.

One last special case is the input itself. Documents can contain text that sounds like an instruction, for example in a forwarded email or a third-party contract. The agent must treat content from documents as material and never as an order. This belongs in the tests: you deliberately feed in test documents containing such sentences and check whether the agent's behavior remains unchanged.

With document agents, the benefit lies in the time between a question and a reliable answer, and in relieving the few people who know everything. Neither can be quantified across the board, which is why we give no figures. Realistically, simple lookup questions are answered faster and follow-up questions to colleagues become less frequent. Complex or disputed cases stay with humans.

Measuring can be done before launch with a test set. You collect thirty to fifty real questions, define the correct answer together with its source with the responsible person, and let the agent answer. Three things are rated separately: was the right source found, is the statement correct, and did the agent cleanly abstain when no answer was available? This separation shows whether the problem lies in the search, in the model or in the documents.

In live operation, simple feedback is a good fit: a thumbs-up for the answer, a field for "source is wrong", and a monthly spot check of logs. It becomes noteworthy when certain questions repeatedly return no hits, because then a document is probably missing. There is no guarantee of error-free answers, and anyone who promises one underestimates the types of error described above.

The most important insight is unspectacular: the agent makes visible the state of the documents that nobody previously had an overview of. Duplicate versions, unclear responsibilities and missing versions stand out as soon as a machine no longer silently compensates for them. Anyone who sees this as a side benefit and tidies up the documents during the project ends up with not just an agent but also better filing.

Begin with a small, clearly defined stock, such as the manual and the work instructions, and add contracts later. The potential for damage is lower there, and you learn how your documents behave. Have one person responsible per document group from the start, because currency cannot be automated; it needs a person who releases versions and archives old ones.

Plan the updating as a fixed component. New and changed documents should be re-indexed automatically, and so should deleted or archived ones, otherwise the agent answers from stock that no longer exists. Permission changes, for example after a staff change, must arrive just as promptly. An index that mirrors yesterday's state can be more dangerous for permissions than for content.

Equally instructive is the question of expectations. Teams that see the agent as a person to ask are quickly disappointed as soon as it once reports "nothing found". Teams that see it as a fast route to the source use it more calmly. A short introduction that explains this role and shows how to open a source and give feedback therefore has more effect than any improvement to the answer style.

An AI agent that searches internal documents pays off where knowledge exists but is hard to find. It replaces neither document maintenance nor the specialist who assesses contracts. It shortens the way to the source and shows where the filing has gaps. Its quality is decided by four points: substantiated sources, checked permissions, marked currency and the courage to abstain.

The next step is small. Collect twenty questions that were asked in the team in recent weeks, and note where the answer was and who ultimately gave it. From that you can see whether a pilot is worthwhile, which documents would be needed and where tidying up would have to come first. Add three questions to the list where a wrong answer would really hurt, because these cases decide how strictly the agent is guided and what is deliberately excluded. We are happy to look at it together on request and will say honestly if better filing is the simpler route.