Retrieval augmented generation (RAG) for company documents, in plain English

Retrieval augmented generation, usually shortened to RAG, is a way of letting an AI model answer questions from your own documents. When someone asks a question, the system first searches your policies, manuals or contracts for the most relevant passages, then gives only those passages to the model and asks it to answer from them and cite them. Answers stay tied to your current documents, and you control which documents each person can reach.

Why a model needs your documents at all

A language model learns from public text up to a certain date. It knows what a leave policy usually says. It does not know what your leave policy says, that your Pune plant follows a different shift allowance from your Chennai office, or that the travel policy changed last April.

You could paste the whole policy into every question. That works for one short document and fails quickly: an HR handbook, forty standard operating procedures and three years of circulars are far too much text for every question, and the cost and delay add up. RAG solves this by sending the model only the few passages that matter for each question.

How RAG works

The steps below run in order every time someone asks a question, after a one time preparation of the documents.

Preparing the documents

You choose which documents the assistant may use: the current HR policy, the IT guides, the quality manual. Old versions are removed or clearly marked. Scanned PDFs are converted to text, and tables are kept readable.

Splitting into passages

Each document is cut into passages of a few paragraphs, with the document name, section heading and date kept alongside. Each passage is turned into a numeric fingerprint, called an embedding, that captures its meaning. These are stored in a search index.

Searching for the question

When an employee asks "How many days of paternity leave do I get?", the question is turned into the same kind of fingerprint and compared with every passage. The closest matches come back, often combined with a plain keyword search so that exact terms such as a policy number are not missed.

Answering with citations

The model receives the question, the top few passages and an instruction: answer only from these passages, say which one you used, and say clearly if the answer is not there. The employee sees the answer with a link to the source section.

A worked example

An engineering components maker near Pune has about 400 staff and a quality manual, HR policies in English and Marathi, and a folder of customer specifications. Supervisors spend time every shift answering the same questions: which torque value applies to a part, what the overtime rule is on a holiday, which gauge to use for an inspection.

The company sets up an internal assistant using RAG with three separate collections: HR policies for all staff, quality procedures for production and quality teams, and customer specifications for the quality and engineering teams only. A supervisor on the shop floor asks on a tablet, "Inspection frequency for bracket 7742 after a die change?" The assistant finds the relevant section of the control plan, gives the frequency and shows the document and revision number. If the control plan has no entry for that part, it says so and suggests asking the quality engineer.

What makes RAG work well

Clean, current source documents

Most poor answers come from the documents. Two versions of the same policy, a scanned page with no text layer, or a rule that lives only in a WhatsApp group will all produce bad answers. Spend the first weeks on the documents themselves.

Access rules carried through to search

If a manager can read salary bands and a trainee cannot, the search must respect that before any passage reaches the model. Set permissions on the collections, and filter by the person asking. Giving an agent your documents without leaking them covers this in detail.

Citations a person can check

An answer with a source link can be checked in seconds. Make citations compulsory, and treat an answer without one as a failure in testing.

A clear "I do not know"

Instruct the assistant to say when the documents do not answer the question and to point to a person. A plain "not covered in the HR policy, please ask HR" is a good outcome.

Regular updates

When a policy changes, the index must change the same day. Name an owner for each collection and give them a simple way to replace documents.

RAG and fine tuning

Fine tuning changes the model itself by training it on examples. It can help with style or a narrow task format. It is a poor way to teach a model your facts, because those facts change and you cannot easily see which document an answer came from. For company documents, RAG is usually the right starting point, and fine tuning is worth considering only later for specific formats.

Questions to settle before you build

• Which documents are in scope, and who owns each one?
• Who may ask questions of each collection, and who may not?
• What should the assistant say when the documents do not answer the question, and who does it point to?
• Which languages will people ask in?
• How quickly must a changed policy show up in answers?

Written answers to these five questions make the build faster and the testing clearer.

Where a person approves

For an internal question and answer assistant, people approve in two places. A document owner approves what goes into each collection and when it changes. And any answer that leads to a decision with consequences, such as granting leave outside policy, accepting a deviation from a quality specification or quoting a contract term to a customer, goes to the responsible person before anyone acts on it.

In AksharAI, knowledge sources, access rules and citations are set up in the agent studio. The HR copilot and research assistant both work this way, and HR policy assistants that employees trust shows one in practice.

Limits to plan for

• Search can miss. If the question uses different words from the document, the right passage may not be found. Test with the phrasing staff actually use, including Hinglish or Marathi.
• Tables and scans are harder. Rate tables, drawings and poor scans lose detail when converted to text. Check them separately.
• Contradictions confuse it. If two documents disagree, the assistant may pick either. Remove the conflict at the source.
• It answers from documents only. It will not know about a decision made in yesterday's meeting until someone writes it down and adds it.
• Personal data needs care. Documents containing employee or customer data are subject to the DPDP Act, 2023 and the DPDP Rules, 2025. Load only what the assistant needs and keep access narrow.