Running AI on premise or in a private cloud

Why the question comes up

A pharmaceutical formulations manufacturer near Ahmedabad wants an assistant for its quality team. The assistant would search standard operating procedures, batch manufacturing records and deviation reports, and help draft investigation summaries for a quality officer to review. The documents include formulation details that the company treats as its most sensitive commercial information, and records that regulators and customer audits expect to be tightly controlled.

The head of IT has a clear position: these documents do not leave our network. That rules out sending passages to a model hosted by an outside provider, at least for this use. The choice is between running AI in the company's own cloud account or on servers in its own data centre.

Banks, NBFCs, hospitals, defence suppliers and government bodies face the same question for their own reasons.

The three ways to deploy

Hosted

The platform and the models run in a provider's environment. You send data over secure connections and receive results. This is the quickest to start, the cheapest at low volumes, and gives you access to the most capable models. Your data leaves your environment, under the provider's terms on retention and use.

Private cloud

The platform runs in your own account with a cloud provider such as AWS, Microsoft Azure or Google Cloud, in a region you choose, including Indian regions. Models are either open models you run on your own cloud servers, or a cloud provider's managed model service inside your account. Data stays in an environment you control, while you still get elastic computing and managed services.

On premise

The platform and open models run on servers in your own data centre or office. Nothing goes out unless you allow it. You carry the full cost and work of hardware, power, cooling, security and upgrades.

AksharAI supports all three, and white label versions of each. Many organisations end up using a mix: hosted for low risk work such as drafting marketing copy, private cloud or on premise for sensitive collections.

How to decide

Ask these questions about each agent separately.

• What data does the agent see? Public information and internal policies are usually fine on hosted models. Personal health data, financial records, formulations and source code may need to stay inside.
• What do your regulators, customers and contracts say? Regulated entities may have specific requirements on outsourcing and data location, such as the Reserve Bank of India's directions on outsourcing of IT services for banks and NBFCs. Customer contracts in pharma and defence often have their own clauses. Check the current text with your compliance team, because these rules are updated.
• How capable does the model need to be? The strongest hosted models still outperform most open models on hard reasoning and long documents. For searching and summarising SOPs, a good open model is often enough. For complex drafting, the gap may matter.
• What volume do you expect? At low volumes, hosted is cheaper. At steady high volumes, owning the hardware can pay back over time.
• Do you have the people to run it? On premise AI needs staff who can manage servers with graphics processors, update models and monitor performance.

What on premise really takes

If you choose on premise, plan for these.

Hardware

Language models run on servers with graphics processors. The size of the model you can run depends on the memory of those processors. Smaller open models run on a single server; larger ones need several. Lead times for this hardware in India can be long, and prices change often, so get current quotes before you budget.

Models and updates

Open models improve quickly. Plan to evaluate new versions a few times a year, test them on your own cases, and switch when it helps. Check the licence of each model for commercial use.

Security and access

The AI stack becomes part of your critical infrastructure. Put it behind your normal identity system, restrict administrative access, log every request, and include it in your security reviews and audits.

Monitoring

Watch response times, error rates and server load. Without a provider doing this for you, a full disk or a failed processor will simply stop the agent.

A sensible path

Most organisations do not need to start on premise for everything. A path that works well:

1. Run a trial on hosted or private cloud using documents that are not sensitive, such as public SOPs or anonymised samples, to prove the agent is useful.
2. Decide which collections must stay inside, judged on data, regulation, capability, volume and staff.
3. Move those agents to private cloud or on premise, test that quality holds with the chosen open model, and compare with the trial.
4. Keep low risk agents on hosted models, where they are cheaper and more capable.

Limits to plan for

On premise and private cloud give you control. Safety still depends on how well they are run. A poorly secured internal server can leak data just as easily as a misconfigured hosted service. They also cost more to run and usually mean using a less capable model. For some tasks that is the right trade; for others the hosted option, under clear business terms, is the better choice.

Keeping data inside also does not settle your obligations under the Digital Personal Data Protection Act, 2023. You still need a purpose, consent or another lawful basis, security safeguards and a way to honour data rights, wherever the processing happens.

Checklist

• Each agent's data classified by sensitivity
• Regulatory and contract requirements checked and dated
• Model capability tested on your own cases
• Volume and cost compared across options
• Staff identified to run and monitor the stack
• Hardware quotes and lead times confirmed
• A plan for model updates and security reviews

The Deployment and governance page sets out the hosted, private cloud, on premise and white label options in more detail.