Enterprise AI governance is the set of policies, roles, review gates and records that let an organisation decide which AI use cases to build, how much scrutiny each one gets, and who is accountable when something goes wrong. Done well, it is a fast, predictable path to production for low-risk work and a deliberate, evidence-based path for high-risk work. Done badly, it is a committee that meets monthly and blocks everything, which pushes teams towards shadow AI. This guide gives engineering and risk leaders a practical AI governance framework: risk tiers, roles, lifecycle gates, documentation and a starter kit you can adopt this quarter.
This article is about policy, process, accountability and risk management. The technical controls (prompt injection defences, tool permissions, output handling) are covered in AI security for enterprises; governance decides when those controls are required and who checks them.
Why governance should enable shipping, not just block it
Most AI governance programmes start as a reaction to an incident, with a blanket rule ("no generative AI without approval") and an approval process nobody has designed. Teams with a real need wait months; teams without patience use personal accounts nobody can see.
Governance that actually reduces risk is:
- Proportionate. A meeting-summary tool should not face the same review as an agent that changes customer records.
- Predictable. Teams know in advance what evidence each tier needs, who reviews it and how long it takes.
- Evidence-based. Approval rests on artefacts (an evaluation report, a data review, a vendor assessment), not on a confident presentation.
The measure of a framework is not how many use cases it rejects but whether high-risk systems ship with the right controls and low-risk ones ship at all. Many of the failure modes in why AI demos fail in enterprise production are governance gaps in disguise: no owner, no agreed success bar, no plan for when the model is wrong.
Frameworks to anchor on
Four references are worth knowing, described here at a general level:
- NIST AI Risk Management Framework (AI RMF). A voluntary framework from the US National Institute of Standards and Technology, organised around four functions: Govern, Map, Measure and Manage. A useful vocabulary in any jurisdiction.
- ISO/IEC 42001. An international, certifiable standard for an AI management system: the policies, objectives, roles and continual-improvement processes for managing AI responsibly, in the familiar pattern of ISO/IEC 27001.
- The EU AI Act. A risk-based regulation that sorts AI systems into categories, from prohibited practices and high-risk systems with substantial obligations to systems with transparency duties and minimal-risk systems. It can matter to Indian companies whose products reach the EU market.
- India's Digital Personal Data Protection Act, 2023 (DPDP Act). Governs the processing of digital personal data, including lawful grounds such as consent, purpose limitation and the duties of data fiduciaries. Any AI use case touching personal data must fit within it.
The framework below maps onto them: inventory and tiering are "Map", evaluation evidence is "Measure", controls and incident response are "Manage", roles and policy are "Govern".
Note: This article is general guidance, not legal advice. Obligations depend on your sector, jurisdictions, role (for example, provider or deployer) and use case, and they evolve. Consult your legal counsel and privacy officer before relying on any framework for compliance.
Use-case inventory and risk tiering
Start with a single register of AI use cases, including pilots and bought tools with AI features switched on. Record the business owner, purpose, users, data classes, model or vendor, whether it can act in other systems, and its stage.
Then tier each use case on impact, not technology: the same model can sit in a low-risk and a high-risk use case. Three tiers are enough; more produce debate, not clarity.
| Tier | Typical characteristics | Illustrative examples | Minimum controls |
|---|---|---|---|
| Low | Internal users, no personal or confidential data, output reviewed by a person before use, no actions in other systems | Drafting internal announcements; summarising public documentation; code suggestions in an approved IDE assistant | Register entry, approved vendor or endpoint, acceptable-use policy, named owner, self-certified checklist |
| Medium | Internal or confidential data, decisions support staff but a person decides, read-only tool access, limited customer exposure | Internal policy assistant over HR or IT documents; ticket triage suggestions; sales call summaries in the CRM | All of the above plus data review, evaluation report against an agreed bar, security review, monitoring and an incident contact |
| High | Customer-facing output, personal or regulated data, influence on decisions about people (credit, hiring, claims, care), or write actions in business systems | Customer service agent that issues refunds; credit or underwriting support; clinical documentation; agents that update records | All of the above plus review board approval, human oversight design, fairness and robustness testing where relevant, legal and privacy sign-off, staged rollout and scheduled re-review |
Two rules keep tiering honest: any one high-risk trait makes the use case high risk (no averaging), and when a read-only assistant gains a tool that writes, it goes back through review.
Roles and accountability
Governance fails most often because accountability is diffuse. Name these roles explicitly:
- AI owner (one per use case). A business leader who approves the success bar, accepts residual risk and can pause the system. Engineering builds; the owner answers for it.
- AI review board. A small cross-functional group (engineering, risk, security, legal/privacy, a business representative) that approves high-tier use cases, sets policy and reviews incidents. Delegate medium-tier reviews so the board is not a bottleneck.
- Security. Owns the security review, threat models, vendor security assessment and what "secure enough" means per tier.
- Legal and privacy. Lawful basis for personal data, contracts and vendor terms, regulatory exposure and customer-facing disclosures.
- Data owners. Decide whether each source may be used, by which systems and under what access rules. They know what is actually in the data.
- Engineering team. Produces the evidence: evaluations, architecture, monitoring and runbooks. This is often where forward deployed engineers earn their keep, translating governance requirements into working controls.
The lifecycle gates
Put governance gates on the delivery path teams already follow. If you use the stage-gate model in how FDEs take AI from POC to production, the governance gates attach to it like this:
Discovery ----- [1] Intake + tier
| [2] Data review
Scoped POC ---- [3] Model / vendor review
| [4] Evaluation evidence
Pilot --------- (re-check tier with real users)
Hardening ----- [5] Security review
|
Launch -------- [6] Launch approval
|
Operate ------- [7] Monitoring
[8] Periodic re-review- Intake. A short form (problem, owner, users, data, model, actions, benefit) producing a register entry and provisional tier, in days, not weeks.
- Data review. Data owners and privacy confirm permitted sources, the lawful basis for personal data, minimisation, retention, and whether data may leave a region or reach a model provider.
- Model and vendor review. Use the approved list or run due diligence (below); record model, hosting and configuration.
- Evaluation evidence. Results on a fixed, representative test set against a bar the owner agreed before testing; high tier adds adversarial, robustness and, where decisions affect people, fairness testing. See LLM evaluation.
- Security review. Threat model, tool permissions, identity, logging and red-team results, proportionate to tier.
- Launch approval. The AI owner signs off on the evidence pack, plus the board for high tier. Approval can be conditional: limited users, capped volume or extra review at first.
- Monitoring. Quality, safety, cost and usage signals with thresholds that trigger action (see AI observability).
- Periodic re-review. Scheduled by tier and triggered by events: a model or vendor change, new data or tools, an incident or new regulation.
Low-tier use cases can clear gates with a self-certified checklist. That fast lane is the most effective control against shadow AI.
Documentation artefacts
Keep artefacts short, versioned and next to the code or register. Four are enough:
- Use-case card. Purpose, owner, users, tier and rationale, data sources, model, tools and actions, known limitations, human oversight design, approval history.
- Model/vendor card. Provider, model family, hosting and region, data retention and training-use terms, sub-processors, security attestations reviewed, contractual protections, known limitations, review date.
- Evaluation report. Test set description and version, metrics and thresholds, results, failure analysis, adversarial results, the system version tested, and who accepted the results.
- Incident log. Every incident and near miss: impact, detection, root cause, fix and follow-ups. It becomes your best input to policy.
Test: could a new reviewer, reading only these, explain what the system does, why it was approved and what would make it unsafe?
If your leadership, engineering and risk teams need a shared working vocabulary for this, Cloudsoft runs corporate training for teams covering AI engineering, evaluation and governance on your own use cases.
Human oversight that actually works
"Human in the loop" often fails quietly: reviewers approve everything because the queue is long and the model is usually right. Design it deliberately:
- Choose the pattern by tier. In-the-loop (a person approves first), on-the-loop (the system acts, a person monitors and can intervene) or over-the-loop (people set policy and sample). Customer-affecting writes usually need the first.
- Give reviewers what they need. Show sources, confidence signals and the specific action proposed, not just the final text.
- Measure the reviewers. Track override rates and sample approvals. Near-zero overrides can mean an excellent model or nobody reading.
- Keep an off switch. Medium and high tiers need a documented way to pause or fall back to a manual process.
- Be transparent with users. Tell people when they are dealing with AI and how to reach a person.
Incident response for AI
In an AI incident the service is up but wrong, unsafe or leaking. Extend your existing incident process:
- Define what counts. Harmful or materially wrong output, data exposure, unauthorised agent actions, successful prompt injection, discriminatory outcomes, runaway cost, sustained quality drops.
- Contain first. Disable a tool, roll back a prompt or model, restrict users or switch off the feature. Pre-agree these levers in the runbook; severity follows tier and actual impact.
- Preserve evidence. Traces, prompts, retrieved context and tool calls, handled as sensitive data.
- Notify. The AI owner, security and privacy. Legal decides whether regulators, customers or individuals must be informed; personal data breaches may carry statutory duties.
- Learn. Add failing cases to the evaluation set so they cannot pass a future release, and review the log entry at the board.
Vendor and model due diligence
Most enterprises build on third-party models and buy software with embedded AI. Ask:
- Data use. Are inputs and outputs retained, for how long, and used for training? Can that be switched off contractually?
- Location and access. Where is data processed, who at the provider can access it, and which sub-processors are involved?
- Security posture. Independent attestations, incident notification and deletion at contract end.
- Model transparency. Documentation of intended use and limitations; how updates are announced; whether you can pin a version.
- Change and exit. What happens when a model is deprecated, and could you switch provider without a rebuild?
- AI features in bought software. Many SaaS tools switch AI on by default; procurement and the register must catch these.
Treat model upgrades as changes requiring re-evaluation: a new version can be better on average and worse on your cases.
Illustrative example: a mid-size bank
Consider a mid-size Indian bank with a technology centre in Hyderabad. Operations wants a policy assistant, credit a document-extraction aid and IT a code assistant. Every request has stalled in a general architecture forum.
The bank names a risk delegate to chair a small AI review board and publishes an acceptable-use policy and three tiers. The new register surfaces unapproved usage, including a team drafting customer letters in a personal chatbot account; it is moved at once to an enterprise-contracted endpoint.
Tiering follows impact. The code assistant is low tier: approved vendor, no customer data, fast lane. The policy assistant is medium: it reads confidential procedures, so it needs a data review, an evaluation on real staff questions, a security review and cited sources. The extraction aid is high: customer personal data feeding lending decisions. The board requires legal and privacy sign-off with the DPDP Act in mind, alignment with the bank's model risk and outsourcing practices and its regulator's expectations, per-field accuracy measurement, and a credit officer confirming every extracted value. It launches in one region with weekly quality reviews.
Later, an unfamiliar scanned format causes extraction errors. Monitoring flags rising officer corrections; the owner restricts the tool to known formats, the failing documents join the evaluation set, and the fix passes re-evaluation before rollout resumes. The process worked as designed.
A lightweight starter kit for teams
If you have nothing today, this is enough to begin:
- Publish a one-page acceptable-use policy: approved tools, prohibited data, disclosure rules, how to request more.
- Create the register and invite teams to declare existing usage under an amnesty.
- Adopt the three tiers with the "any high trait makes it high" rule.
- Name an AI owner for every registered use case. No owner, no production.
- Form a small review board with a fixed meeting slot and a published turnaround target.
- Write templates for the four artefacts: use-case card, model/vendor card, evaluation report, incident log.
- Maintain an approved model and vendor list.
- Agree evaluation bars per use case before testing starts, never after seeing results.
- Add AI incident categories and containment levers to your existing incident process.
- Version prompts, models, retrieval settings and tool definitions, with re-evaluation in the pipeline; see LLMOps for the CI/CD side.
- Schedule re-reviews by tier and list early triggers.
- Review the framework itself periodically using the incident log and team feedback.
FAQ
What is enterprise AI governance?
Enterprise AI governance is the set of policies, roles, review gates and records an organisation uses to decide which AI use cases to build, how much scrutiny each needs and who is accountable for their outcomes and risks, from intake through production monitoring and re-review.
How is AI governance different from AI security?
AI security is the technical discipline of protecting AI systems from attack and misuse, such as prompt injection, data leakage and over-permissioned tools. AI governance is the management layer above it: it decides which use cases proceed, which controls each tier requires, who reviews the evidence and who accepts the residual risk.
Do we need ISO/IEC 42001 certification to govern AI?
No. ISO/IEC 42001 is a useful structure for an AI management system, and some organisations certify when customers ask for assurance. Start with the inventory, tiers, roles and gates, which align with the standard either way.
Does the EU AI Act apply to Indian companies?
It can, depending on whether your AI systems or their outputs are placed on or used in the EU market and on your role, such as provider or deployer. The answer is specific to each product and use case, so ask legal counsel to assess your exposure rather than assuming either way.
How does the DPDP Act affect AI projects?
Any AI use case that processes digital personal data in India must fit the Digital Personal Data Protection Act, 2023 and its rules, including having a lawful basis such as consent, limiting use to the stated purpose and protecting the data. Involve your privacy team at the data review gate, before data is indexed or sent to a model provider.
Who should own AI risk in a company?
Each use case needs a named business owner accountable for its outcome and risk. A cross-functional review board sets policy and approves high-risk use cases, security, legal and privacy review their areas, and data owners approve use of their data. Engineering produces the evidence.
How do we stop shadow AI without banning AI?
Make the approved path faster than the workaround. Provide approved tools, a short intake form, a self-certified fast lane for low-risk use cases and an amnesty to declare existing usage. Bans without alternatives drive usage out of sight.
How often should AI systems be re-reviewed?
Set a schedule by risk tier, with high-risk systems reviewed most often, and also re-review on trigger events: a model or vendor change, a new data source or tool, a significant incident, or a relevant change in regulation.
Governance works when engineering, risk and business leaders share the same picture of how AI systems are built, evaluated and run. Cloudsoft's corporate AI training programmes help teams build that picture on their own use cases, classroom in Ameerpet or live online. For engineers who want hands-on practice taking AI from demo to enterprise outcome, including a Secure Banking AI Assistant project, see Cloudsoft FDE PRO. Call +91 96660 19191 to arrange a conversation or a free demo.
