ThePMGeeks

Artificial intelligence has moved beyond experimentation.

Organizations are deploying copilots, generative AI applications, autonomous agents, retrieval-augmented generation systems, and AI-enabled business processes across finance, HR, engineering, customer service, operations, and software development.

But the conversation around AI risk often starts too late.

Teams build an AI solution, connect it to enterprise data, demonstrate it to leadership, and only then begin asking:

What could go wrong?

AI risk management needs to reverse that sequence.

The better question is:

What could go wrong, how would we know, who owns the risk, and what controls should exist before this system reaches users?

That is the foundation of enterprise AI risk management.

—

AI Risk Is More Than a Security Problem

One of the biggest mistakes organizations make is treating AI risk as another cybersecurity category.

Security is certainly part of it, but AI introduces risks across a much broader surface.

An enterprise AI system can be perfectly secure from an infrastructure perspective and still produce harmful outcomes.

Consider an internal AI agent.

The application might use strong authentication, encrypted connections, approved cloud infrastructure, and enterprise identity controls.

Yet the agent could still:

– provide an incorrect answer with high confidence;
– expose information a user should not receive;
– use outdated knowledge;
– make an inappropriate recommendation;
– invoke the wrong business process;
– generate copyrighted or sensitive material;
– behave differently after a model or prompt change;
– perform an action the user never intended.

These are AI risks even when the underlying infrastructure has not been compromised.

The OECD similarly identifies risks including bias and discrimination, privacy infringement, security and safety issues, and other potential harms. Its AI Principles call for systematic risk management throughout the AI lifecycle.

This means enterprise AI risk management must combine several disciplines:

security + privacy + data governance + model governance + application governance + human oversight + operational monitoring.

—

Eight Categories of Enterprise AI Risk

A practical enterprise risk model should examine at least eight areas.

1. Data Risk

AI systems are only as trustworthy as the information they can access.

Risk appears when systems consume sensitive, poorly classified, outdated, incomplete, biased, or unauthorized information.

For a retrieval-based enterprise agent, for example, giving the model access to an entire SharePoint repository because it is technically possible does not automatically make it appropriate.

Organizations should understand:

– What information can the AI access?
– Who owns that information?
– How is access authorized?
– Is sensitive information classified?
– Can the AI retrieve information outside the user’s intended scope?
– How current is the information?

AI governance therefore begins with data governance.

—

2. Identity and Access Risk

Enterprise AI increasingly acts on behalf of users.

That makes identity architecture critical.

Organizations need to know whether an AI system is operating as:

– the signed-in user;
– a service account;
– an application identity;
– a managed identity;
– or another delegated identity.

The answer determines what the system can retrieve and what actions it can perform.

An AI agent should not become a shortcut around an organization’s existing authorization model.

The principle should be simple:

«AI should not gain access to information or actions merely because the AI application can technically reach them.»

Authorization must remain intentional.

—

3. Model and Output Risk

Large language models generate probabilistic outputs.

That creates risks such as inaccurate responses, unsupported claims, inconsistent reasoning, inappropriate recommendations, and fabricated information.

For low-impact use cases, occasional incorrect output might be tolerable.

For an agent providing safety procedures, financial guidance, legal information, engineering instructions, or employment-related recommendations, the same error rate could represent significantly greater business risk.

This is why AI risk cannot be evaluated independently of the use case and potential impact.

The NIST AI Risk Management Framework was specifically designed to help organizations incorporate trustworthiness considerations into the design, development, use, and evaluation of AI systems.

—

4. Security Risk

AI creates additional attack surfaces.

Depending on the architecture, organizations may need to consider threats such as:

– prompt injection;
– indirect prompt injection;
– malicious documents;
– data exfiltration;
– poisoned knowledge sources;
– insecure plugins or tools;
– excessive permissions;
– compromised connectors;
– unauthorized API invocation.

Traditional application security therefore remains necessary, but it is no longer sufficient.

Security teams must understand how instructions, models, retrieval systems, tools, identities, APIs, and enterprise data interact.

—

5. Privacy and Compliance Risk

AI systems frequently process information that was originally collected for another business purpose.

That creates important questions.

Should this information be provided to the model?

Is personal information being included unnecessarily?

How long are prompts, responses, transcripts, or logs retained?

Can users request correction or deletion where required?

Which geographic or regulatory requirements apply?

The answers depend heavily on the organization, industry, jurisdiction, data involved, and intended use.

This is why AI governance should involve privacy, legal, security, compliance, architecture, and business stakeholders rather than being owned exclusively by IT.

—

6. Agent and Action Risk

Generative AI becomes significantly more consequential when it can act.

There is a major architectural difference between:

Agent A: answers questions about an HR policy.

and

Agent B: updates an employee record, submits a payroll change, creates a purchase order, or approves a business transaction.

The second agent introduces action risk.

Organizations should evaluate:

– what actions an agent can perform;
– whether approval is required;
– whether actions are reversible;
– transaction limits;
– identity used for execution;
– auditability;
– exception handling;
– human escalation.

The more autonomy an AI system receives, the more important these controls become.

—

7. Operational Risk

An AI system that worked correctly during testing can still fail in production.

Knowledge changes.

Models change.

Prompts change.

APIs change.

Business processes change.

Permissions change.

Users discover scenarios developers never anticipated.

Therefore, deployment cannot represent the end of AI governance.

Organizations need continuous monitoring of metrics such as:

– response quality;
– failed actions;
– escalations;
– policy violations;
– abnormal usage;
– user feedback;
– security incidents;
– cost and consumption;
– evaluation scores.

The EU AI Act illustrates this lifecycle approach particularly clearly for high-risk AI systems: Article 9 requires a documented risk-management system operating as a continuous iterative process throughout the system lifecycle, including identification, evaluation, mitigation, and testing of relevant risks.

—

8. Business and Reputational Risk

An AI system can function exactly as designed and still create business risk.

An organization might automate a process that should remain human-led.

A customer-facing assistant could provide technically plausible but brand-inappropriate responses.

Employees might begin relying on an AI system for decisions it was never designed to make.

The final question should therefore not simply be:

“Does the AI work?”

It should be:

“What happens to the organization if people trust this system?”

That question changes the risk conversation considerably.

—

Not Every AI System Needs the Same Controls

A common governance failure is applying identical controls to every AI project.

That creates one of two outcomes.

Governance becomes so restrictive that teams bypass it, or controls become so lightweight that genuinely high-risk systems receive insufficient oversight.

A better approach is risk-based governance.

Consider three simplified tiers.

Tier 1 — Low Risk

Examples might include summarization of non-sensitive information, brainstorming assistants, productivity copilots, or internal FAQ systems using approved public information.

Typical controls might include:

– approved platforms;
– basic security controls;
– documented ownership;
– acceptable-use requirements;
– basic testing.

Tier 2 — Moderate Risk

Examples could include enterprise knowledge agents accessing internal information, customer-facing assistants, or AI connected to operational business data.

Additional controls might include:

– formal risk assessment;
– architecture review;
– access validation;
– evaluation datasets;
– privacy review;
– logging and monitoring;
– human escalation;
– periodic reassessment.

Tier 3 — High Risk

Examples could include systems influencing financial, employment, safety, healthcare, legal, or other consequential decisions, as well as autonomous agents performing sensitive transactions.

Controls may additionally require:

– independent validation;
– formal approval;
– enhanced human oversight;
– adversarial testing;
– detailed auditability;
– strict action boundaries;
– rollback or shutdown mechanisms;
– continuous monitoring.

The exact tiers and controls should reflect the organization’s regulatory obligations, business context, and risk tolerance.

—

NIST Provides a Useful Operating Model

One of the strongest starting points for enterprise AI governance is the NIST AI Risk Management Framework.

NIST organizes the AI RMF around four functions:

GOVERN — MAP — MEASURE — MANAGE

These four words can form the backbone of an enterprise AI governance program. NIST describes the framework as voluntary, rights-preserving, non-sector-specific, and use-case agnostic.

GOVERN

Establish the organizational foundation.

Define policies, responsibilities, accountability, risk tolerance, governance structures, documentation requirements, and escalation processes.

MAP

Understand the AI system in context.

Identify the business purpose, stakeholders, data sources, dependencies, intended users, potential impacts, and reasonably foreseeable misuse.

MEASURE

Evaluate the risks.

Test accuracy, reliability, security, privacy, robustness, bias where relevant, and other characteristics appropriate to the use case.

MANAGE

Decide what to do about identified risks.

Risks may be mitigated, transferred, avoided, accepted, or monitored depending on their severity and business context.

The framework is particularly useful because governance is not treated as a final approval gate.

Risk management becomes part of the lifecycle.

NIST has also published a Generative AI Profile extending the AI RMF specifically for generative AI. The profile identifies risks particular to or amplified by generative AI and provides actions organizations can use across the Govern, Map, Measure, and Manage functions.

—

AI Risk Management Needs an Inventory

An organization cannot govern AI systems it does not know exist.

This makes an enterprise AI inventory one of the most important foundations of an AI governance program.

At minimum, organizations should be able to answer:

– What AI systems exist?
– Who owns each one?
– What business problem does each solve?
– Which models are used?
– Which data sources are connected?
– Which identities are involved?
– Which external services are called?
– What actions can the system perform?
– Who can use it?
– What risk classification has been assigned?
– When was it last evaluated?
– Who approved production deployment?

This inventory becomes the foundation for architecture reviews, compliance assessments, operational monitoring, incident response, and lifecycle management.

Without inventory, AI governance becomes reactive.

—

The AI Risk Register

Organizations already use risk registers for cybersecurity, enterprise architecture, projects, and compliance.

AI should be no different.

A practical AI risk record could contain:

Risk → Cause → Impact → Likelihood → Severity → Control → Owner → Residual Risk → Monitoring Metric

For example:

Risk: Agent provides outdated safety instructions.

Cause: Knowledge source contains obsolete procedures.

Impact: Employee follows an incorrect procedure.

Control: Restrict retrieval to approved documentation, introduce content ownership and review dates, evaluate safety scenarios before deployment, and provide escalation when confidence or evidence is insufficient.

Owner: Business process owner.

Monitoring: Failed evaluation rate and safety-related escalations.

Notice what happened.

“AI hallucination” was transformed from a vague technical concern into a manageable enterprise risk.

That is the goal.

—

Human Oversight Should Be Designed, Not Assumed

Organizations frequently describe humans as being “in the loop.”

But that phrase means very little unless the architecture explains what the human actually does.

Does the human:

– review information?
– approve the recommendation?
– approve an action?
– receive an exception?
– investigate an incident?
– have authority to override the AI?

More importantly:

At what point does the human intervene?

Human oversight should therefore be represented explicitly in architecture and business-process design.

The OECD AI Principles similarly emphasize accountability, traceability, robustness, security, safety, and mechanisms allowing systems to be overridden, repaired, or safely decommissioned where appropriate.

—

AI Risk Management Is Continuous

Perhaps the most important change AI introduces to traditional governance is this:

Approval is not the finish line.

A production AI system is dynamic.

Its environment, information, integrations, users, and sometimes underlying models can change.

Risk assessment therefore needs triggers.

Reassessment might occur when:

– the model changes;
– system instructions change;
– a new knowledge source is connected;
– an API or tool is added;
– permissions change;
– the user population expands;
– the agent gains additional autonomy;
– a significant incident occurs;
– evaluation performance deteriorates.

AI governance must therefore move from:

Review → Approve → Forget

to:

Discover → Assess → Control → Deploy → Monitor → Reassess.

—

Standards Are Beginning to Converge

Organizations do not need to invent AI governance completely from scratch.

Several important frameworks now provide foundations.

The NIST AI Risk Management Framework provides a flexible structure centered on Govern, Map, Measure, and Manage.

ISO/IEC 42001:2023 establishes requirements for creating, implementing, maintaining, and continually improving an Artificial Intelligence Management System. ISO describes it as the world’s first AI management-system standard.

The OECD AI Principles, updated in 2024, emphasize trustworthy AI, human rights, transparency, robustness, security, safety, and accountability.

In 2026, the OECD also published its Due Diligence Guidance for Responsible AI, providing practical guidance for enterprises implementing responsible-business-conduct standards and the OECD AI Principles.

And the EU AI Act establishes legally binding requirements for covered systems in the European Union, including lifecycle risk-management obligations for high-risk AI systems.

These frameworks differ in purpose and legal status, but they point toward a common enterprise principle:

AI risk must be identified, documented, evaluated, controlled, monitored, and owned throughout the system lifecycle.

—

A Practical 90-Day Starting Point

Organizations do not need a massive governance program before beginning.

The first 90 days can focus on three phases.

Days 1–30: Discover

Build the AI inventory.

Identify existing AI applications, agents, models, knowledge sources, integrations, owners, users, identities, and data classifications.

Create an initial risk taxonomy.

Days 31–60: Decide

Define risk tiers.

Establish minimum controls for each tier.

Define who can approve AI systems, when architecture/security/privacy reviews are required, and which use cases require enhanced oversight.

Create the AI risk register and standard assessment process.

Days 61–90: Deploy and Measure

Apply the process to several real AI systems.

Run evaluations.

Document controls.

Establish monitoring.

Measure incidents, exceptions, evaluation results, adoption, and unresolved risks.

Then improve the governance model based on what you learn.

Governance should evolve through evidence rather than becoming a theoretical policy exercise.

—

The Most Important Question

Enterprise AI risk management ultimately comes down to accountability.

For every production AI system, someone should be able to answer:

What can this AI access?

What can it do?

What could go wrong?

What controls prevent that?

How would we know if those controls failed?

And who is accountable when they do?

If an organization cannot answer those questions, the problem is not simply an AI problem.

It is a management problem.

AI risk management is therefore not about preventing organizations from using AI.

It is about creating the conditions under which organizations can use AI confidently, deliberately, and at scale.

The organizations that succeed with enterprise AI will not necessarily be the ones that deploy the most AI systems.

They will be the ones that understand which risks are worth taking, which risks require controls, and which risks they are unwilling to accept.

References

1. National Institute of Standards and Technology (NIST), Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023.

2. NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile, NIST AI 600-1, July 2024.

3. International Organization for Standardization, ISO/IEC 42001:2023 — Information technology — Artificial intelligence — Management system, 2023.

4. OECD, OECD AI Principles, adopted 2019 and updated 2024.

5. OECD, Advancing Accountability in AI: Governing and Managing Risks Throughout the Lifecycle for Trustworthy AI, OECD Digital Economy Papers No. 349, 2023.

6. OECD, Due Diligence Guidance for Responsible AI, OECD Publishing, 2026.

7. European Union, Regulation (EU) 2024/1689 — Artificial Intelligence Act, particularly Article 9, Risk Management System.