Physical Security May Be Asking the Wrong AI Governance Question

As artificial intelligence (AI) becomes more capable, the physical security industry is appropriately spending more time discussing responsible AI. Much of that conversation has focused on important issues such as transparency, privacy, bias, human oversight and accountability. But as AI begins moving from analysis into operational workflows, we need to add a more fundamental question: How much authority are we giving to AI?

Consider the difference between three systems. The first summarizes an incident report for an investigator. The second analyzes available information and recommends that an incident be escalated. The third initiates the escalation, changes an access permission and triggers a security response. All three may rely on similar underlying AI capabilities, but they do not carry the same operational authority and should not require the same level of governance.

This is where our industry can make the responsible AI discussion more practical. Rather than beginning with whether a system uses AI, generative AI, or an agent, we should evaluate risk based on two factors—the authority delegated to the system and the consequence if it is wrong. In other words: AI Risk = Authority × Consequence.

Authority Changes the Risk

Authority exists on a spectrum. An AI system may observe information, analyze it, recommend an action, decide among available actions or be authorized to act. Moving along that spectrum changes the governance requirements because the system is no longer just collecting information, it is increasingly participating in the operation itself.

An inaccurate AI-generated summary can create confusion or send an investigation in the wrong direction. An AI agent independently changing someone’s physical access or initiating a security procedure, however, creates a different level of exposure. The technology underneath those systems may be similar, but the authority granted to them is materially different.

This distinction also makes the familiar concept of “human in the loop” more useful. Human oversight is an important control, but requiring the same human intervention in every AI-enabled process is neither realistic nor necessarily desirable. A person may need to approve every action in one workflow while supervising exceptions in another. What should not change is accountability.

Human accountability is constant. Human intervention is variable.

That distinction becomes increasingly important as organizations automate more routine decisions. The objective should not be to insert a person into every interaction simply so we can say a human was involved. The objective should be to understand where human judgment and approval are required based on the authority being delegated and the potential consequence of an error.

Consequence Determines the Level of Assurance

Authority, as noted above, is only half of the equation. We also need to understand what happens when the system is wrong.

A low-impact administrative recommendation and a decision affecting someone’s safety, privacy, access or freedom of movement may both technically be AI-assisted decisions but treating them as equivalent creates problems in both directions. If every use of AI is governed as though it carries significant operational risk, governance can become an unnecessary barrier to useful adoption. If every use is treated as low risk, organizations may delegate meaningful authority without recognizing the controls that should accompany it.

A more practical approach is proportional: As authority increases, assurance should increase, and as the consequence of error increases, assurance should increase. When both are high, organizations should expect stronger requirements around validation, traceability, permissions, monitoring and human approval.

This also means that confidence should not be confused with authority. A model becoming more accurate does not, by itself, justify granting it greater decision-making authority. Accuracy and confidence can inform the decision to delegate, but the authority still must be explicitly granted within an appropriate governance structure.

AI Agents Make the Question More Urgent

This framework becomes especially important as the security industry moves from AI that gathers information to AI agents capable of participating directly in operational workflows. At that point, simply asking whether the model produces the right answer is no longer sufficient. We also need to understand what the agent is authorized to do, what systems it can access, what decisions it can make, what actions require separate approval and whether we can reconstruct what it observed, recommended, decided and did.

Fortunately, these are not entirely new governance problems. Enterprises already manage identity, access, permissions and auditability for people and software interacting with sensitive systems. AI agents should be treated with the same discipline. If an agent is going to participate in a security process, it should have an identifiable role, explicit permissions and clearly defined boundaries around its authority.

This is also why responsible AI cannot simply be a checklist applied after a model has been introduced. Governance must be designed into the operational process itself. Organizations need to understand the work, determine where judgment and accountability reside, decide explicitly what authority can be delegated and apply controls that are proportional to the consequences.

The impactful decisions supported by security systems make that discipline particularly important in this industry. At the same time, overly restrictive governance could prevent organizations from using AI to improve operations, reduce administrative burdens and help people make better decisions.

The goal, then, should not be to minimize AI or maximize autonomy. It should be to delegate authority deliberately, with assurance proportional to the consequence of getting it wrong. As AI capabilities continue to advance, that may be a more durable foundation for responsible adoption than asking what the technology can do. The more important question is what we are prepared to authorize it to do, and under what conditions.

The views and opinions expressed in guest posts and/or profiles are those of the authors or sources and do not necessarily reflect the official policy or position of the Security Industry Association.

This article originally appeared in All Things AI, a newsletter presented by the SIA AI Advisory Board.