ai · hard
"What safeguards would you build into an agentic AI product?"
What safeguards would you build into an agentic AI product, and how would you decide which ones to prioritize?
This question is not about safety checklists. It is testing whether you can make product trade-off decisions under uncertainty: which safeguard to apply to which action class, how to enforce it outside the model, and how to instrument the product so you know when the safeguards are working. The weak answer recites a list. The strong answer shows you understand where agents actually fail and have a design response to each failure mode.
Why this question matters in 2026
In 2026, feasibility is free. You can build an agent that books flights, manages inboxes, or executes code with less effort than ever before. The constraint is not “can we build it” but “will users actually hand over control.” Trust is the primary PM design problem for agentic products. Safeguards are not a compliance checkbox or a safety team’s problem: they are the product. An agent that acts without legibility loses users in week two. An agent that asks for permission on every read-only lookup gets disabled. The PM’s job is to find the narrowest set of safeguards that preserves user confidence and business liability while letting the agent deliver the value it was built to deliver.
Viable now depends directly on the trust architecture. No enterprise will pay for an agent that cannot produce an audit trail or scope its permissions per workflow.
Structure a strong answer
Start with the classification system, then derive every other decision from it.
strong
"I start by classifying every action an agent can take by its reversibility, because that classification drives every other design decision.
Four tiers. Read-only: search, summarize, retrieve. No confirmation needed, but full traceability. Reversible: create a draft, add a calendar invite, create a file. Show a preview with a diff view and a short undo window (10 to 30 seconds). Partially reversible: send an email, post a comment, submit a form. Explicit confirmation with plain-language consequence description, because you can follow up but you cannot unsend. Destructive: delete data, make a payment, cancel a subscription, revoke access. Hard gate, possibly multi-party approval, mandatory audit log entry.
Second, scope the agent's permission footprint to the minimum needed for the current task, and enforce that at the infrastructure layer, not in the system prompt. This is the most common mistake in agentic product design. Telling an agent 'do not delete production data' is not a safeguard. Revoking the API permission is a safeguard. The model's instruction-following is not reliable enough to be the primary enforcement mechanism. Amazon Bedrock Guardrails' InvokeGuardrailChecks API (2025) takes this approach, applying guardrail checks at any point in the agentic workflow, not just at model output. Anthropic's published guidance on agentic use makes the same point: agents should request only necessary permissions and maintain minimal footprint. CISA/NCSC joint guidance from 2025 is explicit: least privilege must be enforced per task, not per deployment.
Third, handle uncertainty explicitly. Define a confidence threshold below which the agent surfaces a recommendation rather than taking action. This threshold should be tunable per action class and per user trust level. A code agent running against a production branch and a note-taking agent should not share the same default threshold. The threshold is a product parameter, not a model parameter.
Fourth, make the agent's reasoning legible, and layer the display. One-line summary by default, expandable detail on demand. Stanford HAI research found that exposing chain-of-thought reasoning reduced user anxiety by 34% in agentic task settings. Microsoft Copilot Adaptive Cards show the full execution plan before the agent acts, with internal data showing 28% higher acceptance rates compared to agents that act then explain. Google Gemini Live uses a persistent HUD showing microphone state and thinking progress. Transparency is a UI pattern, not a legal footnote at the bottom of a confirmation modal.
Fifth, build for recovery as a fallback, not a primary safeguard. Dropbox's reversible rewind pattern for file changes cut support tickets on accidental agent actions by 54%. Version every write, expose one-click rollback, keep the window open long enough to matter. Recovery design is most important for partially reversible actions where the primary gate already failed.
Sixth, instrument for trust calibration, not just error rate. The leading metrics are approval rate, override rate, and escalation rate. If override rate climbs, my confidence threshold is too low. If no one ever overrides, the gate may be creating false confidence. Repeat usage and approval rate are the product health metrics that tell you whether the trust architecture is actually working.
The meta-principle: safeguards that add friction without value will be turned off or routed around. Good safeguard design is invisible for routine actions and highly visible exactly when stakes are high."
weak
"I would add human oversight, set confidence thresholds, and make sure the AI explains its reasoning." This fails for three reasons. It does not distinguish which actions need which safeguards, treating a file search and a payment authorization as equivalent. "Human oversight" is not a design decision; it is a category. The interviewer needs to know who, when, what triggers it, and what happens to in-flight tasks when it fires. And it ignores enforcement: nothing here prevents the agent from calling an API it was told not to call. The deeper problem is that this answer has no UX dimension. Safeguards that create constant friction destroy the product before the safety risk does. The interviewer hears this and concludes the candidate has read a checklist but has not shipped anything.
The failure modes interviewers are probing for
Interviewers at AI-native companies (Anthropic, OpenAI, Google DeepMind, Sierra, Glean, Harvey) are listening for whether you know where agents actually break, not whether you know the vocabulary. The four failure modes that matter for PM interviews:
Prompt injection. A malicious document or webpage can redirect an agent’s action space. The PM safeguard is sandboxed tool execution environments, not safer prompting. You cannot prompt-engineer your way out of a prompt injection vulnerability.
Scope creep. An agent given a broad goal expands into tool access it was not designed to use. Underspecified scope boundaries are the root cause of most publicized agent incidents. Scope must be declared per task, not per session.
Irreversible actions without gates. The agent executes a destructive action before the user could intervene. Classification-driven approval gates, not general confirmation dialogs, are the response.
Trust miscalibration. Users rubber-stamp because confirmation fatigue has made every approval feel routine. Override rate and approval rate are the signals. If your approval rate is 99%, you have theater, not a safeguard.
Follow-up probes to expect
- “What if the user is offline when the agent needs approval?” Tests async confirmation design. Inline approval dialogs are useless for background agents. The right answer: escalate to a mobile push with a defined time window; if no response, hold the action, do not skip it.
- “How do you enforce scope outside the model?” Tests whether you understand the enforcement distinction. Revoke API access; do not rely on system prompt instructions.
- “How do you know when to expand the agent’s autonomy?” Tests whether you have a measurable ladder of trust. Name specific signals: override rate below X% over N actions, error rate under Y%.
- “What’s your threshold for requiring confirmation vs. proceeding silently?” If you say “anything risky,” you have not answered the question.
The Build5Nines five-tier autonomy model (Observe / Recommend / Prepare / Execute with Approval / Execute Automatically) is a useful PM-facing framework for the autonomy expansion probe. Match each tier to the reversibility classification and you have a concrete answer.
For the full guardrail architecture by component, see design guardrails for an agent. For the antipatterns that safeguards can accidentally create, see obnoxious AI antipatterns. For the cheat-sheet version of agent trust vocabulary, see agent guardrails cheat sheet.