Monitoring existing AI starts by connecting what was requested, who could authorize it, what was executed, and how the result was confirmed. This guide helps turn a general impression of control into verifiable questions.
Choose an operation and follow an action
Define the scope of a process that already uses AI. List the applications or agents involved and ask for a recent example of a request. Follow the path to the target system: did the AI only respond, prepare material, look up information, or change something? The same conversation may contain more than one of these behaviors.
Draw a simple line connecting the request, agent, tool, and system. When there is a stage the team does not know about, mark it as unknown. This map is preparation for investigation; it is not automatic discovery of all AI in the company. Starting with a concrete operation avoids confusing a partial inventory with complete coverage.
Locate authorization before execution
For each action, identify the account used, the required permission, and the rule that determines whether it can take place. Ask where that rule is applied. An instruction in the agent's text, a system permission, and human approval are different controls. Knowing that an access key exists does not explain which changes it permits.
If there is approval, check what was approved: the text of a message, an amount, a recipient, or a specific operation. Ask what prevents something different from being executed after that approval. When the team only has a description of the expected behavior, record “control declared; enforcement to be verified.” This distinction helps request the right evidence without treating the control as proven.
Separate usage limits and action limits
Budget, number of calls, and access to models help manage consumption. Permitted recipients, maximum amounts, types of changes, and execution times limit what can happen in the business. Organize these two groups separately and note the expected response when a limit is reached.
Ask for an example of a block or a safe way to test it. A dashboard that shows spending after the call may support monitoring without preventing the next action. Likewise, blocking a tool does not prove that the budget is under control. What matters is identifying when the check happens, who is responsible, and the alternative path when work needs to stop.
Look for the result in the target system
Connect the request to the execution record and confirmation in the system that received the action. A model call records part of the conversation; a tool call may show the attempt; the target system indicates whether the change was accepted. These records need to be interpreted together when the question is “did the work happen?”
Consider incomplete responses, delays, and repetition. If the tool did not respond, could another attempt duplicate the change? Who checks the situation before resending? If the team cannot connect the records, note the gap and arrange a check with whoever maintains the integration. Do not conclude that everything failed or everything worked just because one piece of evidence is missing.
Illustrative example · no customer data
The agent replied “record updated”
Illustrative example: a person requests an address change. The conversation contains the response “done,” and the dashboard shows a model call. The team still needs to verify which tool made the change, which record was affected, and which record confirms the new address.
If only an attempt that ended without a response appears, the correct status for investigation is “result not confirmed.” Before repeating it, someone checks the record and verifies whether the change has already occurred. The next task is to close that confirmation gap; an invented risk score would not help the team carry out this step.
When the approach needs to change
If only conversation records are available, start by identifying the integrations. If the process only provides information, look for the source and version of the information presented. When another provider executes the action, ask what evidence they can make available; model gateway visibility does not automatically extend to that path.
A point to watch: Treating the response “done” as proof of execution or confusing consumption limits with action limits.
The worksheet is ready to move forward when…
- The map distinguishes a response, an attempt, and a confirmed result.
- Permissions and limits have identified owners and enforcement points.
- Each gap has a concrete question and someone who can help verify it.
Complete the assessment with the information provided and use “I don't know” for open points. The preliminary map organizes the discussion; confirming the controls requires investigating the operation.
Your worksheet
Complete it with your team.
Record what you know and what still needs confirmation. Answers stay in memory and are not submitted.
References for further reading
- RFC 9110 · section 9.2.2: idempotent methods ↗
Defines idempotency and the conditions for retrying HTTP requests. It helps inform discussion of call recovery; authorization and confirmation of the result remain matters specific to the operation.