What is a tool-call risk rating?

A heuristic that labels each tool an agent can call as low, medium, or high based on its blast radius — read-only is low, an outbound request is medium, an irreversible or destructive action is high. Each tier gets a matching control.

How do I control a medium-risk tool?

Typically a rate-limit plus an allowlist — cap how often it fires and restrict the destinations or recipients it may reach (e.g. email only to your domain, HTTP only to approved hosts).

Why does a high-risk tool need human approval?

Because the action is irreversible or broadly destructive (delete, write to a database, transfer funds). A person should confirm before it executes. This is also the practical expression of EU AI Act Art. 14 human oversight.

Does rating tools guarantee the agent is safe?

No. A rating reduces obvious failure modes and makes risk visible, but it depends on a complete, current tool inventory and on the allowlists actually being tight. It is a control, not a guarantee.

What happens to a tool with no rating?

In a well-designed system it should default to blocked or flagged for review. An unrated tool is an unguarded tool — the inventory must be reviewed whenever capabilities change.

Tool-call risk rating: a practical framework for AI agents

The three-tier model

Most agent incidents trace to one tool call that was allowed but unguarded. A tool-call risk rating labels each tool an agent can invoke so the right control attaches automatically:

Low — read-only, no external side effect. Example: a lookup, a read-only DB query with a row cap.
Medium — reaches outside the boundary or mutates local state. Example: an outbound HTTP request, sending an email.
High — irreversible or broadly destructive. Example: delete_record, run_sql with writes, a financial transfer.

The rating is a heuristic view of exposure, not a promise that nothing will go wrong.

Controls per tier

Tier	Example tools	Control
Low	read-only lookup, get_status	Log only
Medium	send_email, http_request	Rate-limit + recipient/domain allowlist
High	delete_record, run_sql (write), transfer	Require human approval + full audit log

The key idea: the control escalates with the tier. A low tool needs a log; a high tool needs a person. This keeps the human in the loop exactly where the blast radius is largest (aligned with EU AI Act Art. 14 oversight).

Worked example

An agent is given four tools:

get_order → Low (read-only).
send_email → Medium (rate-limit to 50/day, allowlist recipients to @yourco.com).
http_request → Medium (domain allowlist; block everything not pre-approved).
delete_record → High (never auto; require a named approver + write the request to an audit log).

With ratings attached, a prompt-injection attempt that says "delete everything" hits the High gate and stops for a human instead of executing.

Limits of the rating

A rating is only as good as the tool inventory behind it. If a new tool is added without a rating, it defaults to unguarded — which is why the inventory itself should be reviewed whenever the agent's capabilities change. And ratings reduce, but do not eliminate, risk: a Medium tool with a weak allowlist can still leak. Treat the rating as the start of control, not the end.

Authoritative references

EU AI Act, human oversight (Art. 14): https://eur-lex.europa.eu/eli/reg/2024/1689/oj
OWASP Top 10 for LLM Applications (tool/function abuse): https://owasp.org/www-project-top-10-for-large-language-model-applications/
AI Act explorer: https://artificialintelligenceact.eu/