Agentic AI Security Testing: Tools, Memory, Permissions and Human Approval

Agentic AI security testing must evaluate how a system converts observations into plans, permissions and real-world effects. A model can produce harmless text in one context and trigger email, code, cloud or financial actions in another. The security boundary is the delegated authority around the agent.

Map the agent control loop

스테이지	Security property	Test evidence
Observation	Untrusted data is labeled and constrained	Source, trust label and parsed content
Planning	The agent cannot silently expand the authorized objective	Task, plan revisions and policy decisions
Memory	Users, tenants and tasks remain isolated	Memory keys, reads, writes and lifecycle
Identity	Delegated credentials are least privilege and task-bound	Principal, scopes, audience and expiration
Tool use	Each tool independently validates arguments and authorization	Call, approval, downstream decision and result
효과	High-impact changes are visible, reversible and auditable	Target state before and after execution

For the assessment framework, consult OWASP LLM Top 10 and map the relevant guidance to the actual environment and authorized scope.

사용 OWASP LLM Top 10, NIST Generative AI Profile 그리고 MITRE 아틀라스 to inform the threat model. The engagement should still be built from the agent’s actual tools, identities and business impact.

Conceptual scope model for an authorized assessment; the engagement must name the actual assets and identities.

Start a Security Assessment

Test goal and scope integrity

Give the agent an authorized objective with explicit exclusions. Introduce ambiguous instructions and untrusted content that encourages scope expansion. Confirm that the agent asks for clarification or refuses rather than interpreting technical reachability as authorization.

Memory testing

cross-user and cross-tenant retrieval;
task completion and deletion lifecycle;
poisoned long-term memories;
secrets copied from tools into broader memory;
stale permissions retained in future sessions;
summaries that lose provenance or policy context.

Delegated identity

Test whether credentials are bound to the correct user, agent, tool, audience, resource and time. Avoid broad developer tokens shared by every task. Revoke access during execution and verify that running and queued actions stop.

Multi-agent systems

When agents delegate to one another, record which agent selected the subtask, what context and permissions were transferred and how the result was validated. A low-privilege planner should not acquire high privilege merely by asking a worker with broader credentials.

Human approval

Approval should name the exact action, target, identity and important arguments. Test approval replay, plan changes after approval and batched operations that hide one dangerous action among many routine steps. A human clicking continue is not meaningful control if the operation has changed.

Stop and rollback

Cancel the agent during planning, tool execution and a retry loop. Confirm that workers, subprocesses and callbacks terminate. For reversible test actions, verify rollback. Record partial completion so operators know which side effects remain.

Evidence requirements

A finding should connect untrusted input or policy failure to the specific delegated authority and downstream effect. Preserve the task, model and policy context, memory access, tool call, approval and result. Label inference and nondeterministic behavior rather than converting one surprising run into a universal claim.

Verification workflow connecting the tested identity and observed result to the deployed fix and adjacent regression checks.

Start a Security Assessment

Vendor acceptance test

Create two tenants, isolated memory and a harmless state-changing tool.
Seed indirect instructions in one tenant’s content.
Attempt cross-tenant retrieval and unauthorized tool parameters.
Change the plan after approval and verify re-approval.
Cancel during a multi-step action and inspect remaining effects.
Require an auditable report and remediation retest.

Final decision

An agentic AI test is successful when it validates delegated authority, not when it merely produces adversarial prompts. Buyers can assess Penligent’s AI testing workflow with the same identity, evidence, approval and stop-control requirements.