Safe & Secure Agentic AI: Governance, Identity, and Red-Teaming Training Course

Foundations: Threat Models for Agentic AI

• Identifying agentic threat vectors: misuse, escalation, data leakage, and supply-chain risks.
• Analyzing adversary profiles and attacker capabilities specific to autonomous agents.
• Mapping critical assets, trust boundaries, and control points for agent operations.

Governance, Policy, and Risk Management

• Establishing governance frameworks for agentic systems, including roles, responsibilities, and approval gates.
• Crafting policies for acceptable use, escalation rules, data handling, and auditability.
• Addressing compliance considerations and preparing evidence for audits.

Non-Human Identity & Authentication for Agents

• Designing agent identities: utilizing service accounts, JWTs, and short-lived credentials.
• Implementing least-privilege access patterns and just-in-time credentialing.
• Managing the identity lifecycle: rotation, delegation, and revocation strategies.

Access Controls, Secrets, and Data Protection

• Implementing fine-grained access control models and capability-based patterns for agents.
• Managing secrets, ensuring encryption-in-transit and at-rest, and practicing data minimization.
• Securing sensitive knowledge sources and PII from unauthorized agent access.

Observability, Auditing, and Incident Response

• Designing telemetry for agent behavior, including intent tracing, command logs, and provenance.
• Integrating with SIEMs, setting alerting thresholds, and ensuring forensic readiness.
• Developing runbooks and playbooks for managing agent-related incidents and containment.

Red-Teaming Agentic Systems

• Planning red-team exercises: defining scope, rules of engagement, and safe failover mechanisms.
• Applying adversarial techniques: prompt injection, tool misuse, chain-of-thought manipulation, and API abuse.
• Conducting controlled attacks to measure exposure and impact.

Hardening and Mitigations

• Applying engineering controls: response throttles, capability gating, and sandboxing.
• Implementing policy and orchestration controls: approval flows, human-in-the-loop, and governance hooks.
• Utilizing model and prompt-level defenses: input validation, canonicalization, and output filters.

Operationalizing Safe Agent Deployments

• Adopting deployment patterns: staging, canary, and progressive rollout for agents.
• Managing change control, testing pipelines, and pre-deploy safety checks.
• Fostering cross-functional governance: coordinating security, legal, product, and ops playbooks.

Capstone: Red-Team / Blue-Team Exercise

• Executing a simulated red-team attack against a sandboxed agent environment.
• Defending, detecting, and remediating as the blue team using established controls and telemetry.
• Presenting findings, remediation plans, and proposed policy updates.

Summary and Next Steps