🛡️
Security Engineers, AppSec, Red Teamers & Security Architects

AI Security Engineering

AI applications have a new attack surface. Learn to test it, break it and defend it.

A hands-on track for security professionals responsible for AI systems. LLM applications, retrieval pipelines and AI agents introduce risks that traditional application security does not cover: prompt injection, data leakage through retrieval and outputs, over-privileged agents and a new supply chain of models and tools. This track teaches you to threat-model these systems, test them the way attackers do, harden them, and monitor them in production, mapped to recognised frameworks. All attack labs run against purpose-built practice applications.

Duration
8 weeks · 6hrs/week
Format
Live online + async labs with HYVE engineer code review
Prerequisite
At least two years in application security, penetration testing or security engineering

Curriculum

01

The AI Attack Surface

6hrs

How LLM applications, retrieval pipelines and agents are built, and where each part can be attacked. What changes when software follows instructions written in natural language. Threat modelling for AI systems, and an introduction to the OWASP Top 10 for LLM Applications and MITRE ATLAS.

Lab: Threat-model a sample AI customer-service application end to end.
02

Prompt Injection, Direct and Indirect

7hrs

Direct injection and jailbreaks, and the more dangerous indirect injection hidden in documents, emails, web pages and tool results that an AI system reads. Why prompt injection cannot be fully prevented by prompting alone, and the layered defences that reduce its impact.

Lab: Exploit and then mitigate injection flaws in a deliberately vulnerable retrieval application.
03

Data Leakage and Privacy

6hrs

How sensitive data escapes AI systems: over-broad retrieval, missing access controls, verbose logs, cached responses and model outputs. PII handling, access-aware retrieval and privacy requirements under data protection laws such as the UAE PDPL and GDPR.

Lab: Find and fix data exposure paths in a sample knowledge assistant.
04

Securing Agents and Tool Use

7hrs

Agents act, so their mistakes and compromises act too. Least-privilege tool design, scoped credentials, sandboxing, rate limits and human approval gates for high-impact actions. Securing MCP servers and tool integrations, and auditing what an agent actually did.

Lab: Harden an over-privileged agent and prove its blast radius is contained.
05

AI Red Teaming in Practice

8hrs

Planning and scoping an AI red-team engagement, building attack libraries, and combining manual testing with automated tooling. Rating severity for AI-specific findings and writing reports that engineering teams can act on.

Lab: Run a scoped red-team exercise on a practice application and write the findings report.
06

AI Supply Chain Security

5hrs

Risks in third-party models, datasets, plugins, packages and MCP servers. Model provenance, dependency scanning and vendor security review. Policies for approving new AI components in your organisation.

Lab: Assess the supply chain of a sample AI application and produce an approval checklist.
07

Guardrails, Detection and Response

6hrs

Input and output filtering, policy enforcement, and their limits. Logging and monitoring that detect abuse, data exfiltration and anomalous agent behaviour. Incident response playbooks for AI systems.

Lab: Build monitoring rules and an incident playbook for a production-style AI service.
08

Capstone: Break and Harden an AI Application

7hrs

Take a complete AI application from threat model through attack, remediation and monitoring, mapping every control to the NIST AI Risk Management Framework and OWASP guidance. Present your assessment to a panel of HYVE engineers.

Lab: Deliver a full security assessment and hardening plan for the capstone application.

Learning Outcomes

  • ✓Threat-model LLM applications, retrieval pipelines and AI agents
  • ✓Test for prompt injection, including indirect injection through documents, web pages and tool outputs
  • ✓Assess data leakage risks across training data, retrieval, logs and model outputs
  • ✓Secure agent tool use with least privilege, sandboxing and approval gates
  • ✓Run structured AI red-team exercises and report findings with severity and fixes
  • ✓Harden the AI supply chain, including models, datasets, plugins and MCP servers
  • ✓Build guardrails and monitoring that detect abuse in production
  • ✓Map AI security controls to the OWASP Top 10 for LLM Applications, MITRE ATLAS and the NIST AI RMF

FAQs