Technology Radar
Threat modeling is an exercise for identifying potential threats to your systems. It’s a core security practice for all software development, especially for generative AI applications, whose distinctive security risks, such as prompt injection and autonomous action, may be missed by rule-based checks and scans. Threat modeling should be a regular, human-led part of the software development lifecycle, with support from agents. The discussion helps surface undocumented behaviors and build stakeholder understanding.
We recommend starting with known threats and mapping them to your context. Useful resources include the MITRE ATLAS catalog of AI threats, MAESTRO as a companion to frameworks such as STRIDE and toxic flow analysis for reasoning about where prompt injection might enter. A session should end with explicit decisions about where nondeterministic model behavior is acceptable, where it isn’t and where deterministic guardrails are needed to limit risk and harm.
En el panorama en rápida evolución del desarrollo de software impulsado por IA, el modelado de amenazas es más crucial que nunca para crear software seguro, manteniendo la agilidad y evitando el security sandwich. El modelado de amenazas; un conjunto de técnicas para identificar y clasificar potenciales amenazas, se aplica en diversos contextos, incluyendo aplicaciones de IA generativa, que introducen riesgos de seguridad únicos. Para que sea eficaz, debe realizarse con frecuencia a lo largo del ciclo de vida del software y funciona mejor junto con otras prácticas de seguridad. Entre ellas se incluye la definición de requisitos de seguridad interfuncionales para abordar riesgos comunes en las tecnologías del proyecto y el aprovechamiento de escáneres de seguridad automatizados para monitoreo continuo.
Seguimos recomendando que los equipos continúen haciendo uso del modelado de amenazas — un conjunto de técnicas que ayudan a identificar y clasificar amenazas potenciales durante el proceso de desarrollo — pero queremos enfatizar que esto no es una actividad a realizar únicamente al comienzo de los proyectos; los equipos deben evitar el security sandwich. Esto sucede porque durante todo el ciclo de vida de cualquier software, nuevas amenazas surgirán y las existentes continuarán evolucionando debido a eventos externos y constantes cambios de requisitos y arquitecturas. Esto significa que el modelado de amenazas se tiene que repetir de forma periódica — la frecuencia de repetición dependerá de las circunstancias y necesitará considerar factores como el coste de realizar el ejercicio y el riesgo potencial para el negocio. En combinación con otras técnicas, como el establecimiento de requisitos de seguridad multifuncionales para manejar riesgos comunes de las tecnologías del proyecto y el uso de escáneres de seguridad automatizados, el modelado de amenazas puede ser un poderoso recurso.
With the number of high-profile security breaches in the past months, software development teams no longer need convincing that they must place an emphasis on writing secure software and dealing with their users' data in a responsible way. The teams face a steep learning curve, though, and the vast number of potential threats—ranging from organized crime and government spying to teenagers who attack systems "for the lulz"—can be overwhelming. Threat Modeling provides a set of techniques that help you identify and classify potential threats early in the development process. It is important to understand that it is only part of a strategy to stay ahead of threats. When used in conjunction with techniques such as establishing cross-functional security requirements to address common risks in the technologies a project uses and using automated security scanners, threat modeling can be a powerful asset.
With the number of high-profile security breaches in the past months, software development teams no longer need convincing that they must place an emphasis on writing secure software and dealing with their users’ data in a responsible way. The teams face a steep learning curve, though, and the vast number of potential threats - ranging from organized crime and government spying to teenagers who attack systems 'for the lulz' can be overwhelming. Threat Modeling provides a set of techniques, mostly from a defensive perspective, that help you understand and classify potential threats. Turned into 'evil-user stories', threat models can give a team a manageable and effective approach to making their systems more secure.
At this point the vast majority of development teams are aware of the importance of writing secure software and dealing with their users’ data in a responsible way. They do face a steep learning curve and a vast number of potential threats, ranging from organized crime and government spying to teenagers who attack systems 'for the lulz'. Threat Modelingis a set of techniques, mostly from a defensive perspective, that help understand and classify potential threats. When turned into 'evil user stories' this can give a team a manageable and effective approach to making their systems more secure.
