The non-profit organization MITER, together with twelve companies and organizations, presented a framework that describes threat scenarios for machine learning applications. Data scientists, software developers, security operation centers and testers should use it to identify attack vectors and prepare their systems for them.

The Adversarial ML Threat Matrix is ​​based on ATT & CK framework (Adversarial Tactics, Techniques & Common Knowledge) from MITER, which describes threats from the outside perspective of the attacker. The word “Adversarial” can be translated as opposing or, better, hostile, since the original ATT & CK matrix is ​​based on the Cyber ​​Kill Chain of the armaments company Lockheed Martin.

Other threat scenarios The motivation for the ML threat matrix is ​​the attacks on systems from companies such as Google, Amazon, Microsoft and Tesla that have become known. In the field of artificial intelligence, attackers use other methods than are known from classic attacks on networks, software or cloud applications. Among other things, the pattern recognition for street signs can be modified through subtle changes so that an autonomously driving car sees a completely different sign than a person.

The neural network interprets the subtly manipulated Speed ​​limit as a stop sign.

(Image: “DARTS: Deceiving Autonomous Cars with Toxic Signs”, arXiv.org)

There are also attacks through so-called data poisoning, that is, poisoned data that convey incorrect information to the ML system during training. Attackers can also, for example, by importing their own samples and templates into the training of an ML system to detect network attacks, impersonate their approach to hacks as normal behavior. Theft of ML models is also one of the increasing attacks.

Seven pillars The Adversarial ML Threat Matrix is ​​divided into seven areas that describe the entire life cycle from reconnaissance through execution to impact. In addition to ML-specific attacks, conventional software attacks and phishing can be found in the matrix.

The Adversarial ML Threat Matrix is ​​in divided into seven areas

(Image: advmlthreatmatrix repository from MITER on GitHub)

The full description of the matrix can be found in MITER’s GitHub repository. In addition to MITER, the organizations and companies Microsoft, Bosch, IBM, Nvidia, Airbus, PricewaterhouseCoopers, Deep Instinct, Two Six Labs, the University of Toronto, Cardiff University, the Software Engineering Institute of Carnegie Mellon University and the Berryville Institute o