MALIAB: MAL-based Imitation for Adversarial Behaviour

MALIAB is a data-driven framework for simulating attacker behaviour based on Cyber Threat Intelligence (CTI) data.

The system under attack is modeled using a Meta Attack Language (MAL) compliant domain-specific language (DSL), producing an instance model. Using the instance model, a scenario is defined for the attacker and acts as input to the MAL simulator. The simulator generates an attack graph, which can be used to simulate cyber attacks against the system. The attack graph consists of nodes, called attack steps, which which correspond to the actions available to an attacker in the MAL simulator. See the page on the MAL simulator for more details on it.

To model attacker behaviour, different action selection policies can be used. The baseline policies implemented are:

• Random: selects a random action from the available actions in the attack surface.
• TTP Weighted Random: selects a random action, conditioned on the TTP (Tactics, Techniques, and Procedures) profile of an Advanced Persistent Threat (APT) group.

Reinforcement Learning (RL) can be used to model the attacker based on a reward function, defined here using Time to Compromise (TTC) in the MAL language. The reward function is formulated as

r(s,a)=−1TTC(a)⋅TTC(a)

for an action a in state s, where 1TTC(a) is an indicator function that is 1 if the action has a TTC defined. See the article on MAL for more information on the definition of TTC. The RL policies implemented are:

• DQN: Deep Q-Network, a value-based RL algorithm that uses a neural network to approximate the Q-value function.
• Rainbow DQN: an improved version of DQN that combines several enhancements, including Double Q-learning, Prioritized Experience Replay, and Dueling Networks.
• PPO: Proximal Policy Optimization, a policy-based RL algorithm that optimizes the policy directly using a surrogate objective function.

Imitation Learning (IL) can be used to model the attacker based on expert demonstrations. The attack graph is further mapped to the actions described in the cyber attack by the CTI data, forming an action sequence over attack graph nodes that serves as expert demonstrations. The IL policies implemented are:

• BC: Behavioural Cloning, a supervised learning approach that learns a policy by mimicking the expert's actions.
• GAIL: Generative Adversarial Imitation Learning, an adversarial approach that learns a policy by training a generator (the policy) and a discriminator (the reward function) in a minimax game.
• AIRL: Adversarial Inverse Reinforcement Learning, an adversarial approach that learns a policy and a reward function simultaneously, similar to GAIL but recovering a reward function that generalizes beyond the training environment.
• IQ-Learn: Inverse Q-Learning, a method using soft Q-learning to learn a policy and a reward function simultaneously.