Is Artificial Intelligence the Shield?

The year is 2023. We have been advancing artificial intelligence (AI) for more than 60 years. We now live in a world where machine learning and humanoids have replaced the need for security analysts, and cybercrime is at an all‐time low.

This would have been a very reasonable prediction in 1984 when Douglas Lenat and his team began work on Cyc, which aimed to encode common sense into a machine.

Using a relational set of 24.5 million rules, Cyc has yet to reach general intelligence. We are also still many years away from AI tools fully replacing the human element in cybersecurity. But can AI and machine learning bring us closer to the “single pane of glass” dashboard we desire as security practitioners?

―

With more than 200 adversary techniques identified in the MITRE ATT&CK Matrix, what detection capabilities does AI provide in solutions that are currently on the market?

I believe we lack the transparency from vendors that would enable security practitioners to thoroughly evaluate the effectiveness of defenses we are deploying. We have seen the power of AI to win chess matches or help detect cancer, but we are surrounded by false advertisements of battle-tested, industry-leading algorithms that detect threats across the enterprise and work around the clock to provide world-class security. These advertisements misrepresent genuine challenges associated with staffing the necessary experts to design and monitor critical infrastructure, asking us to falsely believe that technology can eliminate these challenges.

As we are building the grid of the future, witnessing its digital transformation, and defending against the most sophisticated cyber attacks in history, we need to understand the fundamentals of AI capabilities and limitations.

Before we discuss the challenges and opportunities of AI in cybersecurity, it’s important to set a baseline understanding of deep learning, which is a subset of machine learning, and how it works.

Deep neural networks are considered “deep” because they contain multiple hidden layers. A neural network sums its inputs and applies an activation function to determine its output, which is similar to how our brains work. Inputs are represented as neurons, and each layer is essentially a weighting function. The machine learning algorithm analyzes attributes of the neurons in stages by matching characteristics with parameters of each layer.

A neural network is trained by analyzing the output versus the desired state while incrementally changing the network state and adjusting the weighting factors at each layer. This process requires a pristine data lake of input examples. Industrial control system (ICS) networks have only been successfully attacked by less than a dozen malware packages; this presents a challenge to creating a pristine data lake.

We must also recognize the inherent challenges that AI faces. Let’s review a few:

Brittleness

An AI can only recognize a pattern that it has seen before. A 2018 study [1] found that a state‐of‐the-art AI would correctly identify a school bus right‐side‐up yet fail 97 percent of the time if the image was rotated. We have seen other patterns of brittleness, such as autonomous cars being tricked by stickers on road signs or intrusion detection algorithms’ inability to comprehend the location of traveling employees.

Catastrophic Forgetting

If you visualize AI algorithms as a set of stacked scoring filters, it’s understandable that as you try to train a new desired detection that adjustments to the filters and weighting will cause less accuracy in previous detection capabilities.

A simple example is training an AI to play chess and then trying to add checkers. The skills of detecting solid moves at chess will be overwritten by checkers strategy due to the limited number of analysis layers.

There are methods to deal with catastrophic forgetting, such as creating a specialized neural network for each type of analysis to be performed. However, this is not a scalable solution because there are millions of permutations of techniques, tactics, and procedures of cyber behavioral attacks.

Explainability

While Cyc was an example of symbolic AI, which is primarily used for reasoning and provides explainable AI, neural networks tend to be black boxes. ICSs involve the control of expensive equipment, provide life-preserving public services, and impact the safety of personnel. Without explainability, it will be challenging to accept allowing control of communications or other high‐stakes predictions that come with legal, safety, and other consequences.

―

So, what does this tell us about applying AI to ICS networks? First, we cannot expect to have enough high-confidence attack scenarios to train the system over time using only the traffic on individual networks. We also cannot expect machine learning algorithms to learn new detections as threats evolve without compromising the accuracy of the detection layers.

Most importantly, we will continue to need a human in the loop to make sense of the anomaly detections that the AI will not be able to fully explain.

We are already seeing positive trends in the industry with programs to share information about network behavior to a centralized data lake. We captured ICS malware (Pipedream) that highlights the inherent weaknesses that adversaries are trying to expose, and we have far more computer power to run sophisticated algorithms in real time.

Protecting our nations’ critical services, like water, electricity, and manufacturing, is a very important task. We must also be aware of the safety impact that control system operations have, whether that be chemical levels in water, rpm of a centrifuge, or moving pots of molten slag.

As the field of AI continues to advance, we need to ask tough questions about how the system works, what operational impact it could have, and how we as humans can turn tools into teammates.