The ongoing controversy surrounding Boeing's 737 Max 8 airliner should serve as a cautionary tale—not just for those in aviation, but for the entire transportation safety community. Though plane (and other transportation) crashes are an unfortunate reality, the recent Boeing disasters were unique. Instead of faulty mechanics, pilot error or foul weather, in these cases the culprit was software automation.
Onboard the 737 Max 8 was an advanced automated system meant to account for aerodynamic differences between the 737 Max 8 and its predecessors. Unknown to most pilots, this software adjusted the plane's bearings based on sensor data in accordance with preset rules. Because the software wrested control from human pilots, isolated system errors could spiral out of control. And unfortunately, they did. The potential for similar tragedies is not confined to the skies. Any automated transportation system can introduce similar risks.
Yet, as transportation networks grow ever more complex, the need for automation grows in turn. But the Boeing case shows just how dangerous automated systems that bypass human judgement can be. A far wiser approach would be to use AI to empower trained operators to make better decisions, not override them.
Why automation and AI are becoming key for cybersecurity
Let’s begin with a very natural question: if automated transportation systems introduce risk, why bother with them at all?
To answer that, let’s look at our area of expertise: cybersecurity. For increasingly connected hi-tech trains, planes and automobiles, the answer to that question is simple and reflects a blunt truth: Cyber threats are multiplying far faster and growing far too sophisticated for humans to deal with on their own. Hence the need for automation.
One recent study found that the global labour shortage in cybersecurity amounts to 3 million unfilled positions—which is why Chief Information Security Officers (CISOs) are flocking to AI-powered automation solutions to fill the gap. But then, even after resolving staffing shortfalls, there are still issues with maintaining visibility across ever-expanding networks. According to a recent Cap Gemini study, 61 per cent of executives confessed that their enterprise couldn’t detect breach attempts without the use of AI.
But automated cybersecurity doesn’t come without compromise.
Even fuelled with great training data, false positives (AKA false alarms) in automated systems are a growing concern. In some cases, this downside is acceptable. However, in transportation systems, cybersecurity false alarms are disruptive at best (imagine a train being stopped for hours) and can prove disastrous at worst. AI-based security systems often operate based on trial and error. But in transportation systems, there is no room for error: economies and lives are at stake.
Intrusive vs. non-intrusive AI systems
The solution to this dilemma rests in the two basic kinds of AI-based cybersecurity systems: Intrusive and non-intrusive.
Non-intrusive systems only aim to detect breach attempts, then flag them for an operator to decide whether to investigate any anomalies. Intrusive systems, on the other hand, go further and respond to perceived threats via active measures—automatically imposing controls on any relevant data or connected systems to proactively circumvent what they deem a likely breach in progress.
While intrusive systems provide deeper automation and lighten workloads, this efficiency often comes at far too high a cost for transportation security operators to bear. For example, if an intrusive system reacts unpredictably or registers a false positive, the safety-critical nature of the network means this one ripple could cascade throughout the entire system with potentially disastrous results.
Non-intrusive systems, however, offer a more hands-on approach. When employing such Intrusion Detection Systems (IDS), security operators receive much-needed assistance in spotting potential or actual threats but retain the freedom to exercise their better judgement in deciding upon their response.
In fact, for most transportation security applications, IDS solutions are the only practicable option.
Why only non-intrusive systems work for transportation
Critical systems in transportation, such as signalling and onboard networks, are held to the highest levels of safety.
For example, before any metro operation is approved to transport passengers, it must pass comprehensive tests against identified risks and hazards.
The signalling system offers another illustration: To receive certification, an intrusive system for signalling must be tested against every possible scenario in which it could potentially take direct command of the safety-critical controls. However, since there are endless ways to hack into a system, and therefore endless responses an AI system can have, they simply can’t all be tested. Consequently, such a system could never be certified.
Non-intrusive systems, however, never take control – they simply warn the human operator. Since the only way the train can actually stop in the face of malware is if the human operator deems the issue worthy of interrupting the system’s movement, this significantly limits the eventualities that must be tested to receive certification.
Moreover, there is even uncertainty about the operation of the AI system itself. Especially in more advanced deep-learning AI systems, the algorithm identifies a problem and the system takes appropriate measures to remedy it, but programmers have no idea why.
For example, let’s suppose the AI autonomously identifies a pattern in a transportation system that it judges to be a cyber threat. The entire operation could remain paralysed for days because the intrusive system took an action to stop the line. But no operational person could say if it was a good or bad decision.
AI can help, but humans must be at the wheel
Just like every other sector, the imbalance between defenders and attackers will necessitate some AI automation in transportation systems’ cybersecurity solutions. But the transportation sector must impose certain limitations on the degree of AI automation that can be safely deployed to ensure such security. Transportation operators must be delicate in deploying AI-based cybersecurity, being sure to keep ultimate control in the hands of human operators.
As opposed to intrusive systems which take control out of operators’ hands, advanced IDS solutions are the wiser option – at least for now, during the nascent phases of AI development.
Sophisticated non-intrusive detection systems empower operators to spot exponentially more threats than when unaided, while still retaining ultimate control – an advantage that’s sorely needed, as the recent Boeing disasters attest.
Serge Van Themsche is VP Strategic Partnerships, Cylus