AI Safety, Leaking Abstractions and Boeing’s 737 Max 8

If MCAS is turned off, the pilots will find themselves to be flying an entirely different plane.

When you abstract away interaction with reality, you cannot avoid introducing a process that mediates between a pilot’s action and the actual actions of the plane.

The behavior of the real plane will depend on the environment that it is in.

The behavior of a virtual plane will depend on just the working sensors that are available to render the virtual simulation.

Level 5 automation requires a kind of intelligence that is aware of what sensors are faulty and furthermore is able to navigate a problem with partial and unobserved information.

The smarts to enable this kind of Artificial Intelligence is simply not available in our current state of technological development.

In short, Boeing has decided to implement technology that is simply too ambitious.

Not all software has the same level of complexity.

This is not an issue of insufficient testing to uncover logical flaws in the software.

This is not an issue of robustly handling sensor and equipment failure.

This is an issue of attempting to implement an overly ambitious and thus a dangerous solution.

Air travel is extremely reliable, but introducing software patches as a means to virtualize physical behavior can lead to unintended consequences.

The reason that we still fly planes with pilots in them is that we expect pilots to be able to solve unexpected situations that automation cannot handle.

MCAS like virtualization, handcuffs pilots from differentiating between the real and the simulated.

I would thus recommend to regulators that in the future, MCAS like virtualization should be treated and tested very differently from other automation.

They should be treated as Level 5 automation with a more exhaustive level of scrutiny.

Further ReadingPilot Training, "Alpha Protection" Software and Fly By Wire rules | | AirInsightLast week, Harry Nelson, former Vice President of Airbus flight test department, warned about pilots relying too much…airinsight.

comDid Boeing overplay its hand with the 737 Max?Pilots are expecting a reaction from the plane manufacturer after the Lion Air-disaster.


comTwo Crashes in Five Months: What's Wrong with Boeing's 737 Max 8?.- SPIEGEL ONLINE – InternationalThe first crash occurred on Oct.

29, 2018.

Twelve minutes after an almost brand-new Boeing 737 Max 8 belonging to the…www.


deHow the Boeing 737 Max 8 compares to the world's most fatal commercial aircraftAircraft usually have bumpy debuts; The Boeing 737 Max 8 is an outlier.

After its second fatal crash in less than five…qz.

comBoeing Plans to Fix the 737 MAX Jet With a Software UpdateThe investigation into the crash of an Ethiopian Airlines 737 MAX 8, which killed 157 people on Sunday, is still in its…www.


comMisconception 7: To convince us that they are safe, self-driving cars must drive hundreds of…One of the most difficult questions for self-driving cars concerns their safety: How can we determine whether a…www.



. More details

Leave a Reply