Saturday, May 13, 2017

When automation goes wrong - are we giving humans what they need to fix the problem? QF72

This is a brilliant (long) read about what happened when an automated system went rogue - and a cautionary tale about the risks of #automation when #ai replaces and deskills humans operating heavy machinery, like planes - The untold story of QF72 - What happens when 'psycho' automation leaves pilots powerless.

As organisations and our tools progress through #digitaltransformation and humans are relegated to 'backup systems', but  not given the necessary information or control to address unpredicted computer failure, we may be baking in more risk to human lives and livelihoods during edge situations.

Kudos to Qantas's pilots and crew for saving the lives of all passengers on this flight, and note the flow-on consequences that saw those responsible for saving hundreds of lives so affected by the experience that not all of them may have been able to deal successfully with a repeat of this situation. 

If we burn out good people when computers go bad, we may run out of good people before we run out of faulty computers.


No comments:

Post a Comment