United Blames Human Error For Computer Failure

United Airlines says it was human error that caused a computer failure that grounded all United flights for two hours, snarling air traffic nationwide. From the Chicago Tribune:

“It was human error during routine testing,” United Chief Operating Officer Pete McDonald told airline employees on a recorded call. “An employee made a mistake and caused the failure of both Unimatic and our backup system.”

McDonald did not identify the person responsible, and United officials did not return calls seeking information on how the mistake occurred.

Insert “Skynet” joke here. —MEGHANN MARCO

United blames testing for computer failure [Chicago Tribune]

Comments

Edit Your Comment

  1. Lewis says:

    “”An employee made a mistake and caused the failure of both Unimatic and our backup system.” “

    Oy! “Unimatic?” I am sure this is a very sophisticated system, but it sounds like an ill-conceived robot ticket agent that Pan Am would have demo’d at the 1954 World’s Fair…

  2. rmz says:

    Somebody is so fired. :

  3. B says:

    I’ve done things like this before, although never in such a grand scale. My boss is reasonable about it though, and he feels that instead of firing somebody for making the mistake, you should make sure the mistake can’t happen again.

  4. Invisobel says:

    So one person inadvertently brought down their entire system . Yeah, I trust that system.

  5. toomanyplugs says:

    Donald Norman, the usability guy, might say that any system that allows a slip-up of the operator to cause CATASTROPHIC, system-wide failure is definitely a poorly designed one.

    Users should be allowed to make mistakes without horrible consequences.

  6. QuirkyRachel says:

    I’m thinking this is leading to a negative annual review…

  7. dbeahn says:

    Quite often mistakes like this are the result of systems getting old, and things not being well communicated. Such as there used to be 4 mid-range computers that served as gateways, so it was common practice to take 2 down for back up at a time. Then they eliminate 2 of the machines, but don’t mention that to the LAN guy that worked in the NOC years ago, and he ends up filling in for Joe NOCman who goes on vacation, and the LAN guy cycles it through like he always used to, taking machine 1&2 down for backup.

    Only then, to his utter horror, does he find that there is no longer a quad redundancy as the whole system grinds to a halt.

  8. FatLynn says:

    @toomanyplugs: Not sure I would call it catastrophic. Inconvenient, yes, but it is not as though they let those planes take off with incorrect information or something of that sort. Needs improvements? Definitely. Catastrophic? Probably not.

  9. Fuzz says:

    I’m guessing it was the BOFH. Or perhaps the PFY.

  10. MeOhMy says:

    Routine testing…on a system in production? Bad monkey! No banana!

  11. TWinter says:

    @Troy F.: I suspect these systems are always in use, so there is no completely safe time to do it. OTOH, mid-morning on a weekday is probably about as busy as it ever gets, so definitely not the right time.

  12. FatLynn says:

    @TWinter: I would actually suspect that Wednesday is the *least* busy day of the week, no?

  13. FatLynn says:

    Sorry for the double post, but this just appeared in my inbox:

    Dear [],

    On behalf of United, I want to express our sincere regrets for any disruption to service you may have experienced when flying with us on Wednesday and Thursday this week. We know you expect us to take you where you want to go with on-time departures and arrivals. We failed to meet your expectations on those days.

    As you may be aware, a computer outage, due to human error during routine system testing, significantly impacted our operations systemwide. Working as a team, we were able to get our airplanes and crews back on schedule … and our passengers on their way.

    We greatly appreciated your patience and know that we will make every effort to keep this type of situation from occurring in the future.

    Your satisfaction and business mean a great deal to United, and we look forward to our next opportunity to serve you.

    Sincerely,

    Barbara Higgins
    Vice-President
    Customer Experience
    United Airlines

  14. Xerloq says:

    @Troy F.: Whaddya mean Supreme Commander won’t work on the Unimatic! That’s what I was gonna name my Colossus. Maybe if I add more ram…

    (hammering noises echo from the cavernous insides of the Unimatic)

    Holy crap! Who let the hamsters out!

    (Scene)

    Seriously, What is Barbara Higgins going to do to help those who were inconvenienced? Especially thos hundreds of people stuck on the tarmac at O’Hare?

  15. TWinter says:

    @FatLynn: Wednesday might not be the busiest day of the week, but that time of day is still pretty busy for a major airline like United. The IT folks where I work usually do stuff that could crash important systems at 2:00 AM Sunday mornings. I’m sure the IT folks who do the work hate it, but our systems never go down during prime business hours.

  16. anmlStyl says:

    On a tangent, United’s intranet system is called SkyNet. At least when I worked there until 9/11/01. Chuckled at the irony when it was available outside their firewall.

  17. eli_b says:

    What they really meant was ‘A janitor accidentally kicked the power cord out while mopping and it took two hours for them to find the problem…’

    So let me get this straight, ONE person can accidentally shut down flights for two hours…so what about someone who was INTENTIONALLY trying to stop flights?

  18. Jesse in Japan says:

    Wasn’t “human error during routine testing” what happened at Chernobyl?

  19. observer1959 says:

    As a software developer, I was honestly ROTFLMAO when I read: ‘United Blames Human Error For Computer Failure’. In the computer industry, %90 of all computer failures are based on human error. Very few are traced back to faulty hardware, or ‘random electrons’.

  20. TedSez says:

    I know why this happened: United has made it its mission to keep any of us from ever getting to our destinations on time.

    When I used to fly United, I would regularly find that after various delays, cancellations, missed connections and rude/unhelpful employees got into the picture, trips would almost always take two to three times as long as they were supposed to.

    After about the eighth time this happened, I vowed never to fly United again. But apparently someone there heard I was flying American into Chicago on Wednesday and only had an hour to make my connection to L.A. “Ha-ha-ha!” they said. “Let’s show him that just because he doesn’t fly on our planes anymore doesn’t mean we can’t ruin another trip for him!”

    And so, amazingly, they managed to do it again. Because United isn’t just a bad airline, it’s a villainous entity that aims to ruin all our travels, one passenger at a time.

  21. CumaeanSibyl says:

    @Fuzz: Ah, that’s why it took so long to fix, they’d hooked thickwire to all the doorknobs.