Friday, December 21, 2007

Bugs & Quirks

An old aviation axiom says "Never fly the 'A' model of anything!" While perhaps overstated, it does illustrate the truth that new aircraft designs inevitably have kinks that need to be worked out of them. Some of these are more serious than others. When I was at TWA, they were having a problem with the B717 where ice melting in the forward galley was seeping down into the electronics bay directly underneath and shorting out major components. That's a pretty big deal. Less seriously, I can't count the times I had to reset the FCECU #2 circuit breaker on the Q400 to clear a spurious #3 Rud Hyd caution light right after startup. Until Bombardier finally found a fix last year, there were a few planes that did it every third cycle or so.

NewCo has had pretty good luck with JungleBus reliability thus far. The major flaws seem to be related to cold-weather operation; we've had problems with the potable water system freezing on the ground. Other than that, most of the bugs have been annoyances with little effect on our dispatch rates. I know jetBlue had a lot of problems when they launched the JungleBus+, but the Brazilians seem to have got on top of it pretty quickly and incorporated the fixes into the production line. There are still plenty of little things the JungleBus pilot has to watch out for.

After finishing IOE last week, I had a few days off and then started three days of reserve on Monday. With only about 25 FOs off of IOE compared to 75 Captains, our usage is going to be much higher for a while, which is fine with me as I hope to get some good experience in the airplane before upgrading in a few months. I flew all three days, including NewCo's very first flight to Chicago-Midway on Wednesday. On Monday, I had one flight from Minneapolis to Missoula. I got to experience three JungleBus quirks and bugs on that one flight.

When you first apply AC power to the JungleBus after it's been "cold" (batteries off), it runs a Power-up Built In Test (PBIT) that tests various systems including the fly-by-wire. It takes three minutes and is notoriously sensitive to any interruption during that time, including moving any flight control or putting a load on the batteries. If anything is out of place, you get a "Flight Control No Dispatch" EICAS message that requires a power-down or even maintenance action to clear. The best course is to just leave to cockpit for three minutes after powering up.

On Monday we were about 10 minutes from pushback for Missoula with the APU off since the gate was providing warm air for cabin heating. One of the rampers didn't realize our APU was off and disconnected the external power without asking the captain. Pzheuwww! With only the DC essential busses still powered by the batteries, most of the cockpit went dark. The Captain quickly opened his window and yelled for the guy to plug it back in. In the meantime he started the APU in case it happened again. With AC power back on the airplane, we did a quick damage assessment. Most electronic airplanes really don't like to be depowered unexpectedly. This time we seemed to get away with it...until we saw the dreaded "Flight Control No Dispatch." We realized that the ramper plugging in the external power again had restarted the PBIT just before the Captain put a draw on the batteries by starting the APU. Whoops.

The Captain called maintenance control. They listened to the circumstances behind the PBIT interruption and then directed us to turn to on the #1 and #2 electric hydraulic pumps. After a few seconds, the EICAS message went away. It turns out that not all "Flight Control No Dispatch" messages are created equal. For one resulting from an electrical interruption, supplying hydraulic power to the flight controls is enough to make the computer realize it's not really broken after all. Good to know.

We finished loading, handed out the paperwork, buttoned up, and pushed back. After engine start we ran the before taxi check and I obtained taxi clearance. The Captain engaged the steering handle and pushed up the thrust lever to start moving. DING! "Master Caution, Steering Fail." Hmm. That's great. When you first engage the steering handle, you're supposed to let it go for a few seconds to let the system test itself before reengaging the steering and beginning taxi. The Captain figured he'd just not waited quite long enough. Momentarily engaging the handle again didn't clear the message.

The Captain whipped out his cell phone, turned it back on, and called maintenance control again. This is one of the things that was a lot easier to do at Horizon, where you could simply call over the radio. We were blocking the busy alleyway at MSP until we could get steering back. Maintenance directed us to pull and reset the circuit breaker for the Modular Avionics Unit (MAU) #2. The Master Caution tone began ringing incessantly as various systems pulled themselves offline for self-tests. We waited with bated breath as the messages extinguished one by one until the EICAS was completely cleared. Problem #2 solved.

The flight went very smoothly until we were approaching Missoula. The weather was reportedly decent but not quite enough for a visual approach. We got vectors to the ILS 11 (yes, you can get vectors to the approach in a few parts of Montana). I was flying; while on downwind I called for the approach checklist, which the Captain ran. We were in green needles, meaning that we were set up for ILS guidance rather than FMS guidance. Both Nav radios were tuned to the ILS frequency.

When the controller gave us our final vector and cleared us for the approach, I pushed the APP button to arm the autopilot's approach mode; this was confirmed by the appearance of a white LOC and GS in the flight mode annunciator just above the attitude indicator. This means that the autopilot will retain its previous modes (Heading and Altitude in this case) until it's able to capture and follow the localizer and glideslope. As expected, the LOC annunciation changed to green as the localizer started centering, and the airplane began turning to capture it. The next thing was very unexpected: both my localizer display and the Captain's turned yellow, and LOC and ALT disappeared from the flight mode annunciator, replaced by the default modes of ROLL and FPA (flight path angle).

The JungleBus' autopilot will only capture and follow a localizer signal when each pilot is using on-side navigation guidance; that is, the Captain is using NAV 1 and the FO is using NAV 2. When this is the case, the HSI displays green needles. If either pilot pushes their V/L button to change over to opposite-side navigation guidance, both pilots' needles turn yellow and the autopilot will refuse to capture the navigation signal. If the LOC mode is already active, the autopilot reverts to its basic modes of ROLL and FPA. This is exactly what happened - except neither the Captain's hands nor mine were anywhere near the flight guidance panel. It was like the airplane pushed a V/L button on its own.

Flying the airplane is always Job One. Since the airplane was in a 15 degree bank when it reverted to ROLL mode, it would continue making big turns until I changed the mode. I quickly punched ALT and HDG, confirmed those modes on the FMA, and used the heading bug to guide the airplane back towards the localizer. Next we needed to figure out who had the off-side nav guidance. My nav display still showed LOC 2 like it should, so it had to be the Captain. "Push your V/L button," I said. Voila, green needles. I pushed APP again and this time the airplane obediently captured the localizer but remained in the ALT (altitude hold) vertical mode. I saw why: during all this we had passed through the glideslope and it was now below us. I pushed the FPA button but ALT simply flashed at me in the mode annunciator. Grr, the altitude selector. I spun it up to some random altitude, pushed FPA again, and set it for a 4.5 degree descent to capture the glideslope from above. Now we were high and fast. "Flaps 3, Gear Down." The airspeed wasn't decreasing fast enough so I decreased the descent angle to 3 degrees until the airspeed was just below the Flaps 5 limit. "Flaps 5, Landing Check." I increased our descent angle to 4.5 degrees again and captured the glideslope just before the final approach fix. The airspeed was still high but with a 3 degree descent and the airplane dirtied up it decreased fast enough for me to get the airplane stabilized at Vapp well above the 1000' minimum altitude for doing so. We popped out of the clouds much closer to minimums than the ATIS had led us to believe. I clicked off the autopilot and autothrottles and made my nicest landing so far.

The whole episode left me with a rather uneasy feeling in the pit of my stomach. I'd handled it fine and made a good recovery but furious button-pushing on approach in IMC with big rocks around really isn't an ideal situation, especially with two pilots very new to the airplane. The fact that the airplane's gremlins screwed things up rather than one of us almost made it harder to figure out what was going wrong and correct it. It would've been a lot harder to figure out if I hadn't seen the sim do it several times. I'd been told the bug was not specific to the sim and the airplane could also do it, but this was the first time the Captain and I had seen it in the airplane. I'll be watching for it like a hawk in the future, that's for sure.

Quirks like this are one of the reasons that I'm glad I'll get a few hundred hours as a FO before I upgrade. The current Captains are having a tougher time getting experience because the majority have flown very little on reserve since finishing their IOE. As we get more airplanes and more routes that situation should correct itself, though.


curious said...

Related to the potable water system freezing - I understand some Junglebus operators dump water prior to landing before overnighting the plane in the winter season. It seems this is not the practise at your company?

Btw, thanks for the well written and very interesting stories!

Anonymous said...

As a passenger, should I be nervous about that autopilot gremlin/"bug" or was the situation under control at all times? I assume that if things really started getting nuts with the autopilot, you could safely hand fly all the way down, right?

Anonymous said...

I didn't know you worked at TWA. How long were you employed there? What was your position?

Anonymous said...

i'm with the first anon .. bugs on the ground are annoying, but the autopilot doing something weird during an ils approach is a bit disturbing. is it plausible that the pilots could be distracted by dealing with some other problem, plus possibly tired, and not notice the autopilot had gone wacky, and fly into a hill?

also, what happens if you need to call maintenance in flight? presumably you can't use your cellphone in that case ...

Aviatrix said...

Well done. I hate that new-pilot-calling-mx-twice-before-the-same-flight thing. I'm not sure whether to be reassured or disappointed that this doesn't go away when the support structure and airplanes get bigger.

I'm looking forward to having modern avionics someday, but thanks for the reminder of the drawbacks.

Fred said...

Sam, my friend, that was spellbinding!

The closest I've come to experiencing what you've described is in Flight Simulator (of course) when I was a new pilot. Trying to master the APP a/p mode was maddening. Although in that case, I was the problem, not the machine.
I can't imagine what I'd do in real life.

You have to write a book about all this sometime.

Sam said...

curious - in the past, newco only dumped the water if we were overnighting the airplane. After having airplanes freeze up on daytime layovers of a few hours, they've had us start dumping the water on every approach into MSP. Of course now we're taking delays to wait for potable water refills.

Anonymous 1 and 3 - the situation was under control. If I was unsure what was happening or if we were too far above the glideslope by the time we got things figured out, I would've just got vectored back around as we were still above the minimum vectoring altitude for that area. It IS a pretty serious bug but one that'd be pretty hard to miss, the airplane pretty much smacks you upside the head with the yellow needles and flashing FMA. I don't forsee it leading someone into rocks. I'd say it's something that really needs to be corrected, but not so serious that it warrants grounding the Ejets till it's fixed.

Anonymous 3 - get a phone patch via AIRINC. It's a pain, Horizon's discreet frequencies with SELCAL were much easier.

Anonymous 2 - I was a flight ops intern with TWA in the spring of 2001. They announced the purchase by AA shortly after I got there and it was finalized shortly before I left. Interesting times. I worked in training systems development, supporting and developing computer based training software for the training department.

Anonymous said...

Sam--here's an off-topic question, but as a passenger, I'm interested. I just flew in an airliner and it was a clear day, and we could see all the way from FL390 down to the ground. My question is: as a pilot, can you tell your altitude just by looking at the ground, within 1,000 feet or so? Does the ground-picture look significantly different from, say, FL390 vs. 10,000 feet vs. 1,000 feet?

Anonymous said...

Sam-- what's the protocol for telling your company ("NewCo") about the gremlin/bug on the JungleBus? Do they already know about it? How does a major bug like this get fixed? Is like a computer where a software patch can fix it, or do they have to come in and update the "hardware"?

Anonymous said...

I know you're new in the plane, but never under-estimate the value of disconnecting the auto-pilot and hand flying the ILS.

Sam said...

Anonymous1- I can't estimate it within 1000 ft but there's a noticeable difference between, say, 30000 ft and 35000 ft. It's a huge difference from 1000 to 10000 to FL390.

Anonymous2- Incident report or ASAP report, usually filed by the captain.

Anonymous3- Very true, that's always an option. However, the airlines place a high priority on always keeping the flight director synced up to where you're actually trying to fly; it's frowned upon to fly an approach raw data unless that's your only option (inop FD) or is for proficiency purposes and was briefed beforehand. In a case where you don't know what the airplane is doing, taking it off autopilot is appropriate only if the autopilot is taking it someplace dangerous. Otherwise you're just going to increase workload while trying to figure out what's going wrong.

Xaqman said...

I wasn't there, but based on your description I would have canceled the AP and the FD and just flown the needles. That way, there IS nothing to 'figure out' during such a critical phase of flight. That's not the time for troubleshooting and guessing. Aviate first, program and type later, time permitting. The AP, FD, and FMS are still just labor-saving devices. When they start to ADD labor, ditch 'em, pronto.