Aircraft synthetic-vision systems (SVS), when combined with GPS, gyros, accelerometers and terrain and obstacle databases, provide pilots with a colorful, animated depiction of the world outside the cockpit, matching what they would see looking through the windshield on a clear day. But to really see what is outside in dark or low-visibility conditions, you need an infrared (IR) camera. When you add forward-looking IR to SVS, you get a heat-referenced, real-world view along with a 3-D, database-derived and geo-referenced virtual view. Together they are called enhanced or combined SVS.

In early April, Honeywell offered AIN an opportunity to experience the company’s in-development combined synthetic-vision system (CVS) for helicopters during a demonstration flight (see photo gallery) and video on YouTube. Here is what we learned.

Honeywell delivers synthetic vision for business aircraft under the brand name SmartView. The system uses the terrain database of the company’s enhanced ground proximity warning system (EGPWS) merged with head-up display (HUD) symbology and presents the SVS graphics on an aircraft’s primary flight displays (PFD). SmartView is Part 25 certified for Gulfstreams with PlaneView avionics and Dassault Falcons with EASy II flight decks, and Part 23 certified for Pilatus PC-12 NG turboprops with Apex cockpits. PlaneView, EASy and Apex are all based on Honeywell Primus Epic avionics. Each of these also has an integrated map display.

The situational-awareness advantages of enhanced, synthetic vision for airplanes are obvious, especially for operations on or near the ground, such as taxi, takeoff, approach and landing. How much more valuable, then, would enhanced vision be for aircraft, namely helicopters, that spend much more of their time closer to the ground?

Honeywell’s Advanced Technology Group has the same thought. “Helicopters fly close to the ground, often in obstacle-rich and low-visibility environments and, frankly, they are more likely to hit things than airplanes, and they do,” Trish Ververs, an engineer fellow with Honeywell’s Advanced Technology Group, told AIN. “We think an enhanced synthetic-vision system would be particularly helpful in improving the situational awareness of helicopter pilots in target-rich environments, such as those in metropolitan areas, which are frequently also in congested airspace.”

So it is no surprise that Honeywell has been developing synthetic vision for helicopters for quite some time–since 2006, in fact–and is quite far along in development of the system. The same team has worked on both the airplane and helicopter systems. “We’re in the middle of flight-test, and one of the tests we do is human-factors evaluations,” said Ververs, who specializes in crew interface and platform systems. “Sometimes we do these evaluations in a laboratory. Sometimes we’re talking to the users. Right now with this advanced-technology program, we need to develop scenarios where we can test the system. We want to see how pilots react and adapt to the system.”

This objective is what brought the Honeywell team to Morristown Airport in New Jersey in early April. AIN was one of the lucky outside groups invited for a demonstration flight in Honeywell’s AgustaWestland AW139. What we did not know is that we would also be one of Honeywell’s SVS test subjects.

Research Flights in Target-rich Environment

Morris Township is home to Honeywell’s corporate headquarters, and its company aircraft, including N139H, the AW139 medium twin, are based at Morristown Airport. The Advanced Technology Group received permission to turn the executive twin into a test aircraft for a week. The New York City area provided a particularly target-rich environment for evaluating and demonstrating SmartView for helicopters. Three members of the research team came from other Honeywell locales, so they made good use of their time and kept the helicopter flying as much as possible.

Marc Lajeunesse, whose title is lead pilot (rotary wing), conducted the demonstration flights. His primary job is flying Honeywell’s CEO and other top executives in both the AW139 and the company’s business jets. He has more than 13 years’ experience with head-up display technology, flying the Dassault Falcon 900 EASy and 2000, Gulfstream G450/550 and recently the G650. (He’s also rated in the Bombardier Challenger 300, BAE Jetstream 41, Sikorsky S-76 and Bell 222.) Lajeunesse has done all the flight-testing and demonstrations with the Honeywell SVS for helicopters. Well acquainted with the NYC area, Lajeunesse mapped out a route that would show other pilots and passengers how the system works in its display of obstacles at various altitudes and distances and the warnings it provides the pilots.

“What you’ll see today is not a product; it’s research, although we are at the point right before we transition from research to an actual product,” Ververs said before our 30-minute flight in N139H. From a research standpoint, the SVS project will reach “Technology Readiness Level 6” by year-end, she said. Then the project will transfer to engineering for transformation into a product that can be installed and used.

“Some aspects may make it into the product and some may not,” Ververs continued. “Our job is to make sure the things we develop are useful to the pilot.” The engineering unit will decide what the end product will be, “depending on what our customers want and the features we have.” She anticipates that Honeywell will have a final product within the next two to three years. Because SVS would be offered as part of a larger Primus Epic upgrade, it would not be priced individually, she said.

Honeywell’s AW139 has four large vertical, full-color Honeywell Primus Epic primary flight displays (PFDs), two each in front of both pilot positions, and a fifth display–a horizontally mounted display Safran display–in the middle of the panel. The helicopter also has a Max-Viz forward-looking infrared (Flir) system. Honeywell’s SVS also works with Flir from other companies, including CMC and Kollsman.

Showing the Value of Synthetic Vision Plus Infrared

A key objective of the flight demonstration, Lajeunesse said, “is to show the additional value provided to the pilot by combining SVS with infrared, primarily in low-visibility and night conditions, but also in good visibility.”

To help guest pilots experience this, Honeywell provided a switch by which they could quickly change their left PFD from SVS (synthetic-vision system only) to CVS (combined-vision system, meaning SVS plus infrared) and back again. The guest pilot’s right display could show the experimental moving-map display or other functions of the standard Primus Epic MFDs. Because this flight was primarily about SVS and IR, after engine start I elected to keep my left display on CVS/SVS (switching between SVS+IR and SVS-only when I wanted to see what IR would add to the view) and the right display on the moving map.

Another key objective of the demo flights was pilot evaluation of the flight-path marker (FPM) on the SVS display. Honeywell believes the FPM is key to any SVS display and it is currently available as part of SmartView. The research team brought in other Honeywell pilots to evaluate the FPM-based SVS to determine its suitability and ease of use transitioning from conventional flight director cues. Lajeunesse helped the team define the dynamics of the cue and has confirmed its usefulness as a key component in the design.

The concept is simple: the FPM points to where the aircraft will end up, if the pilot takes no actions to alter the flight path. Keeping this in mind, if the FPM overlays terrain or an obstacle on the SVS display, that’s where the aircraft is going to hit if the pilot does nothing. Increase the collective, pull back on the cyclic or bank into a turn, as needed, and the FPM will soon be pointing to clear sky.

Using an FPM to display flight path helps illustrate one of the fundamental differences the Honeywell design team needed to address as it adapted SVS for helicopters. When an airplane takes off, the pilot raises the nose, and the FPM points up. When a helicopter takes off, the pilot lowers the nose at first and the FPM (before the software was altered) points down. After the helicopter gains airspeed and translational lift and starts climbing, the pilot then raises the nose but it is still below the horizon and the FPM is still pointing down. An airplane with its nose below the horizon is in a descent. Regardless of where the nose is pointing, the FPM had to be able to show where the helicopter would really go.

Sideward hovering presented another FPM problem. During this maneuver, Honeywell decided it is better to indicate aircraft heading, rather then the direction of travel, as the FPM would do. The pilot is typically looking outside the cockpit in the direction of travel anyway and not at instruments in the cockpit. Deciding when and how to transition from FPM to heading indicator as the pilot starts to hover to the left or right was both a human factors and software challenge.

Detailing Obstacles

Another important consideration for a helicopter SVS, or more correctly the EGPWS, database is how and when the obstacles should be displayed, because helicopters typically fly at lower altitudes and in more obstacle-laden environments than airplanes do.

“Airplane pilots tell us that, except for takeoff and landing, they are not really concerned about obstacles that are more than 1,000 feet below the airplane’s flight path,” Ververs said. “But helicopter pilots often fly below 1,000 feet and therefore need better depiction of obstacles, so they have a better idea what to look out for.” As a result, the helicopter obstacle database shows some obstacles in greater detail and adds others.

The standard Honeywell EGPWS obstacle database for airplanes shows vertical, manmade obstacles as black pillars with height numbers. “Instead of showing sticks with heights for a bridge [as the airplane database does], we show the stanchions of a bridge and put a road between them,” said Ververs. “Instead of showing a stick for a wind turbine, we can show a wind turbine.” Honeywell has also overlaid the obstacle database with a power-line database and shows these, too.

However, Honeywell is “not going to tell pilots they can navigate off the SVS, based on the display of obstacles,” Ververs said. “One of the philosophies behind the designing of all the synthetic-vision systems is ‘don’t mess with the primary flight display’,” she continued. “The pilot needs airspeed, altitude and heading. These have to be supported. You can put all sorts of things on the display, but we have to ask, ‘Does the pilot really need this information? How do you include obstacle information without over-cluttering the display?’” One solution is the muting of distant obstacles. “When you are far away from the skyline of a city, for example,” Ververs said, “what you see is a muted representation of the skyline. The depictions of individual buildings don’t start to be recognizable until you get much closer.”

There are also limits to the processing speed of the obstacle database in extremely data-rich environments, requiring a tradeoff between detail and the primary goal of obstacle avoidance. Does it really matter if a tall building is shown as a box on the SVS instead of showing a detailed replica of the building, as one might see in a simulator’s depiction of the same building? Probably not. And this is where an IR view of the building combined with SVS can provide enough real-world detail to make the building easier to identify in all visibility conditions, and could be absolutely critical in some situations, such as when an air-med helicopter pilot is looking for an accident site or hospital helipad at night or in marginal daytime visibility.

Summing up the advantages of combined vision with infrared, Ververs said it provides real-time information for objects not in the terrain or obstacle database, provides detail not available in synthetic vision and it increases the confidence of pilots by confirming the SVS picture.

Flying with SmartView

Lajeunesse’s planned route took us south from Morristown Airport to Raritan Bay, northward up the Hudson River toward New York City (heading directly to the east stanchion of the Verrazano Bridge), past Roosevelt Island toward lower Manhattan at the confluence of the Hudson and East Rivers, along the west bank of Manhattan toward the George Washington Bridge, making a 180-degree turn to the south, back down the Hudson, a right turn across Hoboken and finally picking up Route 280 heading northwest to return to Morristown.

The sky was blue and visibility unlimited (until it became hazy when we turned northward toward NYC), making it easy to match up terrain and obstacles. While the good visibility inhibited the CVS from showing off its stuff in low visibility, it did allow us to make closer approaches to hard surfaces because we easily saw them through the windshield.

Lajeunesse demonstrated a Category A takeoff from Morristown Airport, showing how the flight-path marker leads the pilot through the profile and explaining how it would provide guidance in the event of an engine failure. I have flown or monitored (as the non-flying pilot) several thousand Cat A takeoffs, both from helipads on land and off elevated helidecks on oil installations, using the attitude direction indicator (ADI), airspeed and radar altimeter. Receiving guidance from FPM made sense, although I had trouble following along during Lajeunesse’s takeoff demonstration.

The vast majority of my time has been in aircraft with electromechanical flight instruments or with glass cockpits with digital reproductions of the ADI. I had only a cursory introduction to SVS before. I found it was almost surreal having the animated display as a substitute for the outside world, and I was almost mesmerized at first. I wanted to compare the real-world view outside the cockpit with the virtual and IR version of the view inside, which wasn’t really the point of the flight demonstration.

Along our route, we saw, approached and flew near and very close to numerous obstacles, including power lines and towers, office buildings, manufacturing facilities, bridges, cellphone towers and so on. The color and visual warnings mirror EGPWS warnings, from whence they come. Green is safe (“terrain/obstacle is below the aircraft altitude”), yellow is cautionary (“terrain/obstacle is very near or above the aircraft altitude”) and red is danger (“terrain/obstacle is well above the aircraft altitude”). The audio warnings are also similar, including the “look ahead” alerts based on the flight path (“caution terrain” and “caution obstacle”).

Lajeunesse was purposely flying low and close to obstacles, so there were almost continuous changes in color along our flight path and audio warnings. And we could easily see everything. I found myself slipping into a “cry wolf” attitude toward obstacles and terrain warnings, and had to remember this was a demo flight. In a real flight, these warnings would be most welcome.

We followed highways, rivers (blue in the display) and coastlines. We saw other air traffic on the map display before it was reported by ATC or we could eyeball it through the windows. Lajeunesse described and demonstrated the FPM and I used it without difficulty to do normal turns, climbs and descents, primarily over Raritan Bay.

I found myself flying head down while VFR and in congested airspace (a bit worrisome, but Lajeunesse was flying head up). I marveled at what the SVS was showing and even more so when switching to CVS and adding infrared. Perhaps the most noteworthy difference between the two came when Lajeunesse suggested I switch from CVS to SVS as we flew south along the Hudson River, near the area where Sully and Skiles successfully ditched their Airbus A320 twin-engine glider in 2009.

In only SVS mode, the waterfront on the New Jersey side appeared as a diffused shoreline, like a beach in yellow fog, with tall red, yellow and black buildings indicated a bit inland. Lajeunesse explained that this waterfront area was not a database priority, because it was at such a low level, actually not much above sea level. This was in contrast to the tall buildings on both sides of the Hudson in this area, which were clearly represented by the SVS, because they are resident in the obstacle database. In CVS mode, however, the IR clearly picked up docks, piers, buildings and other structures on the waterfront, in several shades from black to white, depending on their heat signatures. It was like looking at a black-and-white movie instead of just mist.

Back at Morristown Airport, Lajeunesse demonstrated hovering, particularly having me note the transition from flight-path marker to heading indicator during a sidewards hover and back to FPM when he stopped N139H in a stationary hover. The transitions appeared so natural to me that they seemed like a non-event.

Reflections on Helicopter Synthetic Vision

The old expression “taking a drink from a fire hydrant” came to mind after we landed. Ververs asked for my opinion of the SVS and CVS, and I could say only that I was still trying to process what I had experienced. But I did tell her, with frank honesty, that I rather longed for a standard attitude indicator instead of the HUD symbology superimposed on the virtual world display. This is “old guy” thinking, I know, but it’s hard to suppress one’s trust and reliance on the ADI after almost 9,000 hours flying with one.

Still, when asked if I would have liked to have SVS with Flir when I flew helicopters in the North Sea and in the northeast U.S., I replied, “Yes, definitely.” They would have made a huge difference, especially during night and low-visibility operations. I’m certain they will improve helicopters pilots’ overall situational awareness in all conditions and help lead to a reduction of controlled flight into terrain (CFIT) accidents.

A few weeks later as I looked back on my demonstration flight, I found myself warming considerably to having a virtual VMC depiction of the world outside, even if the part in color is synthetically generated from a database and the part in black and white is obtained by detecting the heat radiated by the objects on the ground. After all, isn’t flying visually on a CAVU day a lot easier than using needle, ball and airspeed in the soup?

Helicopter pilots a decade from now will see much better synthetic and infrared depictions in their cockpits. Flir has improved considerably in the last 20 years and both it and SVS will inevitably continue to improve, as well. It is not beyond the realm of possibility that the geo-referenced, 3-D image generated by future enhanced, synthetic-vision systems will be better than the current photo-realistic images produced by simulators today, and the position and height of each obstacle and terrain feature will be perfectly positioned as well. Flying in even nighttime, zero-zero conditions will be no more difficult than flying on the clearest of cloudless days. o

Honeywell’s Moving Map for Helicopters

The SmartView combined vision system was the star of the flight demonstration, so not much time was spent on the helicopter moving-map display, which is not quite as far along in its development as the CVS/SVS. Honeywell envisions the map displaying terrain, water obstacles, flight plan, Tcas traffic, geographic boundaries, roadways with labels, railroads, power lines, airspace boundaries, awareness, info, EGPWS alerting, airports, runways and navaids. o

Tricking Out N139H with Synthetic Vision

For the demonstration/research flights in the AW139, the research team from Honeywell’s Advanced Technology Group removed the middle seat of the front three aft-facing executive seats in the cabin and replaced it with an electronics box that operated the SVS and a map display for helicopters, which Honeywell is also developing. N139H already had Max-Viz forward-looking infrared, so that was not difficult to couple into the system.

A large flat-screen tv monitor, mounted vertically on top of the electronics box, allowed John Suddreth, Honeywell developer (research and development), and other passengers in the cabin to view the SVS and map displays as repeater images of what was available to the pilot. Suddreth also had a keyboard, which he used to boot up and adjust the SVS system, as needed. (Trish Ververs and Gang He, another staff scientist on the Honeywell Advanced Technology team, did not accompany AIN on the demonstration flight, but provided information during the preflight briefing and after the flight.)

For research purposes, numerous parameters were recorded during the flight, including audio and video, so that the entire flight could be “re-flown” afterwards on a computer, showing not only instrument indications, but pilot reactions and responses. “So, I’m to be a guinea pig, too?” I asked Ververs before we took off.

“Yes, of course,” she replied. “That’s the whole point.”