Improving Aviation Safety with Information Visualization: A Flight Simulation Study

Scientists Cecilia R. Aragon and Marti A. Hearst researched the issue of how to provide pilots with complex and dynamic information on airflow in relatively easy-to-access formats using information visualization. The abstract and the link to their article follow.

Many aircraft accidents each y ear are caused by encounters with invisible airflow hazards. Recent advances in aviation sensor technology offer the potential for aircraft-based sensors that can gather large amounts of airflow velocity data in real-time. With this influx of data comes the need to study how best to present it to the pilot – a cognitively overloaded user focused on a primary task other than that of information visualization.

In this paper, we present the results of a usability study of an airflow hazard visualization system that significantly reduced the crash rate among experienced helicopter pilots flying a high fidelity, aerodynamically realistic fixed-base rotorcraft flight simulator into hazardous conditions.



Sikorsky Covered by Augmented Reality Association, The AREA

Each year Sikorsky Innovations, the advanced research organization of Sikorsky, a Lockheed Martin Company, conducts a contest, the Entrepreneurial Challenge. The purpose of the contest is to identify emerging technologies that could be used in the company’s helicopters and to partner with companies which are developing these technologies. In 2015 the challenge identified Augmented Reality for manufacturing as one of the four areas of interest.

The AREA: Thinking Long-Term About Augmented Reality Applications

The Area Logo

Augmented Reality applications will not get introduced and adopted as a workplace solution unless they address a burning business need. While the media hypes Apple’s “secret” AR/VR team and Twitter buzzes with discussions of how Magic Leap raised more money than most ever dreamed possible, the implementation of AR into our daily working lives still faces a few significant barriers to entry.
Enter AREA: The Augmented Reality for Enterprise Alliance, the only global non-profit organization dedicated to widespread adoption of AR in enterprise. AREA members are working together to reduce the barriers that even money can’t easily address. Incorporated in 2013, the AREA’s mission is to help companies improve operational efficiency with AR.
While most of the public focus on AR is about slick games and pre-orders, members of the AREA have been using AR to increase workplace performance right under our noses by encouraging education and collaboration.

One of the ways AREA works together to educate the public is by publishing use cases, such as this abstract on AR being used to assist flight attendants by giving them real-time information on their passengers’s needs. And the AREA continues to collect dozens of Augmented Reality use cases to publish on their website.
“Without the AREA, everything a customer would find on the Web about AR is provided by a sales person who is talking about their products, some of which don’t exist yet. There’s a lot of hype and this fuels high expectations. Customers wonder how much they should be paying, how and where they should start and what options they need.” says Christine Perey, the Executive Director of AREA.
The AREA recommends that companies think of and use Augmented Reality as a new part of existing workflows which involve real world processes and objects. Getting to that level of integration, however, isn’t a straight path.

Augmented Reality Expectations vs. Reality

“Our vision is that we can help to accelerate AR adoption by providing one place where customers and providers of Enterprise AR can come together. It’s a safe place, a place of shared values.”

Translating Augmented Reality

According to Perey, Augmented Reality needs to build upon and leverage a company’s existing enterprise IT system. With AR integrated into the enterprise data delivery systems, workers can lower task interruption and cognitive load. They can also have a positive impact on the bottom line.
Perey notes that a buyer doesn’t want to spend two years testing a multitude of different products, but there is still a steep learning curve for those who want to deploy the nascent technology.
“It’s about reaching operational efficiency. If you’re selling technology and it takes you weeks months or years to educate your customer, that’s not efficient. What they want to know is: what do I need and where do I get it?” If providers and customers are speaking two different languages, the process is slow and inefficient.

Barriers to Entry in Augmented Reality

Partnering for AR Success

The AREA’s 28 members—from two-person startups to multi billion dollar Fortune 100’s like Boeing, Johnson & Johnson and Bosch—are partnering to develop best practices and guidelines.
“While the AREA website is focused on customer education, we also offer programs that help our members collaborate on research products and train more professionals. We speak at events. We promote high quality, vendor-neutral content, so it’s not all sales materials.”
The AREA itself partners with other industry organizations, including RA’pro in France.
“Research consortia, like EPRI, and leading research institutes such as Georgia Tech and the University of Sheffield in the UK are also important to our ecosystem. They provide things that are not for sale, they are not competing with the providers of products or services.”

Members of the AREA Augmented Reality Alliance for Enterprise

But what about organizations who are highly secretive, or don’t believe in collaboration?
“In all honesty, there are people who only see the world in their way, and I think if people don’t feel driven to working with others to build a better, more productive workplace, they are unlikely to join our organization… The companies who do join are those that think about a much bigger picture. They are thinking about the long term… We all know AR is going to be a big leap, so we need to join hands and pool our resources to get there.”

Augmented Reality Project Planning — The Six Questions to Answer

Greater understanding of the information around an object, like a work of art, is an AR benefit.
Greater understanding of the information around an object, like a work of art, is an AR benefit.
While Augmented Reality glasses are not available for consumer use in 2015, many people might be surprised to know that AR Goggles have been already successfully applied
in pilot programs for “desk-less workers” (people who work with their hands and whose workload would benefit from being able to put down handheld devices), such as HPs Visual Remote Guidance program.
At a recent Augmented Reality Meetup in Manhattan, Pete Wassell, CEO of Augment, noted that AR may be in use even more than we think. Thinking of it as a “secret weapon,” he believes some companies may be keeping their AR-investments and pilots under wraps until they are six months or a year into their programs.

Assessing The Need For Augmented Reality

Successful Augmented Reality Project Planning requires mastery of many diverse fields, including everything from hardware and software development, to data entry, UX, graphic design, and even ergonomics.
Luca Bertuccelli, founder of the UX team at United Technologies Research Center, stresses that Augmented Reality is just one tool in a toolbox.
“At its core, any new technology has to add value to what the people are doing at that particular point in time,” Bertuccelli says. “The key is zooming in on some of those valuable use cases and very quickly figuring out if that technology makes sense.”

“When implementing new technology, you need to be aware of the whole ecosystem, starting from the humans that are going to be using it.”
Luca Bertuccelli – United Technologies Research Center

Are Augmented Reality Investments Right For Your Company?  Answer these six questions.

When considering the use of AR in an enterprise setting, the following six questions should be addressed:
  1. Does it create a more efficient work environment?
  2. Does it help lighten individual workloads?
  3. Does it enhance collaboration?
  4. Does it create better awareness?
  5. Does it reduce the probable need for later repairs?
  6. Does it increase speed and efficiency in training?

Codeveloping for Augmented Reality

Before or during the creation of a testbed for Augmented Reality applications, developers must understand what the end users do and how they do it. This is why Bertuccelli stresses the need for co-development.
augmented reality“We need to co-develop the technology and really understand if users are interested in it and if they gain value in using it. Whether it’s medical or industrial, the important part is that at the end of the day, you are talking about helping people do their jobs in a better way.”
Bertuccelli describes working with Augmented Reality Usability as a SocioTechnical System. “When implementing new technology, you need to be aware of the whole ecosystem, starting from the humans that are going to be using it.” “It needs to make sense.”

Challenges Faced By Augmented Reality Designers

“There’s a saying in aerospace,” Bertuccelli says, “Technology needs to earn its way into the cockpit.” He is referring specifically to head-up display technology, but the same concept can be applied to any workspace.
It is important to remember that the introduction of AR goggles into the workplace requires retraining as well as sensitivity. Skilled workers have been doing their jobs for years, and as such, they have a sense of earned pride. They have developed workarounds and shortcuts, and may even be distrustful of being “replaced” by technology.
Like the path once taken by smartphones, Bertuccelli reminds us that many careful hours were put into thinking about who the users would be and how they would experience the technology. How is the data being shared between different users, for example?
If you have ideas, technology or solutions that can be applied towards enterprise augmented reality, we hope you’ll consider applying for the $25K Sikorsky EChallenge.  Here’s what they are looking for:  “Real-time augmented reality for collaborative applications and complex manufacturing processes.”  Click here to apply.


Augmented Reality — Ready to Explore the Details?

As our computing platforms make amazing gains in capacity and convenience, the way we experience visual data has made similar improvements. What once was a 50-pound CRT monitor now fits in your pocket. Many believe the smartphone will be replaced by Augmented Reality glasses within 10 years, which explains Facebook’s 2014 acquisition of Oculus, makers of the Oculus Rift, for $2 billion.
Technically, a smartphone is an Augmented Reality platform—after all, it has a camera, a GPS, and object recognition capabilities—but the results display themselves on a handheld device instead of through immersive, holographic goggles. In this way, society is already acclimated to receiving instant information, often based on geolocation or QR codes.
But are people ready to wear Virtual Reality goggles in their daily lives? And what is the path from virtual games to commercial implementation of Augmented and Mixed Reality?

What is the Definition of Augmented Reality? Virtual Reality?

Here’s our working definition: Virtual Reality delivers an immersive experience of a new environment that has no relationship to your location. Augmented Reality allows you to see the real world in an enhanced way with information superimposed on top of it. It is also known as mediated reality. But let’s explore this idea further.

What is Virtual Reality?

VR is what you imagined as a child would happen if you stuck your head inside a television set. It is an immersive, synthetic environment experienced through VR goggles like Oculus Rift. You cannot manipulate actual reality using VR any more than you can drive your car up the street by playing Grand Theft Auto on a Virtual Reality headset. You can, however, meditate on a virtual beach while sitting on your actual couch.

What is Augmented Reality?

Augmented Reality is different from Virtual Reality, because you are not completely immersed in a synthetic environment. Wearing see-through AR goggles like the Epson Moverio or Google Glass, you still experience your immediate environment but you will also see superimposed data that relates to the images that you see. It’s what might have happened if Pop Up Videos and Wikipedia combined and took control of the world. Much Augmented Reality technology (like the Meta Glasses) are designed to be responsive to instinctive hand gestures, so you can “push” a button or make a swiping motion to move information in real-time. Yes, AR-like capabilities are displayed in the 2002 Tom Cruise movie Minority Report, although future-tech interface designer John Underkoffler would like to see new vocabulary to describe Augmented Reality, since he feels this comparison is overused.

What is Mixed Reality?

Mixed Reality—also known as hybrid reality— would allow you to manipulate a virtual object by using a physical object. In theory, you could scan a physical object using a 3D scanner, see it on the display of your goggles, manipulate a holographic version of the object with your hands, and then reprint it anew using your 3D printer. This is much more like the work of famed industrialist Tony Stark’s work in the first Iron Man movie. (Yes, we know he’s a fictional character – but the tech was very cool.)

When Will You Encounter the Next Advance in Virtual Reality or Augmented Reality?

Depending on your job, you might already have a pair of Augmented Reality goggles. DHL in the Netherlands, for example, just finished a successful pilot program using Augmented Reality in its warehouse.
As an enterprise solution, Augmented Reality could be used to help train surgeons and reduce the number of medical errors each year. Or it could be used to transmit a hologram of a violinist onto your pint of ice cream. While the first scenarios for consumer and corporate use are still in the hands of developers and experts, we expect the majority of the eventual AR opportunities will be conceived once consumer and business users get to play with the tech.
And if you are a die-hard gamer looking for the next generation in VR experience, the Oculus Rift is scheduled to ship in 1Q 2016.
We’ll discuss more about Usability in Augmented Reality along with insights from Luca Bertuccelli, United Technologies Research Center AR Usability Expert, in a blog post titled:  Augmented Reality Project Planning – The Six Questions to Answer

Sikorsky Innovations is looking for Augmented Reality Solutions.

As part of the 6th Sikorsky Entrepreneurial Challenge, the company has made a statement on the types of techniques it is looking for on the path to mission critical Augmented Reality. Specifically, the challenge calls for: Real-time augmented reality for collaborative applications and complex manufacturing processes”