User Interface for Astronauts

Creating a User Interface (UI) with Augmented Reality (AR) on the Microsoft HoloLens to improve astronaut Extra-Vehicular Activities (EVAs).

Photo of Leah Davis
2 10

Written by

Additional categories (optional)

  • Education
  • Technology

Are you employed by T-Mobile or related to an employee of T-Mobile?

  • No

Eligibility: Date of Birth

August 4th, 1999

Help us stay in touch!

(214) 709 -1009 338 Town East Blvd. Texas: Sunnyvale (75182)

Website or social media url(s) (optional):

Date You Started Your Project Started

September 27th, 2018

Project Stage: Select the description below that best applies to your approach.

  • Growth (have moved past the very first activities; working towards the next level of expansion)

1. The Problem: What problem are you helping to solve?

An integral part in every manned mission is astronauts completing Extra-Vehicular Activities (EVAs). Tasks range in complexity, but currently are completed with cuff checklists attached to an astronaut’s wrist. With a skewed view from the spherical helmets, this can’t be the most productive display. EV Astronauts also have a crippling reliance on their Intra-Vehicular (IV) counterparts as well as Mission Control Center (MCC) via a voice loop.

2. Your Solution: How are you planning to solve this problem? Share your specific approach.

The team is taking part in the NASA Spacesuit User Interface Technologies for Students (SUITS) competition through the Johnson Space Center. For this competition, the team is creating an augmented reality User Interface (UI) to be deployed on the Microsoft HoloLens. Displaying tasks, biometrics, and navigating the user through their environment, the team hopes to make EVAs more efficient. The technical team is coding in Unity and the research team uses generated UIs for subjects to go through tasks with specific settings changed. The findings from the research will lend to a more user-friendly device. Now, EVAs are conducted by two astronauts outside the vehicle while assisted by two astronauts inside the vehicle (and MCC). If IV astronauts could obtain Virtual Reality (VR) devices and integrate with AR devices of EV astronauts, something as simple as indicating a part could be done by highlighting an object. Instead of taking the time to explain confusion, the IV astronauts could directly mesh into the Heads-Up Display (HUD) the EV astronaut has. The team is exploring assimilating AR and VR devices, as the more an astronaut can be communicated with, the more productivity.

3. Personal Journey: What’s the story behind why you decided to start this project?

My second semester of college I wanted to get aerospace engineering experience so I joined a Human Bioastronautics and Performance lab. From there, another freshman and I created the Spacesuit User Interface Technologies for Students (SUITS) competition team. Having gone to a performing arts high school for theatre, I was great at communicating but didn’t have a lot of technical skills. I started as a community engagement leader, writing lessons for kids about aerospace and quickly moved into a leadership role on the team. I’m a huge advocate for the unforeseen benefits of space exploration so to be able to work on a real project and interface with real NASA employees everyday is an honor. The goals and achievements we make, NASA makes and implements to the 2024 xEMU new Lunar suit that just passed its PDRs in August.

4. Selfie Elevator Pitch: Include 1-minute video that answers the following “I am stepping up to make change because...”

Changing the way astronauts complete missions for more independence as we venture to further missions and longer duration flights.

5. Example: Please walk us through a specific example of what happens when a person or group gets involved with your project.

Immediately when someone joins, we let them try out our AR space simulation on the Hololens device so that can learn what it is like to be an astronaut. This involves learning about the futuristic design of the NASA xEMU spacesuit by observing real size xEMU holograms floating in front of them. From there, they actually get to run through a virtual space mission simulation. This involves being inside of a lunar habitat and stepping through procedures listed in a floating hologram to help them exit the airlock and step out onto the Moon's surface. From there they are guided through a scenario to replace a tire on a lunar rover. Once that is finished, they’ve successfully passed astronaut training and have the option to help add to our AR simulation be developing 3D models, contributing code, or just giving us ideas of space missions that they think would be interesting to the community.

6. The X Factor: What is different about your project compared to other programs or solutions already out there?

The difference about joining our group is that each and every member gets to interface with NASA engineers and astronauts, as well as help come up with concepts for future spacesuit designs that are actually considered by NASA's design team. However, it doesn't have to be limited to that. Since we are designing concepts in both augmented and virtual reality, any idea can be prototyped and showcased to the world with our help. For example, if someone had an idea for a new molecular compound or robotic assistant, they could utilize our technology to draft the idea as a hologram.

7. Impact: How has your project made a difference so far?

Through our work so far, we've given NASA the concept of a "VR Telepresence Whiteboard", meaning sketches from an astornaut in a VR headset drawn in 3D space could help guide another astronaut wearing an AR visor during a spacewalk. This was one of the ideas the NASA engineering team considered in their design for the future xEMU spacesuit. This past August, they held a Preliminary Design Review (PDR) presentation for the project and the design passed! Many of the NASA employees commented that such a VR whiteboard would be invaluable for future missions, since it help with realtime decision making. In the past, there have been anomaly situations in which the voice/audio loop was corrupted and having such a 3D visual cue in front of their face could have helped assure mission success as well as mitigate potential risk. This idea can also apply to other situations needing collaboration.

8. What’s Next: What are your ideas for taking your project to the next level?

Taking this project to the next step would be including AR holograms that could be run on mobile devices that could be viewed by anyone. Creating a series of VR/AR simulations on STEAM education, many K-12 students would be inspired to learn about rocket science, inter-molecular chemical reactions, electronic circuitry, computer science algorithms, structural mechanics, and other such concepts that would normally be considered boring textbook reading. Additionally, more research is necessary to formulate the most productive human factors entities to incorporate in user interfaces. Simple attributes such as display distance can be studied and compiled into research papers to improve AR products from anyone.

9. Which of the following types of expertise would be most useful for you?

  • Project Plan & Strategy

10. Finances: If applicable, have you mobilized any of the following resources so far?

  • Mentors/advisors
  • Donations between $100-$1k
  • Donations between $1k-$5k

Help Us Support Diversity! Part 1 [optional] Which of the following categories do you identify with?

  • White (for example: German, Irish, English, Italian, Polish, French) (6)

Help Us Support Diversity! Part 2 [optional] Do you identify as part of any of the following underrepresented communities?

  • No, I do not identify with an underrepresented community

How did you hear about this challenge?

  • Social media
  • Search engine
  • Article in the news
View more

Attachments (6)

Final Report Due 6_20_19 (1).pdf

This is the Final Report turned into NASA SUITS administrators on the ideas and conclusions of the team's contributions in the 2019 competition cycle.


At the NASA Johnson Space Center for 2019 NASA Spacesuit User Interface Technologies for Students (SUITS) Competition.


Join the conversation:

Photo of Christina

Awesome work Leah!!!

Photo of Melissa Davis

Great job Leah!!