A virtual assistant for the blind and visually impaired

Photo of Dhruv Khurana
14 27

Written by

Date Started

March / 20 / 2018

Project Stage: Select the description below that best applies to your approach.

  • Start-Up (first few activities have happened)

1. Personal Journey: What’s the story behind why you decided to start this project? NOTE: All applications must include a 1 minute video that answers: “I am stepping up to make change because..."

Before Darsana, I was the co-founder of this startup called iSight, a company that sold devices to correct visual impairments. In the summer of 2016, I was in India doing testing of one of the prototypes in an eye hospital. I met a lot of patients suffering from a variety of visual impairments: everything from macular degeneration to nystagmus. This is where I met Ram. He was a middle-aged man who had recently turned blind and was hoping for the doctors to pull off some miracle and restore his eyesight. After looking at his file, I knew there was nothing I could do for him, and so I turned to the doctor and I asked him, “Doctor, what are you going to do for him?” and the doctor with a straight face replied, “Rehab. We’re gonna teach him how to live with a stick.” I am stepping up to make a change because I know that we can do better than sticks.

2. The Problem: What problem are you helping to solve?

Blindness and visual impairment. There are around 50 million people around the world who are blind, and more than 300 million that are severely visually impaired. That means there are more than 300 million people that need assistance going through day to day life. As of now, they rely on other people or guide dogs to navigate and receive certain sensory information, making them very dependent and their life much more difficult.

3. Your Solution: How are you planning to solve this problem? Share your specific approach.

To solve this problem, we are developing a portable virtual assistant that uses artificial intelligence to accomplish many tasks that would normally require human sight. This device will be wearable, look similar to glasses and will include a computer that fits in your back pocket. Functions: Navigation - We plan to integrate a hands-free GPS system that users can control through voice commands. We are also planning to create an add-on device that can map the user’s surrounding environment on a tactile (braille) screen in real-time so that users can sense obstacles in their surroundings through touch. Illustration - The device can communicate scenes, objects, signs, people’s names, facial expressions, and more all through audio and without the use of the internet. Through voice commands, the user could ask “what is in front of me” and it could say "a laptop, a person, and a chair" Reading - The device can read out all forms of text to users, ranging all the way from digital text to pages from books. Smartphone functionality - Has all functionality of a normal smartphone like calling, texting, reading the news, and every functionality of Google Assistant.

4. Example: Please walk us through a specific example of what happens when a person or group gets involved with your project.

Development: We’re working on shrinking our device down as well as implementing all the functionality we mentioned. On a normal day, we would research on the best ways to implement a function as well as maintain the sleek design of the device. We make sure the software we make is optimized to provide fast and accurate results. We make sure we are keeping the device cheap and carefully pick our hardware. We think and innovate based on our users' problems. Outreach and Testing: We try to reach as many blindness institutions and patients as possible to get feedback on our progress. We go to these institutions in person to interview patients, doctors and staff for advice and testing. So far we have visited 3 different institutions for this purpose. This is important because we need to know how the blind live their life to make this valuable, and we don't want to make assumptions.

5. The X Factor: What is different about your project compared to other programs or solutions already out there?

Functionality and cost. There is no device in the market that does what we’ve done and what we plan for our device to do for our price. The only other company is called OrCam and they have not been able to provide the functions we can. Their device can read a text, identify colors, currency and people but it does so at a cost of $4500. Taking into account all the costs associated with creating and selling one product, our anticipated cost should be a tenth of that. There is also an app called Seeing AI by Microsoft, but it raises privacy concerns because it sends images to Microsoft's servers.

6. Impact: How has your project made a difference so far?

This device will be of value to the blind and visually impaired community because it provides them with a cost-effective manner to increase their independence and perform everyday tasks with greater ease. For example, the navigation system we have thought of will allow our users to get to where they want to go without having to rely on a third party such as a guide dog or another human. The reading functionality will empower users with the ability to understand the text in their surroundings, again, without having to rely on a third party. We’ve tested our prototype zero at 3 different locations so far and have learned a lot about what is valuable to our users. We also have testing scheduled at two of the biggest eye hospitals in the world in June. We have mentors in the form of engineers, doctors, and researchers, and also support from Perkins School for the Blind.

7. What’s Next: What are your ideas for taking your project to the next level?

There is a plan of action. We plan to keep working on the prototype and implement all functionalities as well as make it more accurate and user-friendly. Alongside, we will work to establish contracts with hospitals and rehabilitation centers for sales as well as get medical certification and patents for our technology. We have a business plan in place and our short-term goals are divided into three phases which are: R&D, Production, and Market. We plan to make B2B contracts with hospitals and centers as well as sell to the customer directly (B2C) using our website.

8. Future Support: What are the resources needed to make your vision a reality?

Funding and resources. Since this product requires hardware, we need money for experimentation, R&D, as well as manufacturing in the later phases. The other thing we’re really looking for is resources. Be it a sponsorship, contacts, or a platform, we want to get our name out there so people know we’re working on this. That would not only get us recognition but get us the approval we need. Of course, we would welcome any help that we get to accelerate the process but our core needs are those two.

9. Finances: If applicable, have you mobilized any of the following resources so far?

  • Friend support
  • Family support
  • Mentors/advisors
  • Donations between $100-$1k

10. Ripple Effect: Please share some ideas of how you could partner with other changemakers or involve other young people as leaders in making a difference.

So far we’ve gotten help from professors and classmates as well as an independent study planned for next semester in our university. Our future plan is to build a team of students for outreach and device testing. We also plan to work with other Changemakers who’ve been working in the field of healthcare or more specifically eye care in India or the US.

How did you hear about this challenge?

  • T-Mobile page or contact


Join the conversation:

Photo of Armani Simmons

Wow. You have a very interesting and useful product! I look forward to working with you as a fellow changemaker!

View all comments