Darsana

A virtual assistant for the blind and visually impaired

Photo of Dhruv Khurana
14 27

Written by

Eligibility: Are you employed by T-Mobile or related to an employee of T-Mobile?

  • Yes

Date Started

March / 20 / 2018

Project Stage: Select the description below that best applies to your approach.

  • Start-Up (first few activities have happened)

1. Personal Journey: What’s the story behind why you decided to start this project? NOTE: All applications must include a 1 minute video that answers: “I am stepping up to make change because..."

Before Darsana, I was the co-founder of this startup called iSight, a company that sold devices to correct visual impairments. In the summer of 2016, I was in India doing testing of one of the prototypes in an eye hospital. I met a lot of patients suffering from a variety of visual impairments: everything from macular degeneration to nystagmus. This is where I met Ram. He was a middle-aged man who had recently turned blind and was hoping for the doctors to pull off some miracle and restore his eyesight. After looking at his file, I knew there was nothing I could do for him, and so I turned to the doctor and I asked him, “Doctor, what are you going to do for him?” and the doctor with a straight face replied, “Rehab. We’re gonna teach him how to live with a stick.” I am stepping up to make a change because I know that we can do better than sticks.

2. The Problem: What problem are you helping to solve?

Blindness and visual impairment. There are around 50 million people around the world who are blind, and more than 300 million that are severely visually impaired. That means there are more than 300 million people that need assistance going through day to day life. As of now, they rely on other people or guide dogs to navigate and receive certain sensory information, making them very dependent and their life much more difficult.

3. Your Solution: How are you planning to solve this problem? Share your specific approach.

To solve this problem, we are developing a portable virtual assistant that uses artificial intelligence to accomplish many tasks that would normally require human sight. This device will be wearable, look similar to glasses and will include a computer that fits in your back pocket. Functions: Navigation - We plan to integrate a hands-free GPS system that users can control through voice commands. We are also planning to create an add-on device that can map the user’s surrounding environment on a tactile (braille) screen in real-time so that users can sense obstacles in their surroundings through touch. Illustration - The device can communicate scenes, objects, signs, people’s names, facial expressions, and more all through audio and without the use of the internet. Through voice commands, the user could ask “what is in front of me” and it could say "a laptop, a person, and a chair" Reading - The device can read out all forms of text to users, ranging all the way from digital text to pages from books. Smartphone functionality - Has all functionality of a normal smartphone like calling, texting, reading the news, and every functionality of Google Assistant.

4. Example: Please walk us through a specific example of what happens when a person or group gets involved with your project.

Development: We’re working on shrinking our device down as well as implementing all the functionality we mentioned. On a normal day, we would research on the best ways to implement a function as well as maintain the sleek design of the device. We make sure the software we make is optimized to provide fast and accurate results. We make sure we are keeping the device cheap and carefully pick our hardware. We think and innovate based on our users' problems. Outreach and Testing: We try to reach as many blindness institutions and patients as possible to get feedback on our progress. We go to these institutions in person to interview patients, doctors and staff for advice and testing. So far we have visited 3 different institutions for this purpose. This is important because we need to know how the blind live their life to make this valuable, and we don't want to make assumptions.

5. The X Factor: What is different about your project compared to other programs or solutions already out there?

Functionality and cost. There is no device in the market that does what we’ve done and what we plan for our device to do for our price. The only other company is called OrCam and they have not been able to provide the functions we can. Their device can read a text, identify colors, currency and people but it does so at a cost of $4500. Taking into account all the costs associated with creating and selling one product, our anticipated cost should be a tenth of that. There is also an app called Seeing AI by Microsoft, but it raises privacy concerns because it sends images to Microsoft's servers.

6. Impact: How has your project made a difference so far?

This device will be of value to the blind and visually impaired community because it provides them with a cost-effective manner to increase their independence and perform everyday tasks with greater ease. For example, the navigation system we have thought of will allow our users to get to where they want to go without having to rely on a third party such as a guide dog or another human. The reading functionality will empower users with the ability to understand the text in their surroundings, again, without having to rely on a third party. We’ve tested our prototype zero at 3 different locations so far and have learned a lot about what is valuable to our users. We also have testing scheduled at two of the biggest eye hospitals in the world in June. We have mentors in the form of engineers, doctors, and researchers, and also support from Perkins School for the Blind.

7. What’s Next: What are your ideas for taking your project to the next level?

There is a plan of action. We plan to keep working on the prototype and implement all functionalities as well as make it more accurate and user-friendly. Alongside, we will work to establish contracts with hospitals and rehabilitation centers for sales as well as get medical certification and patents for our technology. We have a business plan in place and our short-term goals are divided into three phases which are: R&D, Production, and Market. We plan to make B2B contracts with hospitals and centers as well as sell to the customer directly (B2C) using our website.

8. Future Support: What are the resources needed to make your vision a reality?

Funding and resources. Since this product requires hardware, we need money for experimentation, R&D, as well as manufacturing in the later phases. The other thing we’re really looking for is resources. Be it a sponsorship, contacts, or a platform, we want to get our name out there so people know we’re working on this. That would not only get us recognition but get us the approval we need. Of course, we would welcome any help that we get to accelerate the process but our core needs are those two.

9. Finances: If applicable, have you mobilized any of the following resources so far?

  • Friend support
  • Family support
  • Mentors/advisors
  • Donations between $100-$1k

10. Ripple Effect: Please share some ideas of how you could partner with other changemakers or involve other young people as leaders in making a difference.

So far we’ve gotten help from professors and classmates as well as an independent study planned for next semester in our university. Our future plan is to build a team of students for outreach and device testing. We also plan to work with other Changemakers who’ve been working in the field of healthcare or more specifically eye care in India or the US.

Help Us Support Diversity! Part 1 [optional] Which of the following categories do you identify with?

  • Asian (for example: Chinese, Filipino, Indian, Vietnamese, Korean, Japanese, Pakistani) (9)

Help Us Support Diversity! Part 2 [optional] Do you identify as part of any of the following underrepresented communities?

  • Religious minority (non-Christian)

How did you hear about this challenge?

  • T-Mobile page or contact

Evaluation results

6 evaluations so far

1. Overall evaluation

5 - This idea rocked my world. It’s awesome! - 66.7%

4 - This idea seems really exciting. With a little more polishing, it’d be among my favorites. - 33.3%

3 - This has some good elements and some areas for development - 0%

2 - This doesn’t inspire me so much - 0%

1 - It didn’t make my heart beat faster. - 0%

2. Changemaker Quality

5 - Absolutely, 100%! - 50%

4 - I feel really good about this - very promising - 50%

3 - This has some good elements and some areas for development - 0%

2 - This doesn’t inspire me so much - 0%

1 - This entry is weak here - 0%

3. Creativity

5 - Absolutely, 100%! - 33.3%

4 - I feel really good about this - very promising - 33.3%

3 - This has some good elements and some areas for development - 33.3%

2 - This doesn’t inspire me so much - 0%

1 - This entry is weak here - 0%

No Answer - 0%

4. Commitment

5 - Absolutely, 100%! - 100%

4 - I feel really good about this - very promising - 100%

3 - This has some good elements and some areas for development - 33.3%

2 - This doesn’t inspire me so much - 0%

1 - This entry is weak here - 0%

No Answer - 0%

5. Connection

5 - Absolutely, 100%! - 33.3%

4 - I feel really good about this - very promising - 66.7%

3 - This has some good elements and some areas for development - 0%

2 - This doesn’t inspire me so much - 0%

1 - This entry is weak here - 0%

No Answer or No Connection - 0%

14 comments

Join the conversation:

Comment
Spam
Photo of Armani Simmons
Team

Wow. You have a very interesting and useful product! I look forward to working with you as a fellow changemaker!

Spam
Photo of Arun
Team

Hi Dhruv! I was astonished by the amount of work you have put in to your idea! It is great to see that the project has taken off from the ground and is past the basic stages. We should keep in touch! I have a friend, who I taught mathematics through skype, who lives in Bangalore and is visually impaired. Feel free to email me if you would like his contact information! Good luck!

Spam
Photo of Dhruv Khurana
Team

Hi Arun!

Thank you for your support on our project, and this sounds wonderful! What is your email address?

Dhruv

Spam
Photo of Arun
Team

arunmohr@gmail.com

Spam
Photo of Tia Johnston Brown
Team

Hello! I love that you are inventing something that could be life changing at such a deep level. You have thought through many details including talking to the government in India about subsidizing the product. We want to support projects that are focused on a social return. I'd love to hear more about your exploration into making this affordable. This is great! One question I had is about the busy streets of India. Won't the device be sensing all of the people and constantly activating?

Spam
Photo of Dhruv Khurana
Team

Hello!

Thank you very much for your interest and support in our project, it really means a lot to us! And that is a great question!

We plan to make it so that user can control the notifications he gets about the objects in his or her environment. For example, a button push could activate the object detection and notify the users about the objects in his or her environment. We can programmatically solve the problem of 'too much' input by collecting the total number of the same objects visible in one image capture. For example, instead of the device telling the user "person detected, person detected, car detected, car detected", we can make it say "2 people and 2 cars detected."

We have currently developed a new prototype in which the hardware costs amount to around 200$. This prototype includes some of the primary features listed above. As of now it can recognize around 20 common objects with fairly good accuracy at .1-.2 seconds, but we plan to increase the accuracy and number of detectable objects in exchange for around .5 seconds of latency in the future.

Once again thank you for your support and if there are any more inquiries, my email is dkhurana@umass.edu !!

Dhruv

Spam
Photo of Brittany Corner
Team

Sight is truly one of the greatest gifts you can give someone and can improve someone's life immensely, especially amongst the poorest.

Spam
Photo of Dhruv Khurana
Team

Thank you very much for such kind words and we are very appreciative of your support!

Spam
Photo of Hannah and Zoha
Team

This is an incredible project! Great job on your success so far and we can't wait to see even more success in the future. It is inspiring to see that this initiative was the result of your work in India– do you plan on targeting the distribution of your devices to the blind population in developing countries mainly? If so, will they be free of charge to your beneficiaries?

Spam
Photo of Dhruv Khurana
Team

Hi! We very much appreciate your interest in our project, your support and your kind words! Thank you!

We want to eventually sell this device everywhere but we are focusing on developing nations first! Although we would like that this product be free, this has proven to be difficult because of the costliness of hardware. We are hoping right now we can get it subsidized for our consumers by the Indian government, so that more people can afford and will have access to this technology.

Thanks,
Dhruv

Spam
Photo of Phillip Mathew
Team

I was unaware how severely blindness affects the world. This is a very advanced and sophisticated idea and I believe it has the potential to become very successful. How would this device be able to differentiate between a chair and a table if the user was to ask, "What is in front of me?". What technology would be used to distinguish shapes and sizes of objects?

Spam
Photo of Dhruv Khurana
Team

Hi! Thank you so very much for your kind words and your interest in our idea!

That is a great question! Detecting objects in photos uses a powerful tool called a neural network that essentially mimics neurons in the brain. How it works:

1. Take thousands of images of a specific object
2. Label them all with the object name
3. Train the neural network to recognize the objects by feeding the images and their labels into it, so that it can LEARN as we do!
4. Take a new picture of the object, and the neural network has a very very high likelihood of detecting it

Neural Networks are fascinating but complex so it would be difficult to explain them in one comment. Also we do not take the 1000's of images ourself, we use pre provided datasets with images and their labels!

Thanks and I hope that helped!

Spam
Photo of Selma N
Team

I am truly inspired by your project and see so much potential! This device could truly change the lives of those who are visually impaired and may not have the ability to afford professional care. Once again, this is awesome!

Spam
Photo of Dhruv Khurana
Team

Hi Selma!

Thank you so much for the kind words, it means a lot to the both of us. I've actually read your submission on the refugee project previously and I think you have a fantastic vision and purpose. It has the potential to change many lives for the better! Good luck and I am rooting for you to win the competition!