Finding multiple sclerosis lesions using machine learning

We wrote a machine learning program that performs automated image segmentation of MS lesions and made an easy to understand teaching video.

Photo of Ethan Ocasio
6 12

Written by

Additional categories (optional)

  • Education

Date You Started Your Project Started


Project Stage: Select the description below that best applies to your approach.

  • Growth (have moved past the very first activities; working towards the next level of expansion)

1. The Problem: What problem are you helping to solve?

Multiple sclerosis (MS) is a debilitating brain demyelinating autoimmune disease. There is no cure, however, early detection and ongoing monitoring of the number, size and locations of MS lesions is critical in the treatment planning and outcome. Finding and measuring MS lesions in MRIs is time consuming, costly and subjective between radiologists. We are helping to solve the time, cost, and accuracy discrepancy.

2. Your Solution: How are you planning to solve this problem? Share your specific approach.

We researched deep neural networks and their applications in medical image processing. Specifically we looked into the application of neural networks to the problem of image segmentation. In standard segmentation applications, the network detects objects in the image, such as people, cars, animals, and is able to precisely outline where the objects are in the image. For MS, the network is trained to detect and outline the lesions. We used lesions pre-segmented by multiple radiologists from publicly available databases in order to train the network. We experimented with different kinds of neural network designs, focusing on variations of convolutional neural networks, which have shown great accuracy in many computer vision applications, including segmentation. Since MRIs produce multiple different images at the same time (for example, T1-weighted, T2-weighted, and FLAIR sequences) the network can use all of the information together to make a segmentation determination. Part of this investigation involved how to best combine the information from these diverse sequences to get the best result. To do this we created a training and testing framework using Python and Keras.

3. Personal Journey: What’s the story behind why you decided to start this project?

We are interested in science, math and coding. I participated in an 8 week summer science internship program at George Mason University (ASSIP) where I was granted permission to chose my own computer science research project. I chose this project, Finding MS lesions using machine learning, because I was able to combine the three areas of my interest in a meaningful way while potentially being able to help tens of thousands of people. Together we are enthusiastic to continue to advance this research. Our main motivator is in the belief that we can potentially make a significant contribution to help people throughout the world. By continuing to write machine learning programs which may detect MS lesions with 100% accuracy within seconds, we would be advancing the quick detection and measuring of multiple sclerosis, reducing cost and removing subjectivity between radiologists.

4. Selfie Elevator Pitch: Include 1-minute video that answers the following “I am stepping up to make change because...”

We are Ethan and Emily Ocasio. We are stepping up to make change because we believe our math, science and computer coding contributions can potentially make a difference for tens of thousands of people. We believe we can use technology in medicine to make a difference by writing computer programs to automate the process of finding multiple sclerosis lesions in MRI's using machine learning. Our research could reduce the cost, time and subjectivity between radiologists.

5. Example: Please walk us through a specific example of what happens when a person or group gets involved with your project.

I have always been willing and passionate about getting involved in areas related to math, science and coding, but lacked direction in where and how to apply my knowledge to make a difference. My involvement with the Girls Computing League (GCL), as well as, research opportunities and competitions such as Ashoka and T-mobile have inspired me to continue stepping up to make change. Networking and asking for feedback has helped me to advance and reflect on other ideas, be creative, and not become tunnel visioned. It has been my experience that when groups work collaboratively they can expand their reach and ultimately make a larger contribution toward their goal.

6. The X Factor: What is different about your project compared to other programs or solutions already out there?

Many different approaches have been described in the scientific literature for segmenting MS lesions. Machine learning, and in particular deep convolutional neural networks, have shown the most promise in recent years. Our approach was to create a testing framework that would allow us to test different combinations of approaches in order to optimize the network's accuracy. We ended up combining a network design known as U-Net with a training technique in which lesion edges are smoothed out, which provided the best overall result.

7. Impact: How has your project made a difference so far?

We measure the accuracy of the network's segmentation output using a variety of metrics. One of the most commonly accepted is the Dice Segmentation Score, which is a ratio of how much the predicted segments align to the ground truth segments manually performed by expert radiologist for the training set. In addition to Dice we use other calculations such as the positive predictive value and volume difference. As we keep improving the design we will follow these metrics in order to detect whether the network is approaching or surpassing the accuracy of human radiologists. As we make our results publicly available for others to use via open source, we will track popularity and utilization of those techniques by others.

8. What’s Next: What are your ideas for taking your project to the next level?

We plan to improve the MS image segmentation framework in three key areas. First, expand the input data used for training. This can be accomplished by a combination of finding more public sources of data to feed the network, data augmentation techniques such as partial rotation, and generating input data via methods such as generative adversarial networks (GANs). Second, we will experiment with using 3-dimensional input volumes in contrast to the 2-dimensional slices used before. Finally, we will expose all the network code and results in open source fashion such that other researchers can take advantage of the work.

9. Which of the following types of expertise would be most useful for you?

  • Research

10. Finances: If applicable, have you mobilized any of the following resources so far?

  • Friend support
  • Family support
  • Mentors/advisors

How did you hear about this challenge?

  • Email
  • Word of mouth


Join the conversation:

Photo of Susan McWilliams

What a fascinating and creative project. This could have a tremendous impact and help so many people. Outstanding work!

View all comments