Published: Oct. 6, 2023 • By Grace Wilson

Esther Rolf and her research team were awarded a United Nations' Digital GameChangers Award for a tool called MOSAIKS that promises to greatly increase who can access and analyze satellite data. 

Rolf, an assistant professor of computer science starting at CU Boulder in the fall of 2024, said "What's so exciting about receiving this award is knowing these large, multinational institutions see the potential for MOSAIKS to change the game for who can use this type of technology and what they can do with it." 

A 10,000 foot view 

MOSAIKS, short for "Multi-task Observation using SAtellite Imagery and Kitchen Sinks," greatly simplifies the expertise and computing power required to analyze satellite imagery, increasing access to more people who need data about the world around them. 

In an era where thousands of satellites hover above, capturing more than 90 terabytes of data daily, the application of machine learning to such a vast trove of remotely sensed data is crucial for disaster aid, deforestation research and food security. Yet, many satellite machine learning methods can inadvertently exacerbate harm, propagate unfairness or simply be ineffective.

This is, in part, because machine learning is often defined by what it seeks to identify. For example, most buildings will look like rectangles from above, so an algorithm would be trained on rectangular features and look at new images to see if they too have rectangles. This makes the model good at seeing rectangular buildings but potentially terrible at recognizing forests. 

MOSAIKS flips this paradigm on its head through unsupervised, multi-task observation using a technique called Random Kitchen Sinks, a tongue-in-cheek reference to the algorithm searching for an array of random features, as in the phrase "everything but the kitchen sink." 

During one computation, the same satellite image is examined for many different features. This results in unnamed feature sets of patterns that humans can identify as meaningful, including forests, roads, mountains, houses and more. 

When it comes to combining this machine computation with human identification, MOSAIKS takes two minutes on a standard laptop to achieve close to what energy-intensive large server architectures like a ResNet take almost eight hours to accomplish, making the technology more accessible to regions with more limited computing power. 

Making haste slowly

Built by a widely cross-disciplinary team, Rolf said it is essential to acknowledge how many people from different backgrounds were involved in making MOSAIKS a reality, and the length of time it took. It was one of the first research projects Rolf was part of during her PhD and the subject of one of the last papers to be added to her PhD thesis. 

"In computer science, we often work very fast, but doing something on this scale, it was important to go slowly and really listen and work together so we could think through the long-term impact of this project," she said.  

MOSAIKS is available to anyone around the world through the project's website, and it has been downloaded by people across 18 countries and 48 academic institutions thus far. Rolf said that her vision for the project is people across the world being able to make meaningful analysis of this data. 

"The idea is that many really inventive, creative users across the world can pair MOSAIKS with their carefully collected data, and run analysis truly specific to their goals." she said.

Tags: