Pictionary for Robots
“You know those AI generation programs where you enter text and it pops out an image?” asks Kailee Shermak ’25, sociology major and digital studies concentrator. “Essentially, we’re studying whether a machine can do that task as well as a human could.”
It’s June, and Shermak is a few weeks into her Mentored Advanced Project (MAP) with Fernanda Eliott, assistant professor of computer science. Eliott develops computational approaches inspired by human decision making. For example, she studies how artificial intelligence can most closely mimic human cognition when it comes to generating images.
To build better AI, Eliott needs to understand how people think visually. Here’s where Shermak and her teammates come in — as both researchers and research subjects.
Innate Imagination, the Slow Way
Over the summer, Shermak picked up the modeling software Blender 3D and learned to create three-dimensional images based on her interpretation of text prompts. By creating these time-consuming models, she’s generated a dataset that captures how humans translate words and abstract concepts into visual representations.
The MAP — and the software Shermak used to complete it — entailed a massive learning curve. “With 3D modeling tools, there’s no way to learn everything about them,” she explained at the outset of her project. “So, I’m in this ongoing process of learning new skills based on what my project and my model demand.”
More than Models
Unlike more traditional laboratory research experiences, Shermak’s MAP with Eliott was largely self-structured. She set her own schedule and pace, determined a research routine, and learned for herself the value of working outside your bedroom.
On top of their independent modeling tasks, Shermak and her teammates read extensively, deeply exploring the existing research on AI data visualization.
Weekly meetings with their lab included literature reviews, presentations, and small-group discussions — all experiences that forced Shermak to expand her academic discussion skills and helped her reach the goals she’d set for herself at the beginning of her MAP. “I wanted to be able to articulate my thoughts and feel confident enough to do so, and I’ve definitely grown in that way by doing so much small-group collaboration in a supportive environment,” Shermak says.
In the last few weeks of the summer, Shermak shifted from independent 3D modeling to a more collaborative workflow, compiling results with her teammates for their final paper. For Shermak, this was a thrilling finale. “During the research process, we didn’t share our models with each other because we didn’t want to influence each other,” she explains. After all, they were using a time-consuming modeling approach to make tangible a process that happens almost instantaneously in our imaginations. Comparing notes would have tampered with the images they produced.
But at the end of their last week, the team finally got to look at all their models. “Seeing those all together on the same page of our paper felt very gratifying,” says Shermak.
Chasing New Horizons
Her MAP has come to an end, but Shermak credits the experience with deepening her interest in research. “It’s been so rewarding to focus on a specific topic and really dive into it,” she says.
Currently, she’s conducting qualitative, interview-based research while studying abroad in the Netherlands, Germany, and the Czech Republic. It’s a different approach to research, but she’s enjoying it. And she’s excited for still more opportunities upon her return to campus — her new proficiency with Blender 3D opens many doors in the field of digital studies. “Even at 鶹ý, there are other research projects that use 3D modeling. I’m hopeful I can now apply that skill to a different realm of study.”
