During the pandemic, lots of newly homebound people got the itch to knit and, in response, scores of knitters recorded how-to videos and posted them on YouTube.

"But videos don't give you feedback," said Dina El-Zanfaly, an assistant professor in Carnegie Mellon University's School of Design. She and Kris Kitani, associate research professor in the School of Computer Science's Robotics Institute, think artificial intelligence offers a better way.

El-Zanfaly and Kitani haven't built a knitting tutor yet, but they did lead the development of Origami Sensei, an AI-powered mixed-reality system that assists beginners in creating origami. Along with graduate students from both the School of Computer Science and School of Design, they recently presented their research at the Association for Computing Machinery Conference on Human Factors in Computing Systems (CHI 2025) in Yokohama, Japan.

Origami Sensei employs a type of AI — computer vision — to monitor the beginner's work. The vision system is tied to a projector, which can project imagery directly on the workspace to identify a problem or demonstrate the next step.

"You don't just get feedback; you get personalized feedback," El-Zanfaly said.

The system might not be as good as a human tutor, but human tutors are a limited resource.

"If you have an instructor, they would typically teach five or 10 people," El-Zanfaly said. "Origami Sensei is like having your own tutor, one on one. You can do things at your own pace."

The system can't yet be paused or put in reverse, but she said those features could easily be added.

What they learned in developing the origami tutoring system applies to other domains — knitting, yes, but also sculpting and physical computing.

What's up next? Welding.    

Learn more about Origami Sensei on the project's website.