A Virtual “What’s Up Doc” on Your Cellphone

Researchers have devised a way to project the image of a cartoon character from a cell phone onto any surface where it can then interact with another projected character from a different cell phone.

The images may wave at each other, shake hands, and even give each other presents. Might be a good way to score a date, say its creators at the MIT Media Lab.

Their recently developed prototype is called PoCoMo, for Projected Collaboration using Mobile Devices, which is the brainchild of principal investigator, first-year PhD student Roy Shilkrot.

PoCoMo uses a Samsung Halo projector phone integrated in a prototype case that employs a mirror to align the camera and the projector, along with unique computer vision algorithms, to create a simulated “ice-breaker” meet-up between an interested cartoon man and a potentially attracted cartoon woman, or vice versa.

“Initially we brainstormed fifty types of games before we came up with this idea,” says Shilkrot. “What finally brought us to this—we tried to create a social interaction in a mobile, projected way.”

Along with collaborators in the Media Lab’s Fluid Interfaces group, first-year PhD student Seth Hunter and group director Pattie Maes, Shilkrot developed PoCoMo as the first interactive game using multiple mobile projectors in an uncontrolled environment—meaning any environment that doesn’t need a lab table and expensive equipment to run in.

By aligning the projected characters on any surface, given the proper lighting conditions, one phone runs software that tracks the position of the other phone’s character. When the potential cartoon couple is close enough, they respond—first by acknowledging the other’s presence and smiling, then waving, shaking hands, and walking together, conceivably off into the techno-sunset. Characters can even leave a present like a picture file for the other character to open, though this aspect of the tool is still in it infancy.

In developing PoCoMo, Shilkrot hoped to create a platform users could tinker with. “We wanted to use commercially available hardware and open source software so people could use this on their own,” says Shilkrot.

Mobile projector phones are commercially available today—you can buy an AT&T projector phone for under $300—and the tracking software is already available on the social coding site Github (https://github.com/royshil/HeadFollower).

Shilkrot says that PoCoMo is part of a growing interest in the tech world to expand the reach of mobile devices. New prototypes like PoCoMo that create new avenues for social interaction could eventually alter our relationships with our cell phones.

University of California-San Diego computer science researcher Lisa Cowen has explored the way in which projector phones are changing how people communicate. She led a 2010 study to investigate how people use projector phones in the real world. Her approach: hand out ten phones to ten people and see how they’re used over a four-week timeframe. Her findings suggest we may be in the market for more than just passive digital data reception.

“Projector phones have been marketed for really passive uses, where you set it down in your living room and you show pictures from your latest trip to your friends. Or they’ve been marketed to people in a business environment for giving presentations on the fly,” says Cowen.

Yet, during the study, people used their phones mostly in jest by projecting playful images like sharks and donuts at walls and tables near their friends, explains Cowen. “We found that people really used these in active ways. They’re very playful. They’re not just setting a projector down. They’re really using them to communicate, not just for the content being presented.”

“(We) try to imagine these devices as not just a teleportal out of your current environment, but as actually being a productive part of that environment,” says Bill Griswold, Cowen’s advisor and head of the Ubiquitous Computing and Social Dynamics Group at UCSD.

Shilkrot sees a similar vision in PoCoMo. “Just in time, just in place,” is the motto of PoCoMo, according to the MIT student. He says interactive mobile projection devices could blur the line between our digital screens and the objects around us. “One of the reasons we created this project was to break free from the screens that we stare at all day,” he says.