Project Brief
Project timeline
Sep - Dec 2018
Project Type
Augmented Reality
Jiacheng Yang

Product Designer
Sara Ferris

Jose Ramirez
My Contribution
AR prototype
Interface Design
Interaction Design
Storytelling Tool Designed for Future Creators
Nebula is a AR product designed to help children actively tell stories. The project was sponsored by Google Daydream with a prompt of designing for future creators. Nebula is a mat with cameras and censors that turns any object into virtual objects or controller of virtual objects so kids can visualize their imaginations. Other than passively watching YouTube all day, we wanted to explore new ways to actively engage children between 4-8 into storytelling.
02 Challenges
Storytelling is crucial, not story watching, storytelling.
Engaging active storytelling in the age of watching
Designing a story telling product and experience
Prototyping a story telling tool to work with imaginations
Play Overview Video
When storytelling turns into "story watching"
Storytelling is a crucial process for children to learn empathy, improve word learning, cultivate creativity, collaboration in family groups and the ability to transfer problem-solving skills learned to the physical world. But with more mobile screens surrounding us, we found kids' screen time have increased dramatically where they are more often to be quiet and just passively consuming content than actively telling stories.
How we tell stories with toys...
Kids between the ages of 4 and 8 naturally use common objects to imagine and create stories. Nebula builds on this natural trait to create a new digital experience utilizing physical objects.
02 Challenges
Finding ways to prototype story telling experiences
Engaging active storytelling in the age of watching
Designing a story telling product and experience
Prototyping a story telling tool to work with imaginations
A magical mat
Nebula contains two essential parts: the mat, and a screen. The mat is fabricated with conductive material for gesture control and 4 cameras pointing up to detect and track objects.

For more product and material design explorations visit Sara's case study on Nebula.
Track objects
Whether its a empty bottle or his favorite toy, turn any object into a virtual object or controller. Expand the realm of play and show everyone his creation.
Conductive fabric
The mat is fabricated with conductive material. The conductive fabric enables gesture controls, turning the mat into a natural interface.
Play with friends
With an intuitive mobile app, setting up a playing session is very easy. Bring favorite toys together and have a toy party :)
Share creation
All the play sessions can be recorded with motion and audio. Player can easily save their story creations.
02 Challenges
From daydream to a new form of augmented reality.
Engaging active storytelling in the age of watching
Designing a story telling product and experience
Prototyping a story telling tool to work with imaginations
Combining real world objects with virtual objects
We started the project exploring the relationship between physical and virtual objects, here is a portal prototype we made to test sending objects into screen and carrying objects with physical objects. The prototype was created with a projector and basic vuforia.
The joy of physical play
In this prototype we made a modular blocks to introduce physics and real world play into AR. What we learned is that the properties of physical objects really do inspire storytelling.
The power of playing together
In this prototype we created a co playing experience to test how will two people change the story telling experience. The video shows how tow people improvised a funny story with scale difference. The prototype was created in Unity with functional VR player network.
User testing with kids
We bind toys with controllers to test how the hand movement can be seen in virtual world. After disguising with the parents, we found the boys in the video did speak out loud a lot more when he is playing with an iPad.
Prototype voice interaction
For other ways of interaction, we tested voice input for adding object into the scene. We were able to identify how we would design our voice interaction system.
An example of stories you can tell with Nebula
We tested Nebula and tried to improve a story and this is what we have :)
Switch between spaces.
When you jump into your work space, Chrome, Slack, Photoshop, Cinema 4D and many more are all there. You can expect this system to work like a great PC with no compromises in an unlimited workspace.
"Play is the only way the highest intelligence of humankind can unfold."
We as a group all believed in this quote from Joseph Chilton Pearce since the beginning of the project. The photo is us presenting to google at our final presentation from left to right is: me, Jose and Sara.

Our project was well received by the sponsor. For me, this is a fantastic experience collaborating with product designer and product design project. We were able to make things we love and learn from each other ❤
Behind the scenes...
We tested multiple prototyping paths: Unreal, Unity, see through AR with Vuforia, Leap Motion and Ze Mini. The product design side also went through so many iterations and test until we found the right solution. If you are curious of more how this concept is created, shoot me a email or text me on LinkedIn.