immersive memory experience in mixed reality
In response to the Microsoft Design Expo prompt of creating “intentional design for positive cultural impact in mixed reality”, we designed totem, a way of recreating and sharing immersive memory experiences.
This project was created as a submission for the Microsoft Design Expo 2017: Intentional Design for Positive Cultural Impact in Mixed Reality. The prompt for this year’s challenge was the following:
Design a product, service or solution that demonstrates the value and differentiation of Mixed Reality. Your creation should demonstrate the best qualities of a Mixed Reality experience that takes into consideration the environment, objects and people that will be participating in your solution. It should meet a clear need and be extensible to wider applications.
Positive Cultural Impact in the Home
"Positive cultural impact" can include many things in many different realms. Our exploration started with the notion of mixed reality in the home. We envisioned that increasingly smaller urban housing spaces could benefit from virtual multi-purpose areas within a home to adapt to and suit tenants’ varying needs. Mixed reality could be a tool to create additional virtual “space”, or populate empty rooms with the appropriate creative or functional tools for a task. However, as our early ideation progressed, we increasingly wanted to stay away from a vision of the future home as a clinical, empty, white space devoid of personality or humanity.
What makes a home?
Instead of replacing and digitizing pieces of a home, we began to look at the physical objects, belongings, and decor in a home that give us joy, comfort and a sense of ownership. We asked ourselves why these objects remain meaningful and important despite a trend towards dematerialization: e-books, streaming music, electronic photo frames).
We recognized these belongings were markers of identity and expression, as well as ways to feel the presence of others within our homes. Research led us to a definition of the home as made up of collections, rituals and memories. This especially resonated with our values, and we began to move in the direction of enhancing the intangible, personal pieces of a home.
Sketches and storyboarding guided our early ideation. Though our design solution wasn’t clear yet, storyboarding helped us define the moments in the experience that showcased the core of our proposed idea. In addition, storyboarding raised new questions and helped us scope our idea and make further decisions.
The storyboard helped us define the imagined experience around the following principles:
- Enhance the home environment without replacing important existing objects or rituals
- Create a streamlined, unencumbered and scalable experience that is voluntary and fully controlled by the user
- Make the experience collaborative and shareable, while maintaining unique points of view
Following the storyboard, we moved into modeling UI interactions. Again, while the concept remained undefined in certain places, the exploration of the interface, gestures and interactions helped us gain clarity and refinement. Our team moved from sketches to 2D mockups and early 3D modeling.
These helped us create a timeline of the experience, incorporating the events before, during and after a memory is recreated. We discussed how collaborative experiences should work, the visual way in which a memory should be represented and the gestural and voice-activated interactions the experience should have.
totem uses mixed reality to allow people to recreate the visual and auditory environments of their memories and anchor them to objects in their home.
How totem works
When using the application on a mixed reality device, users can created virtual memory environments and attach them to objects. When they later pick up that object, they'll be able to see spheres of memory around the object. Memories are user-created recreations, not recordings of previous experiences.
Memories can be created, saved and shared with others. Memories are created through audio and gestural interaction. The immersive memory environments are built using a mixture of online content and users' existing archive of photos and videos.
Physical objects are the anchors that memories are tied to. People already associate memories with cherished objects in an invisible way, and totem allows them to make those connections more tangible. This creates a new virtual layer of rich information tied to one’s possessions that can be turned on when desired.
Modes of Interaction
When users enter memory creation mode, they can begin to build their virtual environment using a mixture of gesture and voice interaction. Users can use commands such as "I was in Hawaii, on the Big Island" or "It was raining really hard" to add to their environment. Gestures are used to scale the size of the memory from a small sphere to a fully immersive experience.
The project prompt was very open-ended, which made it difficult to quickly refine and scope a fully-formed idea. Sketching, storyboarding and generally creating visuals of the concept early on gave us the momentum to continue moving forward and scope little by little. In addition, storyboarding helped us begin to craft the narrative of our concept, which guided many design decisions and refinements. Our team also encouraged all members to sketch concepts early on, which facilitated discussions and helped us gain consensus.
More research was definitely needed, specifically into memory as well as voice and gesture interactions. Our next steps would be to examine the memory creation process more deeply, and making sure that the right mode of interaction is used to create the most effortless process possible for the user. We'd also like to explore visual treatments for the experience, to evoke with the dream-like essence of memory.