Memory Portal explores how the “spatial memories” created by the occupants of shared spaces can build a deeper connections to how they and others experience the space. We prototyped new ways of using data collected by remote sensors to resurface and materialize interactions lost to time.
my role
Interaction Designer
key collaborators
Amrita Khoshoo, Christianne Francovich, Isabel Ngan
timeframe
Sep 2020 - Dec 2020
outcome
overview
This project was completed in Fall 2020 as part of Dr. Dina El Zanfaly’s hyperSENSE: Augmenting human experience in environments in CMU’s School of Design. The findings from this project were recently accepted in CHI 2022 - Late Breaking Work for it’s innovative approach to using time and memory as design materials. We designed Memory Portal with a research through design methodology —adapting to our research as we designed and prototyped the device.
how it works
The Portal runs on a blend of physical computing, computer vision, and LED programming techniques. After the initial implementation, we ran several rounds of user testing and prototyping to refine both the device and the concept.
research question
How can we create physical and digital interactions that bridge the gap between humans and built environments to augment our senses? Increasingly shared public spaces are equipped with the IoT sensors, but the data they collect is intangible and inaccessible. Our project explored how embodied interactions, phenomenology of perception, and data physicalization could materialize this data in ambient ways to enhance the experience in these spaces.
findings
We did an exhaustive literature and precedence review of existing research and projects. We focused on how we could plot connections between time, space, and memory in a social format to further evolve the our understanding of human interactions with intelligent, personalized environments.
key findings
01
Experience of Time
Time is perceived based on a a persons internally created temporality.
02
Social Ambience
Interactive displays can foster new social interactions in public spaces.
03
Perception of Reality
We blend our past memories over the present experienced environment.
ideation
We started with an open-ended brainstorming sessions to connect our research themes to more specific direction. We each wrote down questions and concepts from our research and placed them in a loose affinity diagram to identify our focus.
portal interaction focus
After this each person sketched different types of interactions around the themes. These sketches influenced the design of a physical installation that could capture actions and display memories. My sketches contributed to exploration of the form design and how the installation would fit into the environment.
sketch 1 - interaction with form
sketch 2 - design details
Each team member also created storyboards to explore different ways we could engage the human senses through touch, sound, and vision. My storyboards explore the different interaction states and focal points within the form design. I also started diagramming how the technology would work to create the overall experience.
Each team member brought forward different aspects of how the form could embody active and passive interaction.
key storyboard themes
01
Tangible IxD
Active, tactile interactions for explorative engagement with manipulation of time
02
Audio Affordance
Passive interaction with sound as feedback to enhance interactions with memory
03
Connection Driven
Spatial memories as a social experience provide shared reflection points for building meaning and connection
interaction system
We built out an interaction system to define the possible interaction focal points of the experience space. We outlined how the interactions with time, memory, or space would trigger feedback from the Portal. Our team built out the first prototype with a second team building out more interactions.
We created a user journey map helped us translate the interaction systems goals into how the Portal creates a sense of temporality and build towards the materiality of the experience.
portal user journey
This journey map also reflects a possible exploration into creating a system of connected portals to expand the experience past a single location.
prototyping
We started prototyping with Arduino, ultrasonic sensor and piezo buzzers but to capture the fidelity we later switched Raspberry Pi, XBOX Kinect, surface transducers and RGB LED Matrices. We encountered a few technical dead-ends. But prototyping in small steps helped us isolate the interaction points and refine key parts of the experience before building the final prototype.
building the v1 prototype
This project was completed with a hybrid model. We shipped parts to different team members and prototyped in small bits. Isabel Ngan and I worked to put the final testing prototype together.
building the v2 prototype
A new group of students took over the project and continued to iterate on the design and feedback from early user testing. In initial testing, there wasn't enough interaction space. The next team increased the LED matrices.
user research
I worked with the new team to create a contextual inquiry based research protocol focusing on evaluating the impact and effectiveness of the Portal. Three key themes emerged across participants interactions during user testing that we are using to continue to develop the interaction design and form.
feedback insights
01
Memory IxD
Memory is a fluid definition defined by interaction over time. Each perceived the past interactions as a "moments of time" then specifically a memory.
“There is not much detail, so it feels like a very fuzzy memory. It's feels like ghosts of the space.”
― Participant 06
“It's very magnetic. It really has my attention. Theres not a lot out there in technology that responds to memory and spaces, and I feel like that is a very human thing.”
― Participant 10
02
Scale & Location
People view the Portal not as a neutral element of the environment but rather as a "third observer" in the space. The larger it is the more impersonal and neutral.
“I would avoid this because I don't like being observed... This fits more in a public space, I would feel uncomfortable if this was in my room.”
― Participant 03
“If the scale was larger, would it feel like memories of the people or would it feel like memories of the space.”
― Participant 07
03
Ambience
People needed more explicit feedback to understand their interactions, but they wanted to continue interact to understand their interactions over time .
“It's quite ambient. The discovery of it kept unfolding as I interacted with it.”
― Participant 12
“Right now I feel like there is a depth lacking in the interaction. I need more time to experience the device to develop a relationship over time.”
― Participant 08
conclusion
This project takes pulls forward the unseen in our life and seeks to give it form. As our technologies evolve, research such as this allows designers to proactively design and adapt technologies beyond their original specifications for the benefit of enhancing our interactions with each other and the places we love.
Our team pushed ourselves conceptually to expand our understanding of multidisciplinary design, sensory-based experiences, and technically to learn new tools and technologies. By focusing on spatial memories we sought to explore new ways of manifesting presence and memory in material ways, and how this can be applied in different contexts ( surfacing IoT data, intelligent environments, shared presence).
next steps
Refining the interaction design - Add more feedback and interaction affordances so people have a clearer idea of the interaction space
Enlarging the scale of the design - Adopt feedback to further test how the scale affect the perception of the portal
acknowledgements
I'd like to thank Dr. Dina El-Zanfaly and Matt Geiger for their help in bringing this project to fruition. This was my first time prototyping at this level of complexity and they lent their wonderful knowledge, countless Zoom calls and amazing skills to guide us.