ul{ margin-left: 40px; list-style:none; }

Time [IN]material

BUILDING A SMART DEVICE TO PERSONALIZE PUBLIC SPACES THROUGH "SPATIAL MEMORIES"

The pandemic brought into terrible focus, what we once took for granted in the ambience of everyday life: the lingering memory of presence, the sound of people in another room, groups of friends walking by. The pandemic shut down these “spatial memories” giving a haunting feeling to public spaces that once teemed with life. We designed and deployed, Portal, a smart device that can create personalized interactions based on the memory of a space and its occupants. We used the Portal to research methods of materializing these fleeting interactions lost to time.

My Role

Physical Prototyping, Design Installation, Design Research, Qualitative Interviews, Physical Computing

Collaborators

Amrita Khoshoo, Christianne Francovich, and Isabel Ngan

OUTCOME

From concept to implementation, our team took on a steep learning curve to bring this concept to life. We pushed ourselves conceptually to expand our understanding of multidisciplinary design, sensory-based experiences, and technically to learn new tools and technologies.

Building with space, time, and memory

Our research basis for this project centered on understsanding human interactions with intelligent, personalized environments.  We focused in on the following research questions:

  • How could a space be more personalized through its memories?
  • How can smart devices adapt to their contexts?
  • How can materializing the unseen in the environment change our idea of time?

We approached this project with a research through design methodology —adapting to our research as we designed and prototyped the device. 

UNDERSTANDING THE CURRENT RESEARCH AND PROJECTS

We conducted a literature and precedence review from a variety of sources that looked at connections between memory, space, and time. These developed into the themes we used to center our research and design.

Perception of Reality

We blend our past memories over the present experienced environment.

Social Ambience

Ambient, interactive displays foster new types of social interactions in public spaces.

Non-linear Experiences

More meaningful experiences are more removed from linear time.

Experience of Time

Time is perceived based on a a persons internally created temporality.

Translating the intangible into the tangible

We translated our research themes into goals for the interaction design. Then through our user testing and prototyping explored the form of the Portal and people’s interactions. Combined, these established the guidelines for creating an experience with a “spatial memories. “

GENERATIVE IDEATION AND CONCEPT BUILDING

We started developing the concept with open-ended brainstorming sessions in Miro around re-materializing mood, changing behavior through connection, fostering connectedness, and exploring permanence of time.

Screen-Shot-2022-01-14-at-3.39.31-PM

We narrowed down to two: fostering the connection between two people and exploring temporality. Our design focuses on how time influences how we foster a connection with others.

PARALLEL IDEATING & CONCEPT DEVELOPMENT

We then began parallel ideation, where each person sketched different types of interactions that could materialize the relationship between human connection and time. These sketches influenced the design of a physical installation that could capture actions and display memories.

casestudy_Portal-ConceptDev
casestudy_Portal-ConceptDev_02
My sketches contributed to exploration of the form design and how the installation would fit into the environment. We had to move away from a hanging structure because there were too many wires to hide and the weight of all the technology required more expensive structural support than was needed for a prototype.

EXPERIENCE STORYBOARDING

To explore the interplay between interaction design and form, each team member created storyboards to explore different ways we could engage the human senses through touch, sound, and vision.

casestudy_Portal-ogstoryboarding_01
casestudy_Portal-ogstoryboarding_03
My storyboards explore the different interaction states and focal points within the form design. This is was when as a team we knew we needed to start defining the technical requirements to make these a reality so I also started outlining how the technology would work to create the overall experience.

We each sketched how the form could embody active and passive interaction. Other team members brought forward different aspects of the interaction design but overall we found three overarching themes throughout our storyboards:

Tangible Control Design

Active, tactile interactions for explorative engagement with manipulation of time

Audio Based Affordance

Passive interaction with sound as feedback to enhance interactions with memory

Connection Driven Interactions

Spatial memories as a social experience provide shared reflection points for building meaning and connection

Designing a system of interactions

While the storyboards outline the individual interaction affordances of the Portal, we still needed a system for exploring and defining the possible interaction focal points of the experience space. We developed this table as the beginning of the interaction system. This helped us define the specific interactions with time, memory, or space that would trigger feedback from the Portal.

Interaction System Table
At this stage, we were still exploring connecting to a second space, but after a design critique we decided to narrow the focus on the Portal to interactions within a single space. This ultimately helped refine our concept to focus on creating a personalized environment.

PORTAL USER JOURNEY

The interaction system allows us to begin breaking down the perception of a participant’s temporal experience, but we still had questions about the overall experience. The user journey map helped us translate the interaction systems goals into how the Portal creates a sense of temporality and build towards the materiality of the experience.

casestudy_Portal-UserJourney

Prototyping as experimentation

Scaffolding the complexity of the design throughout the duration of project allowed us to refine our research and concept alongside the prototyping and implementation. Personally, this helped me navigate the new technical knowledge required to create working prototypes.

casestudy_Portal-technical-system

RAPID PROTOTYPING, AKA FAILING FAST FAILING FORWARD

During the prototyping process we encountered a few technical dead-ends. The small steps helped us isolate the interaction points and refine key parts of the experience before building the final prototype. We started prototyping with Arduino and Piezo Buzzers but to capture the fidelity we later switched Raspberry Pi, XBOX Kinect, surface transducers and RGB LED Matricies.

Screen-Shot-2022-01-14-at-10.53.43-PM
Screen-Shot-2022-01-14-at-11.10.47-PM
Screen-Shot-2022-01-14-at-11.11.26-PM
Screen-Shot-2022-01-14-at-10.52.57-PM-1
Screen-Shot-2022-01-14-at-11.11.38-PM
Screen-Shot-2022-01-14-at-11.10.28-PM
Screen-Shot-2022-01-14-at-11.11.12-PM

FABRICATION AND INSTALLATION

The Portal prototype was constructed from foam, acrylic, and wood veneer. We cut out spaces in the foam for all the electronic components to fit in. At this stage during the installation, we struggled to get the device to display as we prototyped it, before discovering we had mixed some of the wires up... after some rewiring we got it working.

Installation Process
Fabrication Process
5ffe628aecf46ddd3ec9418b_DSC_0012
Screen-Shot-2022-01-14-at-11.37.33-PM

Second Round — Testing Impact

Portfolio-Image_1x1_Timeinmemorial
Based off of feedback from the initial user tests, people felt like there wasn't enough interaction space. The next team increased the LED matrices.
Portfolio-Image_1x1_Timeinmemorial-1
Additionally in order to blend the Portal into the environment more, the shape was changed to be more wall-like.

After developing the prototype and some initial user testing, a new group of students took over the project and continued to iterate on the design and feedback from early user testing. In this second round of the portal development, I created a research protocol focusing on evaluating the impact and effectiveness of the Portal. I used think aloud interviews and surveys to capture feedback from people’s experience with the Portal.

Three key themes emerged across participants interactions during user testing that we are using to continue to develop the interaction design and form.

"The fidelity of the display is very low so there is not much detail, so it feels like a very fuzzy memory. It's really emphasizing the people not the things. It's feels like ghosts of the space."
— P06-32

"It's very magnetic. It really has my attention. Theres not a lot out there in technology that responds to memory and spaces, and I feel like that is a very human thing."
—P10-2X

Interactions with Memory

Memory is a fluid definition defined by interaction over time. Each perceived the past interactions as a "moments of time" then specifically a memory.

Sense of Scale and Location

People view the Portal not as a neutral element of the environment but rather as a "third observer" in the space. The larger it is the more impersonal and neutral.

"I would avoid this because I don't like being observed but if I wanted to interact with it then I go closer. This fits more in a public space, I would feel uncomfortable if this was in my room."
— P03-26

"If the scale was larger, would it feel like memories of the people or would it feel like memories of the space."
— P06-32

"Right now I feel like there is a depth lacking in the interaction. I need more time to experience the device to develop a relationship over time."
— P08-28

"I think it's showing us footage of based on some sort of interactions that we are having with it. It's quite ambient. The discovery of it kept unfolding as I interacted with it."
—P12-2X

Balancing Ambience with Affordance

People need more explicit feedback and wanted to continue to interact and view the unfolding of the interactions over time .

Reflection

While still in an early stage, we developed the portal as a way to explore how temporality can be encoded into the environment, and how this might explore new ways of manifesting presence and memory in material ways. Keeping these spaces alive even if we cant be in them together at the same time. I loved this project because it pulls forward the unseen in our life and seeks to give it form, for the benefit of enhancing our interactions with each other.

NEXT STEPS

This project is still in evolving with a new team of students under the direction of Dr. Dina El-Zanfaly, a computational design and interaction researcher and an Assistant Professor in the School of Design at CMU. She directs the hyperSENSE: Embodied Computations Lab.

Future iterations of the portal's design will continue to build towards a greater understanding of the relationship between temporal and spatial memories through technological means.

  • Refining the technical interactions to include touch, adding more clarity to the video interactions, tuning the sound ambience to the right level.
  • Building more feed-forward affordance so that people have a clearer idea of the interaction space
  • Enlarging the scale of the design while considering accessibility factors such as considering the accessibility of color, height, and sensory interactions

ACKNOWLEDGEMENTS

I'd like to thank Dr. Dina El-Zanfaly and Matt Geiger for their help in bringing this project to fruition. This was my first time prototyping at this level of complexity and they lended their wonderful knowledge, countless Zoom calls and amazing skills to guide us.

Designed by Rachel Arredondo