Exploring the role of physical motion in Intelligent Personal Assistants (IPAs)

Project Overview

Our goal for the project was to explore the role physical motion plays in defining the social relationship people have with their IPAs. To accommodate motion in the current IPAs, we designed our own IPA form. Through low fidelity motion sketches, we conducted a user study to find out how users perceive motion in IPAs.

We found that physical motion in IPAs enable people to communicate the social and emotional relationship these IPAs might have with them in the future. People were able to attribute personalities to these IPAs with their own background story. Finally, we also created our own engineered fabrication models for a selected group of motion designs— allowing us to learn the limits and complexities of engineering motion into an IPA.

Team members: Jenny Hu, Tatyana Mustakos, Tedo Chou

Role: Design, User study, Stop-motion Animation, Hardware Prototyping (using Arduino)

We explored five different states of the IPA, three functional states – processing, awareness, and confusion – and two personality based/emotional expressions – sassiness and celebration

DESIGN

Redesigning form to give affordance to motion

In order to generate and study motion as a means of communication for IPAs, we recognized that we would need to create our own IPA form. Something about the form should indicate the intention and availability of motion as a means of communication.

Inspiration from current IPA forms

We were inspired by the Raven H’s expressiveness and how it allowed intensive movement. We were also inspired by the Google mini and Amazon echo dot for their subtlety and size— indicating a much more passive existence for the IPA as a whole. We wanted to maintain the same subtle geometries that these forms already have, and give affordances to motion.

Inspired by the expressiveness and clear affordances of motion in Raven H
Inspired by the size of mini and dot which lends them a passive existence in space
PEBBLE: The final form design, hand made from clay
Clay Model Credits: Jenny Hu

Design features, a head and a body

Our aim was to maintain a similar geometry as mini and dot, but hint at an affordance of movement. We did so by establishing two main features: a head and a body. The head, being the piece that moves the most, and the body being the piece that is stable and establishes the threshold of movement for the head.

PROTOTYPING

Prototyping expressions through stop motion

We prototyped a variety of motion sketches for each of the five expressions: awareness, processing, confusion, celebration and sassiness. Awareness, processing and confusion are functional expressions in that they help the users in understanding when the device is ready to listen, when it is processing and when it did not understand the user. Celebration and sassiness are emotional expressions, they don’t help the user with any information but they do add playful interactions or banter that exists purely for fun.

Awareness: when the user wakes up Pebble by calling its name, or while the user is speaking
Processing: when the Pebble is executing a command or interpreting speech
Confusion: when the Pebble hears the user but does not understand what he/she said.
Sassiness: when the Pebble responds in a mocking way
Celebration: Pebble celebrates when it is directly asked to, or when triggered by an event (e.g. party music is playing, or it is someone’s birthday, etc)

USER STUDY

Comparing the physical movement with light motion 

We conducted an informal study with eight users through casual conversation and interviews to compare existing non verbal communication modality in google mini, light animation with physical movement in Pebble. Following were the research questions of our user study:

  • What does each motion sketch convey to users? How do they describe it?
  • Do the motion sketches convey the respective expression? Which motion sketch user likes the best for each expression?
  • Does physical movement in Pebble define any social relationship users have with their personal assistants?
  • How do users perceive light animation in google mini as compared to physical movement in Pebble as a means of nonverbal communication?
Google mini uses four blinking dots as a means of non verbal communication 
Pebble moves about different axes to communicate nonverbally

INSIGHTS

01.   Motion is more communicative socially and emotionally

  • Comparatively to the Google mini, the motion studies drew statements from interviewees that conveyed motion adding much more ‘life’ to the device. 
  • Using head nods, body gestures, sound and metaphor, they described the sketches with effects like “~~~woooooo~~” and “unce! unce! Unce!” and metaphors such as “my nagging grandma”.
  • Through these statements, we can infer a certain level of complex, dynamic social and emotional communication between the motion of the IPA and the user’s perception.

User quotes

According to Hall, the space between two people during their interaction defines how close their relationship is.

02.   When and where someone interacts with a motion based device matters

  • In cases, when the goal of the users is to get a task done like checking the weather, playing a song etc. they expressed their tendency to not look at the IPA and hence it doesn’t matter what it is doing.
  • They expressed a desire to “have a conversation” with the device if it is doing some motion to see how it would react.
  • They would want their IPA to move only when they are near to it.

Does the presence of physical movement reduce the interpersonal distance between the IPA and its user to personal and intimate spaces ?

03.   People are able to creatively attribute personas to the device

  • People ascribed personalities to the motion sketches, like “my nagging grandma”, “an awkward dancer on the dance floor”, “my cute little pet”
  • While it seems these attributed personas are difficult to control, perhaps due to fidelity, but also perhaps due to motion as a principle of expression, we are begged to ask the question whether the design of the persona of these devices is entirely in the hands of the designers of the IPA.
  • How will a designer differentiate between these personas using motion design is another question.

WINNING DESIGNS

At the end of the user study, we identified one motion sketch per expression which was perceived by the majority as conveying that expression the best.

FABRICATION

Complexities of engineering physical movement into an IPA 

We created our own engineered fabrication models for a selected group of these designs— allowing us to learn the limits and complexities of engineering motion into an IPA.

We had to design two functional models because movement required rotation around two different axes which was difficult to achieve in a single model given our team’s resources and skills
Our fabricated models were nowhere close to our style model because bottom part needed more space to integrate circuitry and also hide it and hence a tissue dispenser looking model instead of a grapefruit-in-a-bowl design
The fabricated movement of awareness, confusion and processing is less expressive as compared to what we designed because we could only move the head of the Pebble along one axes

STUDY LIMITATIONS

  • We were comparing our low fidelity prototypes with a highly finished working product (Google Mini) and this mismatch in the fidelity also affected user’s responses.
  • During the study, first, we showed the participants light motion and physical motion without the audio. After they expressed what each motion conveys to them, we played the audio to reveal the context in which Google Mini and Pebble performed the respective motion. A better approach would have been to play the audio alongside the individual motion.