Smell-Enhanced Human-Computer Interaction?

Opportunities for Odor:

Experiences with Smell and Implications for Technology

Marianna Obrist1,2, Alexandre N. Tuch3, 4, Kasper Hornbæk4

m.obrist@sussex.ac.uk | a.tuch@unibas.ch | kash@diku.dk

1Culture Lab, School of Computing Science Newcastle University, UK

2School of Engineering and Informatics University of Sussex, UK

3Department of Psychology, University of Basel, CH

4Department of Computer Science, University of Copenhagen, DK

Permission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the fullcitation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise,or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Permissions@acm.org. CHI 2014, April 26 – May 01 2014, Toronto, ON, Canada Copyright 2014 ACM 978-1-4503-2473-1/14/04…$15.00. http://dx.doi.org/10.1145/2556288.2557008

ABSTRACT

Technologies for capturing and generating smell are emerging, and our ability to engineer such technologies and use them in HCI is rapidly developing. Our understanding of how these technologies match the experiences with smell that people have or want to have is surprisingly limited. We therefore investigated the experience of smell and the emotions that accompany it. We collected stories from 439 participants who described personally memorable smell experiences in an online questionnaire. Based on the stories we developed 10 categories of smell experience. We explored the implications of the categories for smellenhanced technology design by (a) probing participants to envision technologies that match their smell story and (b) having HCI researchers brainstorm technologies using the categories as design stimuli. We discuss how our findings can benefit research on personal memories, momentary and first time experiences, and wellbeing.

Author Keywords

Smell; smell experiences; odor; olfaction; user experience; smell-enhanced technology; narratives; smell stories; crowdsourcing; design brainstorming; designing for smell.

ACM Classification Keywords H.5.2 Information interfaces and presentation (e.g., HCI): Miscellaneous. General Terms Experimentation, Human Factors, Design.

INTRODUCTION

Smell plays an important role for memories and emotions. Compared to other modalities, memories evoked by smell give stronger feelings of being brought back in time, are more emotionally loaded, are experienced more vividly, feel more pleasant, and are autobiographically older (ranging back to childhood) [15,33]. Smell is incredibly powerful in connecting humans to past events and experiences.

Matsukura et al. [22] recently proposed the Smelling Screen, an olfactory display system that can distribute smells. Earlier work in HCI has proposed other systems that capture and generate smells. For example, Brewster et al. [5] developed a smell-based photo-tagging tool, and Bodnar et al. [4] showed smell to be a less disruptive notification  mechanism than visual and auditory modalities.

Thus, smell technologies are already emerging. Our understanding of how these technologies match the experiences with smell that people have or want to have is surprisingly limited.

First, while technologies such as those mentioned above are often evaluated, the results mainly concern the perception of smell. The evaluations say little about the general potential of smell technologies for  humans or their ability to generate particular experiences.

Second, whereas earlier work states that the subjective experience of smell stimulation is crucial for the success of a system (e.g., [5]), we are unaware of work in HCI that studies the subjective experience of smell (though see [17]). Third, several hundred receptors exist for smell and we cannot rely on any primary smells to stimulate a particular experience, as might be imagined for other human senses [2].

Taken together, these points suggest that we can only link smell tenuously to particular experiences or emotions. This limits our ability to design for a spectrum of experiences.

The present paper focuses instead on experiences and emotions related to smell and links them to potential technologies. Inspired by work on user experience [14,34], we concentrate on personal memorable smell experiences and their links to emotion. From the focus on experience we developed design guidance for smell-enhanced technologies. The goal is to contribute knowledge on subjective smell experiences and their potential for design.

We collected 439 smell stories, that is, descriptions of personal memorable experiences involving smell.We distributed a questionnaire through  crowdsourcing, ensuring a large-scale coverage and variety of smell stories. We analyzed the stories and identified 10 main categories and 36 sub-categories. Each category was described with respect to its experiential and emotional characteristics and specific smell qualities. Besides smell stories associated with the past (e.g., memory of loved people, places, life events) we identify stories where smell played an important role in stimulating action, creating expectations, and supporting change (e.g., of behavior, attitude, mood). Smell can sometimes also be invasive and overwhelming, and can affect people’s interaction and communication. Within the categories, we identify common smell qualities and emotions, which support the exploration of opportunities for design. In particular, we discuss the implications for technology based on feedback from participants and on a  brainstorming session with HCI researchers working on smell technologies.

The main contributions of this paper are (1) an experiencefocused understanding of smell experiences grounded in a large sample of personal smell stories, which allowed us

(2) to establish a systematic categorization and description scheme for smell experiences, leading to

(3) the identification of technology implications by participants,and

(4) the exploration of design potentialities by HCI researchers.

THE HUMAN SENSE OF SMELL

The sense of smell is the most complex and challenging human sense.

Hundreds of receptors for smell exist and the mixing of the sensations, in particular with our sense of taste, is immense [2]. The sense of smell is further influenced by other senses such as vision, hearing, and touch; plays a significant role for memory and emotion; and shows strong subjective preferences. Willander and Larsson [33] showed that autobiographical memories triggered by smell were older (mostly from the first decade of life) than memories associated with verbal and visual cues (mostly from early adulthood). Moreover, smell-evoked memories are associated with stronger feelings of being brought back in time, are more emotionally loaded, and are experienced more vividly than memories elicited through other modalities [15,33]. No other sensory system makes the direct and intense contact with the neural substrates of emotion and memory, which may explain why smellevoked memories are usually emotionally potent [15].

The emotion-eliciting effect of smell is not restricted to the context of autobiographical memories. Smell is particularly useful in inducing mood changes because they are almost always experienced clearly as either pleasant or unpleasant [8]. For instance, Alaoui-Ismaïli et al. [1] used ‘vanilla’ and ‘menthol’ smells to trigger positive emotions in their participants (mainly happiness and surprise) and ‘methyl methacrylate’ and ‘propionic acid’ to trigger negative emotions (mainly disgust and anger). Interestingly, Herz and Engen [15] pointed out that almost all responses to smell are based on associative learning principles. They argued that only smells learned to be positive or negative can elicit the corresponding hedonic response and that people, therefore, should not have any hedonic preference for novel smells. The only exceptions are smells of irritating quality that strongly stimulate intranasal trigeminal structures. Such smells often indicate toxicity. While neuroscientists and psychologists have established a detailed understanding of the human sense of smell, insight into the subjective characteristics of smell and related experiences is lacking.

The exploration of this subjective layer of smell is often understood as going beyond the interest of these disciplines, but is highly relevant for HCI and user experience research.

SMELL IN HUMAN-COMPUTER INTERACTION

Ten years ago, Kaye [17] encouraged the HCI community to think about particular topics that need to be studied and understood about smell. While some attempts have been made to explore smell during recent years, the potential of smell in HCI remains underexplored.

Most work on smell in HCI focuses on developing and evaluating smell-enhanced technologies.

Brewster et al. [5] used smell to elicit memories, and developed a smell-based photo-tagging tool (Olfoto). Bodnar et al. [4] showed smell to be less disruptive as a notification mechanism than visual and auditory modalities. Emsenhuber et al. [9] discussed scent marketing, highlighting the technological challenges for HCI and pervasive technologies. Ranasinghe et al. [24] further investigated the use of smell for digital communication, enabling the sharing of smell over the  Internet. More examples of smell-enhanced technology can be found in multimedia applications [13], games or online games in the olympic kingsway casinos[16], online search interfaces [19], health and wellbeing tools (e.g., http://www.myode.org/), and ambient displays [22]. The exploration of smell-enhanced technologies is mostly limited to development efforts and the evaluation of users’ smell perception of single smell stimuli. The smells used are often arbitrary and not related to experiences. This is because of the lack of knowledge pertaining to the description and classification of smells required for HCI [17]. Kaye points out that “There are specific ones [classification and description schemes] for the perfume, wine and beer industries, for example, but these do not apply to the wide range of smells that we might want to use in a user interface” (p. 653). Thus, previous work has a general and quite simple usage of smell.

THE POTENTIAL OF STUDYING SMELL EXPERIENCES

In contrast to the work reported above, the present paper focuses instead on experiences with smell and links them to potential technologies. We do so through stories of experiences with smell. Stories are increasingly used within user experience research to explore personal memories of past experiences, but also to facilitate communication in a design process [3,34]. Stories are concrete accounts of particular people and events, in specific situations [10],

and are more likely to stimulate empathy and inspire design thinking than, for example, scenarios.

STUDY METHOD

We asked a large sample of participants to report smell experiences that were personal and meaningful.

We refer to the description of these experiences as smell stories. These stories were captured through a questionnaire described below, which included inspirational examples of smellenhanced technologies at its end. Based on the examples we asked participants to reflect on their experience and future technologies. The rationale of this approach was to begin from smell experiences that matter to participants, instead of starting from an application or a particular technology.

Questionnaire

We created a web-based questionnaire consisting of six parts. We started with an open question to stimulate the report of a personal memorable smell experience. This was followed by closed questions aiming to elucidate the relevant emotional and experiential characteristics, as well as the smell qualities. Participants could freely choose the story to report. The questionnaire was administered through a crowdsourcing platform to obtain a large sample of smell stories. Crowdsourcing provides valid and reliable data [20] and has been used for capturing user experiences [31].

Part 1: Smell Story

The smell stories were elicited through an initial exercise, where participants were asked to think about situations and experiences where smell played an important role. The aim was to get participants into the right frame of mind and sensitize them to smell. Next, participants were asked to describe one memorable smell experience in as much detail as possible, inspired by the questioning approach used in explicitation interviews [23]. This questioning technique is used to reconstruct a particular moment and aims to place a person back in a situation to relive and recount it. Part 1 of the survey was introduced as follows: Bring to your mind one particular memorable moment of a personal smell  experience. The experience can be negative or positive. Please try to describe this particular smell experience in as much detail as possible. You can use as many sentences as you like, so we can easily understand why this moment is a memorable experience involving smell for you. Participants were asked to give a title to their story (reflecting its meaning) and indicate if the experience was positive, negative, or ambivalent (i.e., equally positive and negative). They were also asked to indicate how personally relevant the experience was (from ‘not personally relevant at all’ to ‘very personally relevant’).

Part 2: Context

Part 2 asked participants to give further details of their reported experience via open and closed follow-up questions. There were four questions on the context of the described experience, including the social context (who else was present), the place (based on the categories used by [26]), the location (as an open field), and the time when the reported experience took place (days, weeks, months, or years ago).

Part 3: The smell

Specific questions on the characteristics and qualities of the smell were asked in Part 3. Participants characterized the smell itself using a list of 72 adjectives (i.e., affective and  qualitative terms) derived from the ‘Geneva Emotion and Odor Scale’ (GEOS) [7]. Participants could also add descriptions to characterize the smell in an open feedback box. In addition, they rated the smell with respect to its perceived pleasantness, intensity, and familiarity.

Part 4: Experienced emotions

In Part 4 participants had to describe how they felt about the experience as a whole, using a list of affective terms (101 in total). They could go through the list and tick the words that best described their emotions during the experience. The words were derived from Scherer [27]. Participants could also add their own words in a free-text field.

Part 5: Smell technologies

After the participants had selected, thought about, and described a particular smell episode, Part 5 linked their personal experience to technology. The participants were engaged in a envisioning exercise inspired by work on mental time travel [30]. They were shown six inspirational examples of smell technologies, namely: Olfoto: searching and tagging pictures (CHI, [5]); Smelling screen: ambient displays (IEEE, [22]); Digital smell: Sharing smell over the Web (ICST, [24]); Scent dress: interactive fabric with smell stimulation (http://www.smartsecondskin.com/); Mobile smell App: iPhone To Detect Bad Breath and Other Smells (BusinessInsider 01/2013), and Smell-enhanced cinema: Iron Man 3 Smell-Enhanced Screening (Wired 04/2013). These six technologies cover areas of relevance for HCI (mobile, ambient, wearable, personal, and entertainment computing), give realistic examples of smell technologies from research, and include recent, commercial examples. We asked the participants to imagine any desirable change that future smell technology might make (or not) with respect to their personal smell experience.

We asked them the following questions: (1) How could your experience be enhanced? (2) What technology are you thinking about? (3) Why would such a combination of your experience and the technology be desirable, or why would it not?

Finally, the participants could express any other ideas for smell technology in a free-text field.

Part 6: Personal background

At the end of the questionnaire, participants answered questions on their socio-demographic and cultural background. The goal was to try to identify any geographical and cultural influences on smell attitudes (as found by Seo et al. [29]). The participants were also asked to assess their own smell sensitivity. All the questions, except for those on demographics, were mandatory. On average, the survey took 16 minutes to complete (SD = 7.57 minutes). Participants received US$ 1.50 for completing the questionnaire, corresponding to an hourly salary of 5.63 dollars.

Collected data and participants

A total of 554 participants began the questionnaire. Of these, 480 completed the questionnaire and answered three verification questions at its end. These questions required participants to describe the purpose of the study without being able to go back and look at the earlier questions or guidelines. After data cleaning, 41 stories were excluded. Fake entries (n = 11) were identified immediately, while repeated entries (n = 10), incomplete stories (unfinished sentences; n = 6), and incomprehensible stories (which did not make sense on their own; n = 14), were excluded iteratively throughout the coding process. This left us with 439 smell stories.

At the time of the study, all 439 participants (52.8% female) lived in the US; most had grown up in the US (95%). The participants’ age ranged from 18 to 67 years (M = 31.5, SD = 10.0). A majority of participants (84%) indicated being sensitive to smell (rating 4 or higher on a scale from 1 to 5).

Data analysis

The analysis process followed an open and exploratory coding approach [25]. Two researchers conducted the qualitative coding process. After coding an initial 25% of the stories, two more coding rounds (to reach 33% and then 50% of the data), led to the establishment of an agreed coding scheme. The coding scheme contained 10 main categories and 36 sub-categories, and a category entitled ‘not meaningful’ for cases where smell did not seem to have any relevance in the described experience. Based on this coding scheme, one researcher coded the remaining 50% of the data, and the second researcher coded a subsample of 25% of that data, resulting in a good inter-coder agreement (Cohen’s kappa of κ = .68) [12].

Follow up design brainstorming

In addition to the feedback from our participants,

we also explored the design value of the smell stories with experts in the field. We organized a two-hour design brainstorming

session with three HCI researchers, two working on smell technologies and one working on advanced interface and hardware design. None of them were from the same organization as the authors and none were familiar with the details of the study before the session. The brainstorming session aimed to share and interpret the smell stories and followed four stages [11]: (1) prompting, (2) sharing, (3) selecting, and (4) committing. We selected 36 stories (one representative story for each sub category) as brainstorming prompts. All 36 stories were printed on A6 sheets (including the story title, the smell story, context information, and personal background). Each researcher was asked to read through the stories individually before discussing them together. They were asked the same questions as our participants (e.g., how they might imagine a connection between the experience and technology). Each researcher chose the most interesting/inspiring stories to share with the group, then they generated ideas as a group, and selected three to four ideas to be developed in more detail. The outcome of the brainstorming session is presented in the implication section, after the description of the findings from the smell stories.

FINDINGS ON EXPERIENCES WITH SMELL

In the following sections we present our findings according to the 10 identified categories. The 439 smell stories were organized via their primary category, as agreed by the coders. This categorization does not define a strict line between the categories, as they are not wholly independent, but it does enable us to organize the material and generate a useful dataset for design.

Below we provide for each category a rich description of the particularities of the stories, excerpts from example stories, and their associated smell qualities and emotions. Each category also contains information about the participants’ own rating of the stories as positive, negative, or ambivalent.

Category 1: Associating the past with a smell

This category is the largest and contains 157 stories. In these stories, the participants described a past experience in which a smell was encountered during a special event in life (e.g., ‘Wedding Day’, ‘New House’), at a special location (e.g., ‘The Smells of Paris’, ‘Grandma’s House’), or as part of a tradition (e.g., ‘The Smell of Thanksgiving’ or ‘Christmas Eve’). In these stories the smell was described as having a strong association to those particular moments in the past, with no actual smell stimulus in the present. A particularity of this category is the distinction between stories describing personal memorable events versus personal life events (e.g., ‘Disneyland’ versus ‘When my mother died’). Smells were also associated with personal achievement/success (e.g., ‘Scent of Published Book’, ‘New Car Smell’) and other important episodes of change, such as “‘Fresh Start’: I was taking a job in a new city. …. I took a plane trip across the country and the moment I took a step off the plane and took a deep breath will always stick with me. It felt so clean and the air actually smelled fresh and new” [#488]. Within this story, the qualities of the smell were for instance described as fresh, energetic, and invigorating. Some of the emotions experienced at this moment were courageousness and excitement. Although this category is dominated by positive experiences (n = 127), negative experiences were also reported (n = 27), such as ‘Car Crash’.

Category 2: Remembering through a smell

The 40 stories in this category described a recent experience of a smell, which reminded participants about past events, people, locations, or specific times in their life. In contrast to the previous category (where stories describe a direct link from the recollected past smell to the present; e.g., the smell of ‘Grandma’s House’), this category contains stories that describe an indirect link from the present experienced smell stimulus

to the past event, person or place (e.g., the smell of chocolate cookies as sudden reminder about grandma). Most stories in this category contain reminders of childhood described as ‘sweet’, ‘reassuring’ and ‘nostalgic’ with respect to the qualities of the smell. A sub-set of stories in this category (n = 10) also highlight the particular power of smells to take a person back in time. The description of such a flashback caused by a sudden smell stimulus was described as: “‘My first love’: It was the next day, when I was walking through the local Macy’s that I smelled something that threw me back into that situation, I could feel and see everything that had happened the day before when I smelled a perfume in the store” [#630]. Some of the qualities used to describe the smell were attractive, erotic, and fresh. The experienced emotions were described as amorous, aroused, excited, hopeful, and interested. The stories in this category were mainly positive (n = 37), except for three.

Category 3: Smell perceived as stimulating

The 62 stories in this category described experiences with a unique, mostly unknown smell (all stories, except one, were positive). The smells arose from different sources, such as perfume, food, and nature. A particularity of this category is the quality of ‘first time’ encounters with a smell across all origins. One participant described the first time he was at a beach: “The smell was very different from anything I had ever experienced before. At first I was kind of grossed out by the smell, but I grew to love it” [#921]. Another participant described the smell of a tornado experienced for the first time: “It was similar to the smell before rain but had a certain sharpness to it, as if to warn of the incoming danger. I felt like I knew this smell but at the same time, it felt foreign to me. It wasn’t a bad smell, it was just slightly unfamiliar” [#713]. The smell qualities and experienced emotions were often described with mixed attributes (e.g., heavy, imitating, and stimulating; attentive, serious, and calm), but still rated as positive experiences by participants. Most of the other stories in this category reported on the first experiences with food (e.g., ‘Slice of Heaven’) and nature (e.g., ‘Grass’), and were described as desirable, fresh, or pure, and provoked feelings of happiness at the moment they occurred. Although specific memories were established, including unique new associations (e.g., ‘Tornado smell’), the stories in this category did not evoke the kind of strong connections to the past as described in Category 1 and 2.

Category 4: Smell creating desire for more stories (45 positive).

Key to these stories is that the smell grabbed the persons’ attention unexpectedly. The smell was either associated with food (triggering appetite), nature (triggering curiosity), or the scent of other people (triggering attractiveness), which  motivated one to do or get something. In some stories smell was described in relation to the sensation of newness (e.g., “‘The sweet smell of CPU’: …There was the smell of the cardboard boxes it all arrived in, the smell of new metal– perhaps it was a combination of these and other things, but when the building was complete there was just a singular smell that was unique to a new computer built by my own hands” [#685]). The qualities of the smell in this story included beneficial, heavy, sophisticated, energetic, and pleasantly surprising. The experienced excitement was expressed through words such as confident, delighted, enthusiastic, impressed, or triumphant. This category also contained one story where the smell at a funeral stimulated reflection in the moment (e.g., ‘The scent of moving on’). The story was rated as a positive experience and at the same time the smell was described as clean, penetrating, and persistent, and the participant indicated that she was afraid, anxious, discontented, sad, tired, and uncomfortable. Despite the negative situation described in this story, the smell gave hope and a desire to live and move on, looking into the future in contrast to the stories in Category 1 and 2.

Category 5: Smell allowing identification and detection

This category captures the enabling role of smell in certain situations, such as allowing one to identify or detect a smell (e.g., “‘Gas leak’: I was cooking something on a gas stove and went out for a few minutes. When I came back, the fire was extinguished but the gas was still on. My roommate was sat at the table doing schoolwork, completely oblivious to the poisonous gas that was filling the room. I told him to get the hell up and open the windows and doors” [#951]). The qualities used to describe the smell were distinguished, penetrating, dirty, and light. The emotions related to this situation were described as anxious, conscientious, confident, and serious. Although the category is rather small (n = 11), the lesson to be learnt from the shared stories was the immediacy of the smell, allowing the participant to act or prevent something.

Category 6: Overwhelming power of smell

This category includes 37 stories where the smell overwhelmed the person in a positive way (n = 5; e.g., ‘The Chocolate Factory’) but predominately in a negative way (n = 30; e.g., ‘The Smell of Death’). In the latter case, people described the smell as something disturbing, as something that hit them suddenly on their way or during an activity. A subset of the stories was recounted as traumatizing, so that the person could still vividly remember the particular moment in the past although years have passed and no recent similar smell stimulation had occurred unlike in Category 2 (e.g., “‘Visit to a local county jail’: My guide warned me ahead of the time that it was going to be a little foul in there, but nothing could have prepared me for the obscenely acrid stench of hundreds of men crammed into every available space of the jail, right down to windowless storage rooms converted into more cells. … For days afterwards, I couldn’t shake the smell…. There weren’t enough showers to take it away. It’s been several years since then, and my memory of that smell is just a strong as ever” [#604]). In this category, the qualities of the smell were described as heavy, penetrating, dirty, or sickening. Amongst others, the experienced emotions were described as alarmed, anxious, distressed, frustrated, or uncomfortable. In contrast to Category 1 and 2 (where the smell was associated with an event from the past or triggered a specific memory), Category 6 is about the smell as such during the experience and not about the memory associated with this smell. As opposed to the first two categories, in most stories forgetting – not remembering –         the smell was the key element.

Category 7: Smell invading private and public spaces

All the stories in this category (n = 32) described an experience where one could not get rid of the smell. The smell invaded private and public spaces. In contrast to the previous category, the smell entered the person’s personal space (the person did not enter the space where the smell already existed) and took over the space. The loss of control over the smell was linked to the lingering quality of the smell (e.g., “‘Don’t want to smell that twice!’: I woke up one morning suddenly confused and was hit with an odor so horrible I couldn’t figure out what it was. … It was not like the smell you get a whiff of when a skunk stinks up the outdoors” [#530]). In the story the power of the smell, causing them to leave the house for several hours, was described with qualities such as foul, nauseous, penetrating, and persistent. One of the experienced emotions was surprisingly ‘amused’, however it was overruled by other emotions including annoyed, anxious, disgusted, taken aback, and uncomfortable. Despite the glimpse of humor in some stories, this category mainly contains negative experiences and underlines the power of the smell with its sudden and lingering qualities.

Category 8: Social interaction is affected by the smell

Within this category, smell was related to a person’s own smell or to the smell of others. Smell negatively affected the interaction among people and their togetherness (e.g., “‘Dragon breath teacher’: Once a teacher yelled at me during class. She got so close up into my face that I could smell her bad breath. This made the experience much worse because I wanted to get up and walk away but she was grabbing me to keep me focused on her while she was talking” [#744]). The smell qualities were described as nauseous, penetrating, and sickening, and caused negative emotions experienced as bitter, distressed, or insulted. Despite frequent interactions among people, this category only contains 11 stories. This set of stories (overall negative experiences, apart from two) contains interesting elements with respect to a person’s own awareness of body smell and the overbearing effect of other peoples’ smell on one’s comfort.

Category 9: Smell changes mood, attitude and behavior

This category contains 23 stories, which underlined the power of smells to change a person’s mood, attitude, or behavior. Stories reported the active regulating effect of smells with respect to mood, but mostly (n = 14) the change of behavior due to smells (e.g., ‘Accidental vegetarian’ or ‘Saved by the Smell!’). One story showed the active usage of smells to change one’s mood. A participant had recently been divorced and reported on the day her husband had moved out: “‘White Lilac Sheets’: “I made the bed with my lilac sheets and the atmosphere changed. I still remember that scent and how I felt on that day. I was going to be okay. The room didn’t look or feel or smell lonely anymore. It looked and smelled fresh and clean and lovely and a bit romantic and it was mine” [#526]. The qualities of the smell were described as fresh, reassuring, and spring-like, while the experienced emotions were determined, hopeful, longing, tense, but also worried. Overall, the stories in this category were reported as mainly positive (n = 12) experiences, but also as negative (n = 7) and four stories were rated ambivalent, neither positive nor negative.

Category 10: Smell builds up and changes expectations

This category shows the potential of smell to build up expectations and to surprise. In the former case (11 stories) the smell was building up expectations until the actual contact with the trigger, such as food or a perfume (e.g., “‘The Smell of Hungry Anticipation’: “I was trying a new soup for the first time. When it was brought to the table, the soft smell of rosemary immediately hit my nostrils. …It complimented the taste of the soup and built anticipation” [#585]). The smell was described as mouthwatering, healthy, and pleasantly surprising, and was further related to emotions such as conscientious, expectant, and relaxed. In other stories (n = 7), expectations were exceeded to the extent that they surprised and diverted anticipations (e.g., ‘PomVinegar Surprise’: “I could smell the pomegranate and vinegar from about 10 steps away, and it was a very pungent (thought not unpleasant) odor. I almost felt my nose becoming runny and took out a tissue. When I tasted the dish, however, the taste wasn’t nearly as sour as I expected it to be from the smell” [#542]). The smell in this story was described as distinguished and penetrating, and was associated with emotions such as attentive and excited.

Key quantitative facts behind the smell stories

While the above-described categories can be used as an inspiration and as a starting point for exploring design opportunities for smell in HCI,

our quantitative data provides additional background information. Below, the key quantitative information across all the collected smell stories is summarized. The majority of the 439 collected stories were positively valenced (n = 296), 112 were negative, and 31 were ambivalent. On average, negative stories tended to be slightly longer (M = 90 words) than positive stories (M = 79 words), but the difference is statistically not significant (U = 14600, p = .063, r = .09).

Contextual information

Most stories occurred in a context where one or more familiar persons were present (64.2%) or where participants were alone (21.6%). The presence of one or more strangers was reported less frequently (8.7%). With regard to location, most of the experiences happened at the participant’s or a friend’s home (38.1%) or in a public building (20.7%). Quite a few participants reported that their experience took place in the streets or another public space (14.4%), in a natural setting (7.3%), or at work (6.4%). The remaining participants (13.2%) indicated other places (e.g., stranger’s home). On average the reported experiences occurred 8.7 years ago (SD = 10.3), ranging from 1 day to 58 years ago.

The qualities of smell

The most frequent smell qualities reported in positive stories were pleasant (60%), fresh (42%), sweet (38%), clean (31%), and mouthwatering (30%). Smells in negative stories were described as unpleasant (62%), penetrating (55%), heavy (54%), foul (53%), and nauseous (51%). In ambivalent stories the smell was perceived as fresh (39%), pleasant (32%), mouthwatering (32%), attractive (26%), and penetrating (23%).

Experienced emotions

When asked to describe how they felt during their experience, participants’ used the affective terms happy (63%), pleased (53%), joyous (42%), delighted (41%), and excited (39%) in positive stories and uncomfortable (55%), disgusted (51%), distressed (43%), miserable (42%), and taken aback (29%) in negative stories. Ambivalent experiences were most frequently described as happy (42%), excited (39%), enthusiastic (35%), joyous (32%), and serene (29%). An overview of all 10 categories and 36 sub-categories including qualitative and quantitative information (including a full example for each sub-category, used in the design brainstorming session) is provided as supplementary material. All smell stories and related qualities of smell, experienced emotions, and context information, are also available at www.multisensory.info for further exploration.

IMPLICATIONS FOR TECHNOLOGY

This study focused first on experiences and second on the implications for technology. This section turns to technology. Below we summarize the feedback from the participants on how technology would fit with their experience, and describe input from a brainstorming session with HCI researchers working on smell technologies andadvanced interface and hardware design, based on a sub-set of the smell stories (one from each sub-category).

Ideas for technology from participants

Below we summarize the six areas and ideas for desirable future smell technologies mentioned by participants in Part 5 of the questionnaire:

(1) To share smells with family/friends: allow one to participate in a family event through remote smelling; share smells of special moments such as the smell of a newborn baby with distant relatives; share smells with people who they know would appreciate it (such as through social media); allow capturing and sharing of smells to create a common understanding where you can’t explain it. Participants also desired to be able to design and share new smells from a personal database and create a personal smell box/bottle.

(2) To support decision making: use smells for a quick judgment in online shopping (like/dislike can be determined easily); create smell profiles about holiday places and travel destinations; smell match maker in dating websites for allowing a better decision making about going on a date or not with a person (smell enhanced profiles).

(3) To regulate mood actively/passively: smell to relive good experiences whenever you want to get in a better mood; to calm yourself down in stressful moments such as in traffic jam or at work; as a reminder of past memories you would have forgotten otherwise but that can cheer you up when you feel depressed and life seems too difficult.

(4) To combine with other technologies and activities: integrate smell into radio; combine smell with music such as with ‘soundhound’ or ‘shazam’ apps; smell-enhanced advertisement on TV (for food channels); enhance visits of concerts, theater and performances with smell; allow underwater smelling when diving.

(5) To combine with everyday objects: enhance wristband with smell for keeping a preferred perfume lingering; have special glasses to see and smell the beach; smell-enhanced jewelry and clothes. One participant imagined her wedding ring enhanced with the smell of that day.

(6) To make oneself and others aware about body smell: to avoid embarrassing moments; provide invisible cues to a person about her/his smell level; quick smell check after sporting activities.

The first idea matches the experiences in Categories 1 and 2, where particular events/moments in life are associated with a smell. The desire for capturing and sharing these experiences enhanced with smell becomes prevalent and suggests design implications for real-time smell-enhanced technologies (e.g., mobile phone, photo or video camera). The second idea can be linked to Category 5, allowing people to identify and detect a smell. Moreover, smells are seen as very powerful for supporting quick decision-making (e.g., smell-enhanced website navigation and searching).

The third idea shows a direct link to Category 9 and the potential of smell to change mood. Interestingly participants whose story was in Category 1 or 2 were wishing for the possibility to capture pleasant smells, for instance from their childhood, and released to them in the present. This desire for smell-enhanced technologies or products is also apparent in the fourth and fifth ideas, where technology, objects, or even activities can be enhanced through smells from the past, or actual smells sourced through nature (e.g., diving in the ocean). Finally, the sixth idea is linked to Category 8, aiming to avoid embarrassing moments in social interactions.

Participants also expressed concerns about future smell technologies. They were concerned about the possible misuse of smell when sharing it through the Internet or mobile phones (e.g., teasing people with smells, how to trust a smell message), and about the potential manipulation through smell (e.g., TV ads, online shopping). Some participants were also afraid to get sick, catch an allergy, or become addicted if they are exposed to chemical stimulations from technology. Finally, one participant raised the question of copyright and ownership of smells (e.g., ‘can I share others’ smells?’).

Ideas for technology from HCI researchers

Below we outline the ideas that emerged from the two-hour brainstorming session prompted by 36 smell stories (one from each sub-category). Four groups of design ideas emerged from this session and are described below:

(1) Smell-enhanced performance regulator: a technology stimulating smell in order to structure the day, taking activities and moods into account, and combining different smells to avoid habituation (training and evaluation phase needed). Smell as a reminder to take a break or as motivation to keep going a bit longer to meet the deadline [inspired by #526 ‘White Lilac Sheets’, Category 9].

(2) Autonomous smell agent: a technology spreading ambient cues (e.g., a robot) to guide someone to a certain place, to build up expectations, and motivate action. Smell trails in the environment can also make hidden informatio accessible, for instance, before entering a room (e.g., smell warning: tense working atmosphere) [inspired by #801 ‘Don’t forget to check your gas stove before you leave the house’, Category 5].

(3) Reminder alert with smell: a technology to remind us about important events, birthdays, and appointments.

Although we have reminders on mobile phones and computers, they are often ignored, snoozed or in the worst case forgotten about. A smell can provide a pleasant reminder to say ‘it is time to call your mom’ by presenting the smell of your favorite dish your mom makes for you. On the other hand, if more critical, bad smells can be very powerful as a reminder and are not easily ignored [inspired by #530 ‘Don’t want to smell that twice’, Category 7].

(4) Smell-enhanced storytelling: a technology that stimulates storytelling around a digitized version of an incense stick. A stick was imagined with different layers, representing smells related to a loved person who passed away. When friends or family members come together, for instance at an anniversary year, they can add new smells to be shared in the group and thus trigger new stories about the dead person. It is as if looking through a photo album, telling the stories from the past, and using the smells as anchor points for keeping the memory alive [inspired by #672 ‘The Scent of Moving On’, Category 4]. We saw that the smell stories, even if they only provided limited information (story, story title, context, gender, and age), triggered vivid discussions, created empathy, and stimulated the sharing of personal smell experiences. The four ideas described above provide a starting point for exploring smell in HCI. The categorization along with additional background information on smell qualities and experienced emotions (see supplementary material) can inspire further explorations of smell technologies.

DISCUSSION

Our findings about experiences with smell in combination with the ideas for technologies just presented show several design opportunities for smell. Below we do not provide solutions for smell-enhanced technology designs, rather we illustrate where our findings might be relevant to stimulate novel designs in existing areas of interest within HCI. We see three anchor points for smell-enhanced technology.

First, the smell stories in Categories 1 and 2 suggest design opportunities for remembering and recalling the past. Our findings might enrich ongoing research on the design for personal memories. Apart from enhancing research supporting the capturing and sharing of personal experiences (e.g., in family relationships [18]) through smell, our findings support research to support people who are living with memory loss (e.g., patients with dementia [32]), where smell can play an important role in remembering the past. An increasing body of research also explores the potential of digital technologies to support our memory in everyday tasks (e.g., reminder systems), to recall past events and experiences (e.g., life-logging tools), to design end-of-life technologies allowing reminiscence of passed away people [21], and to record and reproduce smells [35]. All this research shows the potential of smell to enrich experiences, for instance by enhancing personal memories such as photos or videos with smell. Based on their study of a smell-based photo-tagging system, Brewster et al. [5] stated that participants asked for personal smells to be added. The information on how to classify such smells was still missing; the present analysis allows us to relate smell qualities to particular types of experiences. When designing with smell, as for any memory-based technology, access to such memories has to be considered carefully to preserve their uniqueness. One participant wrote: “I could see it being desirable in that it would allow me to experience the scent whenever I want, but it’s kind of a two edged sword in that experiencing that scent time and time again will make it common place” [#513]. The power of smell might not persist if always available, thus participants suggested to either restrict the access and retrieval of smells to special times (e.g., at ‘grandmas birthday’) or to link them to a certain social setting (e.g., smelling only in company with ‘your sister’). This way the uniqueness of the smell can be preserved.

Second, the stories in Categories 3 to 8 as well as 10 draw the attention away from past memories and suggest design opportunities for the present moment. Designing for in-situ stimulation, the ability to capture and share smells in the moment, and the capability to mask and neutralize bad smells creates a vast space for smell interaction design. One suggestion made by participants was the combination of smell and social media, such as “An app that would allow me to store smells, send smells, or attach smells to a picture that I could post on social media or Instagram or something”. This supports existing research on the delivery of smells through the Internet [24]. We draw attention to three additional design directions concerned with (1) first time experiences with smell, (2) the power of smell to build up expectations, and (3) the potential of designing for bad smells. User experience designers put a lot of effort into designing ‘out-of-the-box’ and first time experiences to create positive experiences [13]. Our categorization not only provides designers with rich descriptions of such first time experiences, but also describes the related qualities of smells in combination with descriptors of the experienced emotions. This can be used to stimulate positive smellenhanced experiences with technology, build up expectations, and create anticipation as studied within experience research [33]. Typically this anticipation stage is influenced by a variety of aspects (e.g., advertisements, product descriptions, accounts from existing customers). Smell stories in Category 10 provide evidence for the power of smell to build up expectations, create surprise by exceeding anticipated experiences, and enhance momentary experiences through capturing and sharing pleasant smells. Categories 6 to 8 contain stories about bad smells, which are wished to be neutralized or masked to change the experience from something negative to something positive. While the idea of outbalancing smells seems to be desirable, the design brainstorming session stimulated a discussion on the usage of bad smell in design, particularly as part of the design idea (3) Reminder alert with smell. Designing for bad smells might not seem appropriate at first, but through intensity manipulation it can open up an interesting space for design. Similar to a snooze function, which slowly increases volume, smell stimulations could be added to certain events (e.g., reminder for mother’s birthday). Starting with a pleasant smell, it could turn slowly into something unpleasant if you did not act. Category 8 also contained stories recounting social experiences with smell, where the smell of a person or of other people caused embarrassment or discomfort. Despite the importance and frequency of social contact in everyday life, few such stories were shared. They might not seem meaningful enough to be memorable or to be shared. Yet, this set of stories holds potential for personal and social smell-enhanced awareness systems, as well as for wearable technologies, and smart fabrics. Technology could, as stated by a participant, “…make the people in those settings feel more comfortable if I interact with them… My holding my nose could be insulting and impede communication.”

Third, the smell stories in Category 9 suggest design opportunities reaching out to the future through positive stimulation, with potential relevance for wellbeing and behavior change research in HCI. The stories shared in this category were about the power of smell to regulate mood, change attitudes, and behavior. Designing for smell could be combined with behavior change research in HCI (e.g., tools to support healthy nutrition and diet), and thought of in relation to positive psychology and research on wellbeing. Seligman and Csikszentmihalyi [28] suggested that happiness can be learned and cultivated and that positive psychology can help change how a person feels. They point to the power of positive emotions for our health, happiness and wholeness. We would suggest that our findings add an understanding of the positive emotional impact of smells that might be a valuable research strategy in wellbeing research (e.g., for regulating mood).

Smell can have a regulating impact on a person’s mood and can, as in one case explicitly reported, be used to regulate mood

(‘White Lilac Sheets’). The participant wrote, “I guess the experience could have been enhanced by some kind of mood moderator. Something that would have sensed my sadness and filled the room or house with comforting scents” [#526]. The participant pointed out that technology would not change the situation to something more positive, as it just was not a happy time at all, but that it could support the sad moments in this transitional period of life.

Limitations

We would like to acknowledge three limitations of this work. First, by using Amazon Mechanical Turk for recruiting and asking participants to describe personal relevant experiences, we were limited to the US and do not know to what extent the smell stories are representative of more general experiences with smell. We are aware about cultural and geographical differences (as described by Seo et al. [29]), which require further studies with a more diverse group of participants. Second, collecting narratives by means of an online questionnaire has an influence on how people narrate their experience and deprives us of the advantages of an interview situation where we can engage in a dialogue with the participant to explore the meaning behind the shared experience in more depth as described by Bruner [6]. We tried to collect information beyond the initial trigger of the shared smell stories in order to allow the establishment of meaningful categorizations and the creation of a basic understanding of experiences with smell in HCI.

Third, our approach provides an overview on the emerging field of smell-enhanced technologies. Future studies will, we hope, lead to in-depth research into experiences with smell inspired by our identified categories.

CONCLUSIONS

Despite interactive technologies increasingly disappearing into our environment (in ubiquitous and mobile computing) and becoming essential in everyday life, the senses used to interact with technology are still limited.

We have discussed the opportunities for smell in HCI based on an analysis of 439 smell stories. We identified 10 primary categories for stories about experiences with smell, which help discuss the potential implications for technology. Implications were drawn from feedback from our participants envisaging desired connections between their own personal experience and future smell technology. The implications for designing for smell were further enriched through ideas from an initial brainstorming session with HCI researchers. Our findings provide guidance for smell enhanced technology design, not only giving a categorization of the role of smell in personal experiences, but also extracting the qualities of smell across the smell stories and the experienced emotions. We argue that this research enriches existing technology driven research on smell in HCI and provides a fruitful starting point when designing for experiences with smell.

ACKNOWLEDGMENTS

This work is supported by the Marie Curie IEF Action of the European Union (FP7-PEOPLE-2010-IEF) and the Swiss National Science Foundation (PBBSP1 144196). We thank our participants and especially Annika Haas for her valuable support in designing the supplementary material.

REFERENCES

1. Alaoui-Ismaïli, O., Robin, O., Rada, H., Dittmar, A., Vernet-Maury, E. Basic Emotions Evoked by Odorants: Comparison Between Autonomic Responses and Self- Evaluation. Physiology & Beh. 62, 4 (1997), 713–720.

2. Bakalar, N. Sensory science: Partners in flavour. Nature 486, 7403 (2012), S4–S5.

3. Baumeister, R.F., Newman, L.S. How Stories Make Sense of Personal Experiences: Motives that Shape Autobiographical Narratives. Personality and Social Psychology Bulletin 20, 6 (1994), 676–690.

4. Bodnar, A., Corbett, R., Nekrasovski, D. AROMA: ambient awareness through olfaction in a messaging application. Proc. ICMI, (2004), 183–190.

5. Brewster, S., McGookin, D., Miller, C. Olfoto: designing a smell-based interaction. Proc. CHI (2006), 653–662.

6. Bruner, J. Life as narrative. Social Research: An International Quarterly 71, 3 (2004), 691–710.

7. Chrea, C., Grandjean, D., Delplanque, S., Cayeux, I., Le Calvé, B., Aymard, L., Velazco, MI., Sander, D., Scherer, KR. Mapping the Semantic Space for the Subjective Experience of Emotional Responses to Odors. Chem. Senses 34, 1 (2009), 49–62.

8. Ehrlichman, H., Bastone, L. The use of odour in the study of emotion. In S. Van and G.H. Dodd, (Eds.), Fragrance: The psychology and biology of perfume. Elsevier, (1992), 143–159.

9. Emsenhuber, B. Scent Marketing: Making Olfactory Advertising Pervasive. In J. Müller, F. Alt and D.  Michelis (Eds.), Pervasive Advertising. Springer, (2011), 343–360.

10. Erickson, T. Design as storytelling. Interactions 3, 4 (1996), 30–35.

11. Faste, H., Rachmel, N., Essary, R., Sheehan, E. Brainstorm, Chainstorm, Cheatstorm, Tweetstorm: new ideation strategies for distributed HCI design. Proc. CHI (2013), 1343–1352.

12. Fleiss, J.L., Levin, B., Paik, M.C. Statistical Methods for Rates and Proportions. John Wiley & Sons, (2013).

13. Ghinea, G., Ademoye, O. The sweet smell of success: Enhancing multimedia applications with olfaction. TOMCCAP 8, 1 (2012), 2.

14. Hassenzahl, M. Experience Design: Technology for All the Right Reasons. Syn. Lect. on HCI 3, 1 (2010), 1–95.

15. Herz, R.S., Engen, T. Odor memory: review and analysis. Psy. Bulletin & Review. 3, 3 (1996), 300–313.

16. Jonsson, F., Verhagen, H. Senses working overtime: on sensuous experiences and public computer game play. Proc. ACE (2011), 56:1–56:8.

17. Kaye, J. “Jofish.” Making Scents: aromatic output for HCI. Interactions 11, 1 (2004), 48–61.

18. Kazakos, K., Howard, S., Vetere, F. Revisiting the relationship between reunion and technology-mediated separation in periodically transitioning families. Proc. CSCW (2013), 1157–1168.

19. Loumakis, F., Stumpf, S., Grayson, D. This image smells good: effects of image information scent in

search engine results pages. CIKM (2011), 475–484.

20. Mason, W., Suri, S. Conducting behavioral research on Amazon’s Mechanical Turk. Behavior Research Methods 44, 1 (2012), 1–23.

21. Massimi, M., Moncur, W., Odom, W., Banks, R., Kirk, D. Memento mori: technology design for the end of life. Proc. CHI EA (2012), 2759–2762.

22. Matsukura, H., Yoneda, T., Ishida, H. Smelling Screen: Development and Evaluation of an Olfactory Display System for Presenting a Virtual Odor Source. IEEE TVCG 19, 4 (2013), 606–615.

23. Petitmengin, C. Describing one’s subjective experience in the second person: An interview method for the science of consciousness. Phenomenology and the Cognitive Sciences 5, 3-4 (2006), 229–269.

24. Ranasinghe, N., Karunanayaka, K., Cheok, A.D., Fernando, O.N.N., Nii, H., Gopalakrishnakone, P. Digital taste and smell communication. Proc. BodyNets, (2011), 78–84.

25. Saldana, J. The Coding Manual for Qualitative Researchers. SAGE (2012).

26. Scherer, K.R. Appraisal considered as a process of multilevel sequential checking. Appraisal processes in emotion: Theory, methods, research 92, (2001), 120.

27. Scherer, K.R. What are emotions? And how can they be measured? Social Science Inf. 44, 4 (2005), 695–729.

28. Seligman, M.E.P., Csikszentmihalyi, M. Positive psychology: An intro. Am. Psy. 55, 1(2000), 5–14.

29. Seo, H.-S., Guarneros, M., Hudson, R., et al. Attitudes toward Olfaction: A Cross-regional Study. Chem. Senses 36, 2 (2011), 177–187.

30. Suddendorf, T., Corballis, M.C. The evolution of foresight: What is mental time travel, and is it unique to humans? Beh. & Brain Sciences 30, 3 (2007), 299–313.

31. Tuch, A.N., Trusell, R., Hornbæk, K. Analyzing users’ narratives to understand experience with interactive products. Proc. CHI (2013), 2079–2088.

32. Wallace, J., Wright, P.C., McCarthy, J., Green, D.P., Thomas, J., Olivier, P. A design-led inquiry into personhood in dementia. Proc. CHI (2013), 2617–2626.

33. Willander, J., Larsson, M. Smell your way back to childhood: Autobiographical odor memory. Psy. Bulletin & Review 13, 2 (2006), 240–244.

34. Wright, P., McCarthy, J. Experience-Centered Design: Designers, Users, and Communities in Dialogue. Syn. Lect. on HCI 3, 1 (2010), 1–123.

35. Wyszynski, B., Yamanaka, T., Nakamoto, T. Recording and reproducing citrus flavors using odor recorder. Sensors&Actuators B: Chemical 106, 1(2005), 388–393.