ICMI 2018 Multisensory Approaches to Human-Food Interaction

MHFI'18- Proceedings of the 3rd International Workshop on Multisensory Approaches to Human-Food Interaction

MHFI'18- Proceedings of the 3rd International Workshop on Multisensory Approaches to Human-Food Interaction

Full Citation in the ACM Digital Library

TasteBud: Bring Taste Back into the Game

  • Chi Thanh Vi
  • Daniel Arthur
  • Marianna Obrist

When we are babies we put anything and everything in our mouths, from Lego to crayons. As we grow older we increasingly rely on our other senses to explore our surroundings and objects in the world. When interacting with technology, we mainly rely on our senses of vision, touch, and hearing, and the sense of taste becomes reduced to the context of eating and food experiences. In this paper, we build on initial efforts to enhance gaming experiences through gustatory stimuli. We introduce TasteBud, a gustatory gaming interface that we integrated with the classic Minesweeper game. We first describe the details on the hardware and software design for the taste stimulation and then present initial findings from a user study. We discuss how taste has the potential to transform gaming experiences through systematically exploiting the experiences individual gustatory stimuli (e.g., sweet, bitter, sour) can elicit.

Towards Multisensory Storytelling with Taste and Flavor

  • Carlos Velasco
  • Yunwen Tu
  • Marianna Obrist

Film makers, producers, and theaters have continuously looked at ways to embody and/or integrate multiple sensory cues in the experiences they deliver. Here, we present a reflection on past attempts, lessons learnt, and future directions for the community around multisensory TV, film, and multimedia as a historical, though renewed, space of content creation. In particular, we present an overview of what we call "tasty film", that is, film involving taste, flavor, and more broadly food and drink inputs, to influence the audience experience. We suggest that such elements should be considered beyond "add-ons" in film experiences. We advocate for experimentation with new kinds of storytelling taking inspiration from multisensory design research and work on sensory substitution. We position this article as a starting point for anyone interested in multisensory film involving taste, flavor, and foods.

Drink-O-Mender: An Adaptive Robotic Drink Adviser

  • Hannes Ritschel
  • Andreas Seiderer
  • Kathrin Janowski
  • Ilhan Aslan
  • Elisabeth André

Social robots become increasingly important in the domain of healthcare and maintenance. Nutrition is not an exception: research robots are used in several experiments to teach people about healthy nutrition. Moreover, robotic products, whose task is to keep track of the user's nutrition, to provide tips, reminders, and to recognize anomalies in health-related behaviors are on the way to market. To convince users of a robot's recommendations, speech is an important interaction modality. However, automatically adapting a robot's spoken advise depending on users' behaviors is still a challenge. We address this issue by building Drink-O-Mender, an interactive installation, which includes a Reeti robot augmented with additional sensing and adaptation abilities. The robot is designed to offer drinks in a social setting. It aims to convince users of consuming healthy drinks while adapting its spoken advices depending on the users' selected beverage choices. The installation is equipped with custom hardware including a smartscale to sense the type and quantity of consumed drinks. We describe the interactive installation in detail and demonstrate feasibility of generating adaptive spoken advices by reporting on insights gained from exhibiting the installation during a public event and observing interactions of 78 users with the robot.

"Eat What You Want and Be Healthy!": Comfort Food Effects: Human-Food Interaction in View of Celebratory Technology

  • Youjin Hwang
  • Siyoung Lee
  • Hyeong Seok Jeon
  • Jung Han Yoon Park
  • Ki Won Lee
  • Joonhwan Lee

Food craving is one of the fundamental desires of human nature. Many HCI researchers used to define this desire as a problem and has developed its corrective technology. However, positive aspects of food craving and food reward has rarely been in the scope of research. Few studies made its efforts to evaluate how food positively interacts with mental aspects of humans but have shown inconsistent results because of user subjectivity and environmental variations. Therefore, we have evaluated the human-food interactions with an EEG as objective indicator to track the mental activity. We found that when participants have a high craving for the certain food, which makes them feel comfort (hereafter comfort food), their working memory performance and related theta signal increase and stress related high beta signal decrease. The methodology adopted in this study will contribute to the progress in food-related celebratory technologies in HCI research field.

The Virtual Cafeteria: An Immersive Environment for Interactive Food Portion-Size Education

  • Ufuk Celikcan
  • Ahmed Şamil Bülbül
  • Cem Aslan
  • Zehra Buyuktuncer
  • Kübra Işgın
  • Gözde Ede
  • Nuray Kanbur

This work introduces the Virtual Cafeteria, a novel virtual reality simulation of a buffet-style cafeteria for use in food portion-size education of adolescents as an alternative to established approaches such as verbal education and education with fake food replicas. As user behavior is automatically recorded to the millisecond, the application also serves as a tool for assessment of food portion-size perception. The Virtual Cafeteria offers a wide range of energy and nutrient densities in 64 food and 7 beverage options. Addressing the limitations and shortcomings of the immersive environments used in previous studies on food portion-size, the Virtual Cafeteria features not only a considerably wider variety of food but also multisensory feedback coupled with a high level of interaction and photorealism, and imperceptible latency.

Eliciting User Food Preferences in terms of Taste and Texture in Spoken Dialogue Systems

  • Jie Zeng
  • Yukiko I. Nakano
  • Takeshi Morita
  • Ichiro Kobayashi
  • Takahira Yamaguchi

Food preference varies from person to person and is not easy to verbalize. This study proposes a dialogue system that elicits the user's food preference through human-robot interaction. First, as the default knowledge of the dialogue system, we determined the ingredients of each dish from a large-scale recipe database, and collected the taste and texture of each dish and its ingredients by analyzing a large number of Twitter messages. Subsequently, the dialogue system asks questions to elicit the user's preferred taste/texture of the food by using the default knowledge base, while employing frame-based dialogue management. Finally, we created a food vector space that represents the relationship between the dish names, ingredients, and taste/texture expressions. We also discuss the possibility of using this vector space in dish recommendation.

Introducing Flavorlens: A Social Media Platform for Sharing Dish Observations

  • Alexandra Fritzen
  • Frederic Andres
  • Maria Leite

In this paper, we present Flavorlens, a mobile application for sharing dish observations. Each observation consists of one or more photographs, a title, a location tag, a description, a rating, a sensoring experience reporting about flavors, textures, and odors of a particular dish. Finally, we describe the design and implementation of our system which is based on a requirement analysis.

Tasty Art: 'The Scream' as a Burger...

  • Hector Fried
  • Helen Pain
  • Michaela Turner

This paper introduces the methodological exploration of a design led approach towards utilizing human-food-interactions in teaching art appreciation. In workshops participants express emotional and cognitive responses towards works of art, then interpret and re-imagine those responses into the creation of representational-food dishes. From this we deconstruct and analyze the multi-sensory and multi-modal experience of food in its complex network of taste, colour, texture, and emotions.


ICMI 2018 ACM International Conference on Multimodal Interaction. Copyright © 2018-2019