ICMI 2016- Proceedings of the 18th ACM International Conference on Multimodal Interaction

SESSION: Invited Talks

Understanding people by tracking their word use (keynote)

  • James W. Pennebaker

Learning to generate images and their descriptions (keynote)

  • Richard Zemel

Embodied media: expanding human capacity via virtual reality and telexistence (keynote)

  • Susumu Tachi

Help me if you can: towards multiadaptive interaction platforms (ICMI awardee talk)

  • Wolfgang Wahlster

SESSION: Oral Session 1: Multimodal Social Agents

Trust me: multimodal signals of trustworthiness

  • Gale Lucas
  • Giota Stratou
  • Shari Lieblich
  • Jonathan Gratch

Semi-situated learning of verbal and nonverbal content for repeated human-robot interaction

  • Iolanda Leite
  • André Pereira
  • Allison Funkhouser
  • Boyang Li
  • Jill Fain Lehman

Towards building an attentive artificial listener: on the perception of attentiveness in audio-visual feedback tokens

  • Catharine Oertel
  • José Lopes
  • Yu Yu
  • Kenneth A. Funes Mora
  • Joakim Gustafson
  • Alan W. Black
  • Jean-Marc Odobez

Sequence-based multimodal behavior modeling for social agents

  • Soumia Dermouche
  • Catherine Pelachaud

SESSION: Oral Session 2: Physiological and Tactile Modalities

Adaptive review for mobile MOOC learning via implicit physiological signal sensing

  • Phuong Pham
  • Jingtao Wang

Visuotactile integration for depth perception in augmented reality

  • Nina Rosa
  • Wolfgang Hürst
  • Peter Werkhoven
  • Remco Veltkamp

Exploring multimodal biosignal features for stress detection during indoor mobility

  • Kyriaki Kalimeri
  • Charalampos Saitis

An IDE for multimodal controls in smart buildings

  • Sebastian Peters
  • Jan Ole Johanssen
  • Bernd Bruegge

SESSION: Poster Session 1

Personalized unknown word detection in non-native language reading using eye gaze

  • Rui Hiraoka
  • Hiroki Tanaka
  • Sakriani Sakti
  • Graham Neubig
  • Satoshi Nakamura

Discovering facial expressions for states of amused, persuaded, informed, sentimental and inspired

  • Daniel McDuff

Do speech features for detecting cognitive load depend on specific languages?

  • Rui Chen
  • Tiantian Xie
  • Yingtao Xie
  • Tao Lin
  • Ningjiu Tang

Training on the job: behavioral analysis of job interviews in hospitality

  • Skanda Muralidhar
  • Laurent Son Nguyen
  • Denise Frauendorfer
  • Jean-Marc Odobez
  • Marianne Schmid Mast
  • Daniel Gatica-Perez

Emotion spotting: discovering regions of evidence in audio-visual emotion expressions

  • Yelin Kim
  • Emily Mower Provost

Semi-supervised model personalization for improved detection of learner's emotional engagement

  • Nese Alyuz
  • Eda Okur
  • Ece Oktay
  • Utku Genc
  • Sinem Aslan
  • Sinem Emine Mete
  • Bert Arnrich
  • Asli Arslan Esme

Driving maneuver prediction using car sensor and driver physiological signals

  • Nanxiang Li
  • Teruhisa Misu
  • Ashish Tawari
  • Alexandre Miranda
  • Chihiro Suga
  • Kikuo Fujimura

On leveraging crowdsourced data for automatic perceived stress detection

  • Jonathan Aigrain
  • Arnaud Dapogny
  • Kévin Bailly
  • Séverine Dubuisson
  • Marcin Detyniecki
  • Mohamed Chetouani

Investigating the impact of automated transcripts on non-native speakers' listening comprehension

  • Xun Cao
  • Naomi Yamashita
  • Toru Ishida

Speaker impact on audience comprehension for academic presentations

  • Keith Curtis
  • Gareth J. F. Jones
  • Nick Campbell

EmoReact: a multimodal approach and dataset for recognizing emotional responses in children

  • Behnaz Nojavanasghari
  • Tadas Baltrušaitis
  • Charles E. Hughes
  • Louis-Philippe Morency

Bimanual input for multiscale navigation with pressure and touch gestures

  • Sebastien Pelurson
  • Laurence Nigay

Intervention-free selection using EEG and eye tracking

  • Felix Putze
  • Johannes Popp
  • Jutta Hild
  • Jürgen Beyerer
  • Tanja Schultz

Automated scoring of interview videos using Doc2Vec multimodal feature extraction paradigm

  • Lei Chen
  • Gary Feng
  • Chee Wee Leong
  • Blair Lehman
  • Michelle Martin-Raugh
  • Harrison Kell
  • Chong Min Lee
  • Su-Youn Yoon

Estimating communication skills using dialogue acts and nonverbal features in multiple discussion datasets

  • Shogo Okada
  • Yoshihiko Ohtake
  • Yukiko I. Nakano
  • Yuki Hayashi
  • Hung-Hsuan Huang
  • Yutaka Takase
  • Katsumi Nitta

Multi-sensor modeling of teacher instructional segments in live classrooms

  • Patrick J. Donnelly
  • Nathaniel Blanchard
  • Borhan Samei
  • Andrew M. Olney
  • Xiaoyi Sun
  • Brooke Ward
  • Sean Kelly
  • Martin Nystrand
  • Sidney K. D'Mello

SESSION: Oral Session 3: Groups, Teams, and Meetings

Meeting extracts for discussion summarization based on multimodal nonverbal information

  • Fumio Nihei
  • Yukiko I. Nakano
  • Yutaka Takase

Getting to know you: a multimodal investigation of team behavior and resilience to stress

  • Catherine Neubauer
  • Joshua Woolley
  • Peter Khooshabeh
  • Stefan Scherer

Measuring the impact of multimodal behavioural feedback loops on social interactions

  • Ionut Damian
  • Tobias Baur
  • Elisabeth André

Analyzing mouth-opening transition pattern for predicting next speaker in multi-party meetings

  • Ryo Ishii
  • Shiro Kumano
  • Kazuhiro Otsuka

SESSION: Oral Session 4: Personality and Emotion

Automatic recognition of self-reported and perceived emotion: does joint modeling help?

  • Biqiao Zhang
  • Georg Essl
  • Emily Mower Provost

Personality classification and behaviour interpretation: an approach based on feature categories

  • Sheng Fang
  • Catherine Achard
  • Séverine Dubuisson

Multiscale kernel locally penalised discriminant analysis exemplified by emotion recognition in speech

  • Xinzhou Xu
  • Jun Deng
  • Maryna Gavryukova
  • Zixing Zhang
  • Li Zhao
  • Björn Schuller

Estimating self-assessed personality from body movements and proximity in crowded mingling scenarios

  • Laura Cabrera-Quiros
  • Ekin Gedik
  • Hayley Hung

SESSION: Poster Session 2

Deep learning driven hypergraph representation for image-based emotion recognition

  • Yuchi Huang
  • Hanqing Lu

Towards a listening agent: a system generating audiovisual laughs and smiles to show interest

  • Kevin El Haddad
  • Hüseyin Çakmak
  • Emer Gilmartin
  • Stéphane Dupont
  • Thierry Dutoit

Sound emblems for affective multimodal output of a robotic tutor: a perception study

  • Helen Hastie
  • Pasquale Dente
  • Dennis Küster
  • Arvid Kappas

Automatic detection of very early stage of dementia through multimodal interaction with computer avatars

  • Hiroki Tanaka
  • Hiroyoshi Adachi
  • Norimichi Ukita
  • Takashi Kudo
  • Satoshi Nakamura

MobileSSI: asynchronous fusion for social signal interpretation in the wild

  • Simon Flutura
  • Johannes Wagner
  • Florian Lingenfelser
  • Andreas Seiderer
  • Elisabeth André

Language proficiency assessment of English L2 speakers based on joint analysis of prosody and native language

  • Yue Zhang
  • Felix Weninger
  • Anton Batliner
  • Florian Hönig
  • Björn Schuller

Training deep networks for facial expression recognition with crowd-sourced label distribution

  • Emad Barsoum
  • Cha Zhang
  • Cristian Canton Ferrer
  • Zhengyou Zhang

Deep multimodal fusion for persuasiveness prediction

  • Behnaz Nojavanasghari
  • Deepak Gopinath
  • Jayanth Koushik
  • Tadas Baltrušaitis
  • Louis-Philippe Morency

Comparison of three implementations of HeadTurn: a multimodal interaction technique with gaze and head turns

  • Oleg Špakov
  • Poika Isokoski
  • Jari Kangas
  • Jussi Rantala
  • Deepak Akkil
  • Roope Raisamo

Effects of multimodal cues on children's perception of uncanniness in a social robot

  • Maike Paetzel
  • Christopher Peters
  • Ingela Nyström
  • Ginevra Castellano

Multimodal feedback for finger-based interaction in mobile augmented reality

  • Wolfgang Hürst
  • Kevin Vriens

Smooth eye movement interaction using EOG glasses

  • Murtaza Dhuliawala
  • Juyoung Lee
  • Junichi Shimizu
  • Andreas Bulling
  • Kai Kunze
  • Thad Starner
  • Woontack Woo

Active speaker detection with audio-visual co-training

  • Punarjay Chakravarty
  • Jeroen Zegers
  • Tinne Tuytelaars
  • Hugo Van hamme

Detecting emergent leader in a meeting environment using nonverbal visual features only

  • Cigdem Beyan
  • Nicolò Carissimi
  • Francesca Capozzi
  • Sebastiano Vascon
  • Matteo Bustreo
  • Antonio Pierro
  • Cristina Becchio
  • Vittorio Murino

Stressful first impressions in job interviews

  • Ailbhe N. Finnerty
  • Skanda Muralidhar
  • Laurent Son Nguyen
  • Fabio Pianesi
  • Daniel Gatica-Perez

SESSION: Oral Session 5: Gesture, Touch, and Haptics

Analyzing the articulation features of children's touchscreen gestures

  • Alex Shaw
  • Lisa Anthony

Reach out and touch me: effects of four distinct haptic technologies on affective touch in virtual reality

  • Imtiaj Ahmed
  • Ville Harjunen
  • Giulio Jacucci
  • Eve Hoggan
  • Niklas Ravaja
  • Michiel M. Spapé

Using touchscreen interaction data to predict cognitive workload

  • Philipp Mock
  • Peter Gerjets
  • Maike Tibus
  • Ulrich Trautwein
  • Korbinian Möller
  • Wolfgang Rosenstiel

Exploration of virtual environments on tablet: comparison between tactile and tangible interaction techniques

  • Adrien Arnaud
  • Jean-Baptiste Corrégé
  • Céline Clavel
  • Michèle Gouiffès
  • Mehdi Ammi

SESSION: Oral Session 6: Skill Training and Assessment

Understanding the impact of personal feedback on face-to-face interactions in the workplace

  • Afra Mashhadi
  • Akhil Mathur
  • Marc Van den Broeck
  • Geert Vanderhulst
  • Fahim Kawsar

Asynchronous video interviews vs. face-to-face interviews for communication skill measurement: a systematic study

  • Sowmya Rasipuram
  • Pooja Rao S. B.
  • Dinesh Babu Jayagopi

Context and cognitive state triggered interventions for mobile MOOC learning

  • Xiang Xiao
  • Jingtao Wang

Native vs. non-native language fluency implications on multimodal interaction for interpersonal skills training

  • Mathieu Chollet
  • Helmut Prendinger
  • Stefan Scherer

SESSION: Demo Session 1

Social signal processing for dummies

  • Ionut Damian
  • Michael Dietz
  • Frank Gaibler
  • Elisabeth André

Metering "black holes": networking stand-alone applications for distributed multimodal synchronization

  • Michael Cohen
  • Yousuke Nagayama
  • Bektur Ryskeldiev

Towards a multimodal adaptive lighting system for visually impaired children

  • Euan Freeman
  • Graham Wilson
  • Stephen Brewster

Multimodal affective feedback: combining thermal, vibrotactile, audio and visual signals

  • Graham Wilson
  • Euan Freeman
  • Stephen Brewster

Niki and Julie: a robot and virtual human for studying multimodal social interaction

  • Ron Artstein
  • David Traum
  • Jill Boberg
  • Alesia Gainer
  • Jonathan Gratch
  • Emmanuel Johnson
  • Anton Leuski
  • Mikio Nakano

A demonstration of multimodal debrief generation for AUVs, post-mission and in-mission

  • Helen Hastie
  • Xingkun Liu
  • Pedro Patron

Laughter detection in the wild: demonstrating a tool for mobile social signal processing and visualization

  • Simon Flutura
  • Johannes Wagner
  • Florian Lingenfelser
  • Andreas Seiderer
  • Elisabeth André

SESSION: Demo Session 2

Multimodal system for public speaking with real time feedback: a positive computing perspective

  • Fiona Dermody
  • Alistair Sutherland

Multimodal biofeedback system integrating low-cost easy sensing devices

  • Wataru Hashiguchi
  • Junya Morita
  • Takatsugu Hirayama
  • Kenji Mase
  • Kazunori Yamada
  • Mayu Yokoya

A telepresence system using a flexible textile display

  • Kana Kushida
  • Hideyuki Nakanishi

Large-scale multimodal movie dialogue corpus

  • Ryu Yasuhara
  • Masashi Inoue
  • Ikuya Suga
  • Tetsuo Kosaka

Immersive virtual reality with multimodal interaction and streaming technology

  • Wan-Lun Tsai
  • You-Lun Hsu
  • Chi-Po Lin
  • Chen-Yu Zhu
  • Yu-Cheng Chen
  • Min-Chun Hu

Multimodal interaction with the autonomous Android ERICA

  • Divesh Lala
  • Pierrick Milhorat
  • Koji Inoue
  • Tianyu Zhao
  • Tatsuya Kawahara

Ask Alice: an artificial retrieval of information agent

  • Michel Valstar
  • Tobias Baur
  • Angelo Cafaro
  • Alexandru Ghitulescu
  • Blaise Potard
  • Johannes Wagner
  • Elisabeth André
  • Laurent Durieu
  • Matthew Aylett
  • Soumia Dermouche
  • Catherine Pelachaud
  • Eduardo Coutinho
  • Björn Schuller
  • Yue Zhang
  • Dirk Heylen
  • Mariët Theune
  • Jelte van Waterschoot

Design of multimodal instructional tutoring agents using augmented reality and smart learning objects

  • Anmol Srivastava
  • Pradeep Yammiyavar

AttentiveVideo: quantifying emotional responses to mobile video advertisements

  • Phuong Pham
  • Jingtao Wang

Young Merlin: an embodied conversational agent in virtual reality

  • Ivan Gris
  • Diego A. Rivera
  • Alex Rayon
  • Adriana Camacho
  • David Novick

SESSION: EmotiW Challenge

EmotiW 2016: video and group-level emotion recognition challenges

  • Abhinav Dhall
  • Roland Goecke
  • Jyoti Joshi
  • Jesse Hoey
  • Tom Gedeon

Emotion recognition in the wild from videos using images

  • Sarah Adel Bargal
  • Emad Barsoum
  • Cristian Canton Ferrer
  • Cha Zhang

A deep look into group happiness prediction from images

  • Aleksandra Cerekovic

Video-based emotion recognition using CNN-RNN and C3D hybrid networks

  • Yin Fan
  • Xiangju Lu
  • Dian Li
  • Yuanliu Liu

LSTM for dynamic emotion and group emotion recognition in the wild

  • Bo Sun
  • Qinglan Wei
  • Liandong Li
  • Qihua Xu
  • Jun He
  • Lejun Yu

Multi-clue fusion for emotion recognition in the wild

  • Jingwei Yan
  • Wenming Zheng
  • Zhen Cui
  • Chuangao Tang
  • Tong Zhang
  • Yuan Zong
  • Ning Sun

Multi-view common space learning for emotion recognition in the wild

  • Jianlong Wu
  • Zhouchen Lin
  • Hongbin Zha

HoloNet: towards robust emotion recognition in the wild

  • Anbang Yao
  • Dongqi Cai
  • Ping Hu
  • Shandong Wang
  • Liang Sha
  • Yurong Chen

Group happiness assessment using geometric features and dataset balancing

  • Vassilios Vonikakis
  • Yasin Yazici
  • Viet Dung Nguyen
  • Stefan Winkler

Happiness level prediction with sequential inputs via multiple regressions

  • Jianshu Li
  • Sujoy Roy
  • Jiashi Feng
  • Terence Sim

Video emotion recognition in the wild based on fusion of multimodal features

  • Shizhe Chen
  • Xinrui Li
  • Qin Jin
  • Shilei Zhang
  • Yong Qin

Wild wild emotion: a multimodal ensemble approach

  • John Gideon
  • Biqiao Zhang
  • Zakaria Aldeneh
  • Yelin Kim
  • Soheil Khorram
  • Duc Le
  • Emily Mower Provost

Audio and face video emotion recognition in the wild using deep neural networks and small datasets

  • Wan Ding
  • Mingyu Xu
  • Dongyan Huang
  • Weisi Lin
  • Minghui Dong
  • Xinguo Yu
  • Haizhou Li

Automatic emotion recognition in the wild using an ensemble of static and dynamic representations

  • Mostafa Mehdipour Ghazi
  • Hazım Kemal Ekenel

SESSION: Doctoral Consortium

The influence of appearance and interaction strategy of a social robot on the feeling of uncanniness in humans

  • Maike Paetzel

Viewing support system for multi-view videos

  • Xueting Wang

Engaging children with autism in a shape perception task using a haptic force feedback interface

  • Alix Pérusseau-Lambert

Modeling user's decision process through gaze behavior

  • Kei Shimonishi

Multimodal positive computing system for public speaking with real-time feedback

  • Fiona Dermody

Prediction/Assessment of communication skill using multimodal cues in social interactions

  • Sowmya Rasipuram

Player/Avatar body relations in multimodal augmented reality games

  • Nina Rosa

Computational model for interpersonal attitude expression

  • Soumia Dermouche

Assessing symptoms of excessive SNS usage based on user behavior and emotion

  • Ploypailin Intapong
  • Tipporn Laohakangvalvit
  • Tiranee Achalakul
  • Michiko Ohkura

Kawaii feeling estimation by product attributes and biological signals

  • Tipporn Laohakangvalvit
  • Tiranee Achalakul
  • Michiko Ohkura

Multimodal sensing of affect intensity

  • Shalini Bhatia

Enriching student learning experience using augmented reality and smart learning objects

  • Anmol Srivastava

Automated recognition of facial expressions authenticity

  • Krystian Radlak
  • Bogdan Smolka

Improving the generalizability of emotion recognition systems: towards emotion recognition in the wild

  • Biqiao Zhang

SESSION: Grand Challenge Summary

Emotion recognition in the wild challenge 2016

  • Abhinav Dhall
  • Roland Goecke
  • Jyoti Joshi
  • Tom Gedeon

SESSION: Workshop Summaries

1st international workshop on embodied interaction with smart environments (workshop summary)

  • Patrick Holthaus
  • Thomas Hermann
  • Sebastian Wrede
  • Sven Wachsmuth
  • Britta Wrede

ASSP4MI2016: 2nd international workshop on advancements in social signal processing for multimodal interaction (workshop summary)

  • Khiet P. Truong
  • Dirk Heylen
  • Toyoaki Nishida
  • Mohamed Chetouani

ERM4CT 2016: 2nd international workshop on emotion representations and modelling for companion systems (workshop summary)

  • Kim Hartmann
  • Ingo Siegert
  • Ali Albert Salah
  • Khiet P. Truong

International workshop on multimodal virtual and augmented reality (workshop summary)

  • Wolfgang Hürst
  • Daisuke Iwai
  • Prabhakaran Balakrishnan

International workshop on social learning and multimodal interaction for designing artificial agents (workshop summary)

  • Mohamed Chetouani
  • Salvatore M. Anzalone
  • Giovanna Varni
  • Isabelle Hupont Torres
  • Ginevra Castellano
  • Angelica Lim
  • Gentiane Venture

1st international workshop on multi-sensorial approaches to human-food interaction (workshop summary)

  • Anton Nijholt
  • Carlos Velasco
  • Kasun Karunanayaka
  • Gijs Huisman

International workshop on multimodal analyses enabling artificial agents in human-­machine interaction (workshop summary)

  • Ronald Böck
  • Francesca Bonin
  • Nick Campbell
  • Ronald Poppe

ICMI 2016 ACM International Conference on Multimodal Interaction. Copyright © 2015-2017