2nd UbiComp Workshop on Ubiquitous Technologies to Augment the Human Mind

The Workshop on Ubiquitous Technologies for Augmenting the Human Mind (WAHM) will be held on September 8th at Ubicomp 2015 in Osaka.

Workshop submission deadline: July 3, 2015

Feedback to authors: July 10, 2015

Camera-ready version: July 15 2015 (hard deadline)

Workshop at Ubicomp 2015: September 7/8, 2015

All submissions should be sent as PDF to wahm2015@hcilab.org with "WAHM 2015 Submission" as email subject. See below for formating details.

Toward the Knowledge Log

Lifelogging technologies, the use of sensing technologies to analyze and record one’s lives, is on the rise. Products from industry and research in academia currently focus on using the collected data to support health and fitness. Given these trends, it is only a matter of time before we see mobile sens- ing technology applied to cognitive tasks, enabling novel research directions and use cases. In this workshop, we explore the implications of ”knowledge logging”, how to record and track what we read, learn, comprehend and how this impacts research towards mind augmentation. The goal of this workshop is to combine innovations in ubiquitous computing with basic research in psychology and cognitive science. The aim is to bring mind augmentation technologies from a niche application in rehabilitation to a mainstream technology and initiating a major change in the way we use technology to externalize our mind. This workshop will bring together researchers, designers and practitioners at the intersection of technology and cognitive psychology to discuss elements and viewpoints of knowledge logging, inferring cognitive states and extending our perception.


9:30 - 9:40 Welcome
9:40 -10:30 Keynote I "Human Mind and Embodied Media" by Dr. Masashi Nakatani
10:30 -10:50 Coffee Break
10:50 - 11:20 Reading

Reading Similarity Measure Based on Comparison of Fixation Sequences
Eye gaze and text line matching for reading analysis
Multimodal Literacy: Storytelling Across Senses

11:25 - 11:55 Mental State

Quantifying the Mental State on the Basis of Physical and Social Activities
LifeLog-based Estimation of Activity Diary for Cognitive Behavioral Therapy
Affective Wear - Towards Recording Affect in Real Life Settings

11:55 - 12:35 Assistance

Investigating Memory Recall by Visualization of Photo Network
Exploring Sensory Augmentation Through Tactile Interfaces
Towards Augmenting the Nurse Training and Monitoring
Helping Me - I’m Bored! Why Assembly Assistance needs to be Adaptive.

12:35 - 14:00 Lunch
14:00 - 14:50 Keynote II "Augmenting Human Mind by Gaze-Based Technologies" Prof. Dr. Andreas Dengel
14:50 - 15:30 Breakout Session I
15:30 - 16:30 Reports
16:30 - 17:00 Breakout Session II
17:00 - 17:30 Reports


Human Mind and Embodied Media

Although body perception is commonly felt in our daily lives, it is the least understood and appreciated modality in five senses. Since the body perception is rather implicit, we do not consider this is vulnerable and enhanceable. However, recent behavioral neuroscience findings report multiple phenomena that have tight relationship between body and brain activity that leads to our decision making, which are called embodied cognition. In this talk I introduce several interesting topics in body and mind study and possible research direction in embodied media. Hands-on exercise session will be included.

Speaker: Prof. Dr. Masashi Nakatani

Dr. Nakatani works as Project Associate Professor at the Graduate School of Media Design, Keio University, Project Research Scientist at the Institute of Gerontology, University of Tokyo. His expertise and research themes include psychology on tactile perception, physiology on skin sensation (in vitro), haptic and tactile sensors development and haptic design for everyone.

Augmenting Human Mind by Gaze-Based Technologies

In the age of digitalization we are confronted with an overwhelming data and process complexity. As a consequence the cognitive load of knowledge workers is continuously increasing and we are lacking in extensions, which augment our intellect helping us to understand, increase our understanding, enhance our problem solving capacity, or complement our skills. This is not a new phenomenon but recent advances in eye tracking technologies allow to use the eyes both as a source for understanding cognitive states but also as a means for interacting with objects and subjects in the real world. Such new options are of increasing interest for developing „electronic information butlers and amplifiers“ that are able to anticipate what may be required in a given context and offer us all relevant information. In this talk I like to address the various aspects of augmenting human mind by gaze-based technologies. I will propose and discuss a bunch of approaches and technologies aiming at a more-rapid and better comprehension of human activities as well as their support by learning from gaze behavior. I will demonstrate experimental results from the office field, from medicine as well as from physics education, and further show some applications in the area of infotainment.

Speaker: Prof. Dr. Andreas Dengel

Professor Andreas Dengel is a member of the Management Board as well as Scientific Director at the German Research Center for Artificial Intelligence (DFKI GmbH) in Kaiserslautern where he is leading the Knowledge Management research department. In 1993 he was appointed Professor at the Computer Science Department of the University of Kaiserslautern where he holds the chair “Knowledge-Based Systems”. Since 2009 he is also Professor (Kyakuin) at the Department of Computer Science and Intelligent Systems, Graduate School of Engineering, at Osaka Prefecture University.

Call for Participation

To approach the challenges of augmenting the human mind, we will focus on the following themes, depending on participants’ contributions:

Sensing Technologies: How do signals gathered from ubiquitous sensing devices relate to cognitive processes? Defining sensing and interaction modalities to better understand human behavior.

Applied cognitive theories: How can we use these real life recordings of physical, physiological and cognitive signals to augment our mind and, for example, induce wanted long-term behavior change for users?

Building ”knowledge logs”: From an information retrieval and processing perspective we want to discuss potential technologies relevant for cognitive processing. Personal “knowledge logs” can be can be compiled from sensing using adaptive algorithms automated daily summaries.

HCI to extend the mind: we are looking for Innovative User Interfaces for mind augmentation, including technologies for information priming. For example, how can feedback through ambient large displays and personal mobile devices aid personal knowledge acquisition, retention, and attenuation?

Commercial application areas for human mind augmentation: While many of the application domains for such technologies are for the public good, the same technologies can also be employed in the commercial context. For example, technology could be used to augment meetings detecting the comprehension of each participant and help mitigate potential misunderstandings.

Discussing privacy security and social implications.

The goals of the WAHM 2015 workshop picks up on last year’s theme: we want to foster discussions about technologies that nurture the augmentation of the human mind. Given the good reception of the first workshop and the feedback that we should include a broader range of topics (last year focused on memory), we are working on extending our community. The above-mentioned themes will be used as a starting point for the discussion and group analysis (described below). However, we will also pay attention to new themes possibly emerging from morning presentation and discussions. In addition to shaping the research agenda, we will also dis- cuss social impact and potential commercial applications. To this end we will also invite representatives from corporate re- search.

Submission Format

Workshop candidates are requested to send a position paper (4-8 pages in the ACM SIGCHI non- archival Extended Abstracts word template or Latex template (landscape format)) to the organizers about their research and link to the workshop theme. In addition to describing their work candidates will be asked to write about challenges and opportunities they see for technology that augments the human mind, in order to prepare the candidates the workshop theme. Participants will be selected on the basis of the relevance of their work and interests and familiarity with the WAHM workshop topics. Accepted submissions will be included in the UbiComp 2015 Adjunct Proceedings and listed in the ACM Digital Library. All submissions should be sent as PDF to wahm2015@hcilab.org with "WAHM 2015 Submission" as email subject.


Kai Kunze

Kai Kunze works as an associate project professor at Keio Media Design, Keio University. Beforehand, he held an assistant professorship at Osaka Prefecture Uni- versity. He received a Summa Cum Laude for his phD thesis, University Passau. He was visting researcher at the MIT Media Lab. His work experience includes internships at the Palo Alto Research Center (PARC), Sunlabs Europe and the Research Department of the German Stock Exchange. His major research contributions are in pervasive computing, es- pecially in sensing, physical and cognitive activity recognition. Recently, he focuses on tracking knowledge acquisition activities, especially reading.

Tilman Dingler

Tilman Dingler is a researcher at the Institute for Visualization and Interactive Systems at University of Stuttgart. He focuses on concepts and applications in the field of Pervasive Computing, thereby developing embedded devices and software for context-aware systems that put users and their context at the center. Tilman holds a Diploma in Media Computer Science from the University of Munich, a Master’s degree in Web Science from the University of San Francisco and an Honors degree in Technology Management from the Center for Digital Technology and Management in Munich. Before starting his PhD, Tilman was developing software for TinyCo and Yahoo!.

Niels Henze

is assistant professor for Socio-Cognitive Systems in the Institute for Visualization and Interactive Systems and the SimTech Cluster for Simulation Technology at the University of Stuttgart. He received awards from different conferences including CHI and MobileHCI. He is interested in large-scale human subject studies, improvement of inter- active systems through models of human behavior, and smart attention management.

Koichi Kise

is a professor in the Department of Computer Science and Intelligent Systems at Osaka Prefecture Univer- sity. From 2000 to 2001, he was a visiting professor at Ger- man Research Center for Artificial Intelligence (DFKI), Ger- many. He received awards including the best paper award of IEICE in 2008, the IAPR/ICDAR best paper awards in 2007 and 2013, the IAPR Nakano award in 2010, the ICFHR best paper award in 2010 and the ACPR best paper award in 2011. He works as the chair of the IAPR technical committee 11 (reading systems) and a member of the IAPR conferences and meetings committee. His major research activities are in analysis, recognition and retrieval of documents, images and activities.

Yoichi Sato

is a professor at the Institute of Industrial Science, the University of Tokyo, jointly affiliated with the Graduate School of Interdisciplinary Information Studies and Graduate School of Information Science and Technology. He received the BSE degree from the University of Tokyo in 1990, and the MS and PhD degrees in robotics from the School of Computer Science, Carnegie Mellon University, Pittsburgh, Pennsylvania, in 1993 and 1997 respectively.