MindSee: Symbiotic Systems for Everyday Life

Project number: 611570
Call (part) identifier: FP7-ICT-2013-10


MindSee aims to develop an information seeking application that exemplifies the fruitful symbiosis of modern Brain Computer Interface technology with real-world Human Computer Interaction. The result will be a cutting-edge information retrieval system that outperforms state-of-the-art tools by more than doubling the performance of information seeking in realistic tasks. As a novel solution, MindSee proposes to fuse EEG - as the main sensor - with peripheral physiological sensors (EDR, fEMG, eye gaze and pupillometry) and contextual information for unobtrusive acquisition of implicit measures of perception, cognition and emotion.
Real-time estimates of these implicit, or hidden, user states will be used to complement keyboard and gestural input in a real-world application of scientific literature search where the information exploration of the user is guided by co-adaptation with the computer.
The proposed Symbiotic Information Seeking System will provide a wide range of visualization resources that adapt the information retrieved according to its relevance, cognitive ergonomic complexity, and aesthetic properties.
HTLAB will have the task to

(Tasks 1.)

  • 1 Define and explore symbiotic systems potentialities
  • 2 Develop scenarios and collect users requirements
  • 3 Experience symbiotic interaction
  • 4 Check credibility, user acceptance, and impact on society and ethics of mind/brain driven symbiotic systems

(Tasks 2.)

  • 1 Single-trial classification of fixation-related potentials in free viewing tasks
  • 2 Mechanism development for neural correlates of cognitive responses in HCI
  • 3 Context modeling from peripheral physiological and behavioral data for electrophysiological markers
  • 4 Exploring physiological patterns related to emotional states evoked during HCI
  • 5 Physiology-informed optimization of user models for information seeking
  • 6 The Preconscious Pupil

(Tasks 5.)

  • Data and corpora collection for machine learning
  • Test the system performance
  • Measure performances of users’ information exploration
  • Overall evaluate in-field MindSee
  • Validate MindSee as symbiotic interaction system for information explorations

For more information
http://www.mindsee.eu and