MINDSEE

MindSee aims to develop an application for scientific information seeking that exemplifies the fruitful symbiosis of modern Brain Computer Interface technology with real-world Human Computer Interaction. The look for information is characterized by an iterative process of typing queries and skimming through the a list of provided results in order to find relevant scientific data. MindSee utilizes a dataset that includes over 50 million scientific documents from the following data sources: the Web of Science prepared by THOMSON REUTERS, Inc., and from the Digital Libraries of the Association of Computing Ma- chinery (ACM), the Institute of Electrical and Electronics Engineers (IEEE), and Springer.
MindSee will be a cutting-edge information retrieval system that outperforms state-of-the-art tools by more than doubling the performance of information seeking in realistic tasks. As a novel solution, MindSee proposes to fuse EEG - as the main sensor - with peripheral physiological sensors (EDR, fEMG, eye gaze and pupillometry) and contextual information for unobtrusive acquisition of implicit measures of perception, cognition and emotions.

This symbiotic system will allow to predict user intentions and needs based on the quantification of the features extracted from the cognitive and physiological signals processing. Then, these features will be utilized in order to feed the machine-learning algorithm and as a result the interface will show symbiotic adaptations (i.e. displaying new relevant data, changing the ranking of the proposed results, and decreasing the density of information in situations of cognitive overload). Real-time estimates of these implicit, or hidden, user states will be used to complement keyboard and gestural input in a real-world application of scientific literature search where the information exploration of the user is guided by co-adaptation with the computer. The proposed Symbiotic Information Seeking System will provide a wide range of visualization resources that adapt the information retrieved according to its relevance, cognitive ergonomic complexity, and aesthetic properties.

HIT Research Centre will be involved in:
- Defining and exploring symbiotic systems potentialities;
- Developing scenarios and collect users requirements;
- Experiencing symbiotic interaction;
- Checking credibility, user acceptance, and impact on society and ethics of mind/brain driven symbiotic systems;
- Studying the pupil dilation, in order to design a real-time pupil analysis software that will be able to recognize and communicate, to the user, changes in mental states, that are related to variations in pupil size;
- Testing the system performance;
- Measuring performances of users' information exploration;
- Evaluating in-field MindSee;
- Validating MindSee as symbiotic interaction system for information explorations

Related Video

Project Start Date: 1st October 2013
Project Duration: 36 months
Project Coordinator: University of Helsinki, FI, Prof Giulio Jacucci
Project number: 611570
Call (part) identifier: FP7-ICT-2013-10
Total Project Funding: 2,990,000 (3,779,783) €
Partners: University of Helsinki, Aalto University, University of Padova, i2 Media Research,Technische Universität Berlin.
Official website: www.mindsee.eu