MINDSEE

MindSee (Symbiotic Mind Computer Interaction for Information Seeking) had the purpose of the design and developing an application for scientific information seeking that exemplifies the fruitful symbiosis of modern Brain Computer technology with real-world Human-Computer Interaction.

This tool relied on a dataset that includes over 50 million scientific documents from the following data sources: the Web of Science prepared by THOMSON REUTERS, Inc., and from the Digital Libraries of the Association of Computing Machinery (ACM), the Institute of Electrical and Electronics Engineers (IEEE), and Springer.

What were the effects of these users’ unconscious signals on the searching process?

Their cognitive and physiological activity affected the ability of the MindSee system to retrieve relevant information in order to withstand the seeking.

MindSee proposed the fusion of electroencephalography (EEG) - as the main sensor - with peripheral physiological sensors (skin response, facial electromyography, eye gaze, and pupil dilation) and contextual information for acquiring measures related to perception, cognition, and emotions.

This symbiotic system will allow predicting user intentions and needs based on the information evaluation derived from cognitive and physiological signals processing. Then, these features will be utilized in order to feed the machine-learning algorithm and as a result, the interface will show symbiotic adaptations (i.e. displaying new relevant data, changing the ranking of the proposed results, and decreasing the density of information in situations of comprehension difficulties for the users). Real-time estimates of unconscious signals were used to complement keyboard and gestural input in this real-world application of scientific literature search where the information exploration of the user was driven by co-adaptations of the computer.

The computer adapted the information customizing them to each different scientific search of the user.

Detailed Project Information

Project Start Date: 1st October 2013

Project Duration: 36 months
Project Coordinator: University of Helsinki, FI, Prof Giulio Jacucci

HIT Principal Investigator: Prof. Luciano Gamberini

HIT Members and Staff: Prof. Anna Spagnolli, dott. Patrik Pluchino

Project number: 611570

Call (part) identifier: FP7-ICT-2013-10

Total Project Funding: € 3,779,783 (2,990,000)

Partners: University of Helsinki, Aalto University, University of Padova, i2 Media Research,Technische Universität Berlin.

Official website: www.mindsee.eu