2023- Reprints: Technology and Information Systems

Research has revealed the potential of recording people’s eye movements to char[1]acterize and predict their choice decisions. In this paper, we propose the use of a deep learning architecture, called RETINA, to predict multi-alternative, multi[1]attribute consumer choice from eye movement data. Unlike previous research, RETINA directly uses the raw eye-tracking data from both eyes as input. It combines state-of-the art Transformer and Metric Learning methods which capitalize on the key characteristics of the raw eye-tracking data. Using the raw data input eliminates the information loss that may result from first calculating fixations, deriving metrics from the fixations data and analysing those metrics, as has been previously done in eye movement research. While Deep Learning architectures often require very large data sets, using the raw gaze data allows us to apply Deep Learning to eye tracking data sets of the size commonly encountered in academic and applied research. Using a data set with 112 respondents who made choices among four laptops, we show that the proposed architecture outper[1]forms other state-of-the-art machine learning methods (standard BERT, LSTM, autoML, logistic regression) calibrated on raw data or fixation data. The analysis of partial time and partial data segments reveals the ability of RETINA to predict choice outcomes well before a decision has been reached. We provide an assessment of which features of the eye movement data contribute to RETINA’s prediction accuracy. We provide recommendations on how the proposed deep learning architecture can be used as a basis for future academic research.

The need for advanced automation and artificial intelligence (AI) in various fields, including text classification, has dramatically increased in the last decade, leaving us critically dependent on their performance and reliability. Yet, as we increasingly rely more on AI applications, their algorithms are becoming more nuanced, more complex, and less understandable precisely at a time we need to understand them better and trust them to perform as expected. Text classification in the medical and cybersecurity domains is a good example of a task where we may wish to keep the human in the loop. Human experts lack the capacity to deal with the high volume and velocity of data that needs to be classified, and ML techniques are often unexplainable and lack the ability to capture the required context needed to make the right decision and take action. We propose a new abstract configuration of Human-Machine Learning (HML) that focuses on reciprocal learning, where the human and the AI are collaborating partners.