UMUAI Journal

Latest Results for User Modeling and User-Adapted Interaction

28 May 2023

• Correction to: How do item features and user characteristics affect users’ perceptions of recommendation serendipity? A cross-domain analysis
• Automatically detecting task-unrelated thoughts during conversations using keystroke analysis

  Abstract

  Task-unrelated thought (TUT), commonly referred to as mind wandering, is a mental state where a person’s attention moves away from the task-at-hand. This state is extremely common, yet not much is known about how to measure it, especially during dyadic interactions. We thus built a model to detect when a person experiences TUTs while talking to another person through a computer-mediated conversation, using their keystroke patterns. The best model was able to differentiate between task-unrelated thoughts and task-related thoughts with a kappa of 0.363, using features extracted from a 15 second window. We also present a feature analysis to provide additional insights into how various typing behaviors can be linked to our ongoing mental states.

• Justification of recommender systems results: a service-based approach

  Abstract

  With the increasing demand for predictable and accountable Artificial Intelligence, the ability to explain or justify recommender systems results by specifying how items are suggested, or why they are relevant, has become a primary goal. However, current models do not explicitly represent the services and actors that the user might encounter during the overall interaction with an item, from its selection to its usage. Thus, they cannot assess their impact on the user’s experience. To address this issue, we propose a novel justification approach that uses service models to (i) extract experience data from reviews concerning all the stages of interaction with items, at different granularity levels, and (ii) organize the justification of recommendations around those stages. In a user study, we compared our approach with baselines reflecting the state of the art in the justification of recommender systems results. The participants evaluated the Perceived User Awareness Support provided by our service-based justification models higher than the one offered by the baselines. Moreover, our models received higher Interface Adequacy and Satisfaction evaluations by users having different levels of Curiosity or low Need for Cognition (NfC). Differently, high NfC participants preferred a direct inspection of item reviews. These findings encourage the adoption of service models to justify recommender systems results but suggest the investigation of personalization strategies to suit diverse interaction needs.

• How do item features and user characteristics affect users’ perceptions of recommendation serendipity? A cross-domain analysis

  Abstract

  Serendipity is one of beyond-accuracy objectives for recommender systems (RSs), which aims to achieve both relevance and unexpectedness of recommendations, so as to potentially address the “filter bubble” issue of traditional accuracy-oriented RSs. However, so far most of the serendipity-oriented studies have focused on developing algorithms to consider various types of item features or user characteristics, but are largely based on their own assumptions. Few have stood from users’ perspective to identify the effects of these features on users’ perceptions of the serendipity of the recommendation. Therefore, in this paper, we have analyzed their effects with two user survey datasets. These are the Movielens Serendipity Dataset of 467 users’ responses to a retrospective survey of their perceptions of the recommended movie’s serendipity, and the Taobao Serendipity Dataset of 11,383 users’ perceptions of the serendipity of a recommendation received at a mobile e-commerce platform. In both datasets, we have analyzed the correlations between users’ serendipity perceptions and various types of item features (i.e., item-driven such as popularity, profile-driven such as in-profile diversity, and interaction-driven including category-level and item-level features), as well as the influence of several user characteristics (including the Big-Five personality traits and curiosity). The results disclose both domain-independent and domain-specific observations, which may be constructive in enhancing current serendipity-oriented recommender systems by better utilizing item features and user data.

• Gaze-based predictive models of deep reading comprehension

  Abstract

  Eye gaze patterns can reveal user attention, reading fluency, corrective responding, and other reading processes, suggesting they can be used to develop automated, real-time assessments of comprehension. However, past work has focused on modeling factual comprehension, whereas we ask whether gaze patterns reflect deeper levels of comprehension where inferencing and elaboration are key. We trained linear regression and random forest models to predict the quality of users’ open-ended self-explanations (SEs) collected both during and after reading and scored on a continuous scale by human raters. Our models use theoretically grounded eye tracking features (number and duration of fixations, saccade distance, proportion of regressive and horizontal saccades, spatial dispersion of fixations, and reading time) captured from a remote, head-free eye tracker (Tobii TX300) as adult users read a long expository text (6500 words) in two studies (N = 106 and 131; 247 total). Our models: (1) demonstrated convergence with human-scored SEs (r = .322 and .354), by capturing both within-user and between-user differences in comprehension; (2) were distinct from alternate models of mind-wandering and shallow comprehension; (3) predicted multiple-choice posttests of inference-level comprehension (r = .288, .354) measured immediately after reading and after a week-long delay beyond the comparison models; and (4) generalized across new users and datasets. Such models could be embedded in digital reading interfaces to improve comprehension outcomes by delivering interventions based on users’ level of comprehension.