Dietmar Jannach, Pearl Pu, Francesco Ricci, Markus Zanker

The origins of modern recommender systems date back to the early 1990s when they were mainly applied experimentally to personal email and information filtering. Today, 30 years later, personalized recommendations are ubiquitous and research in this highly successful application area of AI is flourishing more than ever. Much of the research in the last decades was fueled by advances in machine learning technology. However, building a successful recommender sys-tem requires more than a clever general-purpose algorithm. It requires an in-depth understanding of the specifics of the application environment and the expected effects of the system on its users. Ultimately, making recommendations is a human-computer interaction problem, where a computerized system supports users in information search or decision-making contexts. This special issue contains a selection of papers reflecting this multi-faceted nature of the problem and puts open research challenges in recommender systems to the fore-front. It features articles on the latest learning technology, reflects on the human-computer interaction aspects, reports on the use of recommender systems in practice, and it finally critically discusses our research methodology.

Harald Steck, Linas Baltrunas, Ehtsham Elahi, Dawen Liang, Yves Raimond, Justin Basilico

Deep learning has profoundly impacted many areas of machine learning. However, it took a while for its impact to be felt in the field of recommender systems. In this article, we outline some of the challenges encountered and lessons learned in using deep learning for recommender systems at Netflix. We first provide an overview of the various recommendation tasks on the Netflix service. We found that different model architectures excel at different tasks. Even though many deep-learning models can be understood as extensions of existing (simple) recommendation algorithms, we initially did not observe significant improvements in performance over well-tuned non-deep-learning approaches. Only when we added numerous features of heterogeneous types to the input data, deep-learning models did start to shine in our setting. We also observed that deep-learning methods can exacerbate the problem of offline–online metric (mis-)alignment. After addressing these challenges, deep learning has ultimately resulted in large improvements to our recommendations as measured by both offline and online metrics. On the practical side, integrating deep-learning toolboxes in our system has made it faster and easier to implement and experiment with both deep-learning and non-deep-learning approaches for various recommendation tasks. We conclude this article by summarizing our take-aways that may generalize to other applications beyond Netflix.

Thorsten Joachims, Ben London, Yi Su, Adith Swaminathan, Lequn Wang 

In recent years, a new line of research has taken an interventional view of recommender systems, where recommendations are viewed as actions that the system takes to have a desired effect. This interventional view has led to the development of counterfactual inference techniques for evaluating and optimizing recommendation policies. This article explains how these techniques enable unbiased offline evaluation and learning despite biased data, and how they can inform considerations of fairness and equity in recommender systems.

Joseph A. Konstan, Loren G. Terveen

From the earliest days of the field, Recommender Systems research and practice has struggled to balance and integrate approaches that focus on recommendation as a machine learning or missing-value problem with ones that focus on machine learning as a discovery tool and perhaps persuasion platform. In this article, we review 25 years of recommender systems research from a human-centered perspective, looking at the interface and algorithm studies that advanced our understanding of how system designs can be tailored to users objectives and needs. At the same time, we show how external factors, including commercialization and technology developments, have shaped research on human-centered recommender systems. We show how several unifying frameworks have helped developers and researchers alike incorporate thinking about user experience and human decision-making into their designs. We then review the challenges, and the opportunities, in today’s recommenders, looking at how deep learning and optimization techniques can integrate with both interface designs and human performance statistics to improve recommender effectiveness and usefulness

Paolo Cremonesi, Dietmar Jannach

Scholars in algorithmic recommender systems research have developed a largely standardized scientific method, where progress is claimed by showing that a new algorithm outperforms existing ones on or more accuracy measures. In theory, reproducing and thereby verifying such improvements is easy, as it merely involves the execution of the experiment code on the same data. However, as recent work shows, the reported progress is often only virtual, because of a number of issues related to (i) a lack of reproducibility, (ii) technical and theoretical flaws, and (iii) scholarship practices that are strongly prone to researcher biases. As a result, several recent works could show that the latest published algorithms actually do not outperform existing methods when evaluated independently. Despite these issues, we currently see no signs of a crisis, where researchers re-think their scientific method, but rather a situation of stagnation, where researchers continue to focus on the same topics. In this paper, we discuss these issues, analyze their potential underlying reasons, and outline a set of guidelines to ensure progress in recommender systems research.

Jon Atle Gulla, Rolf Dyrnes Svendsen, Lemei Zhang, Agnes Stenbom, Jørgen Frøland

The adoption of recommender systems in online news personalization has made it possible to tailor the news stream to the individual interests of each reader. Previous research on commercial recommender systems has emphasized their use in large-scale media houses and technology companies, and real-world experiments indicate substantial improvements of click rates and user satisfaction. It is less understood how smaller media houses are coping with this new technology, how the technology affects their business models, their editorial processes, and their news production in general. Here we report on the experiences from numerous Scandinavian media houses that have experimented with various recommender strategies and streamlined their news production to provide personalized news experiences. In addition to influencing the content and style of news stories and the working environment of journalists, the news recommender systems have been part of a profound digital transformation of the whole media industry. Interestingly, many media houses have found it undesirable to automate the entire recommendation process and look for approaches that combine automatic recommendations with editorial choices.

Stefano Bistarelli, Lars Kotthoff, Francesco Santini , Carlo Taticchi

The Third International Competition on Computational Models of Argumentation (ICCMA’19) focused on reasoning tasks in abstract argumentation frameworks. Submitted solvers were tested on a selected collection of benchmark instances, including artificially generated argumentation frameworks and some frameworks formalizing real-world problems. This competition introduced two main novelties over the two previous editions: the first one is the use of the Docker platform for packaging the participating solvers into virtual “light” containers; the second novelty consists of a new track for dynamic frameworks.

Chris Welty, Praveen Paritosh, Kurt Bollacker

The AI Bookie column documents highlights from AI Bets, an online forum for the creation of adjudicatable predictions about the future of AI. Since the column’s inception 3 years ago, only a few scientific bets have been collected, despite universal approval around the idea of scientific betting. We hope to widen our reach with an additional first batch of seed bets that are of broad interest to the research community including AI bias, fifth sentence prediction, emotion regu-lation, big models, and fake news. For detailed guidelines and to place bets, visit sciencebets.org.

Michael Wollowski

Three panelists, Ashok Goel, Ansaf Salleb-Aouissi and Mehran Sahami explain some of the tools and techniques they used to keep their students engaged during virtual instruction. The techniques include the desire to take one’s passion for the learning materials to the virtual classroom, to ensure teacher presence, provide for cognitive engagement with the subject and facilitate social interactions. Finally, we learn about tools used to manage a large online course so as to move the many active learning exercises to the virtual classroom.

Michael Wollowski

Three panelists, Ashok Goel, Ansaf Salleb-Aouissi and Mehran Sahami explain some of the tools and techniques they used to keep their students engaged during virtual instruction. The techniques include the desire to take one’s passion for the learning materials to the virtual classroom, to ensure teacher presence, provide for cognitive engagement with the subject and facilitate social interactions. Finally, we learn about tools used to manage a large online course so as to move the many active learning exercises to the virtual classroom.

K. Brent Venable, Odd Erik Gundersen

Artificial Intelligence has witnessed an exponential growth in the last decade and, thanks to its many successful and pervasive applications, it has now become a research field with profound societal impacts. The interest in AI has reached an all-time high from all sectors of our modern society, including industry, health, education and government. AI Magazine, founded in 1980, has documented the rise of AI from an elite and almost esoteric field to its current status of key player in modern society. Under the leadership of exceptional scientists with a global vision of the field, David Leake first and then Ashok Goel, it has provided a venue for vibrant discussion on technological transformations, research trends and fundamental breakthroughs.