Recommender Problems for Web Applications

Dr. Deepak K. Agarwal
Principal Research Scientist,
Yahoo! Research, Silicon Valley
July 7, 2010 @ 11:00 pm
BSB 361, Dept. of CSE, IIT Madras

Abstract

Several web applications like content optimization and online advertising involve recommending items from an inventory for each user visit to maximize some yield metric of interest (e.g. click rates). These are instances of large scale recommender system problems that entail several statistical challenges. We provide a mathematical description of the problem followed by modeling solutions for a content optimization problem that arises in the context of Yahoo! Front Page (www.yahoo.com). In fact, we discuss models to a) serve most popular items, b) serve items that are most popular in different user segments and c) provide personalized item recommendations for each user. Our models are based on time series methods, multi-armed bandit schemes and bilinear random effects model. One class of bilinear random effects model we propose extends reduced rank regression to incomplete matrices, the other class extends matrix factorization to incorporate covariates.

Bio
Deepak Agarwal is currently a Principal research scientist at Yahoo! Research. Prior to joining Yahoo!, he was a member of the statistics department at AT and T Research. He is a statistician interested in scalable modeling approaches for large scale applications. He has done extensive research on large scale hierarchical random effects model, computational advertising, modeling massive social networks with applications to call graph that arise in the telecommunications industry and modeling massive dyadic data that arise in applications like recommender systems. He has won four best paper awards (JSM 2001, SDM 2004, KDD 2007, ICDM 2009) that are directly related to the material of the talk. He has also done research in anomaly detection using a time series approach and computational approaches for scaling spatial scan statistic to large data sets. He regularly serves on program committees of data mining and machine learning conferences. He is currently associate editor for Journal of Americal Statistical Association, the top journal in the field of Statistics. He have given two tutorials on Statistical Challenges in Online Advertising at CIKM 2009 and KDD 2009. Deepak has also worked on developing algorithms for real recommender systems that are successfully being used and thus has experience with both practical and scientific issues that arise in such applications.