Recommender systems sit behind almost every feed and “For You” page, yet most teams still optimise them for a narrow set of metrics like CTR or watch time. The result is a gap between what dashboards say is “good” and what users, regulators and society actually care about: satisfaction, trust, and not being trapped in an echo chamber.
In this talk, Lev Fedorov shares lessons from building and operating large-scale recommenders for consumer products. He walks through how offline metrics like AUC and NDCG can quietly conflict with long-term retention, how feedback loops appear when models are retrained on their own decisions, and what really happened when his team tested seemingly harmless ranking rules such as “no duplicates in the feed”. The session also covers practical techniques for injecting diversity, using negative feedback signals, and monitoring when certain topics or viewpoints are being over- or under-represented.
Attendees will leave with a concrete mental model and set of metrics for thinking about “what our recommender is really optimising for”, patterns for designing better dashboards, and a set of low-friction guardrails that make ranking systems more robust, more transparent and easier to discuss with both product stakeholders and governance teams.
Technical Level: Technical practitioner