Welcome to the first Marginalia collection to A Recommancer's Pocket Grimoire, from Gwendydd Research! We are a small independent research lab and consultancy, and we're publishing this newsletter to share our field notes—lessons from our own work, ideas gathered from others, and observations or insights that seem worth revisiting. We aim to remain grounded in the actual practice of building real-world recommender systems, and hope that you will find some useful techniques you can use.
We begin with spring collections from the broader RecSys community.
🧹 Broomstick Bulletins
A quick fly-by of happenings, events, and gatherings
With the solstice just behind us, we are approaching a time of great occult significance. We are referring, of course, to conference season! Submission deadlines have passed, but attendee registration is currently open for these upcoming conferences:
- June 29 - July 3: The Web Conference in fully virtual format
- July 20 - 24: SIGIR in Melbourne, Australia
- Aug 9 - 13: KDD in Jeju, South Korea
- Sept 28 - Oct 2: RecSys in Minneapolis, USA (Early Bird pricing through Aug 17)
🗃️ Curated Curiosities
Blogs, books, and broadcasts from nearby realms
In academic work, you have well-established datasets like MovieLens and just develop an algorithm. It beats SOTA—you are happy! In practice, in the business, the journey starts with this.
—Sasha Fedintsev
The latest episode of the RecSperts podcast includes a great discussion (25:50 - 1:37:57) of practical challenges in a real-world recommendation domain that aren't typically covered by the research literature. Among other things, they discuss the advantages and challenges of more complex interfaces beyond a single list, reasons you shouldn't naively use logged data for offline evaluation, and how to avoid issues with BPR loss.
A user’s interests are diverse – cooking, fitness, travel – but a single static embedding blends them into one indistinguishable vector. When ranking a ski jacket, the model has no way to amplify winter sports signals and suppress cooking signals. Every candidate faces the same blurred snapshot.
—Maggie Zhuang
RS_c has a new series on the role of self-attention in recommender systems and how that differs its role in language modeling, leading to different scaling challenges and solutions. In recommender systems, self-attention helps us reflect multiple interests that shift over time, with more recent interactions providing more information on their own and less recent interactions providing information mostly in aggregate. This allows us to apply some tricks to scale attention to longer sequences that wouldn't work for language modeling.
As a data primitive, user sequences are powerful. They capture temporal behavior instead of just aggregates like "how many clicks" over a period. They enable sequence‑aware models such as Transformers, sequence encoders, or attention‑over‑history architectures. And because they preserve fairly raw behavior, they can be reused across ranking, retrieval, exploration, anomaly detection, and other workloads. The catch is that a high‑quality sequence is not just "the N latest events from a log table."
—Pinterest Engineering
Providing sequences of user interactions for model training and serving is harder than it sounds, because there's a lot more to interactions than just the item ids! Along the way interactions can be enriched with additional information like counts, embeddings, and metadata. In a recent post on their engineering blog, Pinterest walks through some of the design decisions they made in reworking their user sequence platform that allow them to define new events and signals once and use them everywhere, avoiding "split-brain" training-serving skew.
At a high level, serendipity scoring works like this: A candidate title and each item in a member’s viewing history are represented as embeddings in a vector space. For each candidate, we compute its similarity against the history embeddings, find the maximum similarity, and convert that into a "novelty" score. That score becomes an input feature to the downstream recommendation logic.
—Harshad Sane
Sometimes features used by recommendation models and logic that seem trivial offline turn out to be surprisingly difficult to compute efficiently at serving time. In Optimizing Recommendation Systems with JDK’s Vector API , Netflix walks through some of the steps they took to optimize the vector computations involved in producing serendipity scores for candidate items.
📚 A Note from the Grimoire-Keeper
An earlier version of this post had more sections with more of our own commentary and practical guidance. We've extracted those sections out into separate posts, in accordance with our decision to publish smaller, single-topic articles more frequently.
Karl Higley
Karl Higley