Welcome to the first Marginalia collection to A Recommancer's Pocket Grimoire, from Gwendydd Research! We are a small independent research lab and consultancy, and we're publishing this (hopefully) monthly newsletter to share field notes—lessons from our own work, ideas gathered from others, and observations or insights that seem worth coming back to. We aim to remain grounded in the actual practice of building real-world recommender systems, and hope that you will find some useful techniques herein.
We are also working on developing Grimoire Chapters, which we envision as longer-form, more in depth writing oriented around particular themes. The Chapters are higher-effort, so they'll be published on a "when it's done" schedule. We'll be sharing updates and some excerpts/previews in the Marginalia. (Subscribe to keep up and be notified when we publish them!)
In this issue:
upcoming conferences, some tricks for handling explicit negative feedback, an interesting paper about fresh items from Pinterest in The Web Conf proceedings, and a curated assortment of recent podcasts and industry blog posts
🧹 Broomstick Bulletins
A quick fly-by of happenings, events, and gatherings
With the solstice behind us, we are now heading into a time of great occult significance. We are referring, of course, to conference season! Submission deadlines have passed, but attendee registration is currently open for these upcoming conferences:
- June 29 - July 3: The Web Conference in fully virtual format
- July 20 - 24: SIGIR in Melbourne, Australia
- Aug 9 - 13: KDD in Jeju, South Korea
- Sept 28 - Oct 2: RecSys in Minneapolis, USA (Early Bird pricing through Aug 17)
If you'd like to attend and haven't yet made arrangements, send your registration owls soon. The Web Conf starts tomorrow (as we write this) but even if you're not able to attend, you may still be interested in the conference proceedings.
🪄 Handy Hexes
Practical charms and clever tricks for stubborn little problems
As relevance witches, we're well-versed in distilling the essence of positive expressions of interest: clicks, listens, candle-lit invocations, etc. But what about disinterest—what do we do when our users tell us what they don't want? If tomes with those spells exist, they're rare or hidden away in secret archives.
Trying to incorporate negative feedback into an existing recommender creates a bit of a conundrum. We need users to interact with negative feedback controls (e.g. 👎) and generate data that we could use to train a model, but they have little reason to use the controls unless there's a noticeable impact on their recommendations. We have to demonstrate that our negative feedback system is effective so that we can collect the data we need to actually make it so! As tempting as it might be, employing paradox magic here would be dangerous and irresponsible, so what other options do we have?
If we're using a model that represents user preferences as a sequence or set of actions (e.g. clicks, plays), rather than a learned user embedding, we do have some options. One of the many advantages of such models is that they offer more flexibility to shape the outputs. Usually, we try to avoid model features that skew differently between training and serving, to avoid predictive error. However, recommendation isn't really a prediction problem, it's just convenient to frame it that way sometimes. What actually matters is producing useful, relevant recommendations (as evaluated by real humans, not an abstract summary statistic), so manipulating serving-time inputs to mimic the differently-sourced training-time features is absolutely valid if it furthers that goal.
Embracing the old adage that "perfect" can often be the enemy of "good", here are three simple charms to create a functional negative feedback system from existing recommender system components.
1. Use explicit negative feedback to cancel out implicit positive feedback
Implicit-feedback-based recommenders often consider any engagement to be indicative of interest or enjoyment. This is clearly a fairly noisy signal—we've all spent attention on something and later wished to have that time back, right? So, we can help the machine to appreciate the meaning of regret with a simple heuristic: remove or mask interactions from a user's history if they gave negative feedback on those items.
"Canceling out" user engagements offers only rudimentary responsiveness to negative feedback controls, but it's a start. Even after you've collected enough negative feedback data to train an actual model on it, overriding an implicit interest signal with explicit disinterest still makes sense.
2. Estimate irrelevance by turning your existing relevance estimator upside-down
Somewhere in your existing recommender system, you have a model for divining which items are most relevant based on what items a user has previously enjoyed. If that model takes (implicitly positive) interacted items as input features, consider feeding it explicit negatives instead to produce an ir-relevance score. Subtract estimated irrelevance from estimated relevance, and use this net relevance score in place of the raw relevance score in downstream ranking logic.
Once you've collected enough negative feedback data for training, you can consider trying to incorporate irrelevance estimation directly into the model architecture. We wish we had a tried-and-true recipe to recommend for that, but you might take some inspiration from Wu, et al's approach to news recommendation with (implicit) negative feedback.
3. Piggy-back on weaker signals of disinterest
In some recommendation domains, there are user actions—such as abandons or skips—that can be reasonably treated as implicit signals of disinterest. In such cases, you can use that implicit disinterest to help the machine build up a tolerance for negativity before introducing explicit negative controls. Then, you can make your system responsive to explicit negatives right out of the gate, simply by treating them as stronger versions of the implicit negatives.
Prepare your system for this hex by including these negative item interactions as negatively-labeled training examples, with the negative feedback actions encoded among the model input features. We like Google's approach of adding an independent "not to recommend" loss function term for negative labels, which permits the impact of positive and negative labels to be tuned separately. This is not an insignificant change, but since it relies solely on implicit signals, it should be doable with existing data.
Then, at serving time, you can feed explicit negative feedback into the same model input features that were used for implicit negative feedback data during training. There's a few options for doing so, with the simplest being to just treat explicit negatives as if they were implicit negatives. Since we'd like explicit negative feedback to have a stronger effect though, you could try weighting explicit negatives more heavily (e.g. maybe a thumbs-down is twice as strong as a skip?), or converting an explicit negative into multiple implicit negative actions (e.g. maybe a thumbs-down turns into 2 skips?). Just be sure to actually look at recommendation outputs to make sure the resulting behavior is reasonable, because we're deliberately going outside the range of inputs the model saw in training.
This hex not only allows your system to be responsive to explicit negative feedback prior to data-collection, it also sets you up to leverage more-plentiful implicit negatives to get more out of the sparser-but-stronger explicit negatives down the line. As more data becomes available and your model develops a thicker skin, you can blend the explicit negatives back into your model as training examples with strong negative labels (e.g. with larger label weights). You can use the same feature manipulation that you've been using at serving time to represent the explicit negatives in training data, or you can add a new feature for them and update the serving-time featurization accordingly.
Once you've cobbled together a workable negative feedback mechanism and collected enough data, then it becomes straightforward to apply known techniques! the next steps involve wading into murky waters. Even the papers linked above offer only partial answers that may require substantial adaptation to fit your context. If you have any further tips, tricks, or references on this topic—maybe even one of those rare tomes—we'd love to hear about them!
What we can say with confidence is that the above hexes can help you reach that starting point for developing a more sophisticated solution, by unblocking the collection of a negative feedback dataset for training. You'll also have a functional negative feedback system running in production while you explore how to actually make use of that dataset.
📜 Mystic Manuscripts
Freshly-penned findings from pre-prints and peer-reviewed publications
The Web Conference proceedings are out, and, as is often the case, one of our favorite papers is from Pinterest. Warmer for Less: A Cost-Efficient Strategy for Cold-Start Recommendations observes that ranking models often develop a bias toward established items and away from fresh (cold-start) items. Since there's a feedback loop between what is recommended and the logged user interactions that flow into model training and future recommendations, this bias can become self-reinforcing and perpetuate the under-recommendation of fresh items. As a result, recommendations can become stale and can detract from the users’ experience of serendipity.
We love that this paper takes a holistic view of recommendation quality that extends beyond predictive accuracy, identifies specific reasons for the bias toward warm established items, and offers very practical techniques to address each one. The adjustments are to the training data, model architecture, and loss function—all applied at training time—which avoids any additional serving complexity. They are also relatively straightforward to implement, without any new hyper-parameters that would require additional tuning.
1. Cold items are under-represented in logged data
By definition, cold items have few logged interactions, so training losses and gradients are naturally biased toward warm items. Rather than directly addressing this disparity (e.g. through data augmentation for cold items), the authors frame the problem as generalizing from warm to cold items and seek to address it by applying manifold mixup. This method linearly interpolates training examples and labels within each training batch, generating new training examples that are semantic blends of existing examples.
The authors suggest that this improves generalization by discouraging the model from learning trivial patterns using few embedding space dimensions. Since one of the most trivial patterns is prevalence or popularity in the training data, we can kind of squint and see how this might help with cold-start—and indeed, they show that there are practical benefits, even if it's not fully clear why it works.
2. Cold items lack the historical features that models heavily rely on
For items with sufficient user-engagement history, we'd expect that historical engagement features be more predictive than non-historical (e.g. content-based) features. Cold items lack that user-engagement history, so we'd like our model to use the non-historical features make plausible predictions for cold items to help them gain exposure. However, because cold items are under-represented in the training data, models aren't really discouraged from learning to emphasize historical features over everything else and neglecting non-historical features, which hurts predictions for cold items. Learning this shortcut is expedient for boosting overall predictive accuracy, but contributes to the problem of cold items being under-recommended.
The authors address this by allowing non-historical features to bypass their model's feature interaction layer and contribute directly to downstream predictions. They show that this encourages the model to put more weight on such features, benefiting cold-start items that don't have much engagement history yet.
3. Cold items tend to receive lower scores than warm items
Warm items have more user interactions (by definition), which means they appear more frequently as positive examples in training data. This makes it understandable that models tend to learn that warm items deserve higher relevance scores.
That pattern isn't what we want though, and the authors address this issue by adding a term to the loss function to directly regularize the gap between warm and cold item scores. They point out that this approach has significant practical advantages over previously studied methods, because it adds a minimal number of model parameters (<5%) and doesn't substantially increase either training time or serving latency, which makes it suitable for large-scale industrial recommenders.
By combining these three techniques, Pinterest was able to improve the offline hit rate for fresh cold-start items at only a minimal cost to the hit rate for all items. In an online A/B test, they found this increased engagement with lower-ranked items, which they attribute to improved generalization and avoidance of learning dominant patterns from the head of the item popularity distribution. More surprisingly, they also found that this increased engagement from less active, cold-start users. It's not completely clear why, but we speculate that fresher recommendations might be more engaging for new users.
🗃️ Curated Curiosities
Blogs, books, and broadcasts from nearby realms
In academic work, you have well-established datasets like MovieLens and just develop an algorithm. It beats SOTA—you are happy! In practice, in the business, the journey starts with this.
—Sasha Fedintsev
The latest episode of the RecSperts podcast includes a great discussion (25:50 - 1:37:57) of practical challenges in a real-world recommendation domain that aren't typically covered by the research literature. Among other things, they discuss the advantages and challenges of more complex interfaces beyond a single list, reasons you shouldn't naively use logged data for offline evaluation, and how to avoid issues with BPR loss.
A user’s interests are diverse – cooking, fitness, travel – but a single static embedding blends them into one indistinguishable vector. When ranking a ski jacket, the model has no way to amplify winter sports signals and suppress cooking signals. Every candidate faces the same blurred snapshot.
—Maggie Zhuang
RS_c has a new series on the role of self-attention in recommender systems and how that differs its role in language modeling, leading to different scaling challenges and solutions. In recommender systems, self-attention helps us reflect multiple interests that shift over time, with more recent interactions providing more information on their own and less recent interactions providing information mostly in aggregate. This allows us to apply some tricks to scale attention to longer sequences that wouldn't work for language modeling.
As a data primitive, user sequences are powerful. They capture temporal behavior instead of just aggregates like "how many clicks" over a period. They enable sequence‑aware models such as Transformers, sequence encoders, or attention‑over‑history architectures. And because they preserve fairly raw behavior, they can be reused across ranking, retrieval, exploration, anomaly detection, and other workloads. The catch is that a high‑quality sequence is not just "the N latest events from a log table."
—Pinterest Engineering
Providing sequences of user interactions for model training and serving is harder than it sounds, because there's a lot more to interactions than just the item ids! Along the way interactions can be enriched with additional information like counts, embeddings, and metadata. In a recent post on their engineering blog, Pinterest walks through some of the design decisions they made in reworking their user sequence platform that allow them to define new events and signals once and use them everywhere, avoiding "split-brain" training-serving skew.
At a high level, serendipity scoring works like this: A candidate title and each item in a member’s viewing history are represented as embeddings in a vector space. For each candidate, we compute its similarity against the history embeddings, find the maximum similarity, and convert that into a "novelty" score. That score becomes an input feature to the downstream recommendation logic.
—Harshad Sane
Sometimes features used by recommendation models and logic that seem trivial offline turn out to be surprisingly difficult to compute efficiently at serving time. In Optimizing Recommendation Systems with JDK’s Vector API , Netflix walks through some of the steps they took to optimize the vector computations involved in producing serendipity scores for candidate items.
🔮 Portents & Possibilities
A glimpse into what we're gathering for the next issue
We look forward to seeing you next month, when we'll take a look at the KDD conference proceedings (among other things). We have some code, papers, and experiments brewing, and we hope to have more to share then too. If you found this newsletter useful, subscribe to get future issues and consider leaving a tip to support our work. Thanks for reading!