As relevance witches, we're well-versed in distilling the essence of positive expressions of interest: clicks, listens, candle-lit invocations, etc. But what about disinterest—what do we do when our users tell us what they don't want? If tomes with those spells exist, they're rare or hidden away in secret archives.
Trying to incorporate negative feedback into an existing recommender creates a bit of a conundrum. We need users to interact with negative feedback controls (e.g. 👎) and generate data that we could use to train a model, but they have little reason to use the controls unless there's a noticeable impact on their recommendations. We have to demonstrate that our negative feedback system is effective so that we can collect the data we need to actually make it so! As tempting as it might be, employing paradox magic here would be dangerous and irresponsible, so what other options do we have?
If we're using a model that represents user preferences as a sequence or set of actions (e.g. clicks, plays), rather than a learned user embedding, we do have some options. One of the many advantages of such models is that they offer more flexibility to shape the outputs. Usually, we try to avoid model features that skew differently between training and serving, to avoid predictive error. However, recommendation isn't really a prediction problem, it's just convenient to frame it that way sometimes. What actually matters is producing useful, relevant recommendations (as evaluated by real humans, not an abstract summary statistic), so manipulating serving-time inputs to mimic the differently-sourced training-time features is absolutely valid if it furthers that goal.
Embracing the old adage that "perfect" can often be the enemy of "good", here are three simple charms to create a functional negative feedback system from existing recommender system components.
1. Use explicit negative feedback to cancel out implicit positive feedback
Implicit-feedback-based recommenders often consider any engagement to be indicative of interest or enjoyment. This is clearly a fairly noisy signal—we've all spent attention on something and later wished to have that time back, right? So, we can help the machine to appreciate the meaning of regret with a simple heuristic: remove or mask interactions from a user's history if they gave negative feedback on those items.
"Canceling out" user engagements offers only rudimentary responsiveness to negative feedback controls, but it's a start. Even after you've collected enough negative feedback data to train an actual model on it, overriding an implicit interest signal with explicit disinterest still makes sense.
2. Estimate irrelevance by turning your existing relevance estimator upside-down
Somewhere in your existing recommender system, you have a model for divining which items are most relevant based on what items a user has previously enjoyed. If that model takes (implicitly positive) interacted items as input features, consider feeding it explicit negatives instead to produce an ir-relevance score. Subtract estimated irrelevance from estimated relevance, and use this net relevance score in place of the raw relevance score in downstream ranking logic.
Once you've collected enough negative feedback data for training, you can consider trying to incorporate irrelevance estimation directly into the model architecture. We wish we had a tried-and-true recipe to recommend for that, but you might take some inspiration from Wu, et al's approach to news recommendation with (implicit) negative feedback.
3. Piggy-back on weaker signals of disinterest
In some recommendation domains, there are user actions—such as abandons or skips—that can be reasonably treated as implicit signals of disinterest. In such cases, you can use that implicit disinterest to help the machine build up a tolerance for negativity before introducing explicit negative controls. Then, you can make your system responsive to explicit negatives right out of the gate, simply by treating them as stronger versions of the implicit negatives.
Prepare your system for this hex by including these negative item interactions as negatively-labeled training examples, with the negative feedback actions encoded among the model input features. We like Google's approach of adding an independent "not to recommend" loss function term for negative labels, which permits the impact of positive and negative labels to be tuned separately. This is not an insignificant change, but since it relies solely on implicit signals, it should be doable with existing data.
Then, at serving time, you can feed explicit negative feedback into the same model input features that were used for implicit negative feedback data during training. There's a few options for doing so, with the simplest being to just treat explicit negatives as if they were implicit negatives. Since we'd like explicit negative feedback to have a stronger effect though, you could try weighting explicit negatives more heavily (e.g. maybe a thumbs-down is twice as strong as a skip?), or converting an explicit negative into multiple implicit negative actions (e.g. maybe a thumbs-down turns into 2 skips?). Just be sure to actually look at recommendation outputs to make sure the resulting behavior is reasonable, because we're deliberately going outside the range of inputs the model saw in training.
This hex not only allows your system to be responsive to explicit negative feedback prior to data-collection, it also sets you up to leverage more-plentiful implicit negatives to get more out of the sparser-but-stronger explicit negatives down the line. As more data becomes available and your model develops a thicker skin, you can blend the explicit negatives back into your model as training examples with strong negative labels (e.g. with larger label weights). You can use the same feature manipulation that you've been using at serving time to represent the explicit negatives in training data, or you can add a new feature for them and update the serving-time featurization accordingly.
Once you've cobbled together a workable negative feedback mechanism and collected enough data, then it becomes straightforward to apply known techniques! the next steps involve wading into murky waters. Even the papers linked above offer only partial answers that may require substantial adaptation to fit your context. If you have any further tips, tricks, or references on this topic—maybe even one of those rare tomes—we'd love to hear about them!
What we can say with confidence is that the above hexes can help you reach that starting point for developing a more sophisticated solution, by unblocking the collection of a negative feedback dataset for training. You'll also have a functional negative feedback system running in production while you explore how to actually make use of that dataset.