Limitations of Implicit User Feedback
A degenerate feedback loop forms when a model's predictions shape the feedback it later learns from, amplifying whatever biases were present at the start.
A Short Video Ranking That Keeps Climbing
Imagine a team building a recommendation system that suggests short videos. On the first day, two videos sit close together in the rankings and are almost equally good as far as anyone can tell. One of them happens to appear a little higher in the list. Users who see it click on it more often, and the system reads those clicks as evidence that the video deserves its high position. The next version of the system places it even higher. Nothing about the videos changed between those two days, yet the system's picture of them has drifted far apart.
This pattern is what the term degenerate feedback loop describes. In this article you will see how the loop forms, how to trace it step by step, which named outcomes it produces, and why teams that depend on user feedback need to watch for it. Before reading on, try the question below.
What do you think happens?
Two videos start with nearly equal quality, and one is ranked slightly higher. After several rounds of recommending, collecting clicks, and retraining, what happens to the gap between their rankings?
Reveal answer
The small early gap grows into a large gap in ranking.
Because the slightly higher video is shown first, it collects more clicks, and the system reads those clicks as a reason to rank it even higher. Each round widens the difference, even though the videos themselves have not changed.
Why Missing Feedback Hides the Loop
The starting point is a limitation that is easy to forget: user feedback is incomplete. A system only learns how people react to the things it actually displays. If an item never appears on a screen, no one can click it, rate it, or ignore it, so the system collects no data about it at all. Feedback therefore describes the system's own choices as much as it describes users' tastes.
This matters for detection. Items that are rarely displayed generate almost no data, so the loop is hard to see from the feedback alone. The unobserved part of the data is silent. A team looking only at the clicks the system has recorded sees a consistent picture, because the picture was built from exactly the items the system chose to show.
How a Feedback Loop Forms
Normally, a model's behavior changes in response to feedback, and that is the point of collecting it. The trouble begins when the model's predictions themselves shape what users see and therefore what feedback comes back. That feedback then goes into the next round of training, so the model is partly learning from its own earlier output. When this happens, the result is called a degenerate feedback loop.
Read this diagram as a closed circle rather than a straight pipeline. The arrow from retraining back to the model is what makes it a loop. Without that arrow, you would have a one-way process that could be audited step by step. With it, a bias introduced at any stage can travel around the circle and return stronger.
Tracing Two Videos Over Several Rounds
Following Video A and Video B
Two videos, A and B, have initially close quality. A is ranked slightly higher. Trace how their ranking gap changes across rounds of recommending, collecting feedback, and retraining.
- 1Start
Both videos are almost equally good as far as anyone can tell. A sits slightly higher in the list, so the starting gap is tiny.
- 2Round one: display
Because A is ranked higher, it is shown first and is seen by more users than B.
- 3Round one: feedback
A collects more clicks. Users cannot click on B as often because it is shown less, so B produces less feedback.
- 4Round one: retraining
The system reads the extra clicks on A as evidence that A deserves its high position, and retrains. A is now placed even higher.
- 5Repeat over several rounds
Each round repeats the same cycle. The gap that results from the loop becomes large, even though the early difference was small.
A tiny starting difference in ranking grows into a large difference after several rounds, and nothing about the videos themselves changed.
Named Outcomes of the Loop
The source groups the outcomes of degenerate feedback loops under three labels: exposure bias, popularity bias, and filter bubbles. Each label names a way the loop can show up in practice. The source treats them as related consequences of one mechanism rather than as three separate mechanisms, and they overlap. The concept map below shows how each one connects back to the same loop.
Beyond Rankings: Products and Conversational Agents
Imagine a small group of users who say they like cat photos. The system notices and starts generating more photos with cats. That draws in more cat lovers, who give more positive feedback about cat photos, which encourages the system to generate even more of them. Eventually the application becomes what the source calls a cat haven.
The cat example is deliberately harmless. The same mechanism, according to the source, can amplify other biases, including racism, sexism, and a preference for explicit content. The point is not that cats are the danger. The loop does not check whether the direction it is amplifying is desirable.
A loop can also affect the substance of what a system says. The source notes that acting on user feedback can turn a conversational agent into, as the source puts it, a liar. Several studies cited in the source show that training a model on user feedback can teach it to give users what it believes they want, even when that is not the most accurate or beneficial answer. A related study cited in the source reports that AI models trained on human feedback tend toward sycophancy, meaning they are more likely to present responses that match the view of the user they are talking to.
Mistakes When Reading Feedback Data
Common mistake Treating recorded clicks as a direct measure of quality
What it looks likeA team sees that Video A has the most clicks and concludes that A is the best video in the catalog.
Why it failsClicks only exist for items that were shown. A was placed first, so it had more chances to be clicked, and the loop cannot separate earned ranking from placement.
Do this insteadAsk how each item was displayed before interpreting its feedback, and compare items that received similar exposure.
Common mistake Assuming missing feedback means an item is unpopular
What it looks likeA product with no clicks is judged to be unwanted and is dropped from the catalog.
Why it failsItems that are never displayed generate no data at all. Silence reflects the system's display decisions as much as users' interest.
Do this insteadCheck whether the item was ever shown before treating its lack of feedback as evidence about users.
Common mistake Thinking the loop is only a problem for rankings
What it looks likeA team checks only the ordering of results and overlooks that the product's focus is narrowing over time.
Why it failsThe source describes loops that alter a product's focus and an agent's answers, not only its rankings.
Do this insteadMonitor what the product offers and how it responds, not just where items sit in a list.
Common mistake Assuming a loop will be obvious from the feedback alone
Tracing a Three-Book Shelf
A bookstore app ranks three books for a shelf. Book X and Book Y are almost equally good, and Book X starts slightly higher. Book Z is clearly weaker. Describe how the loop would likely widen the gap between X and Y over several rounds of showing, collecting clicks, and retraining. Then explain why the app's click data alone cannot tell you whether X was actually the better book.
Hints
- Start by asking which book gets displayed first, and what that does to its clicks.
- Think about whether Book Z's clicks are the same kind of evidence as X's or Y's, given how often each is shown.
- Recall the callout: the loop treats placement and merit as producing the same clicks.
Key Takeaways
- A degenerate feedback loop forms when a model's predictions shape the feedback it later learns from, amplifying whatever biases were present at the start.
- Because feedback only reflects what is shown, small early differences such as a slightly higher ranking can grow into large gaps over repeated rounds of retraining.
- Exposure bias, popularity bias, and filter bubbles are named outcomes of the same loop, and the same mechanism can amplify other harmful biases.
- Items that are never displayed generate no data, so the loop is hard to detect from feedback alone and must be checked by examining what was shown.