The Long Tail Is Only As Good As The Recommendation System

from the that's-how-it's-supposed-to-work dept

It's been amusing to watch folks like Andrew Orlowski continue to misinterpret Chris Anderson's slight admission that things in "the long tail" aren't exactly they way he'd predicted them to be. Of course, Orlowski entirely misses the point by assuming incorrectly (as many others have done) that the discussion of the long tail meant the death of "the blockbuster." That's not at all true. The idea of the long tail was that it both enabled more content to be produced by opening up more shelf space and then made it easier to find that content.

But, the fact remains that the finding of that content is entirely dependent on the filtering and recommendation systems, which is highlighted in the recent NY Times piece by Clive Thompson about attempts to improve Netflix's recommendation engine (and, yes, this is the second post I've written on that article, but this is discussing an entirely different issue than the first, so it seemed worthwhile). In the article, Thompson notes:
Cinematch has, in fact, become a video-store roboclerk: its suggestions now drive a surprising 60 percent of Netflix’s rentals. It also often steers a customer’s attention away from big-grossing hits toward smaller, independent movies. Traditional video stores depend on hits; just-out-of-the-theaters blockbusters account for 80 percent of what they rent. At Netflix, by contrast, 70 percent of what it sends out is from the backlist — older movies or small, independent ones. A good recommendation system, in other words, does not merely help people find new stuff. As Netflix has discovered, it also spurs them to consume more stuff.
Basically, that entire paragraph explains the issue. A good recommendation system does two things: it gets people to consumer more -- and it introduces them to stuff they might not have heard about otherwise. But, that second part is not necessarily the same as the first part. Many people assumed, incorrectly, that the greatness of such "long tail filters" was that it would drive people to consumer more down the tail -- but as Netflix is seeing, the good recommendation engine drives people to consume more content in both the head and the tail.

And, when you think about it, that makes an awful lot of sense. Popular stuff often is popular for a reason. While some may disagree, things are often popular because they really do appeal to a lot of people, so it should be no surprise that a good recommendation system would increase consumption in the head: it's accurately noting that an awful lot of people will like that content. But that doesn't exclude promoting some of the content from the tail. Since the recommendation system is driving more consumption overall, it's "lifting all boats" as they say, even if (as is likely) it lifts the boats in the head more than in the tail. In the past, that content in the tail wouldn't get any business at all, but these days it can at least make some money, if not a huge amount.

So, no the concept of the long tail is hardly dead or even in trouble (or, as Orlowski notes, downgraded). Instead, it's just being understood better.
Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: big head, long tail, recommendations

Reader Comments

Subscribe: RSS

View by: Thread

  1. identicon
    dieselm, 26 Nov 2008 @ 11:01am

    Remember also that Netflix has an economic bias in favor of steering customers to backlist titles which should be reflected in the design of the algorithm.

    The demand a DVDs in the first six weeks of release is huge and then falls off a cliff. Anything that steers people away from new releases in favor of old titles with similar satisfaction reduces costs dramatically.

    reply to this | link to this | view in thread ]

  2. identicon
    Anonymous Coward, 26 Nov 2008 @ 12:27pm


    How about changing the noun "consumer" to the verb "consume" where used as a verb and then delete this comment.


    reply to this | link to this | view in thread ]

  3. identicon
    Rotkapchen, 26 Nov 2008 @ 12:30pm

    Hardly a Deathknell

    No, Blockbuster's demise is clearly totally attributable to clueless management right at the top. First-hand account: fired all the designers -- wanted to redouble effort on store design using classic (read: outdated) retail models.

    reply to this | link to this | view in thread ]

  4. identicon
    Kiba, 26 Nov 2008 @ 12:50pm


    I would think that distribution of DVD would be subject to economic of scale. I think distributing a thousand DVD of the same movies is a lot easier than picking out the correct movie DVD from the correct shelf.

    So I think economic bias would move toward big hits rather than items in the blacklist.

    reply to this | link to this | view in thread ]

  5. identicon
    hautedawg, 26 Nov 2008 @ 1:12pm

    Blockbuster's demise

    Blockbuster is dying a painful and slow death beecause they simply do not care about the consumer. They don't give a rat's patootie if what you are looking for is in stock or not, they simply carry the newest big name releases, and hold on to them for several months after the release. Then they sell them for a couple of bucks, restocking with the next big movie. If you are looking for a classic, there is a slight chance they will have it, but not a strong one.

    When there was actual competition for your rental money, the stores had to carry a wider variety of movies, including movies that were 10 years old. Now that the small stores have been run out of town, Blockbuster does not carry movies that are more than a year or two old, or if they do, they only have one or two copies.

    In no way does this serve the consumer, unless you are only wanting to rent new movies, or recent releases. If you want a "classic" or an older movie, you need to rent it online from Blockbuster or NetFlix.

    In my way of thinking, the only person that is being served by this is Lackluster and certainly NOT the consumer. We are not being exposed to movies that are small release or movies that have been around for 3 or 4 years.

    Happy Holidaze!

    reply to this | link to this | view in thread ]

  6. identicon
    Michael Long, 26 Nov 2008 @ 1:56pm

    More than recommendations...

    Actually, there's three legs to the making the tail work: browsing, search, and recommendations.

    You discussed recommendations. Search is self explanatory. But browsing is often overlooked, when in fact it's a core component of the long tail, dividing a single long tail into multiple long tails.

    Take NetFlix's "Thrillers" category, which in itself is a smaller long tail with the same most popular/least popular slope as the orginal tail. "Pyschological Thrillers", in turn, is again a smaller yet identical tail. Yet its smaller size makes it much more manageable, relevant, and easier to browse and digest.

    Coming up with the right browse lists (genres, by year, by actor, by director, by awards, and so on) is another powerful way to maximize visibility into the tail.

    reply to this | link to this | view in thread ]

  7. identicon
    bowerbird, 26 Nov 2008 @ 3:37pm

    people have finite time and attention,
    so it's not possible to "raise all boats"
    forever and across all of the mediums.

    if i'm watching more films, then i must
    be reading fewer books, or attending
    fewer plays, or playing fewer games...

    what _is_ being well-served, though, is
    my level of _satisfaction_experienced_.

    instead of wasting my time watching a
    crummy movie, i'm watching one i like,
    regardless if it's in the head or the tail.

    and that's a good thing.

    the movie business used to be able to
    churn out crap, and we'd still watch it,
    because it was the only thing out there.
    now, they have to compete against all
    of the classics of the past, and the niche
    product that's now right at our fingertips.

    all of this is a good thing...


    reply to this | link to this | view in thread ]

  8. identicon
    a4105ii, 27 Nov 2008 @ 1:29pm

    Think of the grandpa effect What NEW film song or whatever does some 60something boomer who just downloaded White Rabbit & Electra Glide in Blue want is there an algorythm to do this It clearly
    works both ends

    reply to this | link to this | view in thread ]

  9. identicon
    Anonymous Coward, 29 Nov 2008 @ 1:44pm

    Maznick entirely misses the point by assuming incorrectly (as only he can) that when others discus the long tail they also assume the death of "the blockbuster".
    That's not at all true.
    But it does mean that reading Maznick's articles always leaves you feeling you've just wasted 5 minutes of your life.

    reply to this | link to this | view in thread ]

  10. icon
    Mike (profile), 1 Dec 2008 @ 1:50am


    That's not at all true.

    Then explain why so many have done so.

    reply to this | link to this | view in thread ]

  11. identicon
    Anonymous Coward, 1 Dec 2008 @ 10:52am

    Re: Re:

    you haven't quoted any examples yet and it's your blog

    reply to this | link to this | view in thread ]

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)


Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat

Warning: include(/home/beta6/deploy/itasca_20201215-3691-c395/includes/right_column/ failed to open stream: No such file or directory in /home/beta6/deploy/itasca_20201215-3691-c395/includes/right_column/ on line 8

Warning: include(): Failed opening '/home/beta6/deploy/itasca_20201215-3691-c395/includes/right_column/' for inclusion (include_path='.:/usr/share/pear:/home/beta6/deploy/itasca_20201215-3691-c395:/home/beta6/deploy/itasca_20201215-3691-c395/..') in /home/beta6/deploy/itasca_20201215-3691-c395/includes/right_column/ on line 8
Recent Stories

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it

Email This

This feature is only available to registered users. Register or sign in to use it.