At some point in our lives, all of us likely learned the art of managing expectations: In certain situations, it’s best to set the bar low so that you can easily surpass it when needed. Netflix has mastered this practice when it comes to data transparency, and on Tuesday, the company dropped its magnum opus—a collection of data so massive that you might just miss how utterly useless it actually is.
From trade publications like Deadline to conversations on Reddit, the release—titled “What We Watched: A Netflix Engagement Report”—has been dubbed a “data dump,” It’s a fitting label, given both the amount of information provided and its utter lack of organization.
The spreadsheet includes more than 18,000 titles, listed in descending order from most hours watched to least. Any title that brought in more than 50,000 viewing hours during the spreadsheet’s six-month sample, January through June 2020, reportedly made the cut. Netflix lists seasons of its television shows separately, and although the spreadsheet notes globally available releases with a “Yes” or “No” column, the titles themselves are all mixed together. There is no way to measure viewership solely within the U.S. or another country, and the spreadsheet’s narrow time sample renders any kind of meaningful data analysis even more impossible.
ADVERTISEMENT
In other words, this release is great if you’ve been dying to know just how many hours people spent watching, say, Ginny & Georgia Season 1, but it’s less useful if you actually want to use it to learn anything about how viewership drives Netflix’s programming decisions—which has been a key factor in writers’ and actors’ calls for transparency.
During the WGA and SAG-AFTRA strikes that dominated Hollywood headlines this summer, the unions both emphasized streamers’ opaque approach to data sharing. The unions both ultimately won streaming bonuses for highly successful series in their new contracts: Now, shows that manage to attract 20 percent of any streaming platform’s subscriber base in their first 90 days will earn writers a 50 percent bonus over their standard residual. Actors, meanwhile, will receive a 100 percent bonus—75 percent of which will go to the actors, and 25 percent of which will pour into a fund run by both their employers and the actors union. The information on this chart will not help clarify which shows might actually meet that threshold.
During a conference call with reporters, Netflix co-CEO Ted Sarandos reportedly said that in the company’s early days, “It wasn’t really in our interest to be that transparent because we were building a new business. We needed room to learn, and we also didn’t want to provide roadmaps to future competitors.”
At the same time, Sarandos granted, “The unintended consequence of not having more transparent data about our engagement was it created an atmosphere of mistrust over time with producers and creators and the press about what was happening on Netflix.” It was that dynamic, he said, that led the company to introduce its top-10 list and, now, this new initiative. But while this does place Netflix a little farther up on the “continuum of transparency,” as Sarandos put it, the information shared on Tuesday largely feels like a mostly empty PR effort.
As a metric, “Hours Viewed” feels strategically imprecise. Films are naturally shorter than television shows, and episode length varies across series, so this spreadsheet provides no equivalent points of comparison between productions. By limiting its data set to six months, Netflix also makes it harder to inspect how viewership affects a show’s longevity.
Are long-running series really bringing in that many more viewers than those canceled after one season? With only a six-month sample to draw from and viewership tracked by the hour and not per-episode, it’s very hard to tell! Then again, I’m sure we’re all relieved to finally know that XO, Kitty Season 1 racked up a collective 200,700,000 hours of people’s time several months ago.
As Sarandos acknowledged, Netflix has historically been less than eager to share its viewership data with the public. When the company does dole out information, it does so selectively—and unlike traditional television networks, which all often engage in similar smoke-and-mirror tactics when touting their successes, there aren’t as many third-party arbiters capable of fact-checking Netflix’s claims. (Nielsen, which measures traditional TV viewership, can only measure U.S.-based figures for Netflix, and it cannot track viewing habits on phones or computers—which naturally limits its streaming insights.)
It was this disparity in measurement that led FX Networks chief John Landgraf to criticize Netflix in 2019 for cherry-picking data and for counting viewers who had watched 70 percent of a series as having watched the whole show. A year later, Netflix lowered the bar even further, claiming in an earnings report that 76 million households watched The Witcher in its first four weeks—a figure based on anyone who had watched the show for at least two minutes. Two minutes!
All of this is by design, and although Netflix is certainly not alone in using carefully selected figures as PR, the company does seem to have thoroughly embraced the practice.
During a recent Department of Justice review of the Paramount Consent Decrees—a set of regulations that dictate the relationships between studios and theater chains—The Hollywood Reporter reports that Michigan State University law professor Adam Candeub told the antitrust office that Netflix “does everything in its power to prevent third parties from learning its viewing data. Netflix encrypts its data to prevent ISPs and web browsers from tracking the use, and it does not share any data with third parties, even the studios whose material it licenses and who naturally want information on [their] own show[s].
“While Netflix has a right to its own data,” Candeub reportedly continued, “…Netflix goes a step too far by using its market power in the [online video] market to require that connected devices not use available data.”
It’s true that with this new swath of data, Netflix is giving the public more insight than ever. At the same time, it feels disingenuous for the company to say, as Sarandos did, that “This is the data we use to run the business.” Although I’ve never set foot in Netflix HQ or worked alongside its analytics department, something tells me they’ve got this information broken down a lot more precisely, and that they usually examine more than just a six-month window at a time. To say that this is the data Netflix uses to run its business feels akin to a Michelin-starred restaurant leading us all to a field of grazing livestock and proudly proclaiming, “This is how our sausage gets made.”
Nevertheless, this is progress—even if only because Netflix has generally kept the bar somewhere musty beneath our feet. The company is set to release updated figures every six months, and over time, that should allow those of us with some spreadsheet skills at least a small window into its data basement. For now, however, it’s worth remembering that when a tech company promises transparency, it’s usually prudent to check that window for holographic film. (Speaking of which, I’m sure you’ll be delighted to know that the 2015 film Jem and the Holograms netted 100,000 hours of viewing this year—whatever that means.)
