Content Moderation At Scale Is Impossible: Google Play Bans Video Player App Over ASS File Extension Support

from the pain-in-the-[removed] dept

As you should know by now as readers here, content moderation at scale is impossible to do well. Examples for how and why this is so are extensive on these pages, but the crux of the matter is that scaling moderation for content across huge platforms and a variety of avenues in a way that everyone both agrees is right and that doesn't create false-positives is, well, self-evidently impossible. Not everyone agrees what should be moderated, for starters, nor does anyone trust these platforms to actually get it right. Meanwhile, some massive amount of the public does agree that these platforms should be doing something.

And that's how you get big platforms trying to automate content moderation in a way that makes everyone look incredibly dumb. Yet another example of this is the Google Play store banning a video player application over "sexual content and profanity", but just not for the reasons you're probably already imagining.

This time, the puritan robot overlords that run the Play Store briefly decided that listing support for common subtitle files is enough to get your app banned. The developer for Just (Video) Player posted their story to Hacker News, writing in the app's bug tracker, "After a tiny unrelated description update, Just Player got suspended from the Google Play Store for "Sexual Content and Profanity policy". Google finds issues with following: Full description (en_US): “* Subtitles: SRT, SSA, ASS, TTML, VTT.”"

Yes, just listing standard video player features like support for the "ASS" subtitle format was apparently enough to temporarily earn a suspension. The developer says they "immediately filed an appeal" and today, the app is back up with the ASS subtitle listing still in the description.

While I admit to appreciating that all of this happened so as to create the situation where a publication writes "with the ASS subtitle listing still in the description", this story is in every other way indicative as to why broad, automated moderation is impossible to do well. It also lays bare the fact that there is clearly zero human review or intervention in this process. Instead, the filter that looks for naughty words and content detects "ASS", can't consider the context for why that word or acronym might appear in the description, and then just outright banned the app. Sure, Google reinstated it... but all that shows is that the human intervention that occurs after the fact can and should occur before the ban.

And, while I'm typically loathe to throw a ton of props Apple's way, it's worth noting that Apple's app review process actually does what Google's doesn't.

Saying Google's app review bots "ban first and ask questions later" would actually be an improvement over the current situation, since the bots can't ask questions. The bots ban, send an automated email, and it's up to the developers to figure out why they were banned and jump through hoops to make the bots happy, often without being able to speak to a human.

Google and Apple both collect a percentage of app sales, which the companies categorize as a necessary tax that pays for the infrastructure of the store ecosystem. Apple uses this money partly to hire an army of human app reviewers, a system that Google Play developers often hold up as an example that Google should follow. Instead, Google only has this bot system—or at best, it has an extremely small team of manual reviewers—and developers frequently complain that they are at the mercy of an illogical bot, with no human to speak to even during an appeals process.

Which moves this beyond mere annoyance and into a competitive issues between the two major app stores in the mobile market. Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation... but I doubt it. The more these stories crop up, the more developers will simply decide Google isn't worth the trouble.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: ass, content moderation, google play store, keyword filters, sexual content, video player
Companies: google

Reader Comments

Subscribe: RSS

View by: Thread

  1. identicon
    Anonymous Coward, 31 Jan 2021 @ 10:15pm

    Re: Advantages and disadvantages

    I was just about to say that relying solely on human review causes delays in apps (and also app updates) going live. You're also at the mercy of how the reviewer interprets the App Store rules (and, as noted, human moderators are unable to agree on everything).

Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here

Subscribe to the Techdirt Daily newsletter

Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Insider Shop - Show Your Support!

Report this ad  |  Hide Techdirt ads
Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Report this ad  |  Hide Techdirt ads
Recent Stories
Report this ad  |  Hide Techdirt ads

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it

Email This

This feature is only available to registered users. Register or sign in to use it.