Content Moderation At Scale Is Impossible: Google Play Bans Video Player App Over ASS File Extension Support

Too Much Free Time

from the pain-in-the-[removed] dept

As you should know by now as readers here, content moderation at scale is impossible to do well. Examples for how and why this is so are extensive on these pages, but the crux of the matter is that scaling moderation for content across huge platforms and a variety of avenues in a way that everyone both agrees is right and that doesn’t create false-positives is, well, self-evidently impossible. Not everyone agrees what should be moderated, for starters, nor does anyone trust these platforms to actually get it right. Meanwhile, some massive amount of the public does agree that these platforms should be doing something.

And that’s how you get big platforms trying to automate content moderation in a way that makes everyone look incredibly dumb. Yet another example of this is the Google Play store banning a video player application over “sexual content and profanity”, but just not for the reasons you’re probably already imagining.

This time, the puritan robot overlords that run the Play Store briefly decided that listing support for common subtitle files is enough to get your app banned. The developer for Just (Video) Player posted their story to Hacker News, writing in the app’s bug tracker, “After a tiny unrelated description update, Just Player got suspended from the Google Play Store for “Sexual Content and Profanity policy”. Google finds issues with following: Full description (en_US): “* Subtitles: SRT, SSA, ASS, TTML, VTT.””

Yes, just listing standard video player features like support for the “ASS” subtitle format was apparently enough to temporarily earn a suspension. The developer says they “immediately filed an appeal” and today, the app is back up with the ASS subtitle listing still in the description.

While I admit to appreciating that all of this happened so as to create the situation where a publication writes “with the ASS subtitle listing still in the description”, this story is in every other way indicative as to why broad, automated moderation is impossible to do well. It also lays bare the fact that there is clearly zero human review or intervention in this process. Instead, the filter that looks for naughty words and content detects “ASS”, can’t consider the context for why that word or acronym might appear in the description, and then just outright banned the app. Sure, Google reinstated it… but all that shows is that the human intervention that occurs after the fact can and should occur before the ban.

And, while I’m typically loathe to throw a ton of props Apple’s way, it’s worth noting that Apple’s app review process actually does what Google’s doesn’t.

Saying Google’s app review bots “ban first and ask questions later” would actually be an improvement over the current situation, since the bots can’t ask questions. The bots ban, send an automated email, and it’s up to the developers to figure out why they were banned and jump through hoops to make the bots happy, often without being able to speak to a human.

Google and Apple both collect a percentage of app sales, which the companies categorize as a necessary tax that pays for the infrastructure of the store ecosystem. Apple uses this money partly to hire an army of human app reviewers, a system that Google Play developers often hold up as an example that Google should follow. Instead, Google only has this bot system—or at best, it has an extremely small team of manual reviewers—and developers frequently complain that they are at the mercy of an illogical bot, with no human to speak to even during an appeals process.

Which moves this beyond mere annoyance and into a competitive issues between the two major app stores in the mobile market. Perhaps Google is so big that it can ignore this clear deficiency in its platform moderation… but I doubt it. The more these stories crop up, the more developers will simply decide Google isn’t worth the trouble.

Filed Under: ass, content moderation, google play store, keyword filters, sexual content, video player
Companies: google