Section 230 Isn't Why Omegle Has Awful Content, And Getting Rid Of 230 Won't Change That

Content Moderation

from the one-size-does-not-fit-all dept

Last year, I co-authored an article with my law school advisor, Prof. Eric Goldman, titled “Why Can’t Internet Companies Stop Awful Content?” In our article, we concluded that the Internet is just a mirror of our society. Unsurprisingly, anti-social behavior exists online just as it does offline. Perhaps though, the mirror analogy doesn’t go far enough. Rather, the Internet is more like a magnifying glass, constantly refocusing our attention on all the horrible aspects of the human condition.

Omegle, the talk-to-random-strangers precursor to Chatroulette, might be that magnifying glass, intensifying our urge to do something about awful content.

Unfortunately, in our quest for a solution, we often skip a step, jumping to Section 230—the law that shields websites from liability for third-party content—instead of thinking carefully about the scalable, improvable, and measurable strides to be made through effective content moderation efforts.

Smaller companies make for excellent content moderation case studies, especially relatively edgier companies like Omegle. It’s no surprise that Omegle is making a massive comeback. After 100+ days of quarantine, anything that recreates at least a semblance of interaction with humans, not under the same roof, is absolutely enticing. And that’s just what Omegle offers. For those that are burnt out on monotonous Zoom “coffee chats,” Omegle grants just the right amount of spontaneity and nuanced human connection that we used to enjoy before “social distancing” became a household phrase.

Of course, it also offers a whole lot of dicks.

When I was a teen, Omegle was a sleepover staple. If you’re unfamiliar, Omegle offers two methods of randomly connecting with strangers on the Internet: text or video. Both are self-explanatory. Text mode pairs two anonymous strangers in a chat room whereas video mode pairs two anonymous strangers via their webcams.

Whether you’re on text or video, there’s really no telling what kinds of terrible content—and people—you’ll likely encounter. It’s an inevitable and usual consequence of online anonymity. While the site might satisfy some of our deepest social cravings, it might also expose us to some incredibly unpleasant surprises outside the watered-down and sheltered online experiences provided to us by big tech. Graphic pornography, violent extremism, hate speech, child predators, CSAM, sex trafficking, etc., are all fair game on Omegle; all of which is truly awful content that has always existed in the offline world, now magnified by the unforgiving, unfiltered, use-at-your-own-risk, service.

Of course, like with any site that exposes us to the harsh realities of the offline world, critics are quick to blame Section 230. Efforts to curtail bad behavior online usually start with calls to amend Section 230.

At least to Section 230’s critics, the idea is simple: get rid of Section 230 and the awful content will follow. Their reason, as I understand it, is that websites will then “nerd harder” to eliminate all awful content so they won’t be held liable for it. Some have suggested the same approach for Omegle.

Obvious First Amendment constraints aside (because remember, the First Amendment protects a lot of the “lawful but awful content,” like pornography, that exists on Omegle’s service), what would happen to Omegle if Section 230 were repealed? Rather, what exactly is Omegle supposed to do?

For starters, Section 230 excludes protection for websites that violate federal criminal law. So, Omegle would continue to be on the hook if it started to actively facilitate the transmission of illegal content such as child pornography. No change there.

But per decisions like Herrick v. Grindr, Dyroff v. Ultimate Software, and Roommates.com, it is well understood that Section 230 crucially protects sites like Omegle that merely facilitate user-to-user communication without materially contributing to the unlawfulness of the third-party content. Hence, even though there exists an unfortunate reality where nine year-olds might get paired randomly with sexual predators, Omegle doesn’t encourage or materially contribute to that awful reality. So, Omegle is afforded Section 230 protection.

Without Section 230, Omegle doesn’t have a lot of options as a site dedicated to connecting strangers on the fly. For example, the site doesn’t even have a reporting mechanism like its big tech counterparts. This is probably for two reasons: (1) The content on Omegle is ephemeral so by the time it’s reported, the victim and the perpetrator have likely moved on and the content has disappeared; and (2) it would be virtually impossible for Omegle to issue suspensions because Omegle users don’t have dedicated accounts. In fact the only option Omegle has for repeat offenders is a permanent IP ban. Such an option is usually considered so extreme that it’s reserved for only the most heinous offenders.

There are a few things Omegle could do to reduce their liability in a 230-less world. They might consider requiring users to have dedicated handles. It’s unclear though whether account creation would truly curb the dissemination of awful content anyway. Perhaps Omegle could act on the less heinous offenders, but banned, suspended, or muted users could always just generate new handles. Plus, where social media users risk losing their content, subscribers, and followers, Omegle users realistically have nothing to lose. So, generating a new handle is relatively trivial, leaving Omegle with the nuclear IP ban.

Perhaps Omegle could implement some sort of traditional reporting mechanism. Reporting mechanisms are only effective if the service has the resources to properly respond to and track issues. This means hiring more human moderators to analyze the contextually tricky cases. Additionally, it means hiring more engineers to stand up robust internal tooling to manage reporting queues and to perform some sort of tracking for repeat offenders.

For Omegle, implementing a reporting mechanism might just be doing something to do something. For traditional social media companies, a reporting mechanism ensures that violating content is removed and the content provider is appropriately reprimanded. Neither of those goals are particularly relevant to Omegle’s use case. The only goal a reporting mechanism might accomplish is in helping Omegle track pernicious IP addresses. Omegle could set up an internal tracking system that applies strikes to each IP before the address is sanctioned. But if the pernicious user can just stand up a new IP and continue propagating abuse, the entire purpose of the robust reporting mechanism is moot.

Further, reporting mechanisms are great for victimized users that might seek an immediate sense of catharsis after encountering abusive content. But if the victim’s interaction with the abusive content and user is ephemeral and fleeting, the incentive to report is also debatable.

All of this is to drive home the point that there is no such thing as a one-size-fits-all approach to content moderation. Even something as simple as giving users an option to report might be completely out of scope depending on the company’s size, resources, bandwidth, and objectives.

Another suggestion is that Omegle simply stop allowing children to be paired with sexual predators. This would require Omegle to (1) perform age verification on all of its users with the major trade-off being privacy—not to mention the obvious that it may not even work. Nothing really stops a teen from stealing and uploading their parents’ credit card or license; and (2) require all users to prove they aren’t sexual predators (???)—an impossible (and invasive) task for a tiny Internet company.

Theoretically, Omegle could pre-screen all content and users. Such an approach would require an immense team of human content moderators, which is incredibly expensive for a website that has an estimated annual revenue of less than $1 million and less than 10 employees. Plus, it would completely destroy the service’s entire point. The reason Omegle hasn’t been swallowed up by tech incumbents is because it offers an interesting online experience completely unique from Google, Facebook, and Twitter. Pre-screening might dilute that experience.

Another extreme solution might be to just strip out anonymity entirely and require all users to register all of their identifying information with the service. The obvious trade-off: most users would probably never return.

Clearly, none of these options are productive or realistic for Omegle; all of which are consequences of attacking the awful content problem via Section 230.

Without any amendments to Section 230, Omegle has actually taken a few significant steps to effectively improve their service. For example, Omegle now has an 18+ Adult, “unmoderated section” in which users are first warned about sexual content and required to acknowledge that they’re 18 or older before entering. Additionally, Omegle clarifies that the “regular” video section is monitored and moderated to the best of their abilities. Lastly, Omegle recently included a “College student chat” which verifies students via their .edu addresses. Of course, to use any of Omegle’s features, a user must be 18+ or 13+ with parental permission.

The “unmoderated section” is an ingenious example of a “do better” approach for a service that’s strapped for content moderation options. Omegle’s employees likely know that a primary use case of the service is sex. By partitioning the service, Omegle might drastically cut down on the amount of unsolicited sexual content encountered by both adult and minor users of the regular service, without much interruption to the service’s overall value-add. These experiments in mediating the user to user experience can only improve from here. Thanks to Section 230, websites like Omegle increasingly pursue such experiments to help their users improve too.

But repealing Section 230 leaves sites like Omegle with one option: exit the market.

I’m not allergic to conversations about how the market can self-correct these types of services and whether they should be supported by the market at all. Maybe sites like Omegle—that rely on their users to not be awful to each other as a primary method of content moderation—are not suitable for our modern day online ecosystem.

There’s a valid conversation to be had within technology policy and Trust and Safety circles about websites like Omegle and whether the social good they provide outweigh the harms they might indirectly cater to. Perhaps, sites like Omegle should exit the market. However, that’s a radically different conversation; one that inquires into whether current innovations in content moderation support sites like Omegle, and whether such sites truly have no redeemable qualities worth preserving in the first place. That’s an important conversation; one that shouldn’t involve speculating about Section 230’s adequacy.

Jess Miers is a third-year law student at Santa Clara University School of Law and a Legal Policy Specialist at Google. Her scholarship primarily focuses on Section 230 and content moderation. Opinions are her own and do not represent Google.

Filed Under: content moderation, greenhouse, lawful but awful, section 230
Companies: omegle