It's Not Personal: Content Moderation Always Involves Mistakes, Including Suspending Experts Sharing Knowledge
from the it-happens dept
I keep pointing out that content moderation at scale is impossible to do well. There are always going to be mistakes. And lots of them. We’ve spent years highlighting the many obvious mistakes that websites trying to make moderation decisions on thousands, hundreds of thousands, or even millions of pieces of content are going to make every day. It’s completely natural for those who are on the receiving end of obviously bogus suspensions to take it personally — though there does seem to be one group of people who have built an entire grievance complex on the false belief that the internet companies are targeting them specifically.
But if you look around, you can see examples of content moderation “mistakes” on a daily basis. Here’s a perfect example. Dr. Matthew Knight, a respiratory physician in the UK, last week tweeted out a fairly uncontroversial statement about making sure there was adequate ventilation in the hospitality industry in order to help restart the economy. At this point, the scientific consensus is very much that good ventilation is absolutely key in preventing COVID transmission, and that the largest vector of super spreader events are indoor gatherings with inadequate ventilation. As such this tweet should be wholly uncontroversial:
To get hospitality working safely in the U.K. a significant investment is required in ventilation systems. The standards should be set ASAP and funding made available. Covid is airborne (as are other infections) and ventilation vital part of prevention. Good air quality vital
— DR MATTHEW KNIGHT MBE (@drmknight) May 19, 2021
And yet… despite this perfectly reasonable tweet from a clearly established expert, Twitter suspended his account for “spreading misleading and potentially harmful information related to COVID-19.” It then rejected Dr. Knight’s appeal.
There?s something wrong with Twitter?s censorship function. An Aerosol Scientist and a Respiratory Physician have both been blocked from accessing their accounts for ?spreading misleading & potentially harmful information related to COVID-19?. How is this misleading/harmful? pic.twitter.com/wodknwKUKw
— Kristen K. Coleman (@drkristenkc) May 27, 2021
Thankfully it appears that Twitter eventually realized its mistake and gave Dr. Knight his account back. Lots of people are (understandably) asking why Twitter is so bad at this, and it’s a fair enough question. But the simple fact is that the companies are all put in an impossible spot. When they weren’t removing blatant mis- and disinfo about COVID-19, they were getting slammed from plenty of people (also for good reason). So they ramped up the efforts, and it still involves a large group of (usually non-experts) having to make a huge number of decisions very quickly.
There are always going to be mistakes. As Harvard’s Evelyn Douek likes to note, content moderation is all about error rates. Each choice you make is going to have error rates. The biggest questions are what kinds of errors are preferable, and how many are you willing to deal with. Should the focus be on minimizing false positives? Or false negatives? Or somehow trying to balance the two? And the answers to that may vary given the circumstances and may change over time. But one thing that is clear is that no matter what choices are made, mistakes inevitably come with them, because content moderation at scale is simply impossible to do well.
Filed Under: content moderation, covid, matthew knight
Companies: twitter
Comments on “It's Not Personal: Content Moderation Always Involves Mistakes, Including Suspending Experts Sharing Knowledge”
The appeal rejection is the big problem
I can understand over-enthusiastic enforcement for dis/mis-info. Its to be expected in the current climate. What I find troubling is the seemingly blanket rejection of appeals that all the social media companies are doing for “sensitive” subjects. If you appeal the automated moderation, it should be reviewed by a person. I don’t know if the rejections are literally automatic, or if the reviewers just reject everything, but many people have reported the rejection emails being near-instantaneous. If they are being “reviewed” by people, they must be payed by the review or something.
Re: The appeal rejection is the big problem
Nice idea, bur difficult to do because of the number of appeals. You can’t allow a backlog to start as it will grow without limit and when the backlog starts it is too late to hire more people because it will continue to grow while you hire and train them..
Re: Re: You still can't win, but that isn't the reason.
Point of order: You don’t just hire/train enough to handle the current rate, you hire enough to also dispose of the backlog.
The "excess judges" will eventually be absorbed as the appeals count increases. So yes, it can be done.
The issue isn’t "you can’t hire enough judges". The issue is "you can’t afford to hire enough judges".
Re: Re: Re: You still can't win, but that isn't the reason.
The problem is not as simple as hire more moderators, you also need the buildings, managers, equipment, support personnel to support them. Meanwhile it is taking longer and longer for an appeal to reach moderation. At the scale of Twitter and Facebook et al. it is not a case of hiring a few more people, but more hire several thousand more people and try and train them before your problem grows even larger.
Overall, the problem is not how does Facebook moderate its users, but rather how do you moderate the whole of humanity.
Re: Re: Re:2 You still can't win, but that isn't the reason.
And more and more people appeal, hoping to crash the system.
This comment has been flagged by the community. Click here to show it.
They Hope You Will Forget
It may be impossible to moderate at scale, but there will not be any improvement or accountability until the system stops being so opaque. Publish the algorithm, and explain why this one got censored.
Re: They Hope You Will Forget
Publish the algorithm, and then you’ll have people manipulating it to the point of worthlessness.
Don’t publish it, and then you have people claiming it’s biased in some way.
There’s no perfect solution.
Re: Re: Rigid and specific rules, a troll's wet dream
‘As the specific rules that were forced upon platforms in the name of ‘clarity’ note saying the word ‘Green’ will get a comment flagged and removed, however I very clearly did not say green I said ‘the primary color of plants on the planet earth’ which is not prohibited and therefore the removal of my comment is against the rules and it deserves to be put back in place until they are further clarified to cover my comment.’
Re: Re: Re:
Always trust a troll to break a rules system by any means necessary.
content moderation is only working in one direction, the one that allows dire consequences on to whoever has offended someone or the software concerned
Scaling
Who is moderating the moderators?
Who is watching the watchers?
Re: Scaling
Quis moderabat ipsos moderatores?
I think that’s how you say it in Latin…
Re: Re: Scaling
Comes from a Roman senator complaining that hiring a virile man to guard your wife while you are away on business is in itself a problem.
Re: Strict liability
In cases such as Dr. Knight’s, there should be strict liability for libel. Claiming untruthfully that a doctor of his credentials is "spreading misleading and potentially harmful information related to COVID-19" is libel per se, and considering the source of the libel, should be cause for substantial punitive damages as well.
Re: Re: Strict liability
There’s a difference when it’s done with intentional malice and when it’s not. This was not.
Swing and a miss.
Re: Re: Re: Strict liability
The malice standard is NYT v. Sullivan and only applies to opinion. If you do a FACT check and say Dr. X is lying that is not an opinion. Its a statement of fact. It says so in the term fact check.
Re: Re: Strict liability
Lol, that’s not how defamation works. And it is absolutely not how "defamation per se" works.
Re: Re: Re: Strict liability
Mike, is that the US law take on it or the UK?
Re: Re: Re:2 Strict liability
US Law.
Re: Re: Re:2 Strict liability
Relevant law.
Re: Re: Strict liability
If that were true you would have been sued to the poorhouse for your bad legal takes.
The problem *is* the scale...
In any large ecosystem that supports any level of controversy, what to moderate out starts to depend heavily on which audience member happens to be looking.
For example, the mythical average techdirt commenter hates what kobe or OOTB says, flags it for boring and doesn’t want to hear it …. but I’m pretty sure one of Mike Masnick’s friends has been studying those very same comments.
and that’s before we get to the idea that we can’t expect a large platform with huge numbers of commenters on X to also have huge numbers of experts on X, for all controversies X.
If twitter (or facebook) wants to do better, it’s going to need to establish some trusted public figures… and good luck, because we collectively can’t decide whether to trust Donald Trump or not, even if every techdirt poster comes to the same decision.
"It’s completely natural for those who are on the receiving end of obviously bogus suspensions to take it personally"
I feel seen. GLARES
Another metric: how many suspensions go over 100 days with no response. There are to many competing goals in the mix & its making things much worse.
In response to a video of someone being ‘kidnapped’ for a party (victim did not know until later it was a prank) friend of mine said if someone grabbed her like that she would punch him.
12 hour timeout, having to remove the tweet, because it was wishing pain or harm on someone.
(HI KAT!)
"If someone assaults me I would hit them." – Banable tweet treated on the same level as someone threatening harm to all the Jews.
Zero tolerance policies still are bad ideas.
More timeouts for people threatening hypothetical imaginary people (who aren’t part of one of the 10K protected groups on twitter) doesn’t make the platform better or "safer".
The CoVid Misinfo AI is bugged, Senator Anti-Vaxx is still promoting all of his crazy ideas (can still see them on the platform) while some sociopath mocks morons who think there is a chip in the vaccines has the tweet hidden and is over 100 days in TwitMo waiting for assistance. I’ve offered to delete the tweet, but prying a phone number from my poor hands matters more than anything else it seems.
I guess if you use a trendy avatar, call yourself an immortal sociopath, you are held to higher standards than members of Congress.
That’s why most people have learned to refer to the Disease-That-Must-Not-Be-Named using euphemisms. Because saying That Word in any context at all is a magnet for moderation "mistakes."
No, I don’t think this is a desireable state of things.
Re: Re:
And now that its the poster child for all the bad on the platform they are overreacting to everything assuming people will understand that ‘out of an abundance of caution’ we gave you a time out & then doubled down on it even if everyone else on the planet says we are wrong.
Re: The Disease-That-Must-Not-Be-Named
syphilis would explain the insanity.
While they can train an AI to spot & flag "bad things", do they ever teach it the mistakes they have to fix?
Re: Re:
@TAC, the problem is, let me take your "mistake" and now hold it up to ridicule with a word…
How you gonna teach an AI to know the difference between
"BS, Vaxxes work!"
and
"Vaxxes work!"
All it took was a word, or a small difference in context, and the meanings are opposite. It ain’t gonna happen!
Re: Re: Re:
I still insist it was because I said they claimed Trump was the messiah.
Re: Re:
Sure they do. But as soon as they do, the groups they want to block change their strategy again. And again. And again.
And now you’re 15 AIs in, and the 3rd "mistake" is now being used by an entirely unrelated group for entirely unrelated reasons, and the 2nd and 7th directly contradict the 11th and 12th respectively, and the 9th is actually being simultaneously used by three different "sides" of the issue, and the the 14th shouldn’t have been included at all because the human moderators messed it up….
and your AI remains just as useless as before
Re: Re: Re:
Their AI now is pretty useless already.
Bad Example of something that is purported to be moderation
https://www.theregister.com/2021/06/01/google_usenet/
The Usenet group comp.lang.tcl vanished from Google Groups for several hours before being restored on Tuesday.
…
Google took over the Usenet archive when it acquired Deja News in 2001.
Almost a year ago, comp.lang.forth and comp.lang.lisp, were also removed from Google Groups. And they remain unavailable. Likewise, comp.lang.c and comp.lang.python are currently inaccessible via Google Groups.
The suppression of Usenet groups has cultural, academic, and technical consequences. Some active systems, for example, still rely on Forth.
I agree with your analysis that moderation on the scale of the world is impossible to do right. But your conclusion that it’s not personal, that moderation is always going to suck and people should accept it as inevitable as death and taxes I can’t support. They should do it in a scale that they can do right.
If they’re doing too much moderation to do it right, then they’re doing too much moderation. At the point where the issue stops being humans disagreeing on what should be moderated (which truly is inevitable) and becomes instead humans being too busy to moderate so they let dumb algorithms auto-ban people and auto-deny appeals because it’s easier than doing their jobs, then they are moderating too much.
Despite what some seem to think, people saying stupid things on Twitter is not the end of the world. It’s not worth it to randomly ban normal people just for discussing a controversial topic in an effort to try to silence some conspiracy nuts’ inane BS.
Re:
They literally can’t — not without spending far more money and hiring far more people than should ever be necessary for a moderation team.
The whole point of saying “moderation doesn’t scale” is to point out how moderating a small community is always going to be easier than moderating a large community. If you’re moderating a group of a couple dozen people or so, you’re gonna have an easier time of telling people to knock their bullshit off — mostly because a community that small will have its own quirks and contexts under which it can be moderated. But a community of a couple…oh, let’s say a couple hundred thousand people will require far more moderators and far harder calls to make because those people can branch into subcommunities and develop their own quirks and contexts that won’t parse universally. The ability to moderate all those people doesn’t scale well when compared to the ability to moderate a much smaller amount of people.
Even people who moderate small communities never get it right 100% of the time. How do you expect a company/service/“community” as large as Twitter to get it perfect?
And yet...
… the Russian bot farms can continue to spread blatant misinformation with impunity. They’re not countered until someone with high visibility makes a case against them.
Why does Twitter protect the Russians?
Re: And yet...
Why do you make statements that has no factual basis?
Btw, Twitter bans thousands of bot-accounts they detect regularly, but you seem to know exactly how to unerringly identify these Russian bot farms. Perhaps you should inform Twitter how you do it so they can implement it.
The problem isn’t so much the moderation but the automated rejection of appeal and the blank generics of “why”.
TOS is nothing more than local civil law, at its most basic construct. When you are to be punished because you break the law you should be informed of exactly what you did. Not generic ‘section 4 subsection b’ but the actual infraction:
“Posting false information, in your case ‘C19 is not contagious’”
Appeals should not be automated. If the poster can show a source then the content, and account, should be restored and marked as “disputed” when the poster adds a link edit.
All of this can be avoided by a tos clause that “we can ban you at any time for any reason”. Your using private property.
When such a clause exists there’s nothing to do.
When it doesn’t and a ban is based on violations such violations need to be spelled out.