Content Moderation Case Study: Dealing With Misinformation During A Pandemic (2020)
from the misinfo-is-hard dept
This series of case studies is published in partnership with the Trust & Safety Foundation to examine the difficult choices and tradeoffs involved in content moderation. Learn more »
Summary: In early 2020, with the world trying to figure out how to deal with the COVID-19 pandemic, one of the big questions faced by internet platforms was how to combat mis- or disinformation regarding the pandemic. This was especially complex, given that everyone — including global health experts were trying to figure out what was accurate themselves, and as more information has come in, the understanding of the disease, how it spread, how to treat it, the level of risk, and much, much, has kept changing.
Given the fact that no one fully understood what was going on, plenty of people rushed in to try to fill the void with information. Most social media firms put in place policies to try to limit or take down misinformation or disinformation using a variety of policies and tactics. But determining what is misinformation as opposed to legitimate truth-seeking can be very tricky in the midst of a pandemic.
In late March, as the pandemic was hitting full swing, an article appeared on the website Medium by Aaron Ginn, a self-described Silicon Valley ?growth hacker,? arguing that the response to COVID-19 was overblown and driven by ?hysteria.? The piece included many citations of credible data and reports, but also included a few quotes significantly downplaying the risk of COVID-19, including saying that its ?transmission rates are very similar to seasonal flu.?
The story started to spread widely, mainly after a number of Fox News hosts started tweeting it. As the story got more and more attention, Carl Bergstrom, a professor of biology at the University of Washington, decided to critique Ginn?s article in great detail via an extended Twitter thread. Bergstrom makes a fairly compelling case that Ginn?s lack of expertise in epidemiology led him to making a number of mistakes in his analysis, in particular, not understanding how viruses spread, and how that information is tracked. He also argued that Ginn cherry-picked certain data to support a thesis. Bergstrom and others started arguing that Ginn?s Medium piece was (perhaps not intentionally) misinformation.
Decisions to be made by Medium:
- Should the piece be allowed to remain on the platform or should it be taken down?
- Are there alternatives to taking the piece down or leaving it as is?
- If the piece is taken down, should the user be banned?
- What policies do a potentially well-meaning, but possibly incorrect, piece about a pandemic violate?
Questions and policy implications to consider:
- How do you distinguish mis- and disinformation from potentially well-meaning, but inexperienced analysis?
- Should earnest but incorrect information be allowed?
- How does that analysis change when human health is at stake?
- Do pieces like this need to be judged against official government and healthcare sources? If so, how is that reconciled with constantly updated information from those sources?
- If articles like this are taken down, does that create more ?credibility? for articles that remain on the site?
- If articles like this are left up, does that create more credibility problems for other articles on the site, that may have a much stronger and more competent analysis?
- Does taking down an article like this create political backlash? If so, does that matter?
- Do the site and its management want to feel responsible if people take bad health advice that was posted to their site, and possibly come to harm from it?
Resolution: Medium chose to quickly take down Ginn?s piece about a day after it went up and 13 hours after it went ?viral.? In fact, the article was taken down while Bergstrom was writing out his tweet thread critiquing it. Indeed, Bergstrom ended his thread early upon learning that the article was taken down.
That was not the end of things, however. The article was reposted to the site ZeroHedge, and copies were stored and reposted in other places as well. It also created a short-lived uproar among those who felt that Medium?s moderation decision was unfair. The Wall Street Journal celebrated Ginn, saying that after being ?targeted for censorship,? it only made Ginn more influential. Other publications, including the NY Times, the Washington Post, and Slate, all wrote about the dangers of amateurish analysis in the midst of a pandemic.
Ginn, at one point, appeared to be fine with Medium?s decision, saying that internet platforms ?are free to associate with whom they want,? (though he has since removed the tweet saying that). He has continued to use other social media to argue that the claims about COVID-19 were overblown.
Filed Under: aaron ginn, carl bergstrom, case study, content moderation, covid-19, misinformation
Companies: medium
Comments on “Content Moderation Case Study: Dealing With Misinformation During A Pandemic (2020)”
Other moderation options
I know (experience) that most of the time moderation is easy, mopping up the spam. There are times when moderation is hard: the post seems serious, it is not obviously trolling, but one can smell it will generate more discussion than it should.
I accept that moderators will make decisions that are subject to discussion; that was so 30 years ago, that’s so today and that will be in the foreseeable future. Every platform should accept that. (But there’s no need to involve a poster in the discussion about the moderation of his post!)
Back to my argument: hiding or deleting a post is a useful tool in a moderator’s toolbox. Banning a user can be too. But there are other, more subtle ways of moderation that can be very useful. The option to add a "one line remark" on top of the post (not editable by the poster), like "please fact check this" or "this post contradicts scientific evidence" would allow a platform to give its opinion about the post. And nothing forbids a moderator to provide his (the platform) rebuttal in a special "moderation reply".
If you think out of the "delete or allow" box there are several ways of making nudges to your audience. And, out of experience, every active moderation decision will make someone unhappy. Live with that and have a thick skin against people that think they are more important than the rest of the world.
Re: Other moderation options
Kudos. Though I would like to add, in re:
What government sources? With the strong (but often wrong) Executive declaiming its own experts, how is one to believe the government as a trusted source? Dr. Fauci and the CDC and the WHO have the capability to provide trusted information, if they were not politically compromised. Each of them has been. Trump would have us believe him (not what I would call a trusted source) over the others, and has been doing much to put down reasonable responses to Covid-19, replaced by his desire to energize the economy (which he has little control over) despite the potential harm to human life.
Then, given that there is much we don’t know about Covid-19, the way forward is little more than a guessing game, though some guesses (example face masks) are better than others. Open schools, or not, is a question about creating more pandemic hot spots or not, and of course the lives of the students and teachers and faculty. The decision though seems to be in the hands of administrators and politicians, who will not be impacted in the classroom or hallways. That…does not seem right.
Re: Re: Other moderation options
Yes, government sources, the first question is which government, there still is considerable disagreement on how to handle Covid-19 between governments on different sides of the Atlantic. What should an international forum take as leading here? Another hot topic is which government opinion to take as leading on the issue of global warming. I would certainly not mind when a platform labels government opinions without scientific backing as "political blather".
And yet, as more gets known about Covid-19, it more and more shows to be a nasty disease, nastier than the flu, able to incapacitate a serious fraction of the patients that survive it. The jury is still out on what’s the best way to handle it, but somehow I feel that "the economy must go on, don’t care about the dead bodies" is not the best way.
Re: Re: Re: Other moderation options
I would be happy with any scientific organization (or person) who made statements, prognostications, or projections, or suggestions, so long as they covered themselves with caveats such as ‘so far as we know, but we are still looking’ or ‘this is our best advice, but we will update you when we know better’ or ‘this is what we have found so far’ or ‘there is some hope in, but we don’t know yet’ and then the promise to keep us up to date, regardless of political considerations.
For me, I don’t care if that info comes from North America, South America, Europe, Asia, Africa, Central America, or Antarctica (unlikely), so long as there is some merit in what they say, they have some prior credibility, and they follow through on their follow through.
Re: Re: Re:2 Other moderation options
I do feel that a crossover between scientific method and free expression is the way to go. Have several institutions give their scientifically based opinions with uncertainties and the data they base their opinions upon. If opinions overlap within uncertainty bounds you (statistically) know there is a scientifically solid basis. if opinions conflict, have science do more research!
Re: Re: Re: Other moderation options
Look at the graphs in this article, and ‘the economy must go on’ looks like the worse way.
Re: Re: Other moderation options
You are correct in that there are too many “experts” who know less than those they presume to lecture.
Same as most Congress critters, as well as the big perv.
Just turn off (everything) and cultivate your own resources and live and die on your own terms.
Word on the street is we all pass eventually at different ages and for a variety of causes. Seems to be the way of the world. The question is do we live on our feet with a measure of dignity or as submissive slave prisoners?
By the way, I hate child predators, bullies, criminals acting under color of authority, and on and on. Does that make me guilty of hate crimes?
He has continued to use other social media to argue that the claims about COVID-19 were overblown.
Still?
(Apologies, this comment wasn’t about moderation, which is difficult. Thanks for the continuing examples and the thoughtful questions raised.)
Good luck with that
Moderation, like statistics is dependent upon point of view and agenda.
Re:
Figuring out what to moderate is like figuring out obscenity: “I know it when I see it.”
Re: Re: Re:
That is until the mob of the vocal minority see it differently. See this as a preamble as it is referenced in this which is a discussion about how perceptions change with a few loud voices, right or wrong,
Screaming at the moderators should not change their decisions. Reason and facts might mitigate their moderation choices, and updated reason or facts, but screaming at the moderators because they don’t fit your sensibilities of the moment should not.
Re: Good luck with that
I disagree about your mentioning of statistics and point of view in a single sentence, proper use of statistics should be objective.
Moderation is execution of platform policy, which is squarely dependent on how a platform wants to position itself in the societal discussion. (I avoid saying political here because in the US that has the connotation of partizan.)
Re: Re: Good luck with that
Statistics should be objective. It is a shame how few actually understand how to create and then apply them. What we see however is a plethora of statistics that support a position, which may be political, or social, or other, but actually have no basis in reality. That they are meaningless does not preclude the position taker from making hay with them.
Re: Re: Good luck with that
In a perfect world, statistics wouldn’t be used and abused, gaining a reputation like many lawyers. I agree, statistics should be objective. What we have seems to be a lot of not that.
Re: Re: Re: Good luck with that
The trouble is, honest statistics (where data has neither been invented nor removed) can be extremely misleading and many people who should know enough to know that they aren’t qualified by ability perform incorrect, but superficially convincing, analyses. And most members of the public are totally incapable of accurate assessment of statistics. Stats have a bad rap because they are DIFFICULT and many, with the best will in the world, inadvertently misuse them. Plus many who aim to deceive deliberately misuse statistics.
Heck, one in ten Americans apparently don’t understand that on a flat Earth, a ship can’t disappear from the bottom as it sails away from you over an open sea. That’s simple and something even the ancient Greeks knew.
Re: Re: Re:2 Good luck with that
Those one in ten Americans (statistics wanting, though there is some certainty that there are some) also know absolutely that when you get to the edge, you fall off. To where is a conundrum that has never be clearly elucidated. I keep hoping for some clarity. Not energetically, but curiously.
All those circumnavigators and astronauts and pilots and ship captains and their passengers are just liars. Mean spirited, non conformist liars.
Re: Re: Re:2 Good luck with that
"The trouble is, honest statistics"
Isn’t that an oxymoron? 😉
Re: Re: Re:3 Good luck with that
Hardly. But lol anyway.
I am not sure what that could possibly mean. Statistics are often considered "objective" by people who do not understand them at all, simply because they look like numbers, which are universally considered "objective" by people who erroneously think they understand numbers. But statistics are not numbers: they are probability distributions–which are "functions" (that is, abstract–mental–constructs that cannot be accurately represented by any finite number of numbers.)
And numbers are generally trickier than they look anyway
And finally, numbers arrogantly intended by some people to define the other people are inherently subjective. And the "other people," even more than probabilities, resist that kind of reduction. Functions can’t resent at all–people will! And after resenting, people begin to subvert. Even such a well-meant scheme as Google’s "page-rank" was subject to mass distributed attacks!
Personal statistics don’t–and can’t ever–provide a valid basis for action at all. Their proper valid use is to suggest what further questions need to be asked…and asking questions is inherently a subjective (mental) activity.
Re: Re:
Nice use of the reply to button where we would be sure of whom you were replying to.
But that is the point, some (entities, persons, political parties, etc.) use what they purport to be statistics, as statistics when they are nothing of the sort. If only there were some way to refute these assertions, but without the source material (which often isn’t forthcoming and and is a lot of work) that isn’t feasible, and that is assuming that the questions asked originally were the correct questions, put correctly. They are often not, but designed to obtain a particular outcome, which is not what the point of the exercise should be, but then…missed opportunities for those seeking the outcome designed.
Unless the presented statistics have been certified by an accomplished statistician, they should be looked askance at. I am not advocating for an additional expense or license or whatever, but unless the entire data set and procedure for establishing the quoted statistic is revealed and questioned, then it is in fact of no value, and should be treated as such, vocally. If we behave this way, then, eventually they will start to present quantifiable information, or present none when they can’t actually qualify it. In that case, there will be some other misrepresentation created. Don’t you doubt it.
Re: Proper use of statistics
Statistics is a branch of mathematics and as such the rules of logic and mathematics apply to statistics. If you do statistics properly you take several steps to verify the that your raw data is not biassed, your processing methodology does not introduce biases and that your final result is statistically relevant. And even then, be careful how to interpret the result of your statistical exercise.
Good statistics is hard!
Yes, statistics have been abused by using biassed data, biassed processing, biassed interpretation and assigning relevance to irrelevant results. (lies, damn lies and …) That hurts my mathematical heart!
Re: Re: Proper use of statistics
Just remember, 90% of statistics are made up on the spot!
Another point to consider when deciding whether to moderate:
What if the official declarations by government or health officials that you are weighing the article against are wrong?
Remember, the US Surgeon General made statements to the media that n95+ masks are useless for regular people for protecting the wearer from catching the virus, and that they should conserve them for health care workers… who need the masks to protect them from catching the virus.
Both statements cannot be true unless healthcare workers are a different species from the rest of us.
But by making those statements, it meant that any site moderating commentary and articles on the official government declaration criteria would censor comments and articles that said that anyone can be protected from the virus with a proper mask – which is factually true.
Re: Another point to consider when deciding whether to moderate:
In the US you’re free to form your own opinion about government declarations, so as a platform you can decide to color the debate with your own opinion. (Techdirt shows an opinion on police quality that differs from what’s proclaimed from the White House.)
I think it’s fair to label a statement like that with "Warning: internally inconsistent", despite all of the backlash of "How do you dare accusing a politician of lying!"
Living in China such an attitude may be a cause for having to spend some time with the Uighurs.
Re: Another point to consider when deciding whether to moderate:
"Both statements cannot be true unless healthcare workers are a different species from the rest of us."
They are. They work in close proximity to high risk patients and in high risk environments, so they require the best safety gear.
The "don’t hoard N95 masks" statement came directly after idiots were buying up enough toilet roll to build a new house with, and there was already a known shortage of N95 masks because the tangerine moron fired the pandemic team that was responsible for ensuring such supplies were adequate. Had they not done this, the US would be even more screwed than it is now with the idiots who refuse to wear any kind of mask.
The DAY..
That we enforce TRUTH,
is the day this nation changes.
How can we demand Truth in anything, without abit of understanding. No one cares. Mos to fus Stopped listening to the BS long ago, after hundreds of lies and falsehoods from news, newspapers, TV, Politicians(state and federal), every corp around us. Fact checking is GONE.
There were regs/laws passed in the past, that got Shot in the foot that would require Things on TV to be More truthful, including the political BS. We watch the games played and wonder WHO the politics are playing(paying) for. The ACA(Obama care), was Adjusted last year, and then has been declared Unconstitutional.. And that it should be dismissed/removed.(If you didnt know).
We have agencies in our gov. that have not Given paper work as to how much they have spent, in over 50 years, and we Keep giving them MORE money. We have a few agencies that are working with computers as Old as I am(60) and upgrading the computers is a Scary idea even to consider. Paying a few million over the trillions another agency gets, means 1 less airplane. The IRS cant do their own job, because their desktop computers are over 10 years old, the main frame is from 1960( the oldest, Pentagon has the 2nd oldest), and the tax code is so Complex most humans cant get it all understood. Not to forget that the amount of info Placed in those computers is HUGE, and they cant compute MOST taxes for the corporations. Which is a great thing for the corps, but who do you tax?
Any time there is a backdoor/trick/escape to use in a game or even at work, WE WILL FIND A WAY to do it.
Re: The DAY..
Truth isn’t enforced, nor does it need to be. It is, despite the best efforts of most of humanity.
There is a hymn:
Though tyrants rule by fear,
Suppress the truth by force,
Bring superstition near,
And give the wrong free course,
No human mind or might
Can redefine the right.
Re: Re: The DAY..
The 1 difficulty..
is Opinion.
The freedom of opinion, right or wrong.
After its said, who defends it? Who retracts it?
A word said is never erased.
Like the old newspaper, the article is on the front page, and the correction are next to the obits.
be the words with a purpose said, to falsify or not.
To cause harm or not.
They get published on the front end and corrections buried on the back.
Would be nice if they had Equal leverage.
Its fun watching Fox news Channel change opinions. And you would think they would get tired of being Wronf so many times.
But do they admit it as much as the Said it in the wrong? NOPE.
Re: Re: Re: The DAY..
How many times in history have we tried to enforce the truth only to end up being wrong?