Techdirt's think tank, the Copia Institute, is working with the Trust & Safety Professional Association and its sister organization, the Trust & Safety Foundation, to produce an ongoing series of case studies about content moderation decisions. These case studies are presented in a neutral fashion, not aiming to criticize or applaud any particular decision, but to highlight the many different challenges that content moderators face and the tradeoffs they result in. Find more case studies here on Techdirt and on the TSF website.

Content Moderation Case Studies: Facebook Removes Militia Event Following A Shooting (August 2020)

from the moderating-groups dept

Summary: Following the shooting of Black man Jacob Blake by Kenosha police officers, protests erupted in the Wisconsin town.

As law enforcement attempted to rein in the damage, citizens aligning themselves with private "militias" discussed taking action during the civil unrest.

Some of this organizing began on Facebook. A Facebook "event" created by the Kenosha Guard account (and promoted by conspiracy theorist/far right website Infowars) possibly caught the eye of 17-year-old Kyle Rittenhouse. Rittenhouse traveled from his home in Antioch, Illinois with his weapons to the protest/riot occurring less than 30 minutes away in Kenosha, Wisconsin. Before the night was through, Rittenhouse had killed two residents and injured one other.

Facebook finally removed the "event" posted by the Kenosha Guard account -- one the account referred to as a "call to arms." Posts by the group asked "patriots" to "take up arms" against "evil thugs." The event was deemed a violation of Facebook's policy regarding "Dangerous Individuals and Organizations." Facebook also claimed it could find no link between the account and this event and Kyle Rittenhouse.

Some viewed this response by Facebook as too little too late. Someone had already apparently heeded the call to "take up arms" and had taken people's lives. According to a report by BuzzFeed, the event had been reported 455 times before Facebook removed it. Four moderators had responded to multiple flaggings with a determination that the event (and the account behind it) did not violate Facebook's rules. During an internal meeting with moderators, CEO Mark Zuckerberg admitted the company should have reacted sooner to reports about the event.

Decisions to be made by Facebook:

  • Should moderators be given more leeway to remove events/accounts/pages (at least temporarily) that have generated hundreds of complaints, even if they don't immediately appear to violate policies?
  • Would a better/more transparent appeal process allow moderators to make more judgment calls that might address issues like this more expediently by allowing them to make mistakes that can be undone if no violation occurred?
  • How does the addition of more forms of content to the "unwanted" list complicate moderation efforts?
Questions and policy implications to consider:
  • Does the seemingly constant addition of new forms of content to "banned" lists invite closer government inspection or regulation?
  • Is a perceived failure to react quickly enough an impetus for change within the company?
  • Are policies in place to allow for judgment calls by moderators? If so, do they encourage erring on the side of caution or overblocking?
  • Does taking credit for actions not actually performed by Facebook make it appear more focused on serving its own interests, rather than its users or public safety in general?
Resolution: The event flagged by hundreds of users was ultimately removed… but not by Facebook, as was earlier reported. The group that posted the event took it down following the shooting in Kenosha. The company appears to realize this delay may have contributed to events that unfolded in Kenosha and has put policies in place to make things clearer for moderators.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: content moderation, kenosha, militia, shootings, wisconsin
Companies: facebook


Reader Comments

Subscribe: RSS

View by: Thread


  • identicon
    Bobvious, 23 Oct 2020 @ 4:35pm

    Perhaps widen the skillset of moderators?

    This seems like a classic case of "increased chatter", that someone with proper military and/or signals intelligence experience could assist with in terms of managing the moderation. I'm not suggesting this will only be one person or is entirely simplistic, but this can't just be left to people who are using "profanity and copyright" filtering algorithms.

    While the platforms are not responsible for morons and stupid behaviour, they have the opportunity to limit the negative outcomes of that, through thinking beyond the next financial incentive.

    reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:38pm

    The KEY criminal act is tried to stab a police officer.

    According to the release, Blake admitted to investigators that he did have a knife at the time and a knife was recovered from the driver's side floorboard of his vehicle. No other weapons were found, the report said.

    But that's simply elided here at dishonest TD and in the Rittenhouse piece:

    It was past 11 p.m. local time Tuesday, the third night of protests after a Kenosha police officer shot Jacob Blake seven times in the back.

    reply to this | link to this | view in chronology ]

    • icon
      Stephen T. Stone (profile), 23 Oct 2020 @ 4:50pm

      Having a knife in your possession doesn’t automatically mean you tried to stab someone with it.

      reply to this | link to this | view in chronology ]

    • icon
      Scary Devil Monastery (profile), 27 Oct 2020 @ 4:07am

      Re: The KEY criminal act is tried to stab a police officer.

      "According to the release, Blake admitted to investigators that he did have a knife at the time and a knife was recovered from the driver's side floorboard of his vehicle."

      This is still the United States we're talking about, where a white man is not to be questioned about carrying a loaded AR-15, a shotgun, and have a crate of handguns or rifles in the trunk, but apparently, a black man possessing a knife in his vehicle is justification for said black man being killed?

      Thank you very much, Baghdad Bob, for giving us yet more evidence you're a hypocritical racist shitbag. As if we needed that given your previous assertions about how law shouldn't require evidence or a courtroom.

      "But that's simply elided here at dishonest TD and in the Rittenhouse piece..."

      "dishonest" for bringing to attention that if Blake had been white he could have had his cars stuffed with shotguns or worn them openly and the officers wouldn't have so much as unholstered their weapons. In fact a white person already known to have committed murder could walk through a police line still carrying his AR without even being questioned.

      reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:39pm

    Another KEY part you omit: "third night of protests"

    Among protesters, the rumor spread: Hundreds of white men with guns had answered an online call from a self-described militia group known as the Kenosha Guard and would be waiting in the park to shoot them.

    Not nearly that many armed men showed up, but they were impossible to avoid. Some joined the marchers and pledged to protect them. Many protesters still felt more afraid than secure.

    The police weren't keeping order. Vigilantes decided to, as is the responsibility of sane people.

    That's why to stop riots immediately. Turned into armed confrontation.

    And of course YOU omit that "the marchers" also had guns on their side.

    reply to this | link to this | view in chronology ]

    • icon
      Stephen T. Stone (profile), 23 Oct 2020 @ 4:51pm

      The police weren't keeping order. Vigilantes decided to, as is the responsibility of sane people.

      I’m glad to see you show your true colors by supporting vigilante violence.

      reply to this | link to this | view in chronology ]

      • icon
        That One Guy (profile), 23 Oct 2020 @ 6:25pm

        Re:

        You missed the best part in that their argument could easily be turned around and used in favor of those they would vilify, as people are pissed off and protesting and rioting because the police have shown that they aren't interested in order or laws but merely protecting their own no matter what, such that people have to step in and try to fix a broken system, 'as is the responsibility of sane people'.

        reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:39pm

    "plans to argue self-defense" but TD clearly has convicted.

    It's absolutely clear that "the left" won't back off. But you're actually just a few weenies. I predict you'll be surprised, rather like Yamato said at start of WW2 "we have awakened a sleeping giant".

    reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    identicon
    Libby Ruhl DeLuzionnes, 23 Oct 2020 @ 4:45pm

    And the sum is that slanted bigotry isn't "moderation".

    You're totally biased and try to stop all other viewpoints, especially for any organizing, formula for conflict.

    Now, flatly, not to stint you because you'll go berserk anyway: I don't find any wrongdoing by police or Rittenhouse, but DO by "the marchers". Leftists just pounce on any pretext without regard to facts.

    reply to this | link to this | view in chronology ]

    • icon
      Stephen T. Stone (profile), 23 Oct 2020 @ 4:56pm

      You're totally biased and try to stop all other viewpoints

      So what? It’s his legal right. Whether you think it’s morally or ethically wrong is irrelevant — doubly so, considering the kinds of voices you do support.

      You, uh…you friendly with the Proud Boys, Brainy?

      reply to this | link to this | view in chronology ]

      • icon
        Scary Devil Monastery (profile), 27 Oct 2020 @ 4:16am

        Re:

        "You, uh…you friendly with the Proud Boys, Brainy?"

        Pretty obvious, innit? Old Baghdad Bob's pulled out all the stops in this one; Blake, being unarmed, gets shot 7 times in the back and old Bobmail justifies it by saying he had a knife in his car.

        Then in the very next sentence Bobmail turns right around and glorifies the right-wing extremist violence as vigilantism.

        With the only notable differences between the violence Baghdad Bob condones and condemns being the skin color of the victim it's pretty damn clear where he's standing.

        Not, bluntly put, that this should be any surprise. We've known since his Torrentfreak days he's a paranoid schizophrenic with an unhealthy attraction to fascism, racism, bigotry and outright sadism.

        Or to summarize, a loser in life who compensates for nothing going his way by inventing a world inside his head where he's the eternal victim of other people and only showing up from that world to write the most recent case of thuggery in on "his" side of that narrative.

        He'd be tragic and pitiable if he wasn't such an obvious and persistent asshole.

        reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 23 Oct 2020 @ 5:08pm

      Re: And the sum is that slanted bigotry isn't "moderation".

      You're totally biased and try to stop all other viewpoints

      Says the man who is so biased that he will not even consider a different viewpoint.

      reply to this | link to this | view in chronology ]

      • icon
        PaulT (profile), 24 Oct 2020 @ 2:36am

        Re: Re: And the sum is that slanted bigotry isn't "moderation".

        The subject at hand is a kid who, inspired and emboldened by the same right-wing echo chamber he lives in, went and killed two people - one of whom was in the process of trying to legally apprehend him after he had been witnessed murdering the first. The kid was seen palling around with cops just before going on his murder spree, who let him get away comfortably with his gun to his home in another state, in stark contrast to the violent deaths handed out to black men who they suspect might have a weapon.

        Dickhead has to take a viewpoint like the one he's taking, because he has no leg to stand on morally or factually if he takes another one.

        reply to this | link to this | view in chronology ]

        • identicon
          Anonymous Rex, 26 Oct 2020 @ 1:38pm

          Re: Re: Re: And the sum is that slanted bigotry isn't "moderatio

          He shot and killed a convicted child molester and a convicted wife beater.

          Nope. Doesn’t bother me.

          reply to this | link to this | view in chronology ]

  • icon
    That One Guy (profile), 23 Oct 2020 @ 6:20pm

    Not the best look...

    Reported literally hundreds of times, reviewed for moderation four times, and a page that called for armed responses to 'thugs' ended up being taken down by the people that put it up in the first place, who apparently realized quicker than Facebook what a bad look keeping it up made for.

    Even if it didn't result in death what did they think that sort of 'event' was likely to result in, and how in the hell did not one but four moderators conclude that a page like that didn't violate rules against 'Dangerous individuals and organizations', because honestly if a page calling for armed responses to a group of people doesn't meet the bar required I struggle to think of anything that could.

    reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    identicon
    Anna, 24 Oct 2020 @ 8:00am

    hi

    reply to this | link to this | view in chronology ]

  • icon
    Toom1275 (profile), 24 Oct 2020 @ 11:37am

    "Rittenhouse acted in self-defense!" Said nobody who has any understanding of self-defense.

    reply to this | link to this | view in chronology ]

    • icon
      Toom1275 (profile), 24 Oct 2020 @ 11:38am

      Re:

      These same semi-ambulatory colostomy bags tried the exact same "self-defense" lie after Charlottesville.

      reply to this | link to this | view in chronology ]

      • icon
        Stephen T. Stone (profile), 24 Oct 2020 @ 12:35pm

        And that worked about as well as one might think in the trial of Some Asshole for the murder of Heather Heyer.

        reply to this | link to this | view in chronology ]

        • icon
          That One Guy (profile), 24 Oct 2020 @ 12:49pm

          Re:

          Oh how I wish the Some Asshole initiative was the rule, I can only imagine how many scumbags wouldn't bother knowing that what they did would be all that was mentioned with the main focus being on how the people/community was healing as all the while they would just be known as 'Some Asshole' and treated as just a minor footnote.

          reply to this | link to this | view in chronology ]

    • icon
      That One Guy (profile), 24 Oct 2020 @ 12:45pm

      Re:

      That or they're using the same twisted definition of 'self-defense' that police like to use, where anything might be a threat to their lives and therefore they are always justified in applying lethal force 'just in case'.

      reply to this | link to this | view in chronology ]

    • icon
      PaulT (profile), 24 Oct 2020 @ 12:53pm

      Re:

      It could be argued that he acted in self defence in the second murder because he killed someone who was trying to physically attack and apprehend him at the time.

      Of course, the fact that the reason why he was being apprehended was because he'd just been witnessed murdering another person might undermine that defence somewhat.

      reply to this | link to this | view in chronology ]

    • This comment has been flagged by the community. Click here to show it
      identicon
      Anonymous Rex, 26 Oct 2020 @ 1:35pm

      Re:

      Only on TechDirt would someone shooting a convicted child molester and a convicted wife beater be seen as a bad thing.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 26 Oct 2020 @ 3:08pm

        Re: Re:

        Would you be ok with shooting a convicted bank robber? Or a convicted burglar? Or a convicted drug dealer? Or a convicted drunk driver? Would they deserve to be murdered in cold blood for their crimes that they have already served their sentence for?

        They have been convicted of their crime and have paid the price for it. It is not the job of everyday citizens to enforce the law. That's what the police are (supposed to be) for.

        Cops get away with murdering people all the time. But ordinary citizens? Murdering someone (no, it was not "self-defense") is a crime. Point blank.

        You can't get away with murdering someone just because you don't like them for whatever reason. Even if you think your reason is valid.

        reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 26 Oct 2020 @ 5:13pm

        Re: Re:

        Saying to hell with due process is a good thing? You Judge Dredd? Extrajudicial killings are not supposed be allowed, unless maybe it is one of those secret laws we do not know about but are expected to not violate.

        reply to this | link to this | view in chronology ]

      • icon
        Toom1275 (profile), 26 Oct 2020 @ 9:35pm

        Re: Re:

        Yeah no, victim-blaming to excuse cold-blooded murder isn't a good look there, slick.

        reply to this | link to this | view in chronology ]

      • icon
        PaulT (profile), 27 Oct 2020 @ 12:07am

        Re: Re:

        Only on Techdirt would an "edgy" contrarian claim that people supporting due process and the rule of law is a bad thing, and support extrajudicial murder instead.

        reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Essential Reading
Techdirt Insider Chat
Recent Stories
.

Close

Email This

This feature is only available to registered users. Register or sign in to use it.