Anti-Vaxxers Countermeasures Show Why It's Not So Simple To Just 'Delete' Anti-Vax Misinfo On Social Media

from the countermeasures-happen dept

It's not a new thing that those without any experience in content moderation assume that it's somehow "easy" to just find and delete misinformation and disinformation online -- but it's often stunning how little they've thought through how all of this plays out. As the White House has stupidly been using its bully pulpit to pressure Facebook into deleting anti-vax misinformation, and elected officials are threatening legislation they must know is unconstitutional, none of them seem to recognize that it's not that easy.

Anyone who has done any work related to content moderation knows this. They know that the vast majority of misinformation is not that easy to spot. First of all, it's not clear what is misinformation. You could have someone who gets something inadvertently wrong. Or, perhaps they just misread something or misunderstand something. Is that misinformation that needs to be deleted? Also, there are things like sarcasm or criticism that frequently repeat the misinformation in order to respond to it. Then there are plenty of things that may seem like misinformation but tend to just be people posting stuff that is technically true, but without the necessary context. Does that need to also be deleted? There are tons of degrees involved in misinformation, and figuring out what should stay up and what should be taken down is not nearly as easy as many commentators make it out to be.

But, on top of that, there's the simple fact that those spreading misinformation know that they may face consequences for it, and thus they adapt their techniques. Ben Collins & Brandy Zadrozny, NBC News' two excellent reporters who focus on misinformation, are noting that anti-vax groups on Facebook are effectively trying to cover their tracks in advance of any possible crackdown on the nonsense and propaganda they spew:

Some anti-vaccination groups on Facebook are changing their names to euphemisms like “Dance Party” or “Dinner Party,” and using code words to fit those themes in order to skirt bans from Facebook, as the company attempts to crack down on misinformation about Covid-19 vaccines.

The groups, which are largely private and unsearchable but retain large user bases accrued during the years Facebook permitted anti-vaccination content, also swap out language to fit the new themes and provide code legends, according to screenshots provided to NBC News by multiple members of the groups.

They also note that the groups have already set up secret "backup groups" in case their primary groups get shut down, they can immediately just switch over to the other group. And if you think that now that NBC News has reported on this, well, then it'll be easy for Facebook to find, that's silly as well. It assumes that no further countermeasures will be taken.

Beating Facebook’s moderation system “feels like a badge of honor,” the administrator wrote, followed by a crying-laughing emoji. At the end of the post, the administrator reminded users to stay away from “unapproved words,” and pointed them to a code legend on the side of the page.

Using code words to evade bans is not new among the anti-vaccine community, and it borrows from a playbook used for years by extremists on Facebook and elsewhere. The practice leans heavily on “leetspeak,” or modified language used by coders and gamers that frequently replaced letters in words for numbers or symbols during online discussions.

And, the groups seem effective at finding words that will make it more difficult to search them out:

Group members have incorporated a range of coded language to mask their discussions, many of which perpetuate debunked theories about the vaccines. “Danced” or “drank beer” mean “got the vaccine.” References to “Pfizer” generally use the terms “pizza” or “Pizza King,” and Moderna is referred to as “Moana.” Users generally play around with unofficial language about dancing to create more coded language.

For example, one group member said her husband had become sick after going on a “cross country trip where we spent 2 nights with dancers,” referring to two people who had just been vaccinated.

None of this is to say that Facebook should just throw up its hands and do nothing. But it remains stunning to me how people who just don't understand the challenges of content moderation always seem to think that (1) these things are easy to find and (2) if Facebook just took down a few accounts, these people would magically go away and the disinformation would stop spreading.

It's not that simple!

There are important questions to ask about how Facebook should handle this stuff, but anyone coming up with simple solutions that don't take into account reality -- both in the difficulty in identifying what is truly problematic and the kinds of countermeasures people will take -- isn't helping at all.

Hide this

Thank you for reading this Techdirt post. With so many things competing for everyone’s attention these days, we really appreciate you giving us your time. We work hard every day to put quality content out there for our community.

Techdirt is one of the few remaining truly independent media outlets. We do not have a giant corporation behind us, and we rely heavily on our community to support us, in an age when advertisers are increasingly uninterested in sponsoring small, independent sites — especially a site like ours that is unwilling to pull punches in its reporting and analysis.

While other websites have resorted to paywalls, registration requirements, and increasingly annoying/intrusive advertising, we have always kept Techdirt open and available to anyone. But in order to continue doing so, we need your support. We offer a variety of ways for our readers to support us, from direct donations to special subscriptions and cool merchandise — and every little bit helps. Thank you.

–The Techdirt Team

Filed Under: anti-vax, code words, content moderation, countermeasures, disinformation, misinformation, vaccines
Companies: facebook


Reader Comments

Subscribe: RSS

View by: Thread


  • identicon
    Anonymous Coward, 26 Jul 2021 @ 9:53am

    And then someone reporting on their secret decoder ring would probably get their post pulled first.

    reply to this | link to this | view in chronology ]

  • icon
    Koby (profile), 26 Jul 2021 @ 9:56am

    Counterproductive

    None of this is to say that Facebook should just throw up its hands and do nothing.

    The folks that use code words for their speech on the big platforms now consider themselves to be edgy and rebellious. The fact that they use lingo and euphemisms in their speech, and the idea that they could get demonetized or taken down at any moment seem to be giving them credibility. The channel numbers are growing.

    reply to this | link to this | view in chronology ]

    • icon
      Mike Masnick (profile), 26 Jul 2021 @ 10:17am

      Re: Counterproductive

      the idea that they could get demonetized or taken down at any moment seem to be giving them credibility.

      No it doesn't, Koby.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 26 Jul 2021 @ 10:32am

        Re: Re: Counterproductive

        I agree with you that it doesn't give them credibility Mike, but to them it does. Getting taken down is "evidence" that their truth is just being suppressed by The Man, and flying under the radar with their codewords is Fighting The Power.

        reply to this | link to this | view in chronology ]

        • icon
          James Burkhardt (profile), 26 Jul 2021 @ 10:44am

          Re: Re: Re: Counterproductive

          Also note he is responding to Koby, the current resident anti-moderationist. Koby is trying to make the arguement that the constant "censorship" gives credibility to the arguments that they are being censored and Q.E.D. social media should be government so they stop censoring. Its been his recent pivot, to suggest the arguement in one post rather than actually make it over 10, which reduces the spam flags he gets.

          And the answer is no. No it doesn't. It doesn't matter if they think it gives them credibility, their claims are not legally or scientifically credible.

          reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 26 Jul 2021 @ 11:01am

        Re: Re: Counterproductive

        You're right but to them it does and only adds more fuel to the fire.

        reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 26 Jul 2021 @ 9:56am

    A very important question though: Are they still spreading misinformation?

    If nobody understands their message, and it's just a private group of crazies, I can't see that it's really misinformation any more.

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 26 Jul 2021 @ 10:16am

      Re:

      Are they still spreading misinformation?

      Yes, and remember that humans are much better at extracting the message than algorithms, ,so what is hidden from the algorithm is easily read by humans. Also, the cult effect often keeps doubters from examining issues and leaving the cult.

      reply to this | link to this | view in chronology ]

      • identicon
        Anonymous Coward, 26 Jul 2021 @ 11:09am

        Re: Re:

        Hmm sounds like Facebook does/needs to (I don't use it) also block any non-approved language (for which they don't have people able to vet content) to combat misinformation. I guess they could also just flat out say: conversation in other languages isn't verified.

        reply to this | link to this | view in chronology ]

        • identicon
          TFG, 26 Jul 2021 @ 12:59pm

          Re: Re: Re:

          That wouldn't do a thing to combat this, and would only cut off a large part of their consumer base.

          Point the first: these are code-words in English. "Dancing" is a word in English, that is used most often to refer to actually dancing. Blocking languages would have zero effect on this tactic.

          Point the second: Facebook is used near globally, and in countries that have languages in the double-digits. For example, Cote d'Ivoire, a country in West Africa, has the national language of French since it is a former French colony. However, there are around 70 different indigenous languages in the country, and Facebook is used by damn near everyone who can read.

          Repeat this scenario across all of Africa - and that's just the one continent. So, any blocking of entire languages is going to cut off access to Facebook for huge communities ... and have zero effect on the tactics used by this group.

          That's before even getting into what would be "approved" or "non-approved" and even how you would implement detection of and therefore blocking of said languages: bear in mind that typos occur and just plain bad spelling rather frequently, wihch culod mkae it vrey dfifilcut to dtecet a lnaugage as Egnlsih for a cmoupetr, even though it remains entirely understandable for humans.

          reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 26 Jul 2021 @ 10:11am

    Maybe a wall isn't such a bad idea...

    For the first time ever, I'm of the belief that maybe a wall isn't such a bad idea. Build it around these anti-vax nuts, so the rest of us can move on. Because if you're anti-vaccination in 2021 after all that's happened, society as a whole needs to consider protecting itself from what are frankly, stupid people.

    I'm firmly convinced that a significant portion of the anti-vax crowd are shunning the vaccine not because they think it's ineffective, but because there's an overwhelming contempt they have against anything sensible. Its almost as if it's a competition between themselves to see who can be the biggest idiot. They'd rather bet on the horse with the broken leg somehow winning the race.

    If you can't be vaccinated because of a medical reason, so be it - they can stay outside the wall with the rest of us, where their chance of survival would be statistically better - if they choose. But if you're thinking that the vaccine contains a microchip, or is part of some big government mind control plot, then chuck that dumbass right over the wall where they can be among fellow idiots patting themselves on the back as to how 'smart' they are.

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 26 Jul 2021 @ 11:08am

    This is just a question, but would the pharmaceutical industry, as powerful as they are, ever allow any studies or their results to be publicized which would show any dangers in their products, including vaccines, and negatively impact their profits?

    reply to this | link to this | view in chronology ]

  • identicon
    Anonymous Coward, 26 Jul 2021 @ 11:10am

    "I'm firmly convinced that a significant portion of the anti-vax crowd are shunning the vaccine not because they think it's ineffective, but because there's an overwhelming contempt they have against anything sensible."

    I disagree. The FDA has approved the vaccines for EUA only, they are not fully approved. Most of this goes away once they are fully approved. So lets get on with it?

    reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 26 Jul 2021 @ 11:19am

      Re:

      most of this will not go away after full approval

      or do you think antivaxxers are only attacking the Covid vaccine?

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 26 Jul 2021 @ 1:04pm

      Re:

      If not for the unvaccinated keeping the emergency alive and well, perhaps we could've already gotten on with it.

      Then again, don't get me wrong - I have no empathy for these anti-vax people. Their arguments are by far, some of the stupidest things I've heard of. And for 'reasonable adults' to believe that north of 600,000 people are dead for nothing, they don't deserve any.

      No, I'm all for shunning these irresponsible assholes to oblivion, with the goal of ridding the gene pool of what will likely get the rest of us killed. And before anyone chimes in on how that sounds like eugenics, does it count when the discrimination is self-inflicted?

      reply to this | link to this | view in chronology ]

      • identicon
        TFG, 26 Jul 2021 @ 1:10pm

        Re: Re:

        I don't have much empathy for them either.

        But unfortunately, their actions harm more than just themselves - their children, for example, who they raise in this fashion, and who can't necessarily get the protection they need until they're "adults."

        The regular folk who can't get proper hospital care because the hospitals are flooded with anti-vaxxers who, surprise surprise, got sick.

        The people who have actual medical reasons that they can't get a vaccine (immuno-compromised, for example), who are then put at risk by the lack of the herd immunity that is part and parcel of vaccine effectiveness.

        And the people who, while not necessarily buying into all the anti-vax nonsense, are fed enough misinformation and disinformation that they hesitate, just long enough to wind up paying for it.

        Not to say that there's anything different to do because of this ... just to keep in mind that they do harm to others as well.

        reply to this | link to this | view in chronology ]

        • identicon
          Anonymous Coward, 26 Jul 2021 @ 1:19pm

          Re: Re: Re:

          ... just to keep in mind that they do harm to others as well.

          Understood. I'm the one who wants to build the wall around them. For some, I'd assume it would be the best thing ever. I'd even go so far as telling them Trump made it happen and Mexico paid for it.

          That would take care of the problem with hospitals being inaccessible to those who aren't mentally deficient, and the immuno-compromised could live life with a higher degree of safety outside the wall.

          The children of these morons would be unlucky for sure, but I'm not sure it would change anything as far as their current exposure.

          reply to this | link to this | view in chronology ]

  • icon
    Bloof (profile), 26 Jul 2021 @ 11:17am

    On the plus side, as we saw with nazi groups and their switch to babytalk to try and mask the hate, it's a lot harder for them to recruit when people need super secret decoder ring to decipher the gibberish they spout. It's why the likes of Koby want an end to content moderation, so they don't need to 'hide their power level' and can just openly normalise hate and lunacy.

    Companies will always have to play whack-a-mole with stupid, doesn't mean they shouldn't try.

    reply to this | link to this | view in chronology ]

    • icon
      That One Guy (profile), 26 Jul 2021 @ 12:55pm

      Re:

      That does seem to be a pretty solid upside to be sure in that their efforts to hide from moderation also means it's a lot harder for them to rope in more converts because anyone who just wanders by is likely to see nothing more than nonsensical rambling but by a bunch of loons(which they are, just not in that way).

      reply to this | link to this | view in chronology ]

  • identicon
    christenson, 26 Jul 2021 @ 11:27am

    Flagging??

    Maybe, just maybe, flagging content as questionable, just like we do on Techdirt, or twitter did, might be a better tool???

    If the consequence of misinformation is only to get flagged as BS, it leaves little incentive to go hide it.

    reply to this | link to this | view in chronology ]

  • This comment has been flagged by the community. Click here to show it
    icon
    restless94110 (profile), 26 Jul 2021 @ 11:30am

    Curious

    Why would you want to delete an opposite point of view? Especially in lite of all of the facts coming out revealing the dangerous effects for some of the vaccinations?

    Always in previous times if even 20 people died after being vaccinated that was sufficient to halt vaccinations. So now, it's in the thousands and you write that we need to censor that news?

    For why?

    Could you start making sense? And making it soon?

    reply to this | link to this | view in chronology ]

    • icon
      Mike Masnick (profile), 26 Jul 2021 @ 12:10pm

      Re: Curious

      Why would you want to delete an opposite point of view?

      It's not an "opposite point of view," it's directly misinformation that is causing people to die. There's a difference between "a difference of opinion" and "blatantly false information that lead people to do things that will kill them."

      Always in previous times if even 20 people died after being vaccinated that was sufficient to halt vaccinations. So now, it's in the thousands and you write that we need to censor that news?

      This is blatantly false misinformation.

      For why?

      Because people like you are literally causing people to die. I don't know if you do it for shits and giggles or if your honestly this stupid, but stop it.

      reply to this | link to this | view in chronology ]

      • icon
        Bloof (profile), 26 Jul 2021 @ 12:41pm

        Re: Re: Curious

        Judging by their post history, they're one of those people who want the world to burn because they feel it'd hasten the return of the GOP to power. Doesn't matter to them how many people die, as long as Trump or someone in his mold return to punishing non whites, women and the gays.

        reply to this | link to this | view in chronology ]

    • icon
      Bloof (profile), 26 Jul 2021 @ 12:38pm

      Re: Curious

      So how many people have been rendered magnetic by vaccines then? How many people have died due to exposure to shed proteins from the vaccinated? How many people have found the microchip that is supposedly inserted with the vaccine? How many people were left infertile? How many people have had their 'god gene' turned off and lost all religion because they took a vaccine? How many of the BS claims you people have made have turned out to be true?

      You can't take a victory lap because a medicine has very rare side effects and claim that means you were right continually claiming the the vaccine is a one hit kill that will have your soul fed-exed to satan.

      reply to this | link to this | view in chronology ]

    • icon
      ECA (profile), 26 Jul 2021 @ 12:46pm

      Re: Curious

      "if even 20 people died after being vaccinated that was sufficient to halt vaccinations. So now, it's in the thousands "

      Lets see, you like numbers?
      ?Out of those Counted and infected the numbers are Near 3% death rate.
      Out of the counted and gotten the Shot? Millions compared to Thousands?

      Its not bad what you are saying, its the numbers and Where you got them. This is like TV, an actor representing a Doctor, ISNT A DOCTOR. Having a person that is a dentist, isnt a Virologist giving the SAME info.
      Looking at a situation, YOURSELF, you must look at everything, NOT 1 opinion of whats happening. Like watching a hockey game and a person says the score Will be for this or that side, but dont tell you how the game is played and HOW the other team lost.

      Get it? Got it? very good.

      I could point fingers at this and that, about how the Virus started, and never tell you HOW it got spread. Understanding that this world has made transmission EASY, world wide, Is abit simple, but Many just want to Blame. ROC/Taiwan, LOCKED UP FAST as soon as they saw a few people that had been to China get sick. Even after the WHO wouldnt answer any questions ROC had of what was happening.
      Go look up Legionaries Disease, what caused it and how it got spread. Then feel proud you werent in those families.

      reply to this | link to this | view in chronology ]

    • identicon
      Anonymous Coward, 26 Jul 2021 @ 1:12pm

      Re: Curious

      Why would you want to delete an opposite point of view?

      Well, to be honest, if it's only the unvaccinated with 'muh freedums' as the reason for their lack of vaccination, I'm fine with leaving it up, and letting Darwin sort it out. We would certainly have been done with this pandemic by now if not for those simple-minded pieces of shit. Believe me, there's a lot of people at this point going 'meh, fuck those morons and their alternate facts.'

      You dumb fucks are sure going to lose a lot of company once natural selection says 'fuck your freedum!'.

      reply to this | link to this | view in chronology ]

  • icon
    ECA (profile), 26 Jul 2021 @ 12:33pm

    strange, very strange

    Its a wonder this type of thing hasnt worked in the past. :)
    A gov. that gets tons of letters or a Corp that 'Says' that no one minds look at all this data. :)
    How many Companies have setup Fake companies to give them faked data?
    How many companies created to create faked data, have Very important names, but are designed to give the customer Exactly what they want in data?

    reply to this | link to this | view in chronology ]


Add Your Comment

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Close

Add A Reply

Have a Techdirt Account? Sign in now. Want one? Register here



Subscribe to the Techdirt Daily newsletter




Comment Options:

  • Use markdown. Use plain text.
  • Remember name/email/url (set a cookie)

Follow Techdirt
Special Affiliate Offer

Essential Reading
Techdirt Deals
Report this ad  |  Hide Techdirt ads
Techdirt Insider Chat
Recent Stories
.

This site, like most other sites on the web, uses cookies. For more information, see our privacy policy. Got it
Close

Email This

This feature is only available to registered users. Register or sign in to use it.