Twitter Testing Secret Filter To Stop Abuse: Is That A Good Thing Or An Attack On Free Speech?
from the this-isn't-easy dept
For quite some time now, it’s become popular for people to point to some (often horrible) abuse on Twitter and demand that Twitter “do something” about it. It’s rather easy to find tons of articles calling for Twitter to do more.
And apparently Twitter is listening. As Sarah Jeong notes in a great article over at The Verge, it appears that Twitter recently deployed a feature to block some very abusive tweets, going further than its past tactics of pulling abusive tweets and killing accounts. In this case, it blocked certain tweets from being sent altogether. It appeared to be a filter that combined certain keywords with an @ symbol towards a person receiving a lot of abuse. Of course, as with many such things, the abusers just sought ways around the filter.
For a while, at least, Berger didn?t receive any tweets containing anti-Semitic slurs, including relatively innocuous words like “rat.” If an account attempted to @-mention her in a tweet containing certain slurs, it would receive an error message, and the tweet would not be allowed to send. Frustrated by their inability to tweet at Berger, the harassers began to find novel ways to defeat the filter, like using dashes between the letters of slurs, or pictures to evade the text filters. One white supremacist site documented various ways to evade Twitter?s censorship, urging others to “keep this rolling, no matter what.”
The assumption, from those with at least some understanding of what happened, is that it was done via Twitter’s spam filter:
A source familiar with the incident told us, “Things were used that were definitely abnormal.”
A former engineer at Twitter, speaking on the condition of anonymity, agreed, saying, “There?s no system expressly designed to censor communication between individuals. ? It?s not normal, what they?re doing.”
He and another former Twitter employee speculated that the censorship might have been repurposed from anti-spam tools?in particular, BotMaker, which is described here in an engineering blog post by Twitter. BotMaker can, according to Twitter “deny any Tweets” that match certain conditions. A tweet that runs afoul of BotMaker will simply be prevented from being sent out?an error message will pop up instead. The system is, according to a source, “really open-ended” and is frequently edited by contractors under wide-ranging conditions in order to effectively fight spam.
Trollish behavior and abusive behavior is definitely a concern. And making people feel welcome online on services like Twitter, without fear of harassment seems like a welcome goal. But there are also some pretty serious concerns about how this is all happening in a non-transparent manner, leaving it open to abuse in its own way:
What?s worrisome to free speech advocacy groups like the EFF about this incident is how quietly it happened. Others may see the bigger problem being the fact that it appears to have been done for the benefit of a single, high-profile user, rather than to fix Twitter?s larger harassment issues. The selective censorship doesn?t seem to reflect a change in Twitter abuse policies or how they handle abuse directed at the average user; aside from a vague public statement by Twitter that elides the specific details of the unprecedented move, and a few, mostly-unread complaints by white supremacists, the entire thing could have gone unnoticed.
Eva Galperin thinks incidents like these could be put in check by transparency reports documenting the application of the terms of services, similar to how Twitter already puts out transparency reports for government requests and DMCA notices. But while a transparency report might offer users better information as to how and why their tweets are removed, some still worry about the free-speech ramifications of what transpired. One source familiar with the matter said that the tools Twitter is testing “are extremely aggressive and could be preventing political speech down the road.” He added, “are these systems going to be used whenever politicians are upset about something?”
It opens up some pretty serious questions, and you enter into the same slippery slope that questions around legislating against harassing speech, or even things like revenge porn, start to raise some concerns. It’s not that anyone wants to support those activities. The harassment here is absolutely deplorable. But there is a serious concern about where it leads when an intermediary suddenly takes it upon itself to determine what is and what is not acceptable speech. Was it used for good reasons in this case? Probably. But will it always be done in that manner? That’s where it gets a lot trickier.
Filed Under: abuse, filter, free speech
Companies: twitter
Comments on “Twitter Testing Secret Filter To Stop Abuse: Is That A Good Thing Or An Attack On Free Speech?”
Every time I try to tweet a link to this article, I get an error message and it doesn’t send.
Stupid computer.
Good first step.
Now if Twitter will start blocking all those stupid liberals they can go a long ways to “making people” like me “feel welcome online.”
Just sayin…
I hope they will at least start banning those “feminist” who joke about hanging white men and pinatas. Most of the time the biggest abusers complain the most.
Wrong approach to the problem
No matter how worthy the goal, trying to suppress undesirable speech with automated filters is a poor use of resources. E-mail spam, and more recently general comment spam, have shown this repeatedly. If automated systems could reliably recognize good and bad speech, spam would not be a problem. Instead, most people hide behind spam filters and hope that the filter does not eat anything important.
Among other problems, what if someone wanted to tweet a supportive (if vacuous) statement like “@Berger is a good person. Don’t call her a rat.” or an informative (in certain contexts) statement like “@Berger, when you travel to #NewYorkCity, beware the rats. They’re mean and usually diseased.” Both of these are harmless to good statements, yet would likely be wrongly flagged by the filter because it happens to contain a word that has been twisted to be a slur.
Given that Twitter is a private company, free speech shouldn’t even be mentioned in the same sentence.
Re: Re:
I dont think twitter is private – wasnt’ there an IPO?
That makes ’em a public company.
Re: Re: Re:
They are not a public entity in the sense a university or other taxpayer funded organization is – they are not the government and therefore they have no capacity to stamp on anyone’s freedom of speech. They are under zero obligation to provide a platform for behavior they do not wish to tolerate.
Re: Re: Re: Re:
“They are under zero obligation to provide a platform for behavior they do not wish to tolerate.”
Kind of like how private phone companies should be able censor what people say on the phone.
Re: Re: Re:2 Re:
So Twitter should be classed as a public utility?
Re: Re: Re:3 Re:
Why should any privately owned company be “classed as a public utility”?
Re: Re: Re:3 Re:
Wait, so do you or do you not say that private companies “are under zero obligation to provide a platform for behavior they do not wish to tolerate”? Back-pedal much?
Re: Re: Re:
Private as in not public (state controlled). The only free speech concerns are if the US government wanted to force this kind of censorship. Users have no free speech protections from Twitter, because the speech belongs to Twitter, not the end user.
Re: Re: Re:
That makes them a “publicly traded company”, a term that is often shortened to just “public company” with the assumption that people know what it means.
Re: Re:
While this is not a first amendment issue, it is something that should be concerning for people that want free speech.
Their motivation is important. If they are doing this kind of filtering under pressure that if they do not they may be held liable, this kind of thing can have a chilling effect not only on the use of Twitter, but on other companies that may want to compete in the same space.
Proponents of free speech should always cheer on companies and services that allow free expression of unpopular ideas and frown upon companies that quell speech in any way.
Re: Re: Re:
Proponents of free speech should not use products and services that reserve the right to censor said speech.
Remember these services are shopping malls, not public squares.
Re: Re: Re: Re:
What public forum that can reach as many people as Twitter but is not privately owned do you propose people start using?
Re: Re: Re:2 Re:
Re: Re: Re:2 Re:
twitter could go away tomorrow, and the only thing that would happen, is people would find other avenues to waste time…
i have NEVER (except excerpted in articles) seen a tweet (sheesh i feel like i’m losing IQ points just using this stupid lingo) or care to; my life is NOT poorer for that…
that there are a few REAL dissidents, occupy types, etc who have used it to good effect is besides the point, it is mostly useless…
Re: Re: Re:2 Re:
I personally have written such a platform – and I can tell you from experience that the main problem with ANY such platform is adoption.
If you can get enough people to start using your platform, it then becomes viable. The problem is, everybody is on Twitter, so nobody wants to go anywhere else because nobody is there.
When they don’t like what Twitter does, people just quit using it instead of pursuing alternatives. It’s the easy (lazy) approach – getting your friends to sign up for another platform is just too much work.
Re: Re: Re: Re:
“Remember these services are shopping malls, not public squares.”
Interesting comment, since courts have ruled that places like shopping malls are public spaces for certain things, including free speech issues.
Re: Fascist nonsense.
The fact that they are a corporation should not let them off the hook for living up to our founding ideals.
Re: Re:
For the nth time, First Amendment =/= the ideal of free speech. One can support free speech outside the applicability of the U.S. Constitution.
Re: Re:
Why not? Just because the first amendment doesn’t constrain them in no way means that the topic of free speech is suddenly forbidden. It only means that they aren’t breaking the law if they restrict it.
Re: Re:
You seem to have confused free speech with the 1st Amendment. They are not the same thing.
Content-based measures don't work and won't work
We — well, some of us anyway — learned that in the anti-spam world years ago. Any sufficiently-clueful adversary can defeat content-driven filters at will, either by (a) performing sufficient testing using their own copy of it (if open-source) or (b) performing sufficient testing to reverse-engineer it (if closed-source). Of course not all adversaries are that clueful, and not all of the ones who are have the motivation to tackle the problem, but that still leaves enough to defeat all such measures. (And some of the enterprising ones will sell the solution to others who are less motivated and/or less clueful, but willing to pay for a ready-made solution.)
Content-based measures do have their uses: properly deployed, they can yield interesting and useful information about possible sources of spam, phishing, typosquatting and other forms of abuse. But the best use for that information is to collect it, collate it, summarize and then bring it to the attention of (well-trained) humans, who will hopefully exercise due diligence and will use out-of-band data to augment and check their conclusions.
That’s not what most people want to hear: they want to hear that the problem can be completely solved via automation, because that’s cheap and easy. But it can’t, and attempts to do so inevitably devolve into a cyclical pattern of (new) attack and (new) defense, followed by (newer) attack and (newer) defense, followed by more of the same. We’ve seen this with anti-virus software. We’ve seen this with anti-spam software. And now we’re seeing with anti-whatever-it-is that Twitter wants to call this.
Marcus Ranum, in his brilliant rant “The Six Dumbest Ideas in Computer Security“, see http://www.ranum.com/security/computer_security/editorials/dumb/ for the whole thing, nailed this nearly a decade ago, as dumb idea #2: Enumerating Badness. He says in part: “One clear symptom that you have a case of “Enumerating Badness” is that you’ve got a system or software that needs signature updates on a regular basis […]”. Bingo.
Pointless and dangerous
This is pretty darn stupid. It’s pretty much impossible to stop harassment with any sort of speech filter whatsoever. Inevitably, people can and will find a way around it, and then it will inconvenience your other users. There’s really no point, and call me paranoid but I can see the National Spying Agency serving them up a National Security Letter forcing them to censor anti-government tweets.
Why is this complicated? Just give spam control to users
Instead of foisting spam filters unilaterally, just allow users to filter and reject tweets based on their own individual filter settings.
It’s what I do with Open Tweet Filter and my choices apply to me and no one else.
Re: Why is this complicated? Just give spam control to users
Agreed. When it comes to social networking applications and communications, I reserve it for what an acquaintance of mine calls “the Three F’s”
Friends, Fun, Family.
Many applications do anyway, but I see no reason why I as a user can’t have the tools to restrict my own coversations to suit what I want from it.
Re: Why is this complicated? Just give spam control to users
Also: 1) don’t completely block the tweet, just have it not show up the the recipient or the recipient’s followers, 2) do this quietly, so the sender isn’t aware of it thus receives no feedback to use to try to get around the recipient’s personalized filters.
sticks and stones may break my bones ...
A big problem here is that this person is a member of Parliament. People who are thin-skinned and easily offended should not be government officials, period. It also should go without saying that people who get relentlessly trolled are usually troll feeders. Many of us learned as young children that anyone who couldn’t take teasing would end up getting a lot more of it. Apparently some adults never learned this simple fact of human nature. And the more extensive the text filtering mechanisms, the more determined trolls will be to crack them, and then rub it in everyone’s face.
But as is usually the case, people who are high up tend to get special privileges not afforded to regular people, and these special privileges have a tendency to increase over time.
“The harassment here is absolutely deplorable.”
And that’s the whole point. There have been numerous cases in the past in which bigoted speech (or misconduct in general) by anonymous people turned out to be a “false-flag operation” done to create sympathy for the victim and demonize the side thought to be responsible. Even in “legitimate” cases, an agent provocateur may be planted behind the scenes. We need to keep in mind that these childish antics are not always what they seem.
Of course, there is genuine harassment on these services but this can and will be abused by ideologues to silence dissent. Feminists, religions, etc. have a history of false flagging videos off youtube, getting twitter accounts banned, doxxing, etc. and there was no abuse or harassment, just criticism and facts proving them wrong. Those groups are often the ones being abusive and yet their accounts stay active. Obvious agenda is obvious.
And on the other hand...
I’ve lost count of how many legitimate emails never got delivered to me because filters like this classified them as spam and dumped them.
Including emails I specifically requested.
not feasible
An inordinate number of people seem to feel that they have a right not to be offended. Interestingly enough many of these same people are incredibly offensive to others, believing that their upbringing and opinions are just the natural order of the universe.
There are some two million words in English, and I would defy any filter to be able to deal with them, never mind the implications of an enormous number of ways to interpret combinations.
Quite a few companies on the net have gone out of business attempting censorship
The same Twitter?
This is the same Twitter that’s using AT&T and Verizon’s “perma-cookies” to track their users, right? Right. If they’re willing to do that, then I’m not surprised if they’re willing to do all kinds of other nefarious things.
To me it is fairly simple. Their servers their rules. If you want free speech then setup your own system for publishing. Then you can control what is published. That is the only way to guarantee free speech. If twitter was going out and hampering your ability to say something elsewhere then I would say the company was violating your free speech, but they are not. So it is completely within their rights to block content they find unacceptable and not be transparent about it.
If you do not like it then use another service. And that is another reason why I do not consider it violation of anyons freedom of speech. It is not the only way to say something on the internet, just one of the most widely used. That does not justify concerns about free speech.
Re: Re:
“To me it is fairly simple. Their servers their rules.”
Who said otherwise?
“So it is completely within their rights to block content they find unacceptable and not be transparent about it.”
Again, who said otherwise. Straw-man much?
“And that is another reason why I do not consider it violation of anyons freedom of speech.”
But if you’re trying to twist that into saying that no one’s speech is being restricted, then you’re just wrong. No one is saying that Twitter is breaking any laws. Just pointing out what how they are restricting speech. And restricted speech is not free speech.
The most powerful tool to fight against such idiocy is simply to ignore them. And if they are dumb enough to directly threat or incite violence then there are laws for that. No filtering needed.
There is no expectation of free speech when using a private website. Why do people think otherwise?
They own the damn thing, you agree to their terms, end of story. Not happy? Use one of the 50 competitors.
Re: Re:
Nobody is saying otherwise. However, why do you think that nobody has the right to complain about a private website’s policies? That’s free speech in action.
Their Business, Their Rules
It’s their company. Their commitment to free speech is whatever *they* decide it needs to be.
They’re publishers. Users are just the saps who write their copy for free.
If you don’t want to be edited, get your own site.
Freedom of speech upheld
The harassment here is absolutely deplorable, but there is a serious concern about where it leads when an intermediary suddenly takes it upon itself to determine what is and what is not acceptable speech.
Actually, Twitter didn’t decide what people are allowed to say, they simply upheld the harassment recipients’ right to freedom of speech by blocking stuff aimed directly at them. While people have a right to say what they like, that right ends when it interferes with the rights of others, and tweeting @[name] to put something directly into someone’s Twitter stream forces them to see it, thus interfering with their right to freedom of speech. Simples!
Re: Freedom of speech upheld
Uh, I see where you’re coming from, but that’s not a freedom of speech issue, it’s a freedom from harassment issue. Freedom of speech =/= freedom to abuse other people.
The Open Tweet Filter recommended by NoahVail seems like the most reasonable response. They can’t bug you if they can’t get to you.
Re: Re: Freedom of speech upheld
Freedom of speech =/= freedom to abuse other people.
And isn’t that what I said?