Content Moderation Case Study: Spam "Hacks" in Among Us (2020)
from the very-sus dept
Summary: From August to October of 2020, as the COVID-19 pandemic had no end in sight and plenty of people were still stuck at home, on lockdown, unable to gather with others, the video game Among Us became incredibly popular as a kind of party game when there were no parties. The game had already been out for a while, but for unclear reasons it became the go-to game during the pandemic. It was so popular that the company behind it, InnerSloth, cancelled its plans for a sequel, promising instead to focus on fixing up the existing game and dealing with some of the bugs that were popping up from such widespread usage.
Among the bugs that InnerSloth had to deal with was the ability to hack the game with various apps and tools that allowed users to possess more powers in the game than they should be able to have.
This came to a head in late October of 2020, when the game was apparently overrun by spam promoting a YouTuber named “Eris Loris.” Some of the spam had political messaging, but all of it told people to subscribe to that user’s YouTube account. Sometimes it came with vaguely worded threats of hacking if you didn’t subscribe. Other times it just told people to subscribe.
While this attack was variously described as both a “hack” and a “spammer,” it appears that it was a combination of both at work. The end result was spamming players in the game and making it impossible to keep playing, but it was also carried out via a hack that filled the game with bots designed to spread the message. The person who goes by the name Eris Loris told the website Kotaku that he did it because he thought it was funny:
“I was curious to see what would happen, and personally I found it funny,” Loris told Kotaku in a DM. “The anger and hatred is the part that makes it funny. If you care about a game and are willing to go and spam dislike some random dude on the internet because you can’t play it for 3 minutes, it’s stupid.” — “Eris Loris” to Kotaku reporter Nathan Grayson
InnerSloth admitted that it was aware of the problem and asked players to “bare with us” [sic] and only play private games or with players they knew and trusted until updates were made to the server. A developer for the game separately warned users that he was rolling out changes using a “faster method than I’ve done before” and, as such, that things might break.
Company Considerations:
- How much effort should be put towards preventative measures to try to block spamming, even before an app or service becomes wildly popular?
- At what level does spamming reach a point that it is critical to change the code of a game, perhaps even using “faster” and less reliable methods to combat the spamming than would normally be used?
- How do you balance resource allocations between having engineers improving the product and adding new features as compared to fighting back against malicious actors?
Issue Considerations:
- When something becomes popular, there are always those with nefarious intentions who want to take advantage of the platform’s popularity. Should companies proactively prepare for the unintended consequences of success? What can companies put in place to anticipate the actions of bad actors?
- Spammers and hackers sometimes go hand in hand with popular games and platforms. What are other risks (beyond just losing players/customers) if companies allow, or are slow at the removal, of those bad actors from the platform?
- Many developers leave platforms somewhat open to encourage third party developers to build on additional tools and services that make a game or service more useful. How does a developer determine the trade-offs between an open system to promote innovation and someone abusing that openness?
Resolution: The rapid updates Among Us developers made to the Among Us servers appeared to do the trick, and the Eris Loris spam quickly diminished soon after. There were some questions about whether or not there would be legal consequences for whoever was behind the attacks, but to date, nothing has happened.
There still remain a number of Among Us hacks out there, and some people have attempted to follow in the footsteps of Eris Loris — including someone going by the name Sire Soril (Eris Loris backwards) — but it appears that none of these have had much success at all, suggesting that InnerSloth’s initial fix was pretty successful in limiting the kinds of attacks that overwhelmed the system in October of 2020.
Originally posted to the Trust & Safety Foundation website.
Filed Under: among us, content moderation, eris loris, games, hacking, spam
Companies: innersloth
Comments on “Content Moderation Case Study: Spam "Hacks" in Among Us (2020)”
How do you account for control? how do you know it didn’t just reach its peak organically, or that other factors outside of the problem area weren’t the cause of the drop (like say, changes in YouTube or maybe reaching the desired sub count)?
Re: Reading carefully.... "**appeared** to do the trick"!
For Techdirt, a site with a delightful post called "lies, damned lies, and audience metrics", almost the entire readership knows that "appeared to" means only "happened shortly after" and not "was actually caused by", and that choice of wording was quite deliberate.
It’s gonna take some footwork to gather evidence for causation, and some really clever tricks or a natural experiment to finally prove it beyond reasonable doubt.
Re: Re:
I’m sure that only the developers have access to that information, and they would be able to see from their logs how many attempts were happening and how many were blocked by their activity. The will know, even if they’re not about to directly share all that information or admit that they didn’t really do anything if that were the case.
From an outside point of view, you can only look at the timing correlation and the fact that future attempts seem to have been mitigated to conclude that what they did at least had some effect.
This is QAs job
Back when I worked in QA 20 years ago, we tested for spam, hacks, penetration and the likes. The QA on games today is non-existent compared to those times. Many games ship with zero QA on payroll and crowdsource testing to Alpha testers – for free. Many games are vulnerable to spam, nowadays. It’s sad, but the reality of indie games.