Daily Deal: Spacebar Monitor Stand And 6-Port USB Hub
Team Prenda Loses Big Again: Told To Pay Over $650k For Bogus Defamation Lawsuit

'Nice Internet You've Got There… You Wouldn't Want Something To Happen To It…'

(Mis)Uses of Technology

from the this-is-no-longer-theoretical dept

Fri, Oct 21st 2016 10:46am -

Last month, we wrote about Bruce Schneier’s warning that certain unknown parties were carefully testing ways to take down the internet. They were doing carefully configured DDoS attacks, testing core internet infrastructure, focusing on key DNS servers. And, of course, we’ve also been talking about the rise of truly massive DDoS attacks, thanks to poorly secured Internet of Things (IoT) devices, and ancient, unpatched bugs.

That all came to a head this morning when large chunks of the internet went down for about two hours, thanks to a massive DDoS attack targeting managed DNS provider Dyn. Most of the down sites are back (I’m still having trouble reaching Twitter), but it was pretty widespread, and lots of big name sites all went down. Just check out this screenshot from Downdetector showing the outages on a bunch of sites:

You’ll see not all of them have downtime (and the big ISPs, as always, show lots of complaints about downtimes), but a ton of those sites show a giant spike in downtime for a few hours.

So, once again, we’d like to point out that this is as problem that the internet community needs to start solving now. There’s been a theoretical threat for a while, but it’s no longer so theoretical. Yes, some people point out that this is a difficult thing to deal with. If you’re pointing people to websites, even if we were to move to a more distributed system, there are almost always some kinds of chokepoints, and those with malicious intent will always, eventually, target those chokepoints. But there has to be a better way — because if there isn’t, this kind of thing is going to become a lot worse.

Filed Under: , , , ,
Companies: dyn

71 CommentsLeave a Comment
If you liked this post, you may also be interested in...
Rate this comment as insightful
Rate this comment as funny
You have rated this comment as insightful
You have rated this comment as funny
Flag this comment as abusive/trolling/spam
You have flagged this comment
The first word has already been claimed
The last word has already been claimed
Insightful Lightbulb icon Funny Laughing icon Abusive/trolling/spam Flag icon Insightful badge Lightbulb icon Funny badge Laughing icon Comments icon

Comments on “'Nice Internet You've Got There… You Wouldn't Want Something To Happen To It…'”

Subscribe: RSS Leave a comment
71 Comments
Ninja (profile) says:

Telling the infra-structure players alone must ‘do something’ is naive at best. The real culprits here are a mix of IoT and other hardware manufacturers that couldn’t care less about security. They need to be hurt for their lack of care where it hurts the most: their pockets.

So yes, the infra-structure portion can help mitigate the problem but unless we start taking security very seriously it won’t matter.

Of course, one must not forget the perpetrators should also be severely punished and if it’s a state actor maybe even cut it entirely from the network to preserve its health.

TKnarr (profile) says:

Re: Re:

It requires a number of things on the infrastructure side. Standard practice with IoT needs to be to have the devices on a separate non-Internet-connected network which requires the cooperation of router makers and users. Consumer routers need to implement RFC 3704 egress filtering by default. ISPs need to implement 3704 filtering on the customer side (the head-ends and/or CPE depending on physical configuration) and on the upstream side. Upstream networks need to implement 3704 filtering even if it means reconfiguring their topology to separate the non-transit parts of their network from the transit network. All parties involved need to stop depending on other parties to do the work and configure their own networks as if their measures are the only thing standing in the way of a massive DDoS attack. And finally, targeted parties need to be able to hold the originating and intermediate networks financially liable for all the costs involved, not just the small fraction of the access bill for the downtime, when those networks failed to enforce 3704 compliance.

That won’t stop all of it, but it’ll stop a huge portion of it. The rest can only really be dealt with by forcing end users (consumer or business) to clean up infected/compromised systems on their networks. Given the intransigence of the average end-user (whether a consumer or a company’s IT management) I don’t see anything short of big sticks wielded effectively having any effect.

Anonymous Coward says:

Re: Re: Re:

It also requires accountability, something we used to have on this network a few decades back, but no longer do.

The people whose infrastructure is responsible for this have to be held personally accountable. Publicly named. Publicly shamed, Publicly fired. Publicly denounced. Publicly humiliated.

Because it’s their fault. They’ve failed to meet minimum acceptable standards for Internet operations and they deserve to pay a steep price for it. Many of them should never work in this industry again.

Yes, that’s harsh, but having a big chunk of the Internet taken out — and the attackers could have done more and done it longer if they wishes — is a pretty big deal. Harsh penalties are appropriate.

And maybe, just maybe, everyone else will pay attention and start doing the things that they should have done 10-20 years ago in order to defend the Internet, not merely defend themselves.

Anonymous Coward says:

Re: Re: Re:

“And finally, targeted parties need to be able to hold the originating and intermediate networks financially liable for all the costs involved, not just the small fraction of the access bill for the downtime, when those networks failed to enforce 3704 compliance.”

One point of contention, it’s probably minor to most. Say I order a private vlan from some IXP. Should the IXP be responsible for BCP38, after all the connection itself is just traversing their network to another provider. They certainly can not filter bogons, and how are they to know what ASNs or IPs should traverse that link.

copbox says:

Re: Re: Re:

I don’t know what 3704 is. Nor do I care.
on my net you will be stripped of IPV6.
any blocking rule should be in THREE unless you got a specific purpose
CUSTOM FORWARD
CUSTOM INPUT
CUSTOM OUPUT

ingress, egress, and forwarding

These devices getting hacked must be directly facing the web? Yes? I have several a SONY blue ray player right it has a 192.168.0.X I got a Marantz it has a 192.168.0.XX
Each IP needs rules to get out-crap works fine here and I got the youtube browser and the Opera browser in these boxes. All working just fine. Another thing is I constantly maintain a list of domain to IP’s so if DNS goes down I can load up techdirt at http://104.25.105.28 if i can punch thru cloudfare insanity.

People that don’t run their own boxes don’t get it. You can quote RFC’s all day long it’s freedom, tcpip and networking creativity that matter.

I seen a LOT of this wireless crap at the hospital, but is it even plugged in? I doubt it.

Anonymous Coward says:

Re: Re:

“Of course, one must not forget the perpetrators should also be severely punished and if it’s a state actor maybe even cut it entirely from the network to preserve its health”

Assuming the identity of the bot-herder is known or can be discovered, It would be wise to shut down the botnet (not just the attack) prior to taking any steps to remove the herder or their network access.

If the botnet is reasonably intelligently designed, cutting the perp off from the internet may make it next to impossible to send a shutdown signal the C&C infrastructure will recognize.

Anonymous Coward says:

Re: Re: Re:

It’s not bread and circuses it was a lot of sites people use to communicate with each other and share news, like Twitter and Reddit. With the internet down people can only get the news from the “government approved sources”. This site was also blocked for me for awhile, btw. Right before an election. Bet it happens again Nov 8.

Anonymous Coward says:

Re: Re: Re:2 Re:

That could be read a few ways…

As a child of the 90’s, there is only one way to read it. I chuckle every time someone says “Do you cyber?” here, because that was exactly the same question folks said on BBS’s and the early internet back in the 90’s, but for entirely different, though very similar reasons.

Thad (user link) says:

Re: Re: Re: Nerd Harder!

There’s an easy way to fix this.

Make companies financially liable for security issues in their products in a way that makes securing their software less expensive than not.

Sure, it’s just that easy if you think laws are vague, handwavy things.

In practice, what does this actually mean? Which companies are financially liable for security issues in which products? How quickly does the vulnerability have to be fixed to avoid liability? What’s the statute of limitations?

If there’s a vulnerability in the Linux kernel that affects Samsung phones, who’s liable? Samsung, Google, the Linux Foundation, all of the above? If the vuln has already been patched upstream, and Google’s already pushed an update, but Samsung isn’t staying up on Google’s updates, then presumably you’d hold Samsung liable but not Google or Linux, right? Okay. What if Samsung’s rolled the updates out on some phones but not others? What should Samsung’s obligation be for supporting its old phones? Should it be defined in terms of age? Userbase?

And you trust legislators to understand all these issues and write reasonable laws that take all of them into account while still being strong enough to discourage companies from releasing insecure devices?

You’re basically saying that legislators need to nerd harder, which isn’t really any better than saying programmers do. Though at least you had a suggestion for a way of fixing the problem, which is more than Masnick gave us in the article.

sigalrm (profile) says:

Re: Re: Re:2 Nerd Harder!

here’s a more solid start, based on use of MITRE’s CVE system.

Assume Samsung is selling IoT enabled toasters, because why not. Everything’s better with a network stack. Anyway, MSRP on this toaster is $100usd and Samsung releases the product Jan 1, 2017, and ships 1000 toasters.

Now, if there are no open CVE’s on any component of the IoT stack on this toaster in the 90 days before Samsung ships, they’re effectively insulated from liability. Oh, and in that world, the sky is Fuscia.

But, If there is an open CVE was announced >= 90 days before Samsung launches the product, and it gets exploited, Samsung is the hook for 5% of the MSRP for each unit sold of said product for every 90 days of age on the CVE.

Example: Samsung begins selling their IoT enabled toaster (MSRP == $100usd) on Jan. 1, 2017. And they sold 1000 of them on day 1. Said toaster has a vulnerability that was announced on Aug. 15, 2016 (just outside the 90 day grace period). If one of these toasters gets exploited and causes trouble, Samsung is going to write a check for (5% of $100) == $5 for each of the 1000 toasters sold as of the date of the CVE being exploited, plus the same fine going forward for each non-patched unit they sell.

Now, pretend that vuln wasn’t released on Aug. 1, 2016, it was release on Aug. 1, 2016. Same ship date, same quantity. Except now instead of 5% per toaster, it’s 10%. Add 5% for every 90 day interval of CVE age. Also, allow the total penalty per unit to exceed 100% of MSRP with no upper bound. So, you release an IoT enabled toaster with a 12 year old ssh vuln, and it gets exploited? assume qty 4-90 day periods / year to make it easy, now your penalty is (48 * $5) = $240 * 1000 = $240k in fines for each $100MSRP toaster you sold.

And why use MSRP as the basis for the penalty? Well, because it’s both easy to validate and publicly verifiable.

No grace period, no appeal, cut a check to a high school to fund a secure coding class, because CVE’s are public and theres no way the organization “couldn’t have known”.

Oh, and multiple CVE’s? 5% per CVE, and scale it out.

If you can verifiably patch these toasters 100% then you restart the clock from the time the patch was pushed to the toaster. If you can’t patch them, well, eventually you’ll get to write a check big enough to make the board pay attention.

Bonus: Specifically disallow said penalties as a loss for tax purposes.

As to your other question: It’s a Samsung toaster running a google code, Samsung pays. It’s their label. If Samsung wants to go back and fight it out with Google based on contract terms, that’s fine, Samsung can attempt to recoup their (already paid) losses from Google.

(yeah, I know. There’s no chance this or anything like it will ever happen.)

Thad (user link) says:

Re: Re: Nerd Harder!

It’s a suggestion that the nerds need to do something, without any information whatsoever on what the nature of that “something” is. It is exactly “nerd harder”. It’s not quite as dire as the encryption backdoor debate (where the “nerd harder” advocates are pushing for things that are mathematically impossible), but it’s still not exactly helpful.

“Suggesting people start giving a damn” is vague to the point of uselessness too. Which people? “The internet community”, apparently. Whatever the fuck that means.

Anonymous Coward says:

So, once again, we’d like to point out that this is as problem that the internet community needs to start solving now.

May I point out to Techdirt that we are (see Hyperboria: http://hyperboria.net/ for an example), but that there is serious difficultieswith deploying any such technology. The vast majourity of people (corporate & individuals) can’t be bothered upgrading (most of whom won’t see the point), and many who can be bothered won’t do so as it (if not engineered correctly) will risk backwards incompatibility.

Engineering around these difficulties is a significant challenge I’ve only seen begin to be solved solved recently (and hyperboria could still be improved here).

Tl;dr Don’t ask us to start solving the problem: we have. Instead do what little you can to help us deploy it.

Christenson says:

Nerding harder...

The fundamentals are that I can’t *trust* my own computer, let alone yours.

Lacking trust in computers, *everything* is going to have to go to a bit-torrent style model with no central host (somebody already did this for websites, I forget the project name) because there are enough broadband IoT devices out there to DDOS any single individual, company, or any device performing a particular function. The biological analog should be obvious.

And, just as with fair use and copyright, the problem of discerning “legitimate” traffic (all of Techdirt’s fans) from “illegitimate” traffic (all of Techdirt’s haters, and 100 million of their bots, coordinated so they look just like its fans) is basically impossible.

Time to break the glass over the emergency tools and prepare for the internet to go down. Probably November 9.

Anonymous Coward says:

Re: Nerd Harder

Honestly, it isn’t nerd harder. For IoT, if the developers are too lazy to patch vulnerabilities than simply use a distro that will and setup a cron job to check and update automatically. For network operators, the BCP38 guidelines and BGP filtering will greatly reduce the possibility of your customers doing this from your network.
IE The tools are there, people just are not using them.

Nick (profile) says:

This isn’t getting nearly enough coverage as it should. I managed to catch an article on yahoo news (yeah yeah, laugh it up) about “temporary” 2 hour outages for some people on the east coast.

However, I cannot access the websites of some pretty major companies, such as soundcloud and twitter. If I used twitter, that might be an issue for me. But I know that a lot of people rely on it for their breaking news, and with a lot of other big name company sites down we cannot get up-to-date info.

This is scary bad. The fact that Amazon’s web service went down is scary. Big companies rely on AWS for their internet connectivity for things, and if that goes/stays down, it can mean a lot of lost income.

Anonymous Coward says:

Re: Re:

“This isn’t getting nearly enough coverage as it should”

Probably because it isn’t hitting everybody. If I wasn’t reading about it on the news sites I’d never have known. Been online in CST since before 6am, have used many of the major sites mentioned (and of course AWS at the back of many) all morning with no indication of any problems. (I don’t use FB but I have been using Amzn, TWTR, NYT, WAPO etc etc etc, major sites for work, and they’ve all been flying. Weird.) Literally except for reading about it I have not noticed anything. I feel left out.

Stupid Genius (profile) says:

Re: Response to: Nick on Oct 21st, 2016 @ 1:36pm

You have never heard of “Frontier” as in the company that just purchased Verizon’s FIos while they were rated 270 out of 278 different customer service providing entities. What good is these government bodies created to help consumers from being ripped off when a company (with nearly the worst CS rating) that has some money can purchase Verizon’s Fios service when Verizon was the internet providers leader in customer service. How the hell is that protecting the consumers.
Yes, it’s bad for Amazon but what about other small businesses that are totally revenue-dependent in their internet services staying up. There were companies in Florida with no internet service for a month and many more for weeks. Frontiers tech’s didn’t show up for appointments and when CS was contacted they just lied. One idiot called the consumer in the same landline he was there to repair to let them know he was there. They provided their cell phone numbers no less than 7 times for these brain-dead idiots. Mean-while they were chastising Warner Cable for over charging and throttling only to implement the exact same pricing structure except worse.
WTH!

Anonymous Coward says:

Fix it: White Hat Hacking

Start scanning and when you find a device with a default password, sign in and change it to something random.
If they can find them, so can we. And if the user can’t get in, they will just reset it to default. And it will be found again. Repeat.
Have done this dozens of times in the large and small companies I’ve worked for. Camera’s, scanners, printers, et cetera. If the customer/employee calls in a tech support ticket, they are talked thru how to reset, configure and set a good password.

Secondly, maybe some enterprising company/person could set-up a simple “Certified Safe Supported”. A small company could get a product, certifiy that it has security in ind, such as a) support for updates b) obvious passwords are not used/repeated c) I really don’t need to list them…

-dsr- (profile) says:

When you outsource to the cloud, you have a SPOF you can't see.

Whether or not Dyn should have been able to withstand this DDOS, whether or not the DDOS should have been prevented, it’s still a problem for all of Dyn’s customers that decided that they didn’t need any other DNS services because Dyn is the cloud.

On the DNS customer side, there’s no reason not to use multiple authoritative DNS providers, including running one yourself. The cleanest way of doing this is to run two or three widely separated DNS servers that only talk to your three DNS services. Even for huge zones, this is a cheap and idiot-resistant method.

On the resolving side, there’s no excuse for not having two or three nameservers listed on each of your computers. If you are small: one from your ISP, one from Google, one from any other service. If you are in any position to run caching DNS servers, do that as well.

Add Your Comment

Your email address will not be published. Required fields are marked *

Have a Techdirt Account? Sign in now. Want one? Register here

Comment Options:

Make this the or (get credits or sign in to see balance) what's this?

What's this?

Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop »

Daily Deal: Spacebar Monitor Stand And 6-Port USB Hub
Team Prenda Loses Big Again: Told To Pay Over $650k For Bogus Defamation Lawsuit
Follow Techdirt

Techdirt Daily Newsletter

Ctrl-Alt-Speech

A weekly news podcast from
Mike Masnick & Ben Whitelaw

Subscribe now to Ctrl-Alt-Speech »
Techdirt Deals
Techdirt Insider Discord
The latest chatter on the Techdirt Insider Discord channel...
Loading...