I never cease to be amazed by this.We’re now nearly two decades into the development of the Internet as a commercial medium. More than a billion people are online. Trillions of dollars of business is conducted online. Hundreds of billions (if not trillions) of dollars have been invested in infrastructure, services, software, and other Internet technology.
And the Internet still sucks.
Our entire business is a hundred per cent dependent on the Internet. If our Internet service goes down, we can’t deliver our product. So we are understandably willing to spend a lot of money to make sure our Internet service never goes down.
And it still goes down.
And when it goes down, it’s still infuriatingly difficult to figure out why it has gone down, and what’s wrong, and when it might come back up.
It’s infuriatingly difficult, I understand, because even our Internet service provider often doesn’t know why the Internet has gone down.
And it doesn’t seem to matter how big a connection we get, or how direct, or how much money we pay, the Internet still goes down.
Four years ago, when we were three people in a loading dock, we lived on the WiFi of another startup. For obvious reasons, our Internet often went down.
Then, three years ago, we raised money, which allowed us to buy “business class” Internet service from our cable company, Time Warner Cable. And that always went down.
Then, two years ago, we raised more money and got a 15MB connection from Verizon, keeping our “Business Class” cable modem as a back-up. Whether it was the modem or the network or the router or the weather, both services often went down.
Then, a year ago, after tearing our hair out over how slow the network was, we finally bit the bullet and ordered a 50MB direct fibre connection from Verizon, keeping the cable service as backup.
Getting our 50MB fibre connection took more than six months.
Photo: Flickr The Consumerist
Because, apparently, the only company we could get it from was Verizon, and Verizon saw no reason to act like anything other than a monopoly.Verizon had to run fibre all the way up our building to our floor. And the building superintendent had to drill holes in walls. And so forth.
But then, eventually, about a month ago, we got our 50MB fibre connection. And we still had our backup.
And for a while it was fine.
And then, earlier this week, in the middle of our “rush hour” (the morning news blitz), our Internet went down.
For about two hours.
We called and yelled.
No one could help us. Because no one knew what the problem was.
And part of the reason no one knew what the problem was was that we don’t actually deal directly with Verizon. We deal with a “reseller” called Paetec that sells us our 50MB fibre connection by bundling services from several other companies including Verizon. And we also, apparently, deal with several other companies (indirectly) that sit between us and Verizon. Anyway, Paetec didn’t know what the problem was. So we sat around and twiddled our thumbs until the Internet finally came back on.
Later, just for the hell of it, I asked for a post-mortem.
After all, we’re paying ~$3,000 a month for a 50MB fibre connection and two back-ups because our business depends on the Internet and we really, really, really hate it when the Internet goes down.
The post-mortem took several days.
And, ultimately, here was the explanation we got–not from our service-provider Paetec, mind you, but from the Internet broker (!) who initially hooked us up with Paetec. The explanation mentioned several companies that we are apparently dependent on for our Internet service, at least one of which I have never heard of (“Starnet”).
There was a transit connection isolated in the STARNET network. When the issue was isolated the impacting carrier network was removed and the issue resolved.
I have requested a specific written *reason* as to why the circuit went down, something a bit more specific than “had an ion overload and had to run it all through the main deflector dish”. :-) From what I have been able to determine to date, it was a failure within that entire area, appears that a massive amount of data (perhaps a DOS attack – Denial Of Service) coming in via a backbone connection on a Level3 circuit, which would have affected *EVERY* Internet connection in your area, including your TWC circuit, from any carrier (including Paetec) connected to that area of the network backbone.
So in other words, while I am still trying to get a more definitive answer for you, what I have determined to date (and not just from Paetec but also other carriers I represent) is that this was an issue for *ANY* carrier, not specific to Paetec, and the techs from *ALL* carriers at that time were scrambling to try to re-route traffic around the affected area of the backbone where the Level3 connection was causing the problem.
Here’s what I took away from that explanation:
- It’s no one’s fault.
- It will happen again and again and again.
- We can call and yell if it makes us feel better but it won’t make a lick of difference to anyone
- The Internet still sucks.
And that last point is the real takeaway: 17 years after Netscape created the first commercial browser and launched the era of the commercial Internet, the Internet still sucks.
So if any of you entrepreneurs out there are looking for a big opportunity and a big problem to solve, you can tackle that one: You can figure out how to provide an Internet service that doesn’t suck.