- Google’s team was recently baffled by a series of frequent short outages on a segment of Google’s powerful network in Oregon, according to a funny story tweeted by Urs Hölzle.
- Hölzle is Google’s eighth employee, the data centre and infrastructure genius who is largely responsible for Google’s bulletproof network and the tech that runs its cloud.
- The shortages were caused by cows stepping on a fallen line, the team discovered.
- The story was much enjoyed on Twitter and inspired many cow puns.
- Visit Business Insider’s homepage for more stories.
Despite its massive scale, Google hardly ever goes down. It has teams of people and advanced technology to pinpoint even the slightest blip in services to fix them before they become bigger problems.
But the team was recently baffled by a series of frequent short outages (a situation they call “flaps”) on a segment of Google’s powerful network in Oregon, where Google built its first data centre, according to Urs Hölzle. Hölzle is Google’s eighth employee, the data centre and infrastructure genius who is largely responsible for Google’s bulletproof network and the tech that runs its cloud.
About a month ago, they determined the problem: the outages were caused by a herd of cows.
The cows were grazing on land underneath Google’s high-speed, fibre cables. One of those cables had fallen to the ground and every time a cow stepped on that cable, the engineers saw the “flap.”
Hölzle shared the story in a series of tweets:
“OK here’s a new one: did you know that cows can cause network outages? Don’t laugh, it happened to us. The beginning of the story: recently, we noticed frequent short outages (“flaps”) on a multi-terabit fibre path through Oregon. This link is an aerial fibre link (fibre is strung along the path of a high-voltage power line). Such links have lower reliability because storms, trees, ice, and the occasional hunter can damage them.
“But this time we found something new: the fibre line had fallen to the ground yet continued to work just fine. But recently a farmer had started grazing a herd of cows nearby. And whenever they stepped on the fibre link, they bent it enough to cause a blip.
“Don’t believe me? Here’s a picture capturing the situation: the power line, the fibre on the ground, and a cow in the background. You heard it here first “
Don't believe me? Here's a picture capturing the situation: the power line, the fiber on the ground, and a cow in the background.
You heard it here first ???? pic.twitter.com/zodIUrbAh5
— Urs Hölzle (@uhoelzle) May 21, 2020
Of course, the funniest part was the response the story got on Twitter. Some people shared their own stories of cow-caused, or lama-caused, or elephant-caused network outages, common problems in places like Boliva, Chile, India and South Africa. Others shared stories of dogs or cats chewing cables.
But most others joined in the fun with puns like …
“You herd it here first”
“Cows do enjoy a high fibre diet”
“Now that’s a story I haven’t herbivore”
“That’s a cow-in-the-middle attack”
and “Udderly amazing, Urs.”
The fix was pretty obvious. Get the power company out there to put the cable back up on the pole where it belongs.
We asked Hölzle about the fix the situation and he sent us this cow-ment: “Since the cable is on the power company poles, they needed to first find a window to de-energize the lines to repair. But yes, ultimately the fix was simple.”
Are you a Google Cloud insider with insight to share? Contact Julie Bort via email at
or on encrypted chat app Signal at (970) 430-6112 (no PR inquiries, please). Open DMs on Twitter
- Now read:
- The corporate office as we know it is dead – long live the ‘dynamic’ workplace, says a CEO who’s been experimenting with this new concept for a year
- How Oracle-rival startup Couchbase raised $US105 million during a pandemic by being brutally honest with investors
- Amazon’s cloud CEO wants the world to know that Zoom mostly runs on AWS, not Oracle’s cloud
Business Insider Emails & Alerts
Site highlights each day to your inbox.