Google’s DoubleClick ad server for publishers (DFP) went down for over an hour on Wednesday, potentially costing publishers across the world millions of dollars in missed advertising revenue as Google scrambled to solve the problem. It also brought much of the web to a virtual standstill as the issue prevented pages from loading.
Brian O’Kelley, cofounder and CEO of AppNexus, the world’s largest independent adtech company, told Business Insider the outage could have cost publishers, collectively, “$US1 million per hour” in lost ad revenue. That’s just an estimate (and O’Kelley is quick to point out that AppNexus has had its share of temporary outages, so he feels Google’s pain). Nonetheless, AppNexus is one of the few businesses that has the kind of scale to inform an educated guess.
A second advertising source, who asked not to be named, also confirmed the impact was likely to be within the region of $US1 million per hour.
It appears the problem first arose at about 9 a.m. ET. Ads began appearing again on sites at about 10:45 a.m. ET. For a brief time much of the web existed only in a rare, ad-free condition.
Aside from lost revenue, the global issue was also bringing some websites to a halt for users as pages waited for ads to load.
This chart provided to Business Insider from Dynatrace shows just how badly the outage affected website load times. Look at that sharp spike on the right-hand side.
Usually when accessing Forbes’ site, for example, users are famously served with a full-page ad before entering the main homepage. (Forbes, of course, is a competitor of Business Insider’s.) However, that ad slot appeared blank and text simply said “waiting for ad.doubleclick.com,” which prevented the page from loading as quickly as it should:
Other affected sites included The Wall Street Journal, BBC.com, The Guardian, Mail Online, Mashable, and Business Insider. Google dominates the global digital advertising market, with eMarketer estimating it has a 31.45% share of the sector, underlining just how huge an issue even just a small amount of downtime is.
The DoubleClick for Publishers Help forum was flooded with questions about the outage.
Some people posting in the forum say their Google account managers have contacted them confirming a “P0 critical issue.” “P0” is a reference to “priority zero,” meaning a priority that must be tackled even before you get to P1, or any other task.
A Google spokeswoman confirmed to Business Insider: “DoubleClick for Publishers experienced an outage this morning impacting publishers globally, across their video, display, native, and mobile formats. Our team has worked quickly to fix the software bug, and it’s now back up and running, so our publisher partners can return to funding their content.”
Google’s DoubleClick for Publishers, DoubleClick for Advertisers, and Dart Sales Manager previously came to a halt back in March 2013, when the services were down for a few hours, according to a report from MediaPost.
Google DoubleClick’s latest headache comes just weeks after researchers at Malwarebytes found the server was exposing millions of users to malware. Google has now resolved this particular issue.