麻豆影视

Skip to main content

Facebook blames outage on error during routine maintenance

Share
LONDON -

The global outage that knocked Facebook and its other platforms offline for hours was caused by an error during routine maintenance, the company said.

Santosh Janardhan, Facebook's vice president of infrastructure, that Facebook, Instagram and WhatsApp going dark was "caused not by malicious activity, but an error of our own making."

The problem occurred as engineers were carrying out day to day work on Facebook's global backbone network; the computers, routers and software in its data centers around the world along with the fiber-optic cables connecting them.

"During one of these routine maintenance jobs, a command was issued with the intention to assess the availability of global backbone capacity, which unintentionally took down all the connections in our backbone network, effectively disconnecting Facebook data centers globally," Janardhan said Tuesday.

Facebook's systems are designed to catch such mistakes but in this case a bug in the audit tool prevented it from properly stopping the command, Janardhan said.

That change also triggered a second problem that made things worse by making it impossible to reach Facebook's servers even though they were operational.

Engineers scrambled to fix the problem on site, but this took time because of the extra layers of security, Janardhan said. The data centers are "hard to get into, and once you're inside, the hardware and routers are designed to be difficult to modify even when you have physical access to them."

Once connectivity was restored, services were brought back gradually to avoid traffic surges that could cause more crashes.

It was an "unforeseen anomaly" for a faulty maintenance update to take down Facebook's backbone network, but the company probably could have avoided a scenario in which its servers were completely taken offline, making it impossible to access the tools needed to fix it, said Angelique Medina, of Cisco Systems' ThousandEyes, a firm that monitors internet outages.

"The big question is why so many internal tools and systems could have a single source of failure," Medina said. "Facebook would still have been down because of the network outage, but they could have resolved the outage sooner if they had internal access."

CTVNews.ca Top Stories

It鈥檚 1998. Google was just born, and dial-up was the primary way to connect to the Internet. New sites were being developed daily on any number of topics, from recipes to shopping. David Yanciw, however, was thinking big - big things, that is.

Canadian musician Jacob Hoggard has been found not guilty of sexually assaulting a young woman in northeastern Ontario eight years ago. The former Hedley frontman had pleaded not guilty to sexual assault.

The Ministry of Labour is investigating a workplace incident that claimed the lives of two people in Georgian Bluffs, south of Owen Sound.

NEW THIS MORNING

NEW THIS MORNING

New data shows the automated speed enforcement camera on King Edward Avenue, between Bolton Street and St. Patrick Street, issued 6,337 speeding tickets in August, the highest number of tickets issued by Ottawa's 40 photo radar cameras.

According to the Canadian Women & Sport's 2022 Rally Report, 76 per cent of girls who participate in sports believe it helps build confidence and enhances their mental health. While those benefits are important for all children, the organization's CEO says, right now, girls have less access.

Local Spotlight

Chantal Kreviazuk is set to return to Winnipeg to mark a major milestone in her illustrious musical career.

From the beaches of Cannes to the bustling streets of New York City, a new film by a trio of Manitoba directors has toured the international film festival circuit to much pomp and circumstance.

A husband and wife have been on the road trip of a lifetime and have decided to stop in Saskatchewan for the winter.

The grave of a previously unknown Canadian soldier has been identified as a man from Hayfield, Man. who fought in the First World War.

A group of classic car enthusiasts donated hundreds of blankets to nursing homes in Nova Scotia.

Moving into the second week of October, the eastern half of Canada can expect some brisker fall air to break down from the north

What does New Westminster's t蓹m蓹sew虛tx史 Aquatic and Community Centre have in common with a historic 68,000-seat stadium in Beijing, an NFL stadium and the aquatics venue for the Paris Olympics? They've all been named among the world's most beautiful sports venues for 2024.

The last living member of the legendary Vancouver Asahi baseball team, Kaye Kaminishi, died on Saturday, Sept. 28, surrounded by family. He was 102 years old.

New data from Greater Vancouver and the Fraser Valley shows a surge in supply and drop in demand in the region's historically hot real estate market.