Rogers restores service for ‘vast majority’ of customers after massive outage

Canadian telecom giant Rogers Communications says it has restored connectivity to the “vast majority” of customers following a major outage that took down its internet, mobile, and home phone services on Friday. In an update posted to Twitter on Saturday morning, Rogers explained that some customers “may experience a delay in regaining full service” as it works to get everyone back online.

Tony Staffieri, the president and CEO of Rogers, wrote a message to customers Saturday afternoon, announcing that services “have been restored” and that its network and systems “are close to fully operational.” Staffieri attributes the outage to a “network system failure following a maintenance update” that caused an issue with the company’s routers.

“We disconnected the specific equipment and redirected traffic, which allowed our network and services to come back online over time as we managed traffic volumes returning to normal levels,” Staffieri adds.

Following our previous updates, we have now restored services for the vast majority of our customers and our technical teams are working hard to ensure that the remaining customers are back online as quickly as possible. pic.twitter.com/IobL7Dze6i

— RogersHelps (@RogersHelps) July 9, 2022

This appears to confirm speculation from the experts at Cloudflare, a company that specializes in content delivery and DDoS mitigation. At the time of the outage, they believed that it may have been caused by an issue with the Border Gateway Protocol (BGP), a system that controls the flow of traffic on the internet. BGP also played a role in an outage that brought down Facebook’s services last year.

The outage began as early as Friday at 5AM ET, lingering throughout the entirety of the day and well into the evening. At the time of the outage, internet disruption tracker NetBlocks found that it brought down about a quarter of Canada’s connectivity.

According to CBC, Rogers has about 9 million mobile customers and provides cable and internet to nearly 3 million people. In addition to affecting customers of Rogers’ mobile subsidiaries, Fido and Chatr, it also unplugged a range of essential services throughout the country, interfering with 911 calls, debit card transactions, ATMs, and government agencies including Canada’s passport offices and Revenue Agency.

A chart shared by NetBlocks, which shows real-time internet traffic data in Canada, indicates that Rogers started to restore connectivity around 12AM ET Saturday morning. Internet traffic data from Cloudflare exhibits a similar pattern, with internet traffic beginning to pick up again at midnight and rising to nearly normal levels.

Interac, one of the banking networks affected by the outage, has stated that its services are once again available. The company powers a large number of debit cards, ATMs, and electronic fund transfers throughout Canada, a service Interac recently announced reached over one billion transactions. Interac says it plans on “adding a supplier” to strengthen its “existing network redundancy” to help prevent its services from going offline in case of another outage.

Data from Cloudflare shows internet traffic slowly returning to normal levels.

“Once again, we sincerely apologize for the disruption this has caused our customers and we will be proactively crediting all customers,” Rogers said. “You do not need to contact us for the credit as it will be automatically applied to your account.”

Update July 9th, 5:07PM ET: Updated to add information from Rogers CEO Tony Staffieri.

Source by www.theverge.com