The incident of NWT data center in 2009. There was a sudden power surge  that broke the electric circuit and there supposed to be a diesel generator to provide the backup power, but guess what? There is not enough diesel fuel in the tank. Wow! This is so similar to SmarTone’s data center  down time and I am surprised that they don’t have auto real-time failover site, this is strange for a listed company with over a few millions of subscribers in Hong Kong, a big question to their so called Level 3 ISO approved data center.

Now the government suggested other carriers should temporary provide the 2G/3G service for any carrier facing downtime, I think it’s pure dreaming, why would they anyway?

I bet One2Fre is the most happy face this time to see SmarTone sinking, as SmarTone was not so nice to them either previously.

Dear SmarTone Customer,

As a valued customer, I would like to apologise to you for Monday’s service disruption and inform you of what actually happened, and what we have done to resume services and minimise disruption.

On 9 April at around 8:00 am, the building in which one of our three switching centres is located, suffered a total power failure disrupting the power supply to the entire building. Our switching centre’s standby battery system immediately substituted the failed power source and there was no disruption to service. Our backup generator then commenced operation, successfully taking over from the standby battery system.

At approximately 10:35am a component in the starter circuit board of the backup generator broke down unexpectedly, causing the generator to stop operating. This also caused a huge surge of electrical current that triggered circuit breakers to disconnect the cellular switching system from the standby battery system. The resulting power outage disrupted cellular service in several areas in Hong Kong and selected MTR stations in Kowloon.

Emergency restoration and recovery procedures started immediately and service was restored progressively from 12:15pm onwards. Aside from a small number of selected services, voice and mobile internet services were largely back to normal at 1:00pm. From 2:30pm onwards, all SMS services had returned to normal. The Company’s two remaining switching centres operated as normal during this incident.

We carry out inspections and tests of all power generation facilities regularly. The backup generator which caused the outage successfully passed recent inspections and tests on 22 February. We will submit an incident report to the Office of the Communications Authority and will re-evaluate our procedures for informing the public and its customers.

We regret the inconvenience caused to you and we will thoroughly investigate both the reasons for the building’s power outage and the failure of our power backup systems in order to ascertain the root cause. We are determined to learn from these findings and make improvements to prevent similar incidents in future.

Yours sincerely,

Douglas Li
CEO