Liverpool primary backhaul path failure - Issue resolved
We are aware of a failure of our primary backhaul to Liverpool. We have raised this with the fibre provider and we will post updates as available. All traffic has re-routed via alternative paths.
We'd like to apologise for the disruption caused by this link failure, and thank customers for their cooperation during this issue.
Regards
Metronet Support
The fibre provider has resolved the problem on the primary path in to Liverpool and after testing, this link has been brought back in to production.
Regards
Metronet Support
We will be migrating to our primary path on Tuesday 31st January between 22:00 and 23:00. We will advise when this has been completed.
Regards
Metronet Support
Due to issues when testing our provider has been unable to migrate back to the primary path. We will continue to run on the temporary path until the primary has been re-tested. We will advise on dates as soon as we receive.
We apologise for this.
Regards
Metronet Support
Connectivity will shortly be migrated back to the primary path. We will advise when complete.
Regards
Metronet Support
Our provider has advised that they have scheduled maintenance on the fibre path for Tuesday 24th January between 22:00 and 02:00 (Wednesday 25th January) to perform the required hardware replacements and light balancing. Metronet are continuing to operate on the temporary service and will continue to do so until 24 hours of stability is reported by our provider.
We will post the next update on Wednesday 25th January unless there are any further events.
Regards
Metronet Support
We have been advised that partial service has been restored on the fibre path over which our primary connectivity operates however the element which delivers our service is still impacted due to failed hardware. The hardware has been requested from vendor and our provider is currently awaiting confirmation of delivery. Once delivered they will arrange for the hardware to be replaced. All activities are being treated as a priority. Our service will continue to operate on the temporary service until primary service has been restored and has been stable for 24 hours.
Regards
Metronet Support
An alternative path has been implemented to reduce load across the alternative backup paths. All paths are now operating below capacity. This is a temporary solution until the full fix is implemented on the primary path. An update on the primary path issue will be posted before the end of the day. We apologise for the delay.
Regards
Metronet Support
We are working on final preparations for the alternative option. Further updates will be posted shortly.
Regards
Metronet Support
We are working on final preparations for the alternative option. Further updates will be posted shortly.
Regards
Metronet Support
We have identified a potential alternative solution and we are working with the provider on implementing this. We will provide further updates on this later this morning.
Regards
Metronet Support
Our provider has advised that they have sourced replacement hardware and are awaiting delivery details. An update will be provided tomorrow.
While this is progressing we are working with them on alternative options.
A further update will be provided tomorrow.
Regards
Metronet Support
Our provider has updated with the following:
"We are continuing to liaise with our suppliers to rectify or supply new equipment. We aim to provide further updates this afternoon or as soon as information is available."
We will update as soon as we receive an update.
Regards
Metronet Support
We have this morning received the following update:
"Unfortunately we are going to have to postpone troubleshooting due to unforeseen equipment failure. During our pre checks and final preparations for this morning it has been found that one of our OSA testers is functioning out of spec which could result in false positive results. We are currently working with our suppliers to rectify or supply alternative equipment, this is a high priority."
We will advise as soon as we receive further updates. We apologise for the delay in the start of this work.
Regards
Metronet Support
We have received an update advising that network engineers are required to perform light rebalancing works across a number of sites in order to resolve this fault. This work is due to take place from tomorrow morning however due to the number of locations involved is expected to take much of the day to complete. This has been escalated to senior management at the supplier.
We will continue to liaise with them and we will post updates as they are available.
We apologise for any service disruption experienced as a result of this incident.
Regards
Metronet Support
We have been advised of the below from our provider:
"We are working through this current outage as a high priority. This is associated with an issue encountered last week on our underlying fibre connections, which was ultimately resolved through the mentioned emergency planned works - which included installing additional amplifiers.
Our specialist fibre engineers are currently investigating the options available to apply a resolve to this issue, which is likely to be conducted under further emergency planned works, and include rebalancing a number of fibre links."
We will advise of any further details.
Regards
Metronet Support
Our systems have detected packetloss affecting some services in Liverpool. Traffic re-routed following the primary path failure and our network infrastructure team are investigating any additional routing options. We apologise for any inconvenience from this.
Regards
Metronet Support