12:20 - 11th Jan (UK) - the network issues at Clouvider Telehouse North and parts of Enfield are known, there is a significant and sustained DDOS attack impacting the network, work is in progress to mitigate it.13:20 - 11th Jan (UK) - the network issues now seem to be isolated to Telehouse North, I have reached out to Clouvider for further ... Read More »
Update 10th JAN 15:37 - The initial planned method for the patching was to save the sates of everyone's virtual server and restore it as if they had never been rebooted however this is taking significantly longer than expected the last few UK nodes that this method was applied to are being restored now and all further patching will be done with ... Read More »
20:25 (UK Time) The issue with KVM2 UK is known, it is being worked on. updates to follow
20:58 (UK Time) All services have been restored, the issue was caused by a kernel panic, specifically to do with the kernel page pool, this will be looked in to in greater detail and a maintenance schedule arranged for a more permanent fix.
Christmas and New Year Support hours for 2017 For self-managed services, i.e. if you do not pay a specific management add-on, support and billing support will be unavailable between the 22nd December 2017 and 1st January 2018 inclusive.For semi-managed or managed customers 24x7 365 support will be available via the managed support portal or via ... Read More »
10:16pm 14th Dec 2017 - A number of issues have been picked up on this node which is seriously impacting performance, the root cause has been quickly identified however the only resolution requires a reboot, this is being done as quickly as possible servers should be up within 20 minutes of this message, sorry for the inconvenience.10:46pm 14th ... Read More »
12:30 - 16th Nov - The network issues in Phoenix are known, it has been escalated to the Network Ops guys and hopefully, it will be resolved soon. Sorry for the inconvenience.12:47 - 16th Nov - The issues were resolved a few minutes ago, the root cause was a locked up line card which impacted some of the VLAN's in the DC, this has been fixed ... Read More »
Due to an OpenVZ issue, there are currently over 30k unkillable processes running on the node which has impacted performance to the degree an emergency reboot is required, sorry for the inconvenience.
Services should be back up by 13:15 UK time, reboot Triggered 12:50 UK time.
Updates regarding the maintenance of node1 and 5 will be placed here. Update 1# 10:45am - Node 5 has been started early due to stability issues, due to the additional time it is expected to take to wait for this server to shut down gracefully that has been started early to avoid delays.Update 2# 12:25pm - The maintenance is complete all ... Read More »
There has been a server hang impacting some customers, the server is back up now and virtual servers are starting.
A crash dump was created and will be analyzed to find the root cause and any necessary work will be done for stability.
Sorry for the inconvenience.
The powerdns cluster has failed beyond recovery, it is being restored from backups now.
Any changes made in the last w hours may need to be re-added.
Sorry for the inconvenience it is expected to be resolved by the end of the day today.
UPDATE: The work has been complete.