Posted in: help-and-feedback
Server Downtime on Monday24 Aug 2010 17:46
Dear All,
Big sorry for the completely useless state of affairs on Monday. We moved data centres over the weekend; a carefully orchestrated event, which went without hitch... or so we thought.
We piled up to London on Saturday evening in vans, and took down the website at 9:45pm. The equipment was loaded into vans, and transported a couple of miles across London into Docklands, where we set about plugging in a spaghetti bowl of wires and a heavy stack of servers.
We finished hardware installation to schedule at 1:30am on Sunday, and then spent the next couple of hours running updates and checks. We left the data centre at 5:30am on Sunday morning, confident that everything was working as it should have been.
Monday arrived, markets opened, and as a good number of you will know, things fell over, hard. Our data feeds all stopped, news froze, and the website was intermittent - not a happy event. We rushed engineers into the data centre, and spent the day checking servers, cables, routers, firewalls, and running diagnostic checks, while the chaps in the office manned the phones and wrote apologetic e-mails.
At 6:30pm after an exhausting day, we found the glitch. Our supplier had configured the speed on their equipment incorrectly, and LSE was effectively chocking on all the information rushing through it. Very frustrating, but gladly a relatively easy fix.
All is back to normal - or slightly better than normal; an improved data centre is in operation, and all set for growth. Ironically, one of the reasons for the data centre move was to further improve stability and the reliability of the service. We're confident, teething glitches aside, this will be the case.
Thank you to all our users for their patience, and good humour, despite the seriousness of the situation.
If you e-mail us in the next couple of days, please use short words, as we're all a bit sleepy!
Kind Regards,
Admin
Sign In
Follow the stocks
