It was a Friday around noon when a few people started reporting slow downs in editing the website. The live site was fine, but the editor interface was failing.
Once I had enough confirmations from reliable reporters, I opened up a ticket with our website vendor to investigate. While I was doing so, it seemed to be getting better – but it was still noteworthy enough that I was hoping for a review of the logs.
They got back to me right away – our website was under attack.
It was a bot swarm and it was overloading our servers. The live site was okay only because we have a CDN running – a Content Delivery Network – that we had put in place after the last attack. The CDN creates geographically separated copies of the site to help protect against this very thing.
Vendor support kept updating the ticket and I tried to keep our editors and colleagues updated – including our own support desk.
And then, things got worse.
Our vendor support cranked up the settings on the CD to try and block the attack, but it got too sensitive and started to block us from seeing our own live website – though the editing was back to normal.
Now the calls and emails really started to roll in. “Did you know the website is down?”
Of course I did.
I quickly updated the vendor support ticket to report the new issue and sent out another wave of emails to update our editors. And updated my boss.
The vendors adjusted the sensitivity back down a bit and our pages started to show up again in live mode. And the editor interface was still working too.
I reported back to the vendor of the success after some testing, but waited a bit longer before updating our site editors with the “all-clear”.
The end result was…nothing. We went partly down for a little bit, but everyone involved worked quickly and efficiently to respond to the attack.
It was a rough way to spend a Friday afternoon, though.
Stupid bots.