IAM Cloud would like to thank all our EU customers for their patience this morning while we resolved unplanned disruption to the Authentication Service in the EU following major infrastructure maintenance this weekend. The scheduled maintenance this weekend was very complex and a major step forward in providing increased stability and efficiencies within the IAM Cloud Platform. All resources and planning was applied to ensure no disruption to our customers so we are extremely disappointed that any of our customers were affected today. We want to assure you that we take these incidents very seriously and have put clear steps in place to ensure we learn from this experience and further improve the IAM Cloud Service.
We hope the below RCA offers some insight and reassurances following this major incident.
Root Cause Analysis (RCA) Issued 10th July 2018
On Tuesday 10th July at 8:33 am BST our Infrastructure team were alerted of errors to the IAM Cloud Authentication Service which caused failed authentications for EU customers.
The root cause of this issue was that following the scheduled maintenance this weekend the scaling algorithm failed to scale successfully meaning that the servers were unable to service the requests.
On Tuesday 10th July at 09:15am manual scaling was applied and existing services were re-cycled to clear the backlog of pending request. Full service was resumed at 09:40am.
Engineers have fixed the scaling metrics and additional alerting and notifications on traffic build up have been applied.
Please do not hesitate to contact me direct if you would like to discuss this or any aspect of our service further.
Kind Regards Victoria Hamblin IAM Cloud's Chief Customer Officer