BEN #5229 - Luminate Online Incident May 13, 2015

We have confirmed that your Luminate Online Services are now fully restored.  We will be actively monitoring the service for the remainder of the night to ensure stability and accessibility.

As stated previously, the root cause of this incident is being fully investigated but we can say with confidence that this issue was caused by a hardware failure which was triggered by a replication inconsistency. There was and is no security vulnerability and we do not anticipate data loss as a result of this incident.

Note: This information originally published in KB article BB

Quality and stability of the Luminate Online service is critical to your organization, mission, bottom line and most importantly the millions of people and causes that you serve. Having a failure of any size or magnitude during any part of the year is unacceptable and Blackbaud is committed to working expeditiously to ensure your complete future success using our products and service.

What caused this issue to last as long as it did?

Due to the magnitude of the replication issue and subsequent hardware failure, several vendors and consultants were called in to help identify, resolve and test the remediation of the issue. This unfortunately took several hours to organize and execute. We understand that the resolution took an extended period of time and we will be working on a new recovery plan to address this in the future. More details will be provided in the Root Cause Analysis (RCA).

Was there any data loss or security vulnerabilities?

We do not believe at this time that there was any data loss or security issues during this incident. Some clients had the need to restart scheduled email tasks or data synchronization operations but that is the only known side effect.

Is the Service 100% restored?

Yes, the system is fully restored and our vendors are working around the clock with Blackbaud’s team of experts to analyze the root cause. We will continue to monitor all systems to ensure stability.

When can we expect a Root Cause Analysis?

There is a significant amount of data to analyze and we are working through this as rapidly as possible. We understand the urgency of providing a quick response to our staff and customers. We also want to confirm the accuracy of our findings before we share them internally and with our customers.  An incident report is available at request and an RCA will be available within the coming weeks.



Was this article helpful?