Follow

RCA - 02/02/2017 - Users Unable to Log In

Root Cause Analysis (RCA)

Date: 02/02/2017

Time: 2:36pm - 4:15 pm Pacific Standard Time

Type: Inability to Login

Affected Clients: 179

 

SUMMARY:

 

  • On February 2nd 2017 at 2:36 pm Pacific Standard Time, Velocify detected a software error affecting one of our redundant systems, which prevented login for some Lead Manager clients. Velocify views system uptime as one of our highest priorities and we sincerely apologize for the impact this problem had on your business. We value all our clients and take our role in your success seriously.

 

DETAILS:

 

  • During login, a session state is stored in a high performance open source memory cache, supported by a server farm. We are capable of handling any errors in this software due to redundancy. However this time, a very rare corner case caused the server farm to be affected, creating a failure of our usual redundancy processes. This required us to trigger a secondary redundancy process. Velocify maintains two full production environments and in response to this error, began migrating clients to the unaffected environment. The migration process was completed at approximately 4:15 pm Pacific Standard Time. Affected clients were seeing service restoration as the migration progressed.

 

CORRECTIVE ACTION:

 

  • In order to prevent this problem from occurring again, a software hot fix was rolled out the following day. We believe this error was an anomaly and do not expect this problem to occur again.

 

During any system wide problem, updates and notifications will be posted on our Now Trending page: Velocify Now Trending 

 

Please contact our Support Team with any additional questions. 

 

Support: 844-327-3296

 

Monday - Friday 6am to 6pm Pacific Standard Time

support@velocify.com

 

Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments

Powered by Zendesk