Root Cause Analysis (RCA)
Dial-IQ Connection Error 02/28
(All times Pacific Standard Time)
02/28/2017 12:30 pm – 2:30 pm
Type: Inbound Dial-IQ Dropped Calls
Affected Clients: 26
On February 28th at approximately 12:30 pm, some Velocify clients experienced an issue during the handling of inbound calls and warm transfers. Callers were hearing a “Sorry – An Application Error has Occurred” message and calls were disconnected.
Velocify is partnered with Twilio, a world class telephony carrier interface that facilitates call routing for our Dial IQ product. During the process of handling inbound calls through distribution or warm transfers, Twillo plays standard ringing and hold music as calls are transferred and distributed. In order to facilitate the playback of these responses, Twilio utilizes Amazon’s Web Services (AWS S3) for file storage. On February 28th, AWS was experiencing availability issues in their US-EAST-1 region, which impacted many companies serviced by Amazon. As a result, file retrieval services were temporarily unavailable causing calls to be ended erroneously. Amazon restored connectivity at roughly 2:10 pm and Dial-IQ services were fully restored by approximately 2:30pm.
Velocify treats all system downtime problems with the highest priority and if an issue arises, our engineering teams are on call to address any technical issues within our control. As was the case with many software companies who rely on Amazon Web Services, we were unable to enact any internal changes to mitigate this problem.