VoIPms - Notice history

This incident has been resolved and the re-route has been cancelled.

Thank you for your patience and understanding.

We are currently investigating this incident.

This incident has been resolved.

After our providers restored the power on the data center and the servers came back, we canceled the re-route over these servers.

Thank you for your patience on this.

We have successfully re-routed all the DIDs that were using Newyork 1 to 4 to backup servers, this is a temporary measure, while our provider gets their issue fixed.

Thank you for your patience on this.

We have just received updates from our provider: They are still working on the issue with their respective vendor. We are still awaiting a firm ETA.

Rest assured that our team keeps monitoring this and that we will provide further updates as soon as possible.

Hello,

Our provider has identified a power issue on the datacenter that hosts these servers. They reported that they are working on fixing this, however, there is no ETA.

We apologize for the inconvenience.

We are pleased to announce that all services and features are now fully restored and functioning as intended. In the coming week, our dedicated team will thoroughly assess the timeline of events and deliver a comprehensive post-mortem analysis.
If you happen to encounter any lingering issues, we kindly request that you reach out to our support staff without hesitation.

We genuinely appreciate your patience and understanding throughout this event. Thank you for your continued support.

We are pleased to inform you that all services are now back up and operational. Our dedicated team is diligently working on finalizing details and ensuring the stability of all our services, while also guaranteeing that all features and options are fully functional. We will closely monitor the system and optimize any process to ensure everything runs smoothly.

If you encounter any difficulties, please feel free to reach out to our support staff who are ready to assist you. Thank you for your patience and cooperation.

With nearly all services fully restored, our team is diligently working to complete the remaining tasks and conduct thorough testing to ensure optimal functionality. We will provide regular updates on the progress and promptly notify you when all outstanding issues are resolved. If you have any questions or concerns in the meantime, please feel free to contact our customer support team.

We appreciate your understanding and continued support.

Our diligent team is actively engaged in the ongoing process of restoring all services to full functionality. While most of the services are now fully operational, we are committed to ensuring that all remaining functionalities of the system are thoroughly tested. Rest assured, we will keep you informed with regular updates on our progress.

We are pleased to inform you that our provider’s recovery process is underway, and systems are gradually coming back online. Our dedicated team has been closely monitoring the situation and is now initiating the transition back to our regular operations from our emergency backup plan. However, please note that certain portal functionalities, including MMS, Recordings, Call Recordings, and Conference Recordings, may still experience limited functionality at this time.

Rest assured, we are committed to keeping you updated on our progress as we work towards complete restoration. We sincerely appreciate your patience and understanding throughout this process.

Thank you for your continued support.

Our provider continues moving forward with the plan to bring the site back online and they are currently at 30% completion. At the moment they are cooling the facility after being able to bring electrical equipment online.

We’ll continue following closely to provide prompt updates.

We have received confirmation that the initial inspection this morning has been successfully completed and the process to power up the Data Center has begun. The estimated time for this process to complete is up to 5 hours, after which, the servers will start to come up online.

We want to confirm that at the moment, all the call traffic is connecting properly since yesterday, and our website and customer portal are fully accessible as well. Some specific portal or email functionalities may not work as intended, however, our team continues working diligently to have the service back up at 100%.

We’ll continue posting any relevant updates. Thank you for your patience.

Following the 2:00 PM EDT meeting, the fire marshal has requested additional remediation actions around the fire system components before the energy can be re-enabled at the Data Center. The follow-up inspection from the fire marshal is at 9:00 AM EDT, Wednesday. The Evocative team is bringing on additional personnel and vendors to comply completely with the additional requests and ensure there are no more delays.

In the meantime, our team continues working diligently on fully mitigating the impact on the customer portal and ensuring all services are working correctly.

At the moment, calls traffic continues to connect properly through our backup solutions and customer portal access has been recovered, however, you may experience issues with some specific functionalities and changes not applying correctly.

We’ll continue proving updates as they come.

At this point we haven’t received a firm ETA from our data center yet, however, our team was able to roll out emergency backup measures and the VoIP.ms website and portal have been brought back up. You may notice some functionalities not working as they should while the team continues to roll out the backup plan to fully restore the website and portal.

Calls continue to connect properly as traffic is currently rerouted to backup solutions.

There is currently a 2PM EDT meeting scheduled, during which Evocative staff and local township officials will perform a walk-through of the site. Depending on their inspections and approvals, Evocative staff will then be able to access the site and initiate the sequence to bring all data center systems back online.

We’ll continue to follow closely and provide more updates as they come.

Thank you for your patience.

We are continuing to work on a fix for this incident.

There has been an isolated fire at the Evocative data center mid-day on 2023-07-10. The local fire department arrived on the scene and had to cut the entire power to the building and disable generators.

As of 2023-07-11 10h00 AM EST, power remains off as per the local fire Marshal. A vendor has been on site since yesterday to proceed with an extensive cleaning in the UPS room as per local fire Marshall requirements. The vendor expects to have completed the cleaning by mid-day, when the fire Marshall can review and sign off on reenergizing the data center (which can take up to 5 hours).

We apologize for any inconvenience caused by this incident and appreciate your understanding and patience.

We have just received updates from our provider: They are still working on the issue with their respective vendor. We are still awaiting a firm ETA.

Rest assured that our team keeps monitoring this and that we will provide further updates as soon as possible.

Hello,

Our provider has identified a power issue on the data center that hosts our main web servers. They reported that they are working on fixing this, however, there is no ETA.

We apologize for the inconvenience.

This incident has been resolved and the re-route has been cancelled.

Thank you for your patience and understanding.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

This incident has been resolved.

The issue has been resolved.

This incident has been resolved.