Rackspace hosted Exchange suffered a disastrous blackout starting December 2, 2022 and is still continuous since 12:37 AM December fourth. At first referred to as connection and login concerns, the assistance was ultimately upgraded to reveal that they were handling a security event.
Rackspace Hosted Exchange Issues
The Rackspace system decreased in the morning hours of December 2, 2022. At first there was no word from Rackspace about what the issue was, much less an ETA of when it would be solved.
Clients on Twitter reported that Rackspace was not reacting to support e-mails.
This has actually been rather the day with#Rackspace Every hosted exchange customer has actually been down for 14 hours approximately. Assistance isn’t reading/responding to tickets. Updates are unhelpful.
I am worried now that they came down with something bad like the ProxyNotShell PoC hack. https://t.co/jchKsAO3Z7
— Joe Sinkwitz (@CygnusSEO) December 2, 2022
A Rackspace client independently messaged me over social networks on Friday to relate their experience:
” All hosted Exchange customers down over the previous 16 hours.
Unsure the number of business that is, however it’s substantial.
They’re serving a 554 long hold-up bounce so individuals emailing in aren’t familiar with the bounce for numerous hours.”
The main Rackspace status page provided a running upgrade of the blackout however the preliminary posts had no info besides there was a blackout and it was being examined.
The very first authorities upgrade was on December second at 2:49 AM:
” We are examining a concern that is impacting our Hosted Exchange environments. More information will be published as they appear.”
Thirteen minutes later on Rackspace started calling it a “connection concern.”
” We are examining reports of connection concerns to our Exchange environments.
Users might experience a mistake upon accessing the Outlook Web App (Webmail) and syncing their e-mail customer( s).”
By 6:36 AM the Rackspace updates explained the continuous issue as “connection and login concerns” then later on that afternoon at 1:54 PM Rackspace revealed they were still in the “examination stage” of the blackout, still attempting to determine what failed.
And they were still calling it “connection and login concerns” in their Cloud Workplace environments at 4:51 PM that afternoon.
Rackspace Recommends Moving to Microsoft 365
4 hours later on Rackspace described the scenario as a “substantial failure” and started providing their clients totally free Microsoft Exchange Strategy 1 licenses on Microsoft 365 as a workaround up until they comprehended the issue and might bring the system back online.
The main assistance specified:
” We experienced a substantial failure in our Hosted Exchange environment. We proactively closed down the environment to prevent any more concerns while we continue work to bring back service. As we continue to resolve the origin of the concern, we have an alternate option that will re-activate your capability to send out and get e-mails.
At no charge to you, we will be offering you access to Microsoft Exchange Strategy 1 licenses on Microsoft 365 up until more notification.”
Rackspace Hosted Exchange Security Occurrence
It was not up until almost 24 hr later on at 1:57 AM on December 3rd that Rackspace formally revealed that their hosted Exchange service was struggling with a security event.
The statement even more exposed that the Rackspace professionals had actually powered down and detached the Exchange environment.
Rackspace published:
” After more analysis, we have actually figured out that this is a security event.
The recognized effect is separated to a part of our Hosted Exchange platform. We are taking essential actions to assess and safeguard our environments.”
Twelve hours later on that afternoon they upgraded the status page with more info that their security group and outdoors specialists were still dealing with fixing the blackout.
Was Rackspace Service Impacted by a Vulnerability?
Rackspace has actually not launched information of the security occasion.
A security occasion usually includes a vulnerability and there are 2 extreme vulnerabilities presently in the wile that were covered in November 2022.
These are the 2 most existing vulnerabilities:
- CVE-2022-41040
Microsoft Exchange Server Server-Side Demand Forgery (SSRF) Vulnerability
A Server Side Demand Forgery (SSRF) attack enables a hacker to check out and alter information on the server. - CVE-2022-41082
Microsoft Exchange Server Remote Code Execution Vulnerability
A Remote Code Execution Vulnerability is one in which an aggressor has the ability to run harmful code on a server.
An advisory released in October 2022 explained the effect of the vulnerabilities:
” A confirmed remote assaulter can carry out SSRF attacks to intensify advantages and carry out arbtirary PowerShell code on susceptible Microsoft Exchange servers.
As the attack is targeted versus Microsoft Exchange Mail box server, the assaulter can possibly access to other resources through lateral motion into Exchange and Active Directory site environments.”
The Rackspace blackout updates have actually not shown what the particular issue was, just that it was a security event.
The most existing status upgrade since December fourth specified that the service is still down and clients are motivated to move to the Microsoft 365 service.
Rackspace published the following on December 4, 2022 at 12:37 AM:
” We continue to make development in dealing with the event. The schedule of your service and security of your information is of high significance.
We have actually devoted comprehensive internal resources and engaged first-rate external competence in our efforts to lessen unfavorable effects to clients.”
It’s possible that the above kept in mind vulnerabilities relate to the security event impacting the Rackspace Hosted Exchange service.
There has actually been no statement of whether client info has actually been jeopardized. This occasion is still continuous.
Included image by Shutterstock/Orn Rin
window.addEventListener( 'load', function() { setTimeout(function(){ striggerEvent( 'load2' ); }, 2000); });
window.addEventListener( 'load2', function() {
if( sopp != 'yes' && addtl_consent != '1~' && !ss_u ){
!function(f,b,e,v,n,t,s) {if(f.fbq)return;n=f.fbq=function(){n.callMethod? n.callMethod.apply(n,arguments):n.queue.push(arguments)}; if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version='2.0'; n.queue=[];t=b.createElement(e);t.async=!0; t.src=v;s=b.getElementsByTagName(e)[0]; s.parentNode.insertBefore(t,s)}(window,document,'script', 'https://connect.facebook.net/en_US/fbevents.js');
if( typeof sopp !== "undefined" && sopp === 'yes' ){ fbq('dataProcessingOptions', ['LDU'], 1, 1000); }else{ fbq('dataProcessingOptions', []); }
fbq('init', '1321385257908563');
fbq('track', 'PageView');
fbq('trackSingle', '1321385257908563', 'ViewContent', { content_name: 'rackspace-hosted-exchange-outage-due-to-security-incident', content_category: 'news security' }); } });