Microsoft has warned thousands of its Azure cloud computing customers, including many Fortune 500 companies, about a vulnerability that left their data completely exposed for the last two years.
A flaw in Microsoft’s Azure Cosmos DB database product left more than 3,300 Azure customers open to complete unrestricted access by attackers. The vulnerability was introduced in 2019 when Microsoft added a data visualization feature called Jupyter Notebook to Cosmos DB. The feature was turned on by default for all Cosmos DBs in February 2021.
A listing of Azure Cosmos DB clients includes companies like Coca-Cola, Liberty Mutual Insurance, ExxonMobil, and Walgreens, to name just a few.
“This is the worst cloud vulnerability you can imagine,” said Ami Luttwak, Chief Technology Officer of Wiz, the security company that discovered the issue. “This is the central database of Azure, and we were able to get access to any customer database that we wanted.”
Despite the severity and risk presented, Microsoft hasn’t seen any evidence of the vulnerability leading to illicit data access. “There is no evidence of this technique being exploited by malicious actors,” Microsoft told Bloomberg in an emailed statement. “We are not aware of any customer data being accessed because of this vulnerability.” Microsoft paid Wiz $40,000 for the discovery, according to Reuters. In an update posted to the Microsoft Security Response Center, the company said its forensic investigation included looking through logs to find any current activity or similar events in the past. “Our investigation shows no unauthorized access other than the researcher activity,” said Microsoft.
In a detailed blog post, Wiz says that the vulnerability introduced by Jupyter Notebook allowed the company’s researchers to gain access to the primary keys that secured the Cosmos DB databases for Microsoft customers. With said keys, Wiz had full read / write / delete access to the data of several thousand Microsoft Azure customers.
Wiz says that it discovered the issue two weeks ago and Microsoft disabled the vulnerability within 48 hours of Wiz reporting it. However, Microsoft can’t change its customers’ primary access keys, which is why the company emailed Cosmos DB customers to manually change their keys in order to mitigate exposure.
Today’s issue is just the latest security nightmare for Microsoft. The company had some of its source code stolen by SolarWinds hackers at the end of December, its Exchange email servers were breached and implicated in ransomware attacks in March, and a recent printer flaw allowed attackers to take over computers with system-level privileges. But with the world’s data increasingly moving to centralized cloud services like Azure, today’s revelation could be the most troubling development yet for Microsoft.
Protecting your environment
As described in their initial blog here on #chaosDB, Wiz Research team found an unprecedented critical vulnerability in the Azure Cosmos DB. The vulnerability gives any Azure user full admin access (read, write, delete) to another customers Cosmos DB instances without authorization.
Outside the scope of the immediate remediation actions. It is important to review our networking & access strategy in the light of the #chaosDB vulnerability. The key question is what can we do to build our environments to be more bulletproof and completely avoid these kind of large scale vulnerabilities in the future.
Using RBAC: getting rid of secrets
The first and most important take away is that shared secrets such as ComosDB primary key are insecure and should be deprecated in favor of modern authentication methods.
The fact that CosmosDB has a primary key that can be shared across users and services, all using the same secret without any clear way to audit, monitor or revoke access is simply unacceptable from a security perspective.
The longer term goal should be to transition from these insecure primary/secondary keys to role based access that does not require any secrets at all. Role base access is already supported for Comsos DB and can in fact block the primary key to allow RBAC auth only. However, according to Microsoft documentation, the RBAC support is currently highly limited and doesn’t provide data plane access. Once fully supported, it should be high priority to transition into this new model.
Using private endpoints: minimizing cross-account exposure
The other exposure aspect which should be greatly improved is network exposure. Although Cosmos DB allows to activate an IP based firewall, in reality this doesn’t help to block cross account access since majority of services still require access. Even worse, services without static Ips require users to practically open their databases to the entire world (“Azure only” Ips is the same as saying “any attacker with an Azure tenant”).
The only valid approach to build CosmosDBs and other services with minimal network exposure is to leverage private endpoints to the extreme and ensure that no assets is externally exposed.
This approach as well is hard to implement as not all services actually support private endpoints. However this should be the goal of every organization as they plan the next network design.