Loading...
Loading...
A utility power failure in europe-west4-a caused prolonged outages for VMware Engine, NetApp Volumes, and Bare Metal Solution, lasting up to 13 hours. Preliminary root cause is loss of cooling leading to emergency shutdowns.
Major reliability impact across multiple services in one region; no direct cost, migration, quality, or compliance impact noted.
"modified": "2026-07-16T12:26:34+00:00", "modified": "2026-07-16T12:26:34+00:00", "modified": "2026-07-16T12:26:34+00:00",
"modified": "2026-07-18T00:22:37+00:00", "created": "2026-07-18T00:22:37+00:00", "modified": "2026-07-18T00:22:38+00:00", "text": "# Preliminary Incident Report\nWe sincerely apologize for the disruption this incident caused to your business. We know how much you rely on Google Cloud, and we regret the impact on your productivity.\nPlease note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. A final Incident Report with preventative actions will be posted once our investigation is complete.\nIf you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support.\n## Date/Time of the Issue (All time US/Pacific)\n**Google Cloud VMware Engine Impact:**\n- Start: 15 July 2026 17:09\n- End: 16 July 2026 2:33\n- Duration: 9 Hours, 24 Minutes\n**Google Cloud NetApp Volumes Impact:**\n- Start: 15 July 2026 16:39\n- End: 16 July 2026 01:10\n- Duration: 8 Hours 31 Minutes\n**Bare Metal Solution Impact:**\n- Start: 15 July 2026 18:37\n- End: 16 July 2026 07:34\n- Duration: 12 Hours 57 Minutes\n## Summary\nOn Wednesday, 15 July 2026, Google Cloud VMware Engine, Bare Metal Solution, and Google Cloud NetApp Volumes experienced service interruptions for a total duration of 14 hours, 55 minutes. We are taking immediate steps to ensure this doesn’t happen again.\n## Preliminary Root Cause\nA regional data center hosting services in europe-west4-a experienced a loss of utility power and subsequent cooling capacity.\nThe sequence of events leading to customer impact was as follows:\n- An electrical fault occurred on the utility grid upstream of the data center, disrupting the electrical distribution gear and cooling equipment.\n- The loss of cooling infrastructure resulted in ambient temperatures rising rapidly within the affected data halls.\nHost servers, storage clusters, and network switches were shut down to prevent equipment damages due to the extreme heat.\n- The high temperatures in the data hall resulted in disruption to customer workloads and control plane operations across the affected services.\nGoogle engineers have begun a full root cause analysis and we will provide additional information once it is available.\n## Remediation\nGoogle engineering teams were alerted to the issue via automated temperature and hardware unreachability telemetry starting at 16:39 US/Pacific.\nEngineers collaborated with the third-party facility provider to safely restore primary utility power and cooling systems, returning ambient room temperatures to safe operational levels. With the environment stabilized, field support engineers and remote teams systematically booted and verified the server hosts, storage nodes, and network fabric in a controlled sequence. Automated recovery playbooks were executed to restore underlying network switches and routing infrastructure, allowing storage appliances and compute clusters to be brought back online and verified for health. The underlying infrastructure has been fully recovered, and normal operations have been successfully restored.\n## Description of Impact\nOn 15 July 2026 from 16:39 to 16 July 2026 07:34 US/Pacific, customers experienced service disruptions across several products in europe-west4-a zone:\n- **Google Cloud VMware Engine:** Customers lost connectivity to their private clouds and workloads as foundational hosts and network switches were powered off for thermal protection.\n- **Bare Metal Solution:** Customers experienced connectivity loss to their database servers and storage appliances as the underlying hardware was powered down.\n- **Google Cloud NetApp Volumes:** Customers in the STANDARD, PREMIUM, and EXTREME service levels were unable to access their storage volumes. Control plane operations, including the creation of new storage pools, volumes, and backups, experienced failures in the affected region.", "when": "2026-07-18T00:22:37+00:00" "created": "2026-07-18T00:22:37+00:00", "modified": "2026-07-18T00:22:38+00:00", "text": "# Preliminary Incident Report\nWe sincerely apologize for the disruption this incident caused to your business. We know how much you rely on Google Cloud, and we regret the impact on your productivity.\nPlease note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. A final Incident Report with preventative actions will be posted once our investigation is complete.\nIf you have experienced impact outside of what is listed below, please reach out to Google Cloud Support using https://cloud.google.com/support.\n## Date/Time of the Issue (All time US/Pacific)\n**Google Cloud VMware Engine Impact:**\n- Start: 15 July 2026 17:09\n- End: 16 July 2026 2:33\n- Duration: 9 Hours, 24 Minutes\n**Google Cloud NetApp Volumes Impact:**\n- Start: 15 July 2026 16:39\n- End: 16 July 2026 01:10\n- Duration: 8 Hours 31 Minutes\n**Bare Metal Solution Impact:**\n- Start: 15 July 2026 18:37\n- End: 16 July 2026 07:34\n- Duration: 12 Hours 57 Minutes\n## Summary\nOn Wednesday, 15 July 2026, Google Cloud VMware Engine, Bare Metal Solution, and Google Cloud NetApp Volumes experienced service interruptions for a total duration of 14 hours, 55 minutes. We are taking immediate steps to ensure this doesn’t happen again.\n## Preliminary Root Cause\nA regional data center hosting services in europe-west4-a experienced a loss of utility power and subsequent cooling capacity.\nThe sequence of events leading to customer impact was as follows:\n- An electrical fault occurred on the utility grid upstream of the data center, disrupting the electrical distribution gear and cooling equipment.\n- The loss of cooling infrastructure resulted in ambient temperatures rising rapidly within the affected data halls.\nHost servers, storage clusters, and network switches were shut down to prevent equipment damages due to the extreme heat.\n- The high temperatures in the data hall resulted in disruption to customer workloads and control plane operations across the affected services.\nGoogle engineers have begun a full root cause analysis and we will provide additional information once it is available.\n## Remediation\nGoogle engineering teams were alerted to the issue via automated temperature and hardware unreachability telemetry starting at 16:39 US/Pacific.\nEngineers collaborated with the third-party facility provider to safely restore primary utility power and cooling systems, returning ambient room temperatures to safe operational levels. With the environment stabilized, field support engineers and remote teams systematically booted and verified the server hosts, storage nodes, and network fabric in a controlled sequence. Automated recovery playbooks were executed to restore underlying network switches and routing infrastructure, allowing storage appliances and compute clusters to be brought back online and verified for health. The underlying infrastructure has been fully recovered, and normal operations have been successfully restored.\n## Description of Impact\nOn 15 July 2026 from 16:39 to 16 July 2026 07:34 US/Pacific, customers experienced service disruptions across several products in europe-west4-a zone:\n- **Google Cloud VMware Engine:** Customers lost connectivity to their private clouds and workloads as foundational hosts and network switches were powered off for thermal protection.\n- **Bare Metal Solution:** Customers experienced connectivity loss to their database servers and storage appliances as the underlying hardware was powered down.\n- **Google Cloud NetApp Volumes:** Customers in the STANDARD, PREMIUM, and EXTREME service levels were unable to access their storage volumes. Control plane operations, including the creation of new storage pools, volumes, and backups, experienced failures in the affected region.", "when": "2026-07-18T00:22:37+00:00" "modified": "2026-07-18T00:22:37+00:00",
"modified": "2026-07-18T00:22:37+00:00",
Check your models and usage for free. See the relevant official changes, estimated cost, and a practical next step.