incident.json

[eph-raim]

<title>Google Cloud Service Health Updates</title>2022-04-01T20:13:25+00:00 Google Cloud https://status.cloud.google.com/ <title>RESOLVED: Global: Cloud Monitoring Metrics may be unavailable or underreported for Cloud Pub/Sub</title> tag:status.cloud.google.com,2022:feed:B1hD4KAtcxiyAWkcANfV.T3VTiKN96GAYJ2R68X6b 2022-04-01T20:13:25+00:00

<p> Incident began at <strong>2022-03-31 08:30</strong> and ended at <strong>2022-03-31 15:54</strong> <span>(all times are <strong>US/Pacific</strong>).</span></p><div class="cBIRi14aVDP__status-update-text"><p>We apologize for the inconvenience this service disruption/outage may have caused. We would like to provide some information about this incident below. Please note, this information is based on our best knowledge at the time of posting and is subject to change as our investigation continues. If you have experienced impact outside of what is listed below, please reach out to Google Support by opening a case using <a href="https://cloud.google.com/support">https://cloud.google.com/support</a> or help article <a href="https://support.google.com/a/answer/1047213">https://support.google.com/a/answer/1047213</a>.</p> <p>(All Times US/Pacific)</p> <p><strong>Incident Start:</strong> 31 March 2022 08:30</p> <p><strong>Incident End:</strong> 31 March 2022 15:54</p> <p><strong>Duration:</strong> 7 hours, 24 minutes</p> <p><strong>Affected Services and Features:</strong></p> <p>Google Cloud Pub/Sub, Google Cloud Monitoring</p> <p><strong>Regions/Zones:</strong> Global Locale</p> <p><strong>Description:</strong></p> <p>Google Cloud Pub/Sub customers experienced issues with metrics in Google Cloud Monitoring for a duration of 7 hours, 24 minutes. The issue was caused by a configuration change to the backend for Cloud Monitoring that affected Cloud Pub/Sub metric recording. The issue was mitigated by reverting this change.</p> <p><strong>Customer Impact:</strong></p> <ul> <li>Cloud Pub/Sub metrics in Cloud Monitoring for times during the incident may be missing or underreported. - The metric values lost in this timeframe will not be recoverable.</li> <li>Any alerting based on these metrics might have fired erroneously or not fired when they should have during the time of the incident.</li> <li>Any auto scaling of Google Kubernetes Engine (GKE) based on these metrics may not have functioned as expected during the time of the incident.</li> <li>Cloud Pub/Sub administrative, publish, and subscribe operations were not affected by the incident.</li> </ul> </div><hr><p>Affected products: Cloud Monitoring, Operations, Google Cloud Pub/Sub</p><p>Affected locations: Global</p>