Resolved -
Between approximately 20:56 UTC on July 8 and 01:40 UTC on July 9, customers using the V2 Config Delivery API may have experienced incorrect evaluation of launched Feature Gates. During this window, launched Gates could return false instead of their expected default. V1 consumers were not affected.
We identified the root cause as a recent change to config-spec generation. We rolled back the affected services across all regions and regenerated config payloads. We have verified that launched Gates are again returning their expected defaults, and all Config Delivery API components are now operational.
No action is required to restore correct evaluation going forward. However, if your systems persisted data or took actions based on evaluations returned by the V2 Config Delivery API during the impact window, we recommend reviewing and reconciling that downstream data, as affected launched Gates may have returned false during that time.
Jul 8, 19:36 PDT
Monitoring -
A fix has been implemented and we are monitoring the results.
Jul 8, 19:15 PDT
Update -
We are continuing to work on a fix for this issue.
Jul 8, 18:39 PDT
Update -
We are continuing to work on a fix for this issue.
Jul 8, 18:35 PDT
Update -
We are continuing to work on a fix for this issue.
Jul 8, 18:29 PDT
Identified -
Since approximately 1:56 PM PT on July 8 (20:56 UTC on July 8), customers using the V2 Config Delivery API may have experienced incorrect evaluation of launched Feature Gates. Specifically, launched Gates may have returned false instead of their expected default value.
Our team identified the root cause as a recent code change and started a rollback of the affected services. We were actively working to confirm that evaluations returned by the V2 Config Delivery API were again showing expected defaults for launched Gates.
V1 consumers were not affected. At the time of this update, no immediate customer action was required.
We would provide the next update as more information became available.
Jul 8, 18:28 PDT