tag:www.dynstatus.com,2005:/historyOracle + Dyn Status - Incident History2024-03-19T02:30:32ZOracle + Dyntag:www.dynstatus.com,2005:Incident/202791112024-03-18T12:52:39Z2024-03-18T12:52:39ZDyn Standard DNS and Remote Access Resolution Degradation in South Africa<p><small>Mar <var data-var='date'>18</var>, <var data-var='time'>12:52</var> UTC</small><br><strong>Resolved</strong> - This incident is resolved as of March 17,2024 18:35 UTC.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>19:22</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>19:22</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>19:21</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>19:21</var> UTC</small><br><strong>Investigating</strong> - As of around 13:00 UTC on March 16th, 2024, we are investigating reports of change propagation delays which may be impacting users in and around South Africa accessing hostnames/zones hosted in our Dyn Ecommerce services: Standard DNS and Remote Access/DynDNS Pro. Impacted users may be receiving resolution failures or misrouting for recently updated records, as the service may be returning the old (pre-update) record. We are actively investigating this issue, and will provide updates here as available.<br /><br />Customers with any questions are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/197271032024-01-16T17:23:20Z2024-01-16T17:41:11ZOracle Dyn Update Client | Host Connectivity<p><small>Jan <var data-var='date'>16</var>, <var data-var='time'>17:23</var> UTC</small><br><strong>Resolved</strong> - On January 16th, 2024, around 15:30 UTC, we mitigated the issue affecting Oracle Dyn services.</p><p><small>Jan <var data-var='date'>12</var>, <var data-var='time'>05:00</var> UTC</small><br><strong>Identified</strong> - Around January 12, 2024, we identified the root cause of the problem and are working towards a resolution to fixing the DDNS connectivity issue.</p><p><small>Jan <var data-var='date'>11</var>, <var data-var='time'>16:30</var> UTC</small><br><strong>Investigating</strong> - Starting around 16:38 UTC on January 11th, 2024, customers using the Dyn Update Client app may be observing update endpoint resolution failures when attempting to update their Dyn hostname. This issue impacts any customer using the Update Client app for Remote Access or Standard DNS. Engineers are currently investigating and updates will be provided here as available.<br />Customers with questions are encouraged to reach out to Dyn Customer Support.</p>tag:www.dynstatus.com,2005:Incident/185019682023-09-14T15:28:01Z2023-09-14T15:28:01ZDynID Login Failures: Loops back to login<p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>15:28</var> UTC</small><br><strong>Identified</strong> - On September 13th, 2023, we discovered a bug causing DynID portal login failures for Managed DNS customers using IP access protections. We have identified the source and are working with an upstream provider to resolve. Customers who need priority access to their Managed DNS account should contact Dyn Technical Support to request guidance. Customers may have begun experiencing issues on September 6th, 2023 and retries may be successful.<br /><br />Customer with questions are encouraged to contact Dyn Technical Support.</p>tag:www.dynstatus.com,2005:Incident/185019042023-09-14T15:21:37Z2023-09-14T15:21:37ZDynID 2FA SMS Failures: US & Canada<p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>15:21</var> UTC</small><br><strong>Identified</strong> - Since September 1, 2023, customers logging into the Oracle Dyn Managed DNS or Email Delivery portals via DynID may be experiencing failures to receive 2FA codes. We have identified the issues and are working on a fix with priority. Implementing the fix involves external parties which may result in an extended time to resolve. We will update here as available.<br /><br />Customers with questions are encouraged to contact our Dyn Technical Support team.</p>tag:www.dynstatus.com,2005:Incident/172585202023-05-31T19:51:05Z2023-05-31T19:51:05ZEnd of Life Reminder: Managed DNS, Email Delivery, Dyn Secondary DNS<p><small>May <var data-var='date'>31</var>, <var data-var='time'>19:51</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>May <var data-var='date'>15</var>, <var data-var='time'>19:51</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>May <var data-var='date'>15</var>, <var data-var='time'>19:50</var> UTC</small><br><strong>Scheduled</strong> - This is a reminder of the previously communicated, upcoming End of Life of the Dyn Managed DNS, Email Delivery and Secondary DNS services on May 31st, 2023. If you are using these services, you must take action before May 31st to prevent service disruption. Information on this End of Life, including directions for migrating to Oracle Cloud Infrastructure, can be found at https://www.oracle.com/corporate/acquisitions/dyn/technologies/migrate-your-services/<br /><br />This notice does not apply to our Dyn Remote Access (aka DynDNS Pro/Dynamic DNS) or Standard DNS products which do not have an end of life.<br /><br />Customers with questions are encouraged to reach out to migrations@dyn.com or your Dyn Product Support team.</p>tag:www.dynstatus.com,2005:Incident/172563522023-05-15T18:42:22Z2023-05-15T18:42:22ZOracle Dyn Managed DNS and Portal | UI Degradation<p><small>May <var data-var='date'>15</var>, <var data-var='time'>18:42</var> UTC</small><br><strong>Resolved</strong> - As of 15:19 UTC, engineers have confirmed that the Manage DNS and Managed DNS portal is now accessible again via the UI. Engineers confirm that the issue was caused by a recent upgrade that has been reverted back.</p><p><small>May <var data-var='date'>15</var>, <var data-var='time'>15:44</var> UTC</small><br><strong>Investigating</strong> - Starting at 15:05 UTC on May 15th 2023, Dyn Managed DNS engineers began investigating issues impacting the web portal and Manage DNS login access. Customers may be unable to login and Manage DNS and Portal UI. We will update with further information as it becomes available.</p>tag:www.dynstatus.com,2005:Incident/165130862023-03-15T18:45:39Z2023-03-15T18:45:39ZTelephone/Chat Provider Issues<p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>18:45</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>17:56</var> UTC</small><br><strong>Identified</strong> - Our telephone provider continues to experience service degradation which are prevent Dyn Support agent from receiving inbound calls or chats. We will follow-up with further information as available.</p><p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>15:42</var> UTC</small><br><strong>Investigating</strong> - On March 15th, 2023, beginning around 15:25 UTC, our telephone provider began having intermittent issues receiving inbound phone calls to our Technical Support phone numbers and Dyn Self Service (https://account.dyn.com) chat support. We are working with our provider to get these issues resolved as quickly as possible. Updates will be posted as they become available. <br /><br />Customers with questions are encouraged to reach out to our Product Support Team by email and request a callback if required.</p>tag:www.dynstatus.com,2005:Incident/161264512023-02-21T16:01:53Z2023-02-21T16:01:53ZOracle Dyn Managed DNS | UI & API Degradataion<p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>16:01</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>21</var>, <var data-var='time'>16:01</var> UTC</small><br><strong>Identified</strong> - Engineers have confirmed that the Managed DNS portal, API and Advanced Services have continued stability. We are now setting this incident to resolved.<br /><br />(We apologize for any confusion, our service status alerts should have been set to Operational at our last update.)</p><p><small>Feb <var data-var='date'>17</var>, <var data-var='time'>21:51</var> UTC</small><br><strong>Update</strong> - Engineers have confirmed Advanced Services are now back to fully operational status and Managed DNS portal accessibility via the UI and API remain stable. DNS resolution remains unaffected. At this time, our engineers will continue to monitor this issue. Customers with questions or continuing to see issues are encouraged to contact Dyn Product Support.</p><p><small>Feb <var data-var='date'>17</var>, <var data-var='time'>03:35</var> UTC</small><br><strong>Monitoring</strong> - As of 02:02 UTC, engineers have confirmed that the Managed DNS portal is now accessible again via the UI and API. However, customers may still experience some service degradation with Advanced Services. DNS resolution remains unaffected. Customers with questions or continuing to see issues are encouraged to contact Dyn Product Support at this time.</p><p><small>Feb <var data-var='date'>16</var>, <var data-var='time'>22:49</var> UTC</small><br><strong>Update</strong> - Engineers continue work to restore the Managed DNS portal and Advanced Services monitoring. The continued stability of the platform is our ultimate priority while we are work to restore all function as quickly as possible. At this time, we recommend customers make any needed changes via the API. Customers requiring assistance with reseting their API passwords are asked to reach out to Support from their DynID account (which has remained accessible) with their request to expedite assistance.<br /><br />Customer with questions are encouraged to reach out to the Dyn Product Support team.</p><p><small>Feb <var data-var='date'>16</var>, <var data-var='time'>13:55</var> UTC</small><br><strong>Update</strong> - Engineers are continuing to monitor the stability of the API and change propagation. With continued performance stability, engineers will next work to carefully reintroduce the Managed DNS portal and restore Advanced Services. The continued stability of the platform is our ultimate priority, but we are working to restore all function as quickly as possible.<br /><br />Customer with questions are encouraged to reach out to the Dyn Product Support team.<br /><br />(We apologize that the affected services was not updated to report Advanced Services degradation at impact start. This has been now been corrected.)</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>22:09</var> UTC</small><br><strong>Update</strong> - At this time, we are seeing improved performance on our backend processing and continuing to open up API capacity until they reach normal operating levels. We will continue to monitor the stability of connections via API as our first priority. The Managed DNS portal remains unavailable (in maintenance) at this time.<br /><br />As part of the service protection strategy, Advanced Services are not currently honoring monitoring agent health status to make changes to record eligibility. Customers using Advanced services who require record changes are recommeded to use the API to make endpoint updates directly to your service at this time.<br /><br />Further updates will be provided here as available.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>21:50</var> UTC</small><br><strong>Update</strong> - Our engineers have implemented an adaptive mitigation approach using real-time backend optimizations and fine-tuning of DNS Load Management to handle current requests. These actions are being done in parallel as we continue to monitor the progress of clearing the backlog of requests and overall service health. We will provide an update as soon as we have one available.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>15:24</var> UTC</small><br><strong>Update</strong> - Update: Engineers continue work to carefully restore API function. At this time, the API is available but experiencing significant latency/timeouts related to limitations imposed in correction efforts. The Managed DNS portal is in maintenance mode (unavailable). DNS resolution is not impacted, and changes are propagating with intermittent high latency (> 1 min).<br /><br />This incident is our highest priority. There is no ETA for restoration at this time but we are focused on incrementing service functional improvements (mitigation) while also working toward the goal of total restoration. We will follow-up as updates are available.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>13:26</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Feb <var data-var='date'>15</var>, <var data-var='time'>05:54</var> UTC</small><br><strong>Identified</strong> - We are performing additional backend optimizations to help mitigate the issue and clear the backlog of requests to restore overall service health. Further updates will be provided upon mitigation or as events warrant.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>23:21</var> UTC</small><br><strong>Update</strong> - Engineers continue work to restore the Managed DNS API and Portal service stability. <br /><br />At this time, restorations efforts are being carried out as a controlled introduction of backlog and live traffic to the service. API availability is improved, however latency and/or timeouts may still be encountered due to temporary limits related to corrective efforts. Similarly, the Managed DNS UI has not yet been restored. Our engineers are actively engaged in monitoring and adjusting workload flows to ensure that we restore service stability as quickly as possible.<br /><br />We understand the impact your business and have been continuously addressing this incident as a highest priority. We will continue to be "all hands on deck" as we work to restore the service.<br /><br />Customers with questions or experiencing impact other than listed are encourage to reach out to Dyn Product Support.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>19:04</var> UTC</small><br><strong>Monitoring</strong> - Our engineers have deployed a software patch and performed a series of backend configuration adjustments to help resolve the issue with Managed DNS. Currently, we are seeing significant signs of recovery to backend processing. We will continue to monitor the progress of this recovery. Further updates will be provided upon mitigation or as events warrant.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>05:17</var> UTC</small><br><strong>Update</strong> - Additional investigation and troubleshooting steps are in progress towards the mitigation. Our engineers continue their efforts to restore the service.</p><p><small>Feb <var data-var='date'>14</var>, <var data-var='time'>01:44</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>23:49</var> UTC</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>20:55</var> UTC</small><br><strong>Identified</strong> - We are continuing to address this incident. At this time, we understand that customers are unable to access their Managed DNS via API or UI. Additionally, there is evidence that changes made shortly before or since the impact start time (17:37 UTC) may be experiencing propagation delays. DNS resolution is not impacted.<br /><br />We are addressing this incident as a Sev1 and will provide further information as it is available</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>19:40</var> UTC</small><br><strong>Update</strong> - We are continuing to investigate this issue. At this time, customers are unable to access the Managed DNS service via UI (manage.dynect.net) or API (api.dynect.net or api2.dynect.net). We are working on this incident with priority and will provide updates as available.<br /><br />Customer with questions are encouraged to reach out to Dyn Product Support.</p><p><small>Feb <var data-var='date'>13</var>, <var data-var='time'>18:55</var> UTC</small><br><strong>Investigating</strong> - Starting around 17:37 UTC on February 13th, 2023, Dyn Managed DNS engineers began investigating issues impacting the web portal and DNS records updates. At this time, we are seeing intermittent failures (500 errors) accessing the Managed DNS UI and API for records updates. We will update with further information as it is available.</p>tag:www.dynstatus.com,2005:Incident/158029262023-01-13T21:06:38Z2023-01-13T21:06:38ZPlanned maintenance for Dyn Managed DNS (All Regions)<p><small>Jan <var data-var='date'>13</var>, <var data-var='time'>21:06</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>15:06</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'> 9</var>, <var data-var='time'>15:03</var> UTC</small><br><strong>Scheduled</strong> - On January 9th, 2023 starting at 15:00 UTC, we will begin performing maintenance on the Managed DNS Anycast edge. Maintenance will be completed on every region in a rolling deployment over 3 days. During maintenance, users accessing sites in and around the targeted region may observe intermittent latency related to query rerouting affecting a single regional NS. Customers using Traffic Manager may observe intermittent unexpected routing (based on specific regional rules due to traffic routing changes) which will likely be successful on retry.<br /><br />At the end of this maintenance, query internet routing adjustments which remain will be expected, and any concerns should be forwarded to Dyn Product Support.<br /><br />Customers with questions or concerns are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/157808642023-01-10T20:01:53Z2023-01-10T20:01:53ZPlanned maintenance for Dyn Managed DNS: Advanced Services<p><small>Jan <var data-var='date'>10</var>, <var data-var='time'>20:01</var> UTC</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Jan <var data-var='date'>10</var>, <var data-var='time'>19:01</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Jan <var data-var='date'> 5</var>, <var data-var='time'>21:20</var> UTC</small><br><strong>Scheduled</strong> - On January 10th, 2023 between 19:00 and 20:00 UTC, we will be performing maintenance on our Oracle Dyn Managed DNS Advance Services infrastructure. During the maintenance, propagation delays will be observed for any changes made manually, via API, or due to eligible failover events for Advanced Service records; including Traffic Director, Traffic Manager, Active Failover and HTTP Redirect. Updates made to these records during the maintenance window will be queued for propagation at the completion of the maintenance work.<br /><br />Customers with questions or concerns are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/156641432023-01-04T14:56:49Z2023-01-04T14:56:49ZRemote Access/DynDNS Pro Service Domain Issue: Selfip.biz<p><small>Jan <var data-var='date'> 4</var>, <var data-var='time'>14:56</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>20</var>, <var data-var='time'>15:24</var> UTC</small><br><strong>Identified</strong> - Starting on December 19, 2022, Remote Access (DynDNS Pro) customers may start to observe intermittent issues to some of their DDNS hosts. The following domain was identified to be impacted by this incident: <br /><br />* Selfip.biz <br /><br />No additional services or Remote Access Domains are expected to be impacted.<br /><br />UPDATE: The issue has been identified and our engineers are actively working to implement a fix.<br /><br />ETA for resolution is currently unknown. However, as a workaround solution, users may create a new host name using an alternate domain for any time sensitive connections.<br /><br />This post will be updated when a fix is in place.<br /><br />Customers with questions are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/150891832022-12-14T22:30:20Z2022-12-14T22:30:20ZPlanned maintenance for Dyn Managed DNS (All Regions)<p><small>Dec <var data-var='date'>14</var>, <var data-var='time'>22:30</var> UTC</small><br><strong>Completed</strong> - This maintenance is being rescheduled and will be announced shortly via dynstatus.com.</p><p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>14:15</var> UTC</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Dec <var data-var='date'>13</var>, <var data-var='time'>14:10</var> UTC</small><br><strong>Scheduled</strong> - On December 13th, 2022 starting at 14:00 UTC, we will begin performing maintenance on the Managed DNS Anycast edge. Maintenance will be completed on every region in a rolling deployment over 3 days. During maintenance, users accessing sites in and around the targeted region may observe intermittent latency related to query rerouting affecting a single regional NS. Customers using Traffic Manager may observe intermittent unexpected routing (based on specific regional rules due to traffic routing changes) which will likely be successful on retry.<br /><br />At the end of this maintenance, query internet routing adjustments which remain will be expected, and any concerns should be forwarded to Dyn Product Support.<br /><br />Customers with questions or concerns are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/136615132022-11-21T14:11:54Z2022-12-13T14:12:18ZDyn Managed DNS | Advanced Services Monitoring Degradation<p><small>Nov <var data-var='date'>21</var>, <var data-var='time'>14:11</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>13</var>, <var data-var='time'>15:00</var> UTC</small><br><strong>Investigating</strong> - Starting around November 13, 2022, Dyn Managed DNS engineers began investigating an intermittent issue with our monitoring service used by the Active Failover and Traffic Manager advanced services. Impacted customers will observe ??? next to logs in their portal, with reduced or no log details within the monitoring logs for each endpoint. Efforts to correct this issue may temporarily cause unexpected monitoring locations for endpoints. Our engineers are currently working to identify the issue and mitigate any impact. <br /><br />Customers should not make any changes at this time. Updates will be posted as they become available<br /><br />Customers with questions are encouraged to reach out to our Technical Support Team.</p>tag:www.dynstatus.com,2005:Incident/131466942022-11-15T19:22:13Z2022-11-15T19:22:13ZDyn Managed DNS Resolution Degradation<p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>19:22</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>18:52</var> UTC</small><br><strong>Monitoring</strong> - As of 18:30 UTC, this incident is resolved. We are closely monitoring for stability at this time.</p><p><small>Nov <var data-var='date'>15</var>, <var data-var='time'>18:16</var> UTC</small><br><strong>Investigating</strong> - Beginning at 18:00 UTC, November 15th, 2022 , we began investigating DNS resolution issues with Managed DNS within multiple global regions. Users accessing Managed DNS hosted zones may experience intermittent DNS resolution failures and latency. Our engineers are currently working to identify the issue and mitigate any impact.<br /><br />Customers with questions are encouraged to reach out to the Dyn product support team.</p>tag:www.dynstatus.com,2005:Incident/129340802022-11-12T19:28:15Z2022-11-14T15:33:39ZOracle Dyn Managed DNS: Emergency Maintenance<p><small>Nov <var data-var='date'>12</var>, <var data-var='time'>19:28</var> UTC</small><br><strong>Resolved</strong> - Engineers have completed the emergency maintenance and full functionality has been restored. <br /><br />Customers with additional questions are encouraged to contact Dyn Product Support.</p><p><small>Nov <var data-var='date'>12</var>, <var data-var='time'>18:49</var> UTC</small><br><strong>Identified</strong> - Starting at 17:25 UTC on November 12, 2022, engineers began emergency maintenance on the Managed DNS infrastructure. At this time, customers may be observing the following intermittent issues with Managed DNS:<br /><br /> - Propagation delays for new and updated records<br /> - DynID Portal 503s (unavailable)<br /> - API timeouts<br /> - Query Reporting delays<br /><br />Engineers are investigating this issue with priority and will provide updates here as available.<br />Customers with questions are encouraged to contact Dyn Product Support.</p>tag:www.dynstatus.com,2005:Incident/127174292022-11-10T16:43:50Z2022-11-13T10:24:36ZDyn Support Phone Issue<p><small>Nov <var data-var='date'>10</var>, <var data-var='time'>16:43</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Nov <var data-var='date'> 7</var>, <var data-var='time'>12:16</var> UTC</small><br><strong>Identified</strong> - We have identified the issue and partially restored service. Agents continue to experience intermittent issues and may utilize email to expedite response to voicemails left for Technical Support. We will provide further updates as available.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>14:58</var> UTC</small><br><strong>Investigating</strong> - Starting around 12:00 UTC on November 4, 2022, we are currently experiencing issues with accepting or returning calls to our phone queues. Voicemails to our support call queues are being received and response will be provided via email/case. At this time, we recommend customers with urgent needs leave a detailed message including an email address for response.<br /><br />We are investigating this issue and will provide updates here.<br /><br />Customers with questions are encouraged to email Dyn Product Support.</p>tag:www.dynstatus.com,2005:Incident/127491922022-11-08T06:25:14Z2022-11-08T06:25:14ZDyn Managed DNS Name Resolution Issues in Tokyo<p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>06:25</var> UTC</small><br><strong>Resolved</strong> - Engineers have resolved network routing relationships which were causing some Managed DNS traffic originating from Tokyo to fail due to time outs. Partial resolution resulting in impacted customers receiving response on failed requests upon retry was observed at 04:47 UTC; with full resolution around 06:16 UTC. Customers with questions should reach out to Dyn Technical Support.</p><p><small>Nov <var data-var='date'> 8</var>, <var data-var='time'>02:01</var> UTC</small><br><strong>Investigating</strong> - Beginning at 22:30 UTC, November 7, 2022, we are investigating an issue where some users in and around Tokyo, Japan may be experiencing failing name resolution when attempting to monitor or access sites hosted on Dyn Managed DNS. We are investigating this issue and looking into a resolution as quickly as possible. <br /><br />Customers with any questions are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/126950802022-11-02T15:44:27Z2022-11-02T19:44:52ZPropagation delays with Standard DNS & Remote Access<p><small>Nov <var data-var='date'> 2</var>, <var data-var='time'>15:44</var> UTC</small><br><strong>Resolved</strong> - This incident is resolved as of 15:40 UTC.</p><p><small>Nov <var data-var='date'> 2</var>, <var data-var='time'>14:33</var> UTC</small><br><strong>Investigating</strong> - On November 2nd, 2022 beginning around 10:37 UTC, we began investigating an issue with Dyn Standard DNS & Email Delivery. Our engineers have identified an issue in the service core host site causing delays in pushing updates to the edge responders. Customers impacted by this issue will observe delays in the global propagation of new or updated records. This issue does not affect any other Dyn services.<br /><br />Customers should not make any changes at this time. Updates will be posted as they become available<br /><br />Customers with questions are encouraged to reach out to our Technical Support Team.</p>tag:www.dynstatus.com,2005:Incident/112284832022-09-23T04:00:02Z2022-09-23T04:00:02ZDyn Managed DNS Emergency Maintenance in Milan, IT<p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>04:00</var> UTC</small><br><strong>Resolved</strong> - As of 03:58 UTC on September 23rd, 2022, this incident is now resolved. Customers with questions are encouraged to reach out to the Product Support Team.</p><p><small>Sep <var data-var='date'>23</var>, <var data-var='time'>03:06</var> UTC</small><br><strong>Investigating</strong> - On September 23rd, 2022 at 00:10 UTC, we began performing emergency maintenance on our data center in Milan, IT. Users may experience SERVFAILs when accessing Managed DNS hosted zones in and around Milan. Our engineers are actively working to restore service. Updates will be provided here as available.<br /><br />Customers with questions or concerns are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/111350922022-09-15T00:32:50Z2022-09-15T00:32:50ZOracle Dyn | Manage DNS Query Reporting down<p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>00:32</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Sep <var data-var='date'>14</var>, <var data-var='time'>22:51</var> UTC</small><br><strong>Investigating</strong> - Starting around 19:20 UTC on September 13th, 2022, the Managed DNS query reporting stopped populating for customer accounts. Customers attempting to access their query reporting via the UI or API will return no results.<br /><br />Engineers are working to restore this service as soon as possible. Updates will be posted here as they are available.<br /><br />Customers with questions are encouraged to reach out to our Technical Support team.</p>tag:www.dynstatus.com,2005:Incident/111208542022-09-13T16:10:25Z2022-09-13T16:11:18ZDyn Managed DNS | Portal and API degradation<p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>16:10</var> UTC</small><br><strong>Resolved</strong> - As of 16:03 UTC, this issue is now resolved. Customers with questions or continuing to see issues are encouraged to contact Dyn Product Support at this time.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>15:46</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Sep <var data-var='date'>13</var>, <var data-var='time'>15:43</var> UTC</small><br><strong>Investigating</strong> - On September 13th, 2022 around 15:10 UTC, we began investigating intermittent issues affecting the Dyn Managed DNS API and DynID/Service portals. Impacted customers may observe timeout errors on the API and web portal 503s which may be successful on retries. Our engineers are actively investigating at this time. Updates will be provided here as available.<br /><br />Customers with questions are encouraged to reach out to the Dyn Support team.</p>tag:www.dynstatus.com,2005:Incident/110916602022-09-10T04:00:00Z2022-09-12T13:24:34ZDyn Standard DNS and Remote Access Resolution Degradation: Europe and North America<p><small>Sep <var data-var='date'>10</var>, <var data-var='time'>04:00</var> UTC</small><br><strong>Resolved</strong> - This incident is resolved as of 22:52 UTC on September 9th, 2022 incident. During investigation, engineering discovered propagation failures which would have caused old records to be served for users accessing Standard DNS and Remote Access hostname in both the Toronto, Canada and Frankfurt, Germany DNS regions which had been updated within the duration of this event.</p><p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>19:33</var> UTC</small><br><strong>Investigating</strong> - As of around 15:30 UTC on September 9th, 2022 , we are investigating reports of change propagation delays which may be impacting users in and around Frankfurt, Germany accessing hostnames/zones hosted in our Dyn Ecommerce services: Standard DNS and Remote Access/DynDNS Pro. Impacted users may be receiving resolution failures or misrouting for recently updated records, as the service may be returning the old (pre-update) record. We are actively investigating this issue, and will provide updates here as available.<br /><br />Customers with any questions are encouraged to reach out to our Product Support Team.</p>tag:www.dynstatus.com,2005:Incident/110845182022-09-09T12:33:03Z2022-09-09T12:34:49ZDyn Managed DNS Emergency Maintenance in Ashburn, VA<p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>12:33</var> UTC</small><br><strong>Resolved</strong> - Managed DNS resolution and API services stabilized September 9th, 2022 around 00:18 UTC. Customers may have experienced a brief period of API latency and/or timeouts as services rebalanced on September 8th, 2022 around 23:54 UTC.</p><p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>00:05</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p><p><small>Sep <var data-var='date'> 9</var>, <var data-var='time'>00:03</var> UTC</small><br><strong>Update</strong> - Engineers for our hosting service in Ashburn, VA have identified the source of the site degradation and are working on recovery. Managed DNS announcements in and around the US East are automatically rerouting queries for successful response, however the extended rerouting has resulted in some additional latency, timeouts and retries on internal services which may be causing occasional API latency and/or failures. We are working to resolve this incident and will follow up with information here as available.</p><p><small>Sep <var data-var='date'> 8</var>, <var data-var='time'>21:56</var> UTC</small><br><strong>Investigating</strong> - On September 8th, 2022 starting around 20:33 UTC, Dyn Managed DNS engineers began emergency maintenance in Ashburn, VA (IAD). At this time, users accessing zones hosted in Managed DNS in and around US-East may experience additional DNS latency as queries are rerouted for response. Traffic Manager services with IAD regional routing may experience unexpected DNS response based on the region receiving the redirected query. No impact is expected to other regions, services or to DNS change propagation.<br /><br />Customers with questions are encouraged to reach out to Dyn Technical Support.</p>tag:www.dynstatus.com,2005:Incident/109589632022-08-30T13:33:34Z2022-08-30T13:33:34ZDyn Dynamic DNS and Standard DNS Zone Propagation Delays<p><small>Aug <var data-var='date'>30</var>, <var data-var='time'>13:33</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'>26</var>, <var data-var='time'>19:32</var> UTC</small><br><strong>Investigating</strong> - Starting around August 23rd 2022, Dyn Dynamic DNS and Standard DNS have been experiencing zone propagation delays. Our engineers are currently working to identify and mitigate the issue. <br /><br />Customers with questions are encouraged to reach out to the Dyn product support team.</p>tag:www.dynstatus.com,2005:Incident/107853262022-08-08T21:57:45Z2022-08-08T22:24:54ZDyn Managed DNS | DNS Resolution Degradation<p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>21:57</var> UTC</small><br><strong>Resolved</strong> - The investigation is complete and we've determined that DNS resolution with Managed DNS is healthy in the Ashburn region. However, as part of our efforts to improve service resiliency, we've implemented additional configuration changes to the back-end systems on August 8th, 2022, at 20:38 UTC.</p><p><small>Aug <var data-var='date'> 8</var>, <var data-var='time'>20:01</var> UTC</small><br><strong>Investigating</strong> - On August 8th, 2022, beginning around 19:22 UTC, we began investigating DNS resolution issues with Managed DNS within the Ashburn region. Users accessing Managed DNS hosted zones in and around the US-East may experience intermittent DNS resolution failures and latency. Our engineers are currently working to identify the issue and mitigate any impact.<br /><br />Customers with questions are encouraged to reach out to the Dyn product support team.</p>