tag:status.scaleflex.com,2005:/historyCloudimage by Scaleflex Status - Incident History2024-03-29T04:55:22+01:00Cloudimage by Scaleflextag:status.scaleflex.com,2005:Incident/189869582023-11-01T09:14:03+01:002023-11-01T09:14:03+01:00Network issue<p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>09:14</var> CET</small><br><strong>Resolved</strong> - 2023-11-30 09:55 CET<br /><br />Dear Valued Customer,<br /><br />We are pleased to inform you that the recent issue impacting our services has been successfully resolved. Our dedicated team identified the root cause and restored normal operations.<br /><br />We genuinely appreciate your patience and continued support during this time. If you have any further questions or concerns, please do not hesitate to reach out to us.<br /><br />Best Regards,<br /><br />Major Incident Team</p><p><small>Nov <var data-var='date'> 1</var>, <var data-var='time'>08:59</var> CET</small><br><strong>Investigating</strong> - Dear Customers,<br /><br />Please be advised that at 09:35 CET we've detected a network issue with one of our EU core servers that affects our services. <br /><br />Our engineering team is actively working on restoring the Service to normal operations.<br /><br />We regret any inconvenience this may have caused.<br /><br />Regards,<br /><br />Major Incident Team</p>tag:status.scaleflex.com,2005:Incident/189670402023-10-30T14:00:16+01:002023-10-30T15:40:35+01:00Network issue - 502 error<p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>14:00</var> CET</small><br><strong>Resolved</strong> - 2023-10-30 14:00 CET<br /><br />Dear Valued Customer,<br /><br />We are pleased to inform you that the recent issue impacting our services has been successfully resolved. Our dedicated team worked diligently to address the situation and restore normal operations.<br /><br />Currently, our focus remains on optimizing the performance of our infrastructure to enhance your experience further. We are committed to providing you with a seamless and reliable service, and these efforts are a testament to that commitment.<br /><br />We genuinely appreciate your patience and continued support during this time. If you have any further questions or concerns, please do not hesitate to reach out to us.<br /><br />Warm regards,<br /><br />Major Incident Team</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>13:42</var> CET</small><br><strong>Monitoring</strong> - Dear Customer,<br /><br />Please be advised that we activated one of our spare servers into our infrastructure to mitigate the impact of Error 502. <br /><br />Our team, with the support of our hosting provider, is working diligently to address any remaining issues and optimise the performance of the infrastructure.<br /><br />We expect to fully resolve the situation in 20-30 minutes or earlier.<br /><br />Thank you for your continued support.<br /><br />Regards,<br /><br />Major Incident Team</p><p><small>Oct <var data-var='date'>30</var>, <var data-var='time'>12:33</var> CET</small><br><strong>Identified</strong> - At approximately 08:15 UTC, we detected via our monitoring systems that one of our EU load balancer servers was down, and we have been working with our hosting provider to resolve the issue. <br /><br />Since 10:00 AM UTC, Users may experience intermittent errors (essentially 502 Error) and delays while accessing the platform. <br /><br />Our engineering team is actively working on restoring the Service to normal operations.<br /><br />Regards,<br /><br />Major Incident Team</p>tag:status.scaleflex.com,2005:Incident/162953942023-02-27T21:47:47+01:002023-02-27T21:48:30+01:00Network issue<p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>21:47</var> CET</small><br><strong>Resolved</strong> - Dear Customers,<br /><br />Please be advised that we've identifed the root cause of the network issue which is randomly affects our services. <br />Our Infrastructure team implemented a fix and we were cautious in our statement while monitoring the network.<br />All services have been back to normal for a couple of hours or more.<br /><br />We regret any inconvenience this may have caused.<br /><br />Regards,<br /><br />Scaleflex Incident Management team</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>20:38</var> CET</small><br><strong>Monitoring</strong> - Dear Customers,<br /><br />Please be advised that we've identifed the root cause of the network issue which is randomly affects our services. <br />Our Infrastructure team implemented a fix earlier, and the situation is getting back to normal. We are now moving to monitoring mode until we are satisfied with the situation.<br /><br />We regret any inconvenience this may have caused.<br /><br />Regards,<br /><br />Scaleflex Incident Management team</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>18:52</var> CET</small><br><strong>Identified</strong> - Dear Customers,<br /><br />Please be advised that we've identifed the root cause of the network issue which is randomly affects our services. <br />Our Infrastructure team is implementing the fix, and an update will come shortly.<br /><br />We regret any inconvenience this may have caused.<br /><br />Regards,<br /><br />Scaleflex Incident Management team</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>17:41</var> CET</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>17:40</var> CET</small><br><strong>Update</strong> - Dear Customers,<br /><br />Please be advised that we've detected a network issue that randomly affects our services. <br />Our Infrastructure team is currently conducting an investigation, and an update will come shortly.<br /><br />We regret any inconvenience this may have caused.<br /><br />Regards,<br /><br />Scaleflex Incident Management team</p><p><small>Feb <var data-var='date'>27</var>, <var data-var='time'>17:37</var> CET</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.scaleflex.com,2005:Incident/157086102022-12-25T06:30:38+01:002022-12-25T12:50:26+01:00Cloudimage and Filerobot Admin UI - Unscheduled Upgrade Maintenance<p><small>Dec <var data-var='date'>25</var>, <var data-var='time'>06:30</var> CET</small><br><strong>Resolved</strong> - Dear Valued Customers,<br /><br />Please be advised that the maintenance has been performed successfully, and you should now have normal access to the Cloudimage and Filerobot Admin UIs.<br /><br />We appreciate your patience.<br /><br />Happy Christmas to everyone!</p><p><small>Dec <var data-var='date'>25</var>, <var data-var='time'>06:00</var> CET</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p><p><small>Dec <var data-var='date'>25</var>, <var data-var='time'>00:00</var> CET</small><br><strong>Investigating</strong> - Dear Valued Customers,<br /><br />We would like to notify you that the Cloudimage and Filerobot Admin UIs will be unavailable for few hours.<br />We have opted to execute an unscheduled upgrade maintenance due to low usage on this particular day.<br />There will be no disruption to the Service, and your end-users will not experience any downtime.<br /><br />Maintenance will take place from 00.00 to 06:00 am CET.<br /><br />Merry Christmas to everyone!<br /><br />Regards,</p>tag:status.scaleflex.com,2005:Incident/127065962022-11-04T04:00:42+01:002022-11-04T04:00:42+01:00Infrastructure improvement v2022-1104-1<p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>04:00</var> CET</small><br><strong>Completed</strong> - The scheduled maintenance has been completed.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>00:00</var> CET</small><br><strong>In progress</strong> - Scheduled maintenance is currently in progress. We will provide updates as necessary.</p><p><small>Nov <var data-var='date'> 4</var>, <var data-var='time'>00:00</var> CET</small><br><strong>Scheduled</strong> - To our value customer,<br /><br />Since the beginning of the year, we have begun the process of onboarding a number of significant accounts, and in order to maintain the high quality of our service, our infrastructure has undergone continual improvement. Meanwhile, we have released a large number of new features, all of which call for increased processing power; thus, we need to upgrade a couple of our servers in order to keep up with the growing demand.<br /><br />Impact Classification: Minor<br />During this period of maintenance, we strongly advise against uploading any new content because there is a chance that it will not be processed and will instead return an error message.<br />The delivery of any and all content previously managed by cloudimage will proceed as usual.<br /><br />Maintenance <br />2022-11-04 00:00:00<br />It is possible that the maintenance will take up to 4 hours<br /><br />Customer Action<br />There is no need for any action on your part.<br /><br />Questions<br />Please reach out to the support team if you need any further details.</p>tag:status.scaleflex.com,2005:Incident/111397932022-09-15T14:09:39+02:002022-09-15T14:09:39+02:00Cloudimage Console behaviour<p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>14:09</var> CEST</small><br><strong>Resolved</strong> - To Our Valued Customers,<br /><br />We would like to make you aware that we have located the issue, and the solution is now being deployed.<br />We would advise you to hold off on accessing the dashboard until the following day in order to confirm that the issue that was found is fully addressed.<br /><br />Please accept our apologies for any inconvenience.<br /><br />Regards,<br /><br /><br />Jean-Francois Lecas<br />VP of Global Services & Operations | DPO | Scaleflex</p><p><small>Sep <var data-var='date'>15</var>, <var data-var='time'>12:30</var> CEST</small><br><strong>Investigating</strong> - To Our Valued Customers,<br /><br />Please be advised that we have discovered an issue with the server that manages our Customer dashboard in Cloudimage, and as a direct consequence of this, you can find information that is inconsistent.<br /><br />Please be aware that the delivery of the Cloudimage Service is not impacted in any way by this issue because it is isolated.<br /><br />Our Backend Operations team is currently investigating the issue, and they are working to find a solution to it as quickly as possible.<br /><br />Regards,<br /><br /><br />Jean-Francois Lecas<br />VP of Global Services & Operations | DPO | Scaleflex</p>tag:status.scaleflex.com,2005:Incident/110627562022-09-06T12:53:11+02:002022-09-06T12:53:11+02:00Random 429 Error message<p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>12:53</var> CEST</small><br><strong>Resolved</strong> - To Our Valued Customers,<br /><br />Please be advised that the problem has been resolved.<br /><br />Our technical team has been collaborating closely with our partner, and the workaround was implemented within thirty minutes of the problem being identified. Since then, we've been monitoring the service delivery status, and we're satisfied that everything has returned to normal.<br /><br />Please submit a request to our Support team if your environment is still affected in any manner. Our team will prioritise your case.<br /><br />We sincerely apologise for the inconvenience this has caused.<br /><br />Regards,<br /><br /><br />Jean-Francois Lecas<br />VP of Global Services & Operations | DPO | Scaleflex</p><p><small>Sep <var data-var='date'> 6</var>, <var data-var='time'>10:42</var> CEST</small><br><strong>Investigating</strong> - To Our Valued Customers,<br /><br />One of our premium providers is having problems, and we are collaborating closely with them to find a solution that will have the least amount of negative effect on your environment. It is possible that you will see a significant number of "429 Error" messages, and please know that we are doing all in our power to get the situation back to normal.<br /><br />If your environment is affected in any way, please submit a ticket to our Support team so that we can prioritise your issue as quickly as possible.<br /><br />Please accept our sincere apologies for the trouble this has caused, and be assured that our Technical team is working diligently to find a solution to the problem.<br /><br />Regards,<br /><br /><br />Jean-Francois Lecas<br />VP of Global Services & Operations | DPO | Scaleflex</p>tag:status.scaleflex.com,2005:Incident/92097622022-02-01T11:52:14+01:002022-02-01T11:52:14+01:00Customer Admin Portal (Cloudimage Console & Filerobot Hub) upgrades<p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>11:52</var> CET</small><br><strong>Resolved</strong> - Dear Value Customers,<br /><br />We have completed our upgrade and you should not experience any further issue with the portal.<br /><br />Thank you for your patience.<br /><br />Regards,<br />Scaleflex Support team</p><p><small>Feb <var data-var='date'> 1</var>, <var data-var='time'>11:15</var> CET</small><br><strong>Monitoring</strong> - Dear Value Customers,<br /><br />We would like to inform you that during the course of the day, the Scaleflex Support team will be performing some upgrades on our Customers Portal. During that time, you may not have access to your environment.<br /><br />Please note that the delivery of the service is not affected, only the portal access is.<br /><br />If you have any emergency, please do not hesitate to log a ticket via the Scaleflex Help Center portal (https://scaleflex.zendesk.com/hc/en-gb).<br /><br />Thank you for your patience.<br /><br />Regards,<br /><br />Scaleflex Support team</p>tag:status.scaleflex.com,2005:Incident/86876462021-11-25T12:53:41+01:002021-11-25T13:09:30+01:00Network issue - 502/503 Error<p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>12:53</var> CET</small><br><strong>Resolved</strong> - Dear Customers,<br /><br />Please be aware that the incident related to the 502/503 Error you experienced is resolved.<br /><br />This was a network issue between our Hosting and CDN providers and we had to action our contingency plan.<br /><br />During the incident, you may have experienced that some images were not delivered and then delivered again. In principle, this phenomena would have affected the images which were not CDN cached yet.<br /><br />We apologise for the inconvenience and we will continue to work with our hosting provider to ensure that this kind of incident would not occur again.<br /><br />We are available if you need further assistance.<br /><br />Regards,<br />Scaleflex Major Incident team</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>11:15</var> CET</small><br><strong>Monitoring</strong> - Dear Customers,<br /><br />Please be aware that the incident is solved and we are closely monitoring our service delivery.<br />If you experience some issue, please inform the Scaleflex Support team as soon as possible.<br /><br />We do apologise for the inconvenience caused.<br /><br />Regards,<br />Scaleflex Major Incident team</p><p><small>Nov <var data-var='date'>25</var>, <var data-var='time'>10:00</var> CET</small><br><strong>Identified</strong> - Dear Customers,<br /><br />Our Infrastructure team has identified a network issue between our Hosting and our CDN providers.<br />As a result, some customers may experience a high level of 502/503 Error.<br /><br />We are actively working on resolving this incident by activating our Contingency Plan.<br /><br />We will keep you informed on progress.<br /><br />Regards,<br />Scaleflex Major Incident team</p>tag:status.scaleflex.com,2005:Incident/82168812021-10-13T10:46:11+02:002021-10-13T11:04:01+02:00OVH Cloud - Global outage<p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>10:46</var> CEST</small><br><strong>Resolved</strong> - Dear Valued Customers,<br /><br />Please be aware that we have completed the balancing on the workload on our multi-cloud infrastructure to mitigate the OVH DC outage impact.<br /><br />As per the information we received this morning, OVH planned an internal maintenance on their network which impacted and took down their entire and global delivery. <br />We will continue to monitor the OVH DC outage closely but you should not experience further issue.<br /><br />Regards,<br /><br />Scaleflex Incident Management team</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>09:53</var> CEST</small><br><strong>Update</strong> - We are continuing to investigate this issue.</p><p><small>Oct <var data-var='date'>13</var>, <var data-var='time'>09:52</var> CEST</small><br><strong>Investigating</strong> - Dear Valued customers,<br /><br />Please be aware that one of our datacenters hosted by OVH is currently experiencing an outage.<br />OVH Cloud Incident team has not communicated on the outage yet but we are seeing a slow recovery.<br /><br />You may experience some issue or slow response time as our infrastructure is balancing the workload to our other datacenters.<br /><br />We will keep you updated on the recovery.<br /><br />Regards,<br /><br />Scaleflex Incident Management team</p>tag:status.scaleflex.com,2005:Incident/71963932021-06-08T16:10:40+02:002021-06-10T11:34:17+02:00FASTLY -- Global CDN Disruption<p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>16:10</var> CEST</small><br><strong>Resolved</strong> - Dear Customers,<br /><br />Please find hereunder the latest feedback from FASTLY<br /><br />Jun 8, 12:41 UTC | Resolved - Fastly has observed recovery of all services and has resolved this incident. Customers could continue to experience a period of increased origin load and lower Cache Hit Ratio (CHR).<br /><br />This incident is now closed.<br /><br />Regards,<br /><br />Scaleflex Support team</p><p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>16:09</var> CEST</small><br><strong>Update</strong> - We are continuing to monitor for any further issues.</p><p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>13:04</var> CEST</small><br><strong>Monitoring</strong> - Dear Customers (FASTLY users only),<br /><br />Please be aware that FASTLY status page is back online [https://status.fastly.com].<br /><br />-----------------------------------<br />FASTLY - Global CDN Disruption<br />-----------------------------------<br /><br />Jun 8, 10:57 UTC | Monitoring - The issue has been identified and a fix has been applied. Customers may experience increased origin load as global services return.<br />Jun 8, 10:57 UTC | Monitoring - The issue has been identified and a fix has been applied. Customers may experience increased origin load as global services return.<br />Jun 8, 10:44 UTC | Identified - The issue has been identified and a fix is being implemented.<br />Jun 8, 10:26 UTC | Update - We are continuing to investigate this issue.<br />Jun 8, 10:23 UTC | Update - We are continuing to investigate this issue.<br />Jun 8, 10:21 UTC | Update - We are continuing to investigate this issue.<br />Jun 8, 10:11 UTC | Update - We are continuing to investigate this issue.<br />Jun 8, 10:07 UTC | Update - We are continuing to investigate this issue. <br />Jun 8, 09:58 UTC | Investigating - We're currently investigating potential impact to performance with our CDN services.<br /><br />We will monitor the service on our side.<br /><br />Regards,<br /><br />Scaleflex Support team</p><p><small>Jun <var data-var='date'> 8</var>, <var data-var='time'>12:44</var> CEST</small><br><strong>Identified</strong> - Dear Customers,<br /><br />If you are using the FASTLY network, you may encounter a strong degradation of services.<br />Unfortunately FASTLY has not communicated yet on the outage and we have no visibility on when the FASTLY service will be recovered.<br /><br />Scaleflex Support team is working actively to mitigate the situation.<br /><br />We will update you as soon as we have more visibility.<br /><br />Regards,<br /><br />Scaleflex Support team</p>tag:status.scaleflex.com,2005:Incident/64945692021-03-15T09:16:22+01:002021-03-15T09:16:22+01:00Potential Service Degradation<p><small>Mar <var data-var='date'>15</var>, <var data-var='time'>09:16</var> CET</small><br><strong>Resolved</strong> - Dear Customers,<br /><br />The incident has been resolved.<br /><br />We have carefully checked all the components and we can confirm that the performance return to normal levels.<br /><br />Regards,<br />Scaleflex IM Support team</p><p><small>Mar <var data-var='date'>12</var>, <var data-var='time'>09:10</var> CET</small><br><strong>Update</strong> - Dear Customers,<br /><br />Please be aware that we are in the final check of our API services.<br /><br />We have identified all the components which have been affected and Scaleflex Backend team are applying the appropriate remedy. Due to the complexity of our actions and careful approach, we are expecting the service to be back to normal by this Monday 15th.<br /><br />Rest assured we are monitoring the situation 24/7 and will continue to act in order to maintain the availability of your service.<br /><br />Kind regards,<br />Scaleflex IM Support team</p><p><small>Mar <var data-var='date'>11</var>, <var data-var='time'>14:44</var> CET</small><br><strong>Monitoring</strong> - Dear Customers,<br /><br />We are pleased to inform you that Scaleflex Business Continuity Plan has allowed us to mitigate the impact related to this major incident.<br /><br />We have identified all the components which have been affected and Scaleflex Backend team are applying the appropriate remedy. Due to the complexity of our actions and careful approach, we are expecting the service to be back to normal by this Friday 12th.<br /><br />As of now, we are running a service with some partial degradation related to:<br />- Invalidation API<br />- Folder sharing (public link) API<br />- New SSL Certificate generation<br />- Public documentation not available.<br /><br />Rest assured we are monitoring the situation 24/7 and will continue to act in order to maintain the availability of your service.<br /><br />Kind regards,<br />Scaleflex IM Support team</p><p><small>Mar <var data-var='date'>11</var>, <var data-var='time'>08:49</var> CET</small><br><strong>Update</strong> - We are continuing to work on a fix for this issue.</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>12:54</var> CET</small><br><strong>Update</strong> - Dear Customers,<br /><br />The incident in Strasbourg datacenters is now under control by local authorities and our provider OVH.<br /><br />Due to the embedded redundancy of our systems and swift actions of our teams, there was close to no impact on the service delivered, despite having servers indeed in the OVH datacenters which are still at this time not reachable.<br />Up to now, only minor degradations have seldomly been perceived, mainly due to the re-caching of images into new and available servers.<br /><br />We continue to closely follow-up on the evolution and expect some missing servers to be back online in the next 2 days, depending on further investigations and actions to reconnect the shut-down datacenters.<br /><br />Meanwhile and as a matter of precaution, we already started to provision new servers in different regions in order to cope with potential traffic spikes and maintain a high level of availability and resiliency.<br /><br />Again, would you witness any issue with our asset's acceleration service or asset management hub, please contact in the usual manner to our support, who is monitoring the services around the clock.<br /><br />Regards,<br />Scaleflex IM Support team</p><p><small>Mar <var data-var='date'>10</var>, <var data-var='time'>07:54</var> CET</small><br><strong>Identified</strong> - Dear Customers,<br /><br />One of our providers is experiencing unavailability of its infrastructure located in Strasbourg/France (OVH datacenter SBG), following a major outage on the site that started around 3am local time. This implies degraded performance issues on this provider's infrastructure overall.<br /><br />Since our delivery is entirely redundant and able to absorb traffic peaks over our multi tenant architecture, this incident should not significantly impact your Cloudimage or Filerobot services.<br /><br />However, some specific administration functions might be degraded or unavailable, such as, but not limited to, upload via api and auto-tagging.<br /><br />Our team Vietnam and Singapore have been handling this incident since the very beginning, taking proactive measures to maintain normal performance levels through activation of a BCP (Business Continuity Plan) and we are already evaluating mitigation action for impacts on the administration if the situation would be lasting.<br /><br />Please let us know if you experience any disruption in the delivery of your images: our support teams are, as always, available.<br />We shall send you an update in 4 hours or as soon as our provider is fully back on.<br /><br />We apologise for any inconvenience.<br /><br />Regards,<br /><br />Scaleflex IM Support team</p>