Active Incident

Updated a few seconds ago

For detailed Network status please visit: https://www.sunet.se/tickets

Network




Operational

Box




Operational

eduID




Operational

eduroam




Operational

eduSign




Operational

Kaltura




Operational

Mailfilter-ng




Operational

Ping




Operational

SWAMID




Operational

SUNET Web




Operational

Survey




Operational

Zoom




Operational

SUNET Drive




Operational

External Services

Incident Status

Operational

Components

eduSign

Locations

eduSign validator



October 31, 2024 08:46 CET
October 31, 2024 07:46 UTC
[Investigating] We are experiencing issues with eduSign validator and investigating it.

October 31, 2024 09:20 CET
October 31, 2024 08:20 UTC
[Monitoring] Service is restored. Monitoring.
Issue with Kaltura Live streaming Partial Service Disruption

Incident Status

Partial Service Disruption

Components

Kaltura

Locations

Live streaming



October 3, 2024 12:29 CEST
October 3, 2024 10:29 UTC
[Investigating] Kaltura Live streaming is down. We are investigating.

October 3, 2024 14:21 CEST
October 3, 2024 12:21 UTC
[Resolved] Issue has been resolved.

October 4, 2024 08:35 CEST
October 4, 2024 06:35 UTC
[Investigating] Kaltura Live streaming is down again. We are investigating.

October 4, 2024 08:45 CEST
October 4, 2024 06:45 UTC
[Monitoring] Issue resolved with Kaltura Live streaming. Monitoring further.

October 14, 2024 11:18 CEST
October 14, 2024 09:18 UTC
[Investigating] Underlying cause is still not found so streams can still crash. They should now restart the service automatically and quickly, but it'll still interrupt running streams.

Incident Status

Operational

Components

Zoom

Locations

Recording



September 2, 2024 14:22 CEST
September 2, 2024 12:22 UTC
[Investigating] There's an issue with zoom Recording. You may notice delays and the recordings take longer than normal to appear. This has been escalated to Zoom but please send in a ticket to media@sunet.se anyway so we know how many that is affected.

September 16, 2024 15:20 CEST
September 16, 2024 13:20 UTC
[Investigating] When utilisation increased after the summer holidays, we have encountered an intermediate issue with the Zoom on-premise recording service. This has resulted in the delay of the processing and making the recordings available and has unfortunately led to the loss of recordings that cannot be recovered. During the period from August 27 to September 5, the average loss of recordings was around 5%, in total around 250 recordings were lost. The night between September 5 and 6 NORDUnet implemented a workaround changing the disc configuration on the recording server VM’s. After that the number of lost recordings has dropped to less than 0,5% and a total of 5 recordings have been lost since. The root cause for the problem has so far not been identified. There are currently two different lines of investigations. NORDUnet are in constant contact with Zoom and provide them with all requested information. If there are critical recording needs, it is recommended to use local recording until the issue has been fully resolved.

September 27, 2024 08:25 CEST
September 27, 2024 06:25 UTC
[Investigating] The root cause for the problem has still not been identified. Zoom has deployed a new version of the recorder with added debugging, to help with the investigations. The level of lost recordings is still in the 0,5% range. Some of the lost recordings can be partly recovered manually, others are just very late (+48Hours) If there are very critical recording needs, it is recommended to use local recording until the issue has been fully resolved.

October 14, 2024 13:33 CEST
October 14, 2024 11:33 UTC
[Identified] Zoom believe that they have identified the root cause of the problem. It has to do with that the recording connector needs to migrate files during processing. (If the disk is busy at the time the file movement might fail and this is resulting in a “dirty” file, and this also affects subsequent files and is the reasoning for the generic slowness.) Zoom is working on a new release of the recording connector, that will be deployed as soon as available.

October 29, 2024 12:34 CET
October 29, 2024 11:34 UTC
[Monitoring] A new version of the Recording Connector has been deployed. The new version includes a permanent fix for the recording problem. During the next weeks we will monitor closely to verify that no recordings are being lost and that the recording flow is stable.

November 4, 2024 10:58 CET
November 4, 2024 09:58 UTC
[Monitoring] Unfortunately the new version of the Recording Connector did not resolve the problem. Zoom is continuing to work on a solution at their end.

3

Upcoming Maintenances

3

Active Incidents

1

Incidents Last 7 Days

ping.sunet.se

100.00%




ping.sunet.se

100.00%




ping.sunet.se

100.00%




Scheduled Maintenance

EduID Service MaintenancePlanned Maintenance

Schedule

November 14, 2024 09:00 - 17:00 CET
November 14, 2024 08:00 - 16:00 UTC

Components

eduID

Locations

eduID Web, signup, dashboard, login, DNP

Description

We will be performing scheduled maintenance on the EduID system. During this period, we will upgrade operating system and database services. We expect the EduID service should remain operational; however, please be aware of reduced redundancy.

Schedule

November 23, 2024 08:00 - November 24, 2024 21:00 CET
November 23, 2024 07:00 - November 24, 2024 20:00 UTC

Components

Kaltura

Locations

Live streaming, Analytics, Transcoding, Mediaspace, REACH

Description

There will be an infrastructure maintenance in a datacenter in Denmark. The operation requires all nodes of the cluster to be in maintenance mode at the same time, which in practical terms means that the cluster will be "shut down". And in order for this to happen, all VMs in the cluster will have to be powered off for the duration. Downtime is expected during the maintenance window on Saturday. Sunday is reserved.

Schedule

November 30, 2024 08:00 - December 1, 2024 21:00 CET
November 30, 2024 07:00 - December 1, 2024 20:00 UTC

Components

Kaltura

Locations

Live streaming, Analytics, Transcoding, Mediaspace, REACH

Description

There will be an infrastructure maintenance in a datacenter in Denmark (another datacenter than the previous week). The operation requires all nodes of the cluster to be in maintenance mode at the same time, which in practical terms means that the cluster will be "shut down". And in order for this to happen, all VMs in the cluster will have to be powered off for the duration. Downtime is expected during the maintenance window on Saturday. Sunday is reserved.

Locations

History (Last 7 days)

EduID Service MaintenancePlanned Maintenance

Description

We will be performing scheduled maintenance on the EduID system. During this period, we will upgrade operating system and database services. We expect the EduID service should remain operational; however, please be aware of reduced redundancy.


Components

eduID


Locations

eduID Web, signup, dashboard, login, DNP


Schedule

November 12, 2024 09:00 - November 12, 2024 17:00 CET
November 12, 2024 08:00 - November 12, 2024 16:00 UTC



November 12, 2024 09:00 CET
November 12, 2024 08:00 UTC
[Update] Scheduled maintenance is starting.

November 12, 2024 17:00 CET
November 12, 2024 16:00 UTC
[Update] Scheduled maintenance is complete.
Edusign LB MigrationPlanned Maintenance

Description

We will migrate to a new load balancers during the allocated time window.


Components

eduSign


Locations

eduSign web, eduSign signer, eduSign validator


Schedule

November 7, 2024 14:00 - November 7, 2024 16:00 CET
November 7, 2024 13:00 - November 7, 2024 15:00 UTC



November 7, 2024 14:00 CET
November 7, 2024 13:00 UTC
[Update] Scheduled maintenance is starting.

November 7, 2024 16:00 CET
November 7, 2024 15:00 UTC
[Update] Scheduled maintenance is complete.
Zoom disturbanceOperational

Incident Status

Operational


Components

Zoom


Locations

On-Prem




November 6, 2024 14:41 CET
November 6, 2024 13:41 UTC
[Monitoring] Due to a network issue the Zoom Zone on Örestaden, Denmark got disconnected from the world around 13:03 CET. Functionality was restored around 40 minutes later. Meetings might have been disconnected and some might have got unable to join meeting error message. The issue has been resolved.

November 7, 2024 08:53 CET
November 7, 2024 07:53 UTC
[Resolved] Issue resolved.
Sunet Drive - MaintenancePlanned Maintenance

Description

På grund av ett planerat strömarbete som påverkar ett av datacentren som infrastrukturen för Sunet Drive nyttjar kommer delar av infrastrukturen att stängas ner. Detta kommer medföra minskad redundans för fullnodskunder under den aktuella perioden. Due to a planned power maintenance that affects on the the datacenters that is used for Sunet Drive, some parts of the infrastructure will be turned off. This will result in reduced redundancy for full node customers during this timeframe.


Components

SUNET Drive


Locations

GiH, HH, KAU, LNU, LTU, MDU, SciLifeLab, SU, SUNET, VR


Schedule

November 6, 2024 22:00 - November 7, 2024 06:00 CET
November 6, 2024 21:00 - November 7, 2024 05:00 UTC



November 6, 2024 22:00 CET
November 6, 2024 21:00 UTC
[Update] Scheduled maintenance is starting.

November 7, 2024 06:00 CET
November 7, 2024 05:00 UTC
[Update] Scheduled maintenance is complete.