Power failure on October 29 closed

Large parts of Uppsala saw a power loss at 14.45. Our battery backups for critical infrastructure did not recover as expected. We will first bring this infrastructure back and then focus on restoring all user-facing systems.

We apologize for the inconvenience of another serious outage due to external circumstances after the cooling error earlier this week. This time, it’s clear that some of our equipment and processes didn’t function as intended and we’ll try to address that after the immediate work to restore operations.

Almost all systems are down, but actual equipment damage, if present, seems to be limited.

Update 2023-11-01 09:00

UPPMAX Cloud is back in production.

Update 2023-10-31 14:00

We have opened up UPPMAX cloud, however, there remains an issue with volumes when creating new instances. We are investigating and hoping to have this resolved soon.

Update 2023-10-31 09:43

Bianca is now back in production. Final tests for UPPMAX Cloud in progress.

Update 2023-10-30 16:30

Bianca and the UPPMAX Cloud is expected to need more time. We aim to have both systems back in production by Tuesday morning.

Update 2023-10-30 09:27

Miarka is working again and the login nodes for Rackham were just reopened. Bianca will take longer due to the more complex structure of the secure file system there.

Update 2023-10-29 22:12

We now believe that it’s possible that the power spike in connection with the outage was unusually strong. If that was the case, it would have been hard to fully protect against.