UPPMAX cloud issues with external network (FIP) open

We are working on resolving an issue with external access to VMs. If you start your VM, some users might not be able to connect to it. This is most likely falldown from the power down from Monday, May 29th. It does not affect all users.

Update 2023-06-02 17:00

There are unfortunately still remaining issues preventing the cloud from being fully operational. We will continue to work to have the cloud back in production as soon as possible.

Update 2023-06-02 12:00

The problem is confirmed related to the hard shutdown. We have found corrupted state caused by the abrupt power down. We have managed to clean one part and some of the virtual networks are now working better, however, we still have remaining issues that appears to not heal itself even after cleaning up the old state files. The cloud will remain unavailable as we continue to work diligently to restore the last networks.

Update 2023-06-01 17:00

This problem appears to be related to the hard shutdown, leaving the virtual network configuration in a broken state. At this time, roughly 20 virtual networks are affected. We are making slow progress recovering the networks, and hope to have this issue resolved before the weekend. We apologize for this inconvenience.

Update 2023-05-31 17:00

We have identified this issue as a problem with neutron, however, we have unfortunately not found the root cause yet. The problem however is obvious, as neutron is not creating the necessary virtual infrastructure needed to allow external access to the VMs. We will continue to work to have this solved as soon as possible.

Update 2023-05-31 11:00

We have restarted the network service in OpenStack. All users might temporarily lose connection to their VMs. We apologize for this inconvenience.