We are now in the process of failing the servers over to the good working database so service can be resumed quicker.
This will allow us to work on the other databases without continued loss of service.
Posted Nov 05, 2021 - 17:02 GMT
Update
Soft-Switch Cluster 1 - stores its data on 3 seperate databases. Last night we tried to introduce a 4th Database which failed to sync.
A fix was applied however casued the other databases replication to crash. The resync is taking its time due to the size of the databases.
We continue to work on this until the service is restored. Updates to follow.
Posted Nov 05, 2021 - 16:59 GMT
Update
We are continuing to monitor for any further issues.
Posted Nov 05, 2021 - 16:41 GMT
Monitoring
We found a fault with the Databases which stores all the data for Soft-Swtich Cluster 01 and waiting for the databass to catch up with each other before the service will return back to normal.