USA2 Value network has gone down due to a hard drive failure. The hard drive is going to be replaced and the server reinstalled.

We are hoping to recover the data and get the server back online as soon as possible.

We’re having issues getting the server back online but are working on getting it back as soon as we can.

There are very few customers on this server so the recent problems haven’t affected many.

The hard drive and RAM had been replaced twice and still the crashing continued, so we have set up a brand new server which is up and running now. This should be the last we hear of the problems with usa2 vn.

There is another issue unrelated to the previous one.

This is affecting a small number of shoutcast and autodj streams and looks like it’s due to a flood coming in to the switch that the servers are connected to.

The issue has been resolved. It only affected 2 out of our 15 servers and lasted for about 15-20 minutes.

sorry for any inconvenience caused

There is some packet loss on the UsA Value Network – this is being worked on and should be resolved soon.

This seems to be resolved now, and lasted no more than 5-10 minutes.

Packet loss is back

The datacentre are working on the issue and should have it resolved soon.

It’s been back to normal for at least an hour now, if you are still experiencing problems then please open a support ticket at support@mixstream.net from your registered email address.

Due to some slowness, USA5-VN had to be rebooted a few minutes ago. All streams are now back online. If you are still having issues then please open a support ticket.

Dear Customer,

You may have noticed that the server you are hosted on (USA5-VN) has started to run a little slowly compared to normal. This only seems to be affecting the control panel pages and doesn’t affect any of the streams hosted there.

We are looking in to the problem and will upgrade the server if necessary. No more new customers will be put onto the server from now and we will be setting a new one up by tomorrow for new orders due to very high demand recently.

I apologise if the slowness of the control panel has caused any inconvenience and for the unusually unreliability over the last few days.

The server that will be opened tomorrow will be a UK based server, so if you would prefer to have your stream based in the UK (better for European customers) then please email sales at bellonline.co.uk to request a move – please note that this will mean that you receive new stream details. If you prefer to remain on the current server USA5-VN then the issue will be resolved within the next few days.

The server will be rebooted shortly to see if that helps the situation, that will cause about 2 minutes of downtime.

One more reboot on this and I’ll be taking the accounts off the server today which have been moved.

This should free up resources. I’m also going to check that there are no resellers on there that have set up an unreasonable number of servers which goes against the FUP.

I’ve now moved deleted the unused accounts from the server.

Also, it does look like some resellers have been setting up large numbers of servers which were not in use, so all servers that were not streaming have been shut down (this won’t affeect those that are streaming). If your server has been shut down then you can restart it from the panel – this had to be done because there were so many shoutcast servers running that were never used and just using up resources.

This has made a huge differece to the server speed, 2 minutes after shutting down unused streams, the server load average has reduced by 10 times and the CPU usage from 60% to 2%.

New systems are going to be put into place to stop this from happening in future.

We will be expanding the uplinks between three of our core routers to increase the uplink capacity. We do not expect any service outage during this maintenance, but this involves a risk that is higher than normal operation.

Date: 10 October 2008
Window: 01:00 for 1 Hour
Duration: < 40 minutes

Only customers on UK1-PN and some UK dedi customers may be affected by this.