yep, I think that 9 are very lucky that this facility was developed in 2012 and so is based on more conventional broadcast technology.
… and I think you’re spot on with that … hopefully it will be a wake-up call for 7 as they start thinking about their move of news from Martin Place to Eveleigh …
This bit here is a very important point - after something happens, it’s often the IT Team blamed and “why did you allow this to happen?” etc… ALL users now need to be vigilant and aware of risks, one slip, one failure and it can all come undone. The number of times users hate having to have complex passwords or multi-factor authentication etc… but these events are the reason why and will only become more and more common. We’ve heard about Nine being hit this week, there were probably hundreds of other Australian companies hit this week also to varying degrees.
Editing suites may need access in order to get footage ie off YouTube. Unless they place the files downloaded to a central file server. Even doing that exposes the network to the internet and is a risk. But they should have endpoint randomware protection company wide, but it is possible it may have failed. But to still be affected after virtually a week is a bit of a worry and suggests some major issues with Nine’s network. As with appropriate backups one would think they would be back online mostly unaffected as far as a viewer would see by now.
This shows a (typical) misunderstanding of how slow it is to restore from backups.
Typical restorations are just small amounts of data, maybe one or two virtual machines.
Plus that restoration can’t really start until you’re sure you’ve cleaned off all malware from the network, otherwise if one infected computer (client or server) is powered on it re-infects the network & all the restoration work to-date is undone.
This is full-scale, and will take weeks, at least. Their focus will be on base infrastructure & critical business systems, with others behind delayed until later.
Trying to do too much at the same time will cause contention on the network, storage and servers, slowing down the entire process, plus there’s also a limit to how much can be done without making systems that have been restored unusably slow.
Sure but it should not be affecting day to day operations to the point where it is noticeable to the viewer for example 5pm weekend news not going ahead. That is why it sounds as if the malware got into backup infrastructure and may have been there for quite some time as if you had daily backups and you found when the system was compromised you can restore from the last full backup and then daily backups after that. For example if you do a monthly full backups, daily incremental backups and weekly differential backups you just need to determine the time it happened just say 28th March so you restore the monthly backup from March each differential backup up to the 28th and daily backups since then. It’s highly unlikely the previous backups ie March would be compromised as it would be on its own media or virtual media that would have been rotated. There is no need to access this backup on a daily basis.
But if it can’t be determined when the last backup was compromised then it’s much harder to be sure you aren’t restoring compromised files. It sounds to me their backups were compromised or some didn’t have adequate backups to begin with to have this affecting the viewer experience after a week. For a broadcast automation server wipe it ie reformat, restore from last monthly backup which should not be compromised and you should be back running again. But if you don’t have proper backups or the infection has been there a while then it’s harder to be sure. Also the server might be managed directly by the vender which means Nine would have little control over the backups.
This is the direct quote from the Wide World OF Sports Facebook page:
“Due to ongoing technical difficulties, Sports Sunday is unable to air at this time.”