Most of us are looking forward to the end of 2020, it has been a challenging year for IT departments. Many projects went on hold as teams grappled with the immediate need to enable remote working throughout their organization, spending was cut, or worse still frozen and sometimes painful staffing decisions made.
Critical projects that were able to move ahead were complicated by lockdown, supply challenges and quite rightly the importance of observing social distancing rules when working in the data center. We found ourselves adapting many of our working practices, in order to ensure key projects in hospitals and telecoms operators still took place and were delivered safely on time. There is nothing quite like trying to figure out how to replace large mechanical and electrical plant equipment whilst maintaining social distancing and good hygiene!
At this time of year attention usually turns to planning for 2021, those maintenance tasks that should have been happening but there never seems to be time for, and finally how data center monitoring and callouts will work throughout the Christmas and New Year holidays.
Lockdown of course, forced us into a remote monitoring and call-out regime much earlier in the year. For many, this came as a shock, but preparations for Christmas 2020 meant they were able to adapt quickly. Others had to muddle through, in some cases activating features in their data center infrastructure that they simply had not used before, enabling better alerts, greater flexibility and control over the tasks they could undertake remotely.
Whilst we are in some respects better prepared than ever before, it doesn’t mean that we shouldn’t get on top of checking the basics are all in order for the end of this year. Especially as we cannot be sure how requirements will change in the coming weeks and months around the country as we get used to the new national tiering system. The news of progress with a vaccine is great, but it will be well into 2021 before the vast majority of the population is vaccinated, and therefore means restrictions of some kind are going to be with us for the foreseeable future.
Data centers will not be exempt from the protocols designed to reduce the spread of Covid-19, so what are the checks we should be making now, which will take longer than usual, to ensure a smooth holiday season?
Maintenance checks should of course be running throughout the year on the data center, and the frequency will depend on the type of equipment and manufacturers recommendations as well as your own policies. If in place, this regime will of course reduce the chance of unexpected problems, but it is wise to run a set of pre-holiday maintenance tasks. Some companies simply stagger standard checks in such a way that a set happens in late November, for example. But if on 15th December it will be three months since any maintenance tasks have taken place – then you are leaving yourself open to risk.
Tasks should include everything from checking the installed mechanical & electric Infrastructure, UPS batteries, coolant and filters, to searching for signs of rodents, who may decide to have your cables for Christmas lunch. There will no doubt be other tasks depending on the nature of your infrastructure – the important thing is to assess what they should be and get them done in good time. As a starter, definitely look at the following:
- Uninterruptable Power Supplies (UPS) - For such a critical piece of data center equipment the UPS can often only receive the most rudimentary of maintenance attention. Batteries need to be checked, periodic impedance tests as well as checking for software updates. It is also important to check that configuration of the servers connected to the UPS has not changed.
- Air Conditioning Units (ACU) – Inspections and refreshing of consumables should take place regularly, along with reviews of the condenser locations and critical condenser cleaning. All associated internal and external pipework should be integrity checked.
- Generators – These should be routinely run. This is not simply a case of saying ‘it works’, fuel analysis and load testing should also be reviewed at regular points.
- Fire suppression systems – These should be routinely inspected and tested. Both in terms of the detection systems themselves, outlets, fire suppression cylinders that may need to be removed for pressure vessel stretch testing. Room integrity should also be checked to ensure no service entry ducts have been left open, as well as the automatic alert systems designed to switch to fail over, alert teams, and shut down servers.
- Cabling – Checks should be conducted to ensure cabling is not under undue tension or showing excessive signs of wear on terminators that are regularly manipulated. Also, whilst rare it is possible for rodents and other unwanted guests to find their way into underfloor cavities, so checks for signs of dining and nesting should be looked for.
Get your monitoring right
Now is a good time to check what your monitoring systems are reporting on, as well as the conditions and parameters that will trigger automatic actions or alerts to staff. Are they tight enough, or in place at all? Use this time to fully assess them against your processes and IT ‘red list’ of problems. Check alerts are going to the right people – it is more common than most would like to admit that someone that left the company two years ago, is still in the monitoring software.
Remember to get your monitoring software to bring you good news too. Better to get a daily report and know all is well, rather than be left wondering because your system is only configured to send alerts with bad news! Silence breads fear, and you’ll just worry about whether the DC has checked out for New Year!
Setting the stage for 2021
2020 has forced every IT department to focus on immediate challenges, but in doing so many have streamlined and realized there is technology in the data center they just don’t need. Whilst contingency planning will be a factor in data center plans for 2021 like never before, many are looking forward and thinking about getting back on track the projects that are going to make a difference to the data center and business for years to come. There is no doubt that continuing to improve energy efficiency and reducing costs in the data center are going to remain high priorities in the year ahead, but for many it will also be time to get back to what they do best – keeping their businesses striding forward.
Stay safe and well everyone!
Chris Wellfair, Projects Director, Secure I.T. Environments Ltd