The Challenges Of Retrofitting A Mission Critical Facility

Download PDF

As technology continues to advance, managers are increasingly aware that facilities of the past were not designed to accommodate today’s equipment—equipment that requires more power, takes up less physical space, and emits a tremendous amount of heat. Most facilities were designed for mainframes rather than servers.  And many were designed with the intention of providing cooling to just above the access floor, rather than up to seven feet high.  Despite these design challenges, many facilities still have ample physical room and infrastructure available.  In such situations, a newly- constructed facility is not required.  Rather, a retrofit is in order.

A mission critical facility retrofit is the process of upgrading an existing facility that no longer meets the company’s operational goals. These goals can include capacity, reliability, and efficiency. The objective of a facility retrofit is to effectively maximize current infrastructure while adapting to the latest IT hardware technology.

In most cases, a facility retrofit involves intrusive changes. Server rows may need to be rearranged, cabling may need to be removed or re-run, power distribution and air handling units may need to be added or relocated, and raised access flooring may need to be adjusted. And all of this must be carried out without the slightest interruption to operations.

Think of it like this: A mission critical facility retrofit must be handled with the same precision that you would need to reconstruct an airborne 747.  Fortunately, 747s land for reconstruction.  Mission critical facilities, however, do not.

Before a retrofit can take place, a manager must first understand the outage window. The outage window is calculated by taking the length of time to push the old component out and pull the new component in and adding the time to shutdown and startup the IT equipment that this component is supporting. This outage window is the total time it will take to complete the retrofit of that particular component.

IT equipment shutdown time

+   Component replacement time

+   Component testing time

+   IT equipment power up time

=  Total Outage Window

The outage window should be acceptable to all stake – holders, IT personnel, customers, business units, etc.

Many mission critical facilities have planned outage window dates. The duration of an outage window may be different depending on the date.  For example, for some companies, major holidays allow for a longer duration than a weekend day.

Once the outage window has been determined, a manager must then develop an implementation plan. Some companies put together a formal document, called a Method of Procedure (MOP). This document is largely driven by the outage window that was previously deter- mined.  For example, if the outage window is eight hours, a team of two may make the retrofit.  If, on the other hand, the window is two hours, then ten may be required (assuming, of course, that physical space is available).

Once the outage window has been determined and the implementation plan is in place, the retrofit may begin. There are two approaches to retrofitting live mission critical facility components. The first involves a strategy where the replacement component is brought in to the facility and installed while the original component continues to support the facility.  Once the replacement is ready, the cutover is made. This approach is commonly used in facilities that have adequate physical space and is the faster of the two approaches.

The second is a push-pull strategy in which the component is shut off and pushed out of the facility and its replacement is pulled in to its place. This approach is commonly used in facilities when the replacement component would be best placed in the same location or if there is not adequate space to install it elsewhere in the facility. If space is an issue, we recommend considering consolidating racks, temporarily removing racks, or getting rid of old and outdated equipment before the retrofit implementation. This may provide you with enough space to use the first strategy and therefore experience less downtime.

It is also important to remember that in mission critical facilities, components are interrelated and dependent on one another. Therefore, an action made to one component during a facility retrofit may have an effect on others. Before making an upgrade, have a clear understanding of what the dependencies between systems are and plan accordingly.

For example, electrical capacity may be added to the facility, but not the physical space for additional computer room air conditioning (CRAC) units, which will be required when the power profile increases. If a fire suppression system is upgraded, the HVAC dampers, building fire alarm controls, and facility monitoring system will all need to be adjusted accordingly.  A new Uninterruptible Power System (UPS) will require connection to the existing Emergency Power Off (EPO) circuit.

Technology will always change, causing manufacturers to alter their server and networking device designs.  It is imperative that mission critical facilities be designed and operated with the anticipation of future change- otherwise, in another five years, the facility will be right back where it is today. When implementing a facility retrofit, keep in mind those upgrades that did not make the cut due to budget restrictions.  Plan as if those upgrades may occur next year. This can reduce future retrofit costs and enable more to be accomplished. Remember, the more a manager concentrates on possible future issues, the less likely history will repeat itself.

Once a manager has decided that a facility retrofit is in order, the first step is to consult an expert to help deter- mine the best method of implementation.  Because of the dependencies discussed, the consultant should be experienced in all areas of mission critical facilities, including, but not limited to, architectural, electrical, mechanical, fire protection, building controls, security and monitoring, and maintenance. When done correctly, a facility retrofit will maximize current infrastructure and adapt to current and future technology changes—making the facility more reliable and cost efficient.

Bick Group has subject matter experts in this and many other topics. Talk to a Bick Consulting Services expert by emailing: tdavies@bickgroup.com.