WDTalk

Five nines or 99.999% relates to - the up time availability of data centers. Prospective clients look for uninterruptable uptime as a prerequisite to constitute reliability of a data center, but what is acceptable to one may be inadequate to another. Accordingly, this expectation of uptime relates to a demand for computer reliability. Data center infrastuctures include a variety of critical physical layer environmental sub-systems that must integrate cohesively. Among those are power and cooling. If a system is deemed ‘fault tolerant,” this means it can sustain a worst case, unplanned event and NOT disrupt the end user.

The Four Tiers of Data Centers

Tier 1

It has non-redundant capacity components and single non-redundant path distribution paths serving the site’s computer equipment.

Tier 2

A tier 2 data center has redundant capacity components and single non-redundant distribution paths serving the site’s computer equipment.

Tier 3

A tier 3 data center has redundant capacity components and multiple distribution paths serving the site’s computer equipment.

Tier 4

This is a fault tolerant data center with redundant capacity systems and multiple distribution paths simultaneously serving the site’s computer equipment.

A tier rating for an entire site is limited to the rating of the weakest subsystem that will impact their operations.

I read a thread a while back on a forum from an OP who wondered how much it would cost to build his own data center. A lot of variables come into play determining those costs. Certainly, just gaining approval to build a data center is one of those variables. How much expertise does your staff possess in the area of conceptualizing design and build-out? Is there a hard and fast template for architects? Are the people who are going to manage the new data center the same as those who will be building it? I’d heard it said that building a data center is a once in a lifetime undertaking. I suspect that’s generally true across the board.

How do you know what to ask or who to turn to for guidance?

How far out should you plan? When you figure that most data centers end-of-life cycle is 15-20 years, should you build out to year seven and continue building out as you grow? Or do you build out to year 15 to begin? Unfortunately, that question is normally answered by the amount of funding available.

Tier Level?

Most data centers are generally defined by their Tier Level, so much of the estimation of funds is directly tied to which Tier Level to build out to. Tier 4 data centers will cost significantly more than Tier 1 data centers. Tier 1 is the lowest rated Tier and is prone to downtime – and Tier 4 is the most robust, redundant, and functional.

Tier I: A single path for power and cooling distribution, without redundant components, providing 99.671% availability

Tier II: A single path for power and cooling distribution, with redundant components, providing 99.741% availability

Tier III: Multiple active power and cooling distribution paths but only one path active, redundant components, concurrently maintainable, providing 99.982% availability

Tier IV: Multiple active power and cooling distribution paths, redundant components, fault-tolerant, providing 99.995% availability

Energy Costs?

Consider how far we’ve come in the last 5 years with regard to energy footprints. How can anyone possibly predict the state of this industry in another 15 years? Thus, energy needs to be scalable. Will our racks become more dense? Will new processors and RAM modules consume massive energy, or just the opposite? What is today’s trend, and what looms on the horizon? Part of your business plan must be trend analysis.

Green Technologies?

What is Green? Does Green relate to carbon footprints? Would it pay to design Green, both in energy savings and marketability? A recent Google search for Green Data Centers turned up 288,000,000 results. Some green data centers are powered entirely through renewable wind energy – and designed to operate at approximately 60% lower energy utilization per square foot than comparable data centers of the same size. This translates directly into more computing power using less energy in a given building.

Who will actually build the data center?

Determining the engineering firm, construction company, their sub-contractors, etc. and working through the build out is a monumental undertaking. Murphy’s Law still prevails. What can go wrong, will go wrong.

Millions of dollars are at risk

A couple of years ago, a study determined the cost to build a 150,000 square foot data center with 150 employees averaged from $16-$17 million on the low end to $19-$22 million on the high end. The most expensive were obviously on the East and West coasts. The Midwest and Southeast came in on the low end.

Data centers are – incredibly complex, with interactions between servers, rack and power systems, data storage, meet-me rooms for cross connects, cooling systems, monitors, alarms and alerts, and much more.

As technology advances with – higher performance hard drives, quad core processors and such, power density has become a critical link for maximum performance and cost-effectiveness. For example, dual core processor technology operates in the same power envelope as single core, delivering increased performance per watt.

In order to effectively manage Total Cost of Ownership (TCO) in – managing a data center, look to high performance and low power consumption solutions. Lowering power consumption relates to greater realized density – without being forced to retrofit / upgrade existing cooling systems. As most data centers are designed with a 20 year life span, and given the expense to build out new centers, heat loads of 40 to 70 watts per square foot of yesteryear do NOT begin to address today’s power consumption of 200 watts per square foot.

What does all of this mean – to the data center client? As competition in the industry “heats up,” prices are driven down, lowering net profits. For your business to successfully colocate servers in a data center, that data center has to turn a net profit. It’s a partnership shared to leverage investments on both sides of the rack. I read hundreds of threads started by Ops in forums with questions similar to, “What’s the cheapest offer in data center colocation?” Their question should be, “What is the most secure and cost-effective data center colocation solution offered.” You need to balance the value of your data to the services of your colocation provider – their infrastructure, forethought, perception of value, reliability, flexibility and professionalism. Do they run a tight ship? Ask to take a tour. Ask questions related to your specific requirements. Do your homework. It’s a very competitive market, and as we’ve seen in years past, not everyone survives the test of time. Is your data center growing?

Data centers are – not all created equally. Their variances range in a wide array of dimensions, e.g. security policies (SAS70), power availability, cooling scheme, monitoring (securtiy and network health), alerts (fire, smoke, moisture), phyical security, accessibility and ……..

When querying prospective – data centers, what defining attribute leads you to one versus the other? Is it primarily price? Is it because they seem to be very hands-on (and you find that comforting)? Not all data centers have security guards controlling access behind bullet proof glass enclosures, nor do they need to be, but they all possess a common thread of price, oversight, infrastructure and connectivity – to varying degrees.

Not every feature is – as important to you as it may be to your competition, or the business down the street. Compare this to a copying machine where no one ever uses all of its features, yet some features must be there – just in case. Making double sided 11×17 prints may not be important to you, but your marketing department considers this feature to be a ‘must have.”

In sales, every new sales rep is – taught the Benjamin Franklin close. It goes something like this. List all the good reasons to buy on the left side and all the reasons not to buy on the right side. Add up the two sides. The side with the most points wins. Take this approach one step further. Assign weight values to the features that are most important to you, on a scale from one to ten. For features you absolutely must have, disqualify the prospect if they don’t offer it.  For features you’ve really like to have but don’t absolutely need, assign a negative value if the prospect doesn’t offer it.

List everything from – price for IPs, bandwidth, power, rack and cabinet space to hours of customer support, availability of livechat or phone support, billing flexibility, their Terms of Service provisions, accessibility, level of physical security and network monitoring, comfort level talking to their reps, which leads with solutions rather than price, and whatever else is important to your business.

Do this for – each prospective data center. Compare the totals. One should jump out and hit you square in the forehead.

Power requirements

Ok, we all know that power is a huge consideration in building out or maintaining a data center. A 1MW data center consumes 177 million kWH, worth approximately $17 million dollars over ten years, at $0.10 per kWH. So it should be no surprise that energy costs have become the second highest expense for data centers.

We’re already seeing more and more relocations and outsourced applications, primarily to reduce power expenses. During the next five years, it’s projected that one in four businesses will experience a significant business interuption. Couple that with the explosive growth of data. 161 Exabytes of data were created in 2006, which is approximately three million times the information of all the books ever written.  

Data center space

Going back a few years to the dot com boom and subsequent collapse, a lot of space suddenly became empty and available for pennies on the dollar. Millions of square feet of space that was built out to a specification of 100 watts per sqare foot was sold at bargain basement prices to enterprises seeking regulatory compliance. That space is long gone.

The subsequent advent of metered power started squeezing power margins, making reasonable return on investments even harder.

Thinking of relocating?

New data centers need 200 to 400 watts per sqare foot to be competitive. Unfortunately, local utilities are unable to deliver power in some markets, as the costs of producing that energy has quadrupled in the last few years. How do you combat unmanageable power expenses? Many companies are colocating to data centers in less expensive power areas, for example from the West or East coast to the Midwest. Many are seeing drops from $0.07 to $0.02 per kWH.

Other factors

Fluctuations in rack pricing follows suit, from $400 for low KW sites to over $1500 for high KW sites – not including power. Cooling those racks adds another dimension of expense. Data centers are increasingly offering shorter term contracts to leverage clients for pricing increases at renewal.

Consider this.

In the year 2000, there were approximately fifteen million servers. That grew in 2005 to twenty seven million servers installed worldwide, and it took approximately $100 Billion to manage them. It’s projected by the end of 2009, there will be thirty five million servers – that’s an increase of eight million servers in just four years! What will these eight million additional servers consume in power?  A single CPU processor can consume as much as 130 watts (more than most standard light fixtures), and each server itself consumes two to four times more power than five years ago. Older data centers were designed to support four kilowatts per rack, but current requirements can be as high as fifteen to twenty kilowatts per rack. Cooling all these servers adds an additional expense.

Utility load versus critical load

Data centers consume power for IT equipment (critical load) and to operate cooling and lighting (utility load). Most data centers consume equal amounts of power for each. Thus, a small data center with a 4MW feed would consume 2MW of that power before any IT equipment is accounted for. On the utility side, cooling consumes 25%, air movement 12%, electricity (transformer/UPS) 10% and lighting 3%. Larger data centers have the ability to improve this percentage.

What is the cost to build a new data center?

A tier I or tier II data center (20,000 square feet) can cost $15.4 million. A tier III or IV data center can cost $48.4 million.

What about other complications, such as lead times?

A typical lead time to build a top tier 200W per square foot data center is one year (75,000 sq ft and up). Consider lead times for a UPS, generators and PDUs. Add time for land acquisition, design, permits and build-out.

The majority of entry level and mid-range servers are configured with 120v plugs even though by design they will work with 120 or 208v. Larger towers and rack mounted servers tend to be configured with 208v.

Why 208v?

The primary reason is for power supply energy efficiency, which is a measurement of the amount of power lost between the input and output of a server’s power supply. You could expect a two to three percent increase in power supply efficiency by simply increasing the input voltage, resulting in a significant saving in power expense. Some have reported as high as a 30 percent increase in efficiency.

This energy savings is significant to any data center’s bottomline.

Can 120v be retrofitted to 208v? Absolutely. Whereas 120v has one hot, one neutral and one ground, 208v simply has 2 hot lines versus one.  Do your servers run 120v or 208v? Will your prospective data center customize their existing 120v to match your 208v requirements? These are important questions to ask.

Why is 208v more energy efficient?

For a given amount of power, as the voltage increases, less current is required. Conversely, as voltage decreases, more current is necessary to maintain the same amount of power (what we pay for).

I see a ton of posts in threads on forums asking about data center infrastructure. Are cross connects available? What providers are available? What uplink speeds are offered? What kind of redundancy do they offer? What’s the best procedure to control airflow? What kind of alarms are available? How are they monitored? What are recommended monitoring programs? What kind of managed services are available? Do they employ certified on-hand engineers?

Is your data center up-to-date?

Against what benchmarks do you measure one data center versus another? Surely, the cost of power speaks directly to the efficiency of the data center itself. One such calculation multiplies the efficiency of one’s technology by the efficiency of the physical facility. Simply calculate IT efficiency by multiplying a data center’s IT asset utilization rate by the energy efficiency of the servers. This determines the facility’s efficiency by calculating the amount of space used and multiplying that by the energy efficiency of the building. The higher the number, the more efficient your data center.

Does your provider have multiple data centers?

Are they tied together via fiber optics? Do they have failover? What is the history of their uptime? Are they growing?

Data Center Infrastructure – How important is it?

The real question should be, “How important is your business to you?” Can you trust your business to a data center that is not top-of-the-line?