WDTalk

Is your mission critical data backed up and protected?

A quick Google search for remote backup software returned 6,810,000 results. I’d say that’s significant. I think everyone agrees that mission critical data needs to be backed up, but how is debatable. In the hundreds of businesses I’ve serviced over the years, most in-house IT departments used DAT tapes. Very few actually physically removed those tapes from their premises every day. Even fewer remotely backed up their data. So maybe the better question to ask would be, “To what degree is your mission critical data backed up and protected?”

As an ex-RMA Manager (for a local networking firm), I witnessed quite a few defective DAT drives doing hard time on my shelves. I’ve also seen my share of managers scrambling to recover lost data following “unscheduled events” like virus contamination or hacks. Do you think it can’t happen to you? Keeping your fingers crossed isn’t the wisest strategy to ensure your business’s continued success.

Disaster Recovery and Business Continuity Plans are Important
I always recommend incorporating comprehensive disaster recovery and business continuity plans, then periodically reviewing their effectiveness. One part of that plan should be remote offsite backups. Very often, incorporating a remote backup is as easy as downloading a software client onto your network server or personal computer. Many have setup wizards to walk you through the steps of connecting to the backup server, setting up your backup sets, creating a backup schedule and setting a secret encryption key. Typically, backup sets can be configured to run in a variety of ways – backing up data files at the end of the week or your My Documents folder multiple times per day.

Remote backups traveling across the Internet need to be encrypted so that you and only you have the ability to decrypt your data. I recommend programs that use DES, Triple-DES, Blowfish or Twofish algorithms for encryption.

Measuring the success of the data transfer is important. Look for programs with email notification of successful backups or backups with warnings (with log files attached).

Once your data is remotely backed up
Ok, you’ve backed up your data, but now have a need for one file, or an entire volume of data from two months ago. Is this possible? Simply answered – Yes. There are programs that allow instant access to any version of your data files, from the initial backup to the last incremental backup and EVERY version in between.

Locking down clients
Locking down clients simply refers to implementing procedures to protect critical backup sets from being accidentally changed or deleted, while flexible enough for administers to view and change those settings that control the level of usage each client is offered.

When to backup?
Most organizations schedule backups in the evening, during lulls in their business operations. Some programs allow you to run in silent modes (in the background) without displaying any Windows or Task Bar icons – allowing you to run backups throughout the day.

What if my backup gets interrupted?
Let’s say you start a backup and you lose power. Will the remote server retain the ongoing transfer, or bite the bullet? Features like event managers allow you to resume interrupted backups.

Does remote backup software offer file filters?
Most do offer file filters that allow you to include or exclude files from the backup selection, mostly via file extensions.

Just the tip of the iceberg
There are so many things that can and do go wrong in business every day. One thing is for sure – if you have hardware, particularly IT hardware, it will go down sooner or later. Power supplies fail, memory modules flake out, hard drives crash, DAT drives melt down – stuff happens. Some issues can be resolved in minutes or hours, but others may take days or weeks.

Backing up your mission critical data is – an integral ingredient to averting disaster, but just the tip of the iceberg in developing and managing a comprehensive disaster recovery and business continuity plan that will ensure your business’s continued success. Step back and ask yourself, “What if?” What if a disgruntled employee, possibly a sysadmin, corrupted your main servers, then disappeared? What if your building burnt to the ground? What if that DAT drive refuses to release last night’s tape – holding it hostage with a strangle hold on its recording heads? What if?

First, RAID is not a backup solution – always employ disaster recovery options.

I field queries about – RAID 1 (mirror) software versus hardware solutions from time to time. Both have their advantages and disadvantages. Some say adding a hardware RAID controller introduces another point of failure, but the most popular reason by far to use software RAID is cost. If you do go with hardware RAID, make sure that card is compatible with your motherboard.

With BIOS (basic input output system) RAID – you may encounter a few quirks. Essentially, it’s a poor man’s version of software RAID. Motherboards do fail, so if yours gets fried, you’ll probably have to replace it with a similar board. Taking a mirrored drive to another system won’t work if that system can’t track the drive’s volumes.

Do you want – redundancy or performance? Most of us would prefer both. I think software and hardware RAID solutions work fine for redundancy, but hardware RAID holds the advantage in performance, especially if your server is database intensive.

There are other perks to – hardware RAID solutions as well, like notification of specific drive failures. Software RAID solutions typically have problems identifying bad drives, and normally do NOT support hot swaps. The most obvious drawback to software RAID is that it uses cycles from your CPU to manage the array, and while this isn’t quite so bad in RAID 1, it can significantly slow down your server in RAID solutions that involve striping with parity.

For those of you who use multiple operating system environments, hardware RAID is the way to go. If you set up RAID using a specific operating system, only that operating system can (normally) access that array.

Software boot volume limitations – since the operating system has to be running to enable the array, (obviously) the operating system cannot boot from the array.

My recommendation – do what makes sense for your pocketbook and priorities. Software RAID is definitely less expensive, but has performance drawbacks. Hardware RAID costs more, but is feature rich and generally worth the investment.

Is your in-house IT department prepared – for brown outs, black outs, personnel shortages, ISP issues, internal and external sabotage, equipment failures, new regulations (email retention) and a whole host of other issues?

Recently, an Internet forum with – thousands of members was hacked, and in the process the criminals deleted their backups, stole email and credit card information, then disseminated it across the Internet. Could this happen to your in-house network? Hackers are constantly on the prowl looking for openings to exploit internal networks as well. Is yours protected?

How would you as an owner know if your IT department was prepared? Do you have disaster recovery and business continuity plans in place? Are they routinely reviewed and updated? Do you strategize with your IT personnel, or do they basically run the show? If they said you needed to be PCI DSS compliant, would you know what they were talking about?

Managing IT for multiple departments requires some give and take - Is your IT department empowered to make smart decisions? Not all IT geeks are business savvy, so my question to you would be, “Do your departments talk to each other?” HR certainly has unique requirements, as do Sales, Customer Support and Admin. Do they operate in a vacuum or together as a cohesive unit? An appropriate amount of give and take between departments increases awareness of your business’s core focus and mission.

Of course not, but I read threads every day from businesses (on various Internet forums) that have lost their data - because their website violated the Terms of Services (TOS) of their host. Often their sites are taken down without notice. Some scenarios were because the client didn’t keep their security patches up-to-date, then were hacked. Others were because they were using a shared IP and that IP was blacklisted for spam violations – maybe not that specific IP – just in that range.

So are you prepared to lose your data? Seems like a ridiculous question, but many aren’t prepared because they have no plan beyond simply trusting that their web host will provide back ups if necessary. I write about disaster recovery more so than any other topic because of the severity related to losing mission critical data. More often than not, if you lose your data, you lose your business – or it’s severely impacted.

When selecting a web host, read their Terms of Service carefully – they’re there to protect the host and you, spelling out legal expectations. Regardless, use due diligence to formulate a disaster recovery and business continuity plan that includes routinely scheduled remote offsite backup. Prepare for a worst case scenario.

I relate this to car or health insurance. I hate to pay that bill each month, but I know it’s for my own protection. If you’re the owner or president of your company, you owe it to your clients and employees to secure your business. Stuff happens. It can and does happen to businesses just like yours everyday.

• Multiple hard drives in a RAID array fail simultaneously (defective lot). You thought RAID was your backup solution, but turns out – it wasn’t.
• Fire destroys your servers and DAT tape drive. You forgot to take that tape offsite last night.
• Web host locks access to your server because your bookkeeper didn’t pay the bill. I see lots of posts related to this where the recommendation generally is – be nice to the host and maybe they’ll let you have access to your data.

Bottom line - Set aside some time to review and update your disaster recovery and business continuity plan if you have one. If you don’t have one – keep your fingers crossed and hope that Murphys Law passes you by and hits that business down the street first.

Once again, I’ve won the battle – combatting an onslaught of trojan horses, viruses and malware, but this time the fight dragged on for three days. The victim this time was my wife’s desktop, even though we had Malwarebytes, Microsoft Security Essentials and Safe Returner running – with up-to-date definitions.

The fight began when my wife clicked on – a Facebook link, which was then followed by Home Security 2011 security alerts popping up every few seconds that her desktop was compromised. She thought the alert was genuine and clicked on the tab to run a scan -  to remove the dozens of threats it said were infecting her computer (bad move).

So what was the cure? For over two days, I wasn’t sure there was a cure, as everything I tried to do failed. Malwarebytes wouldn’t run, nor would Microsoft Security Essential or Safe Returner. Ending processes didn’t work as they popped right back up as soon as I ended them.

I was unable to run any commands – like msconfig or regedit, or download any anti-malware programs from the Internet (which wasn’t working either). Safe mode didn’t work. Downloading new anti-malware programs to a thumb drive on my desktop, then attempting to install them on hers didn’t work either.

I finally found a tip on a Google search – that said entering a specific key code on manual registration would stop the pop ups. To my surprise, that worked – but the malware remained. After downloading and installing a program that temporarily ended malicious processes, I was able to run Malwarebytes in Safe Mode and remove a portion of the threats. From there, I rebooted and was able to remove more threats, but with each scan, more threats appeared and I was never was able to run Microsoft Security Essentials. I was able to access and search the Internet now though and went back to AVG, which I had used for years earlier.

Even with a new install of AVG 2011 – and successful scans, there still remained two trojan infections it did not remove, even after multiple scans. To my surprise, I left AVG 2011 run a full scan one last time overnight and awoke the next morning with no threats detected. From there, I deleted the existing Malwarebytes and MS Security Essentials programs, downloaded current versions, reinstalled them and ran both without problems.

Lesson learned - you need real time protection, especially if you frequent social networking sites. Keep your definitions current – one slip can cost you hours in recovery.

In one of my earlier articles – “Disaster Recovery Strategies,” I talked about cold, warm and hot sites and the pros and cons of each. While financially justifying a DR investment has always been a major challenge, especially when you’re trying to convince people to spend money on something they hoped they’d never have to use, with cloud-based products, costs drop dramatically.

Planning a disaster recovery and business continuity solution is – so much easier today, from risk analysis to implementation time. You still need assess all the “what ifs.” What if you have a flood (and as I write this, they are predicting high flood levels here again because of all the snow up North this winter), a fire, an earthquake or a severe wind storm? We had a wind storm in St. Louis a few years back that took out thousands of trees and cut power to a large portion of the metropolitan area for a week. What if a disgruntled employee sabotaged your account receivables?

So you’ve exposed risk – what next? You need to have a plan to mitigate those risks. How likely are each and what impact will they have your business? Even in a cloud solution, if your business is critically impacted by being down for even a few minutes, you might still consider ‘hot’ in the cold, warm & hot solution, for that specific aspect of your business.

Dealing with natural disasters is – the sweet spot for cloud solutions. For those risks that aren’t critical in nature, why build out a cold or warm solution, when you can take advantage of others work and infrastructure? Cloud based solutions allow you to redirect that DR investment to what you do best – drive your business.

Once again, let me preface this – with, “RAID arrays are NOT backup solutions.”  Critical data should always be backed up remotely. Ok, do RAID arrays break? Absolutely, YES. 

Hard drives and RAID cards fail – not often, but it does happen. If you’re thinking of (maybe) hanging an external hard drive on your server, so if the server does go down, you could just pop in a new hard drive, reload the OS and backups – think again. Doing an OS reload and restore can take hours, plus you’ve lost all the data since the last backup. Compare that scenario to a RAID array where it takes seconds or minutes to swap a failed drive, and then the server is online and rebuilding the array while serving client data. 

I don’t think it’s a matter of if – a hard drive will fail, rather when. In a RAID-1 array, it is possible for both hard drives to fail at the same time, especially if from the same lot. Hard drives have moving parts, so by their very nature, tend to fail more often than RAID controllers. 

RAID-1 means mirroring – so you can mirror data across more than 2 drives. With RAID-10 (minimum 4 drives), striping won’t help if you have 2 drives simultaneously fail, unless you lose both drives on opposite sides of the stripe. RAID-5 can be accomplished with 3 drives to provide a stripe and a parity drive. In this case, any 2 drives could rebuild the third drive. 

If you have a failure of a single drive – in RAID-1, all you would have to do is put a new drive in and rebuild the array. Of course, it helps tremendously when you have hot swappable drives. 

My recommendation – if you’re going to use RAID arrays, go with RAID levels 1, 5, 6 or 10, and always have a hot spare.