By Stephen Matheson, Vice President, Product Management and North American Sales, BridgeHead Software
Preparing for a disaster is a wasted exercise if you are already in the middle of it. The time to act is now, before it happens. Does this sound like common sense? Yes, maybe. But, I’ve seen disasters occur time after time across the 30+ years I’ve worked in technology – and it’s worse in healthcare because there’s more at stake. From the costs involved to the loss of reputation and, in some cases, the negative impact on patient care, a disaster can significantly impact a hospital; in the most severe cases causing damage that they are unable to recover from.
No question, technology has transformed the delivery of patient care for the better. But, due to their criticality, some systems are particularly vulnerable in disaster recovery (DR) scenarios. For example, in the case where electronic patient records may not be accessible, hard copies have to be printed and physically transferred with the patients. And, don’t assume ‘the cloud’ is the answer to all ills. It isn’t. If your hospital can’t access the cloud as your connection has been severed – what then?
To help, we’ve put together a nine step guide to enable you to lay the groundwork for a robust disaster recovery plan to you give yourself the best possible outcome should you be subject to data loss, corruption, system outage, or worse.
1. Communication and collaboration
Staff need to have a framework for communication during a disaster scenario. Nurses, clinicians, operations, leaders, as well as IT – everyone needs to understand how to interact should a disaster strike, including who takes the lead and how to establish the whereabouts and availability of key staff. The first step in any disaster is to determine who’s available to help (though they may not always be willing).
2. Prioritize the systems for recovery
Ensure your business and clinical leadership team agree priorities based on the disaster type. A system outage, fire or flood at your hospital will impact operations differently than a problem that affects your geographical region. Consider the most likely disasters you can expect, as well as the potential consequences. Which systems are critical and, thereby, the most important to restore? Where should your team be focusing their efforts? What can wait?
Also, remember, the ability to recover systems may depend on the available personnel. Also, consider how staffing will affect your priorities, e.g. you may target different systems for recovery if you had 75 percent of your IT staff available, compared to 50 percent or just 25 percent.
3. Which systems are required to restore specific functions?
Hospitals have hundreds of IT solutions in their production environment, many with dependencies on other systems. Recovering one system may not be sufficient to restore a mission-critical application for a department. Your IT staff have to understand the relationship between solutions for a particular department or function in order to understand priorities for recovery.
4. Use your vendor community
In some cases, your internal staff may not deal with certain mission-critical software, so vendor involvement in developing your DR strategy is vital. Which disaster plans do your suppliers have in place to support your hospital’s operations during a crisis? Your IT staff must understand which systems third parties are responsible for in a disaster scenario, and which they are able to manage internally.
5. Decide on a location for each disaster scenario
A localized disaster might not impact your datacenter, but a regional event, such as flooding, could. Evacuation of the hospital may require multiple recovery sites because of cost and space considerations.
6. Which records need to go with the patient?
Your electronic medical record system (EPR, EMR) is likely to be in the top recovery tier. But, what constitutes a patient record, and how many systems contribute to it? A power outage in a server room requires different consideration than a partial evacuation of elective surgery patient that, in turn, requires different consideration to a hospital-wide evacuation, where systems may already be compromised.
7. Don’t put all of your eggs in the ‘cloud’ basket
More hospitals are adopting cloud and software-as-a-service offerings that feature stringent service level agreements (SLAs) for uptime as well as data security, such as the UK Data Protection Act or adherence to the US Health Insurance Portability and Accountability Act (HIPAA). And, although cloud adoption is increasingly recognised as best practice, hospitals must consider how data will be accessed in a disaster situation?
We previously worked with a customer who had three outbound communications lines and three different switches to ensure access. However, a sinkhole in hospital’s grounds swallowed the whole communications infrastructure. Unfortunately in this case, the cloud was well beyond reach when the hospital needed it most.
8. Look at cloud/local hybrid software installations
If you use Microsoft Office 365, you are probably already familiar with the ability to work offline, i.e. working with previously sent and received email despite not being able to send/receive new messages. For data continuity, why not do something similar for cloud-based applications? By backing up a predetermined amount of data locally, the information remains available even if your hospital’s internet connections went down.
A hybrid model such as this could include a local on-premise appliance that would enable the movement of data to the cloud while also facilitating backup to another medium, such as tape. In the case of a disaster, you then have a number of days/weeks’ worth of data available with the possibility of utilizing the backup. Admittedly, the transfer of information to and from the cloud will be slightly hindered by the local backup device. But, that small delay will ensure a local copy of critical information is available should disaster strike.
9. Something is better than nothing
Irrespective of the type of disaster, basic planning remains the same. But, it is important that expectations are set that not even the most comprehensive disaster recovery plan can cover all possible scenarios. While that is a daunting prospect, having a plan that covers most scenarios (and vetted by the appropriate hospital stakeholders) is far better than having no plan at all.