By Brendan O'Leary, Training Development Manager, BridgeHead Software.
Disasters, in healthcare IT terms, can strike unexpectedly in any part of the world at any time. They can appear in the form of natural occurrences such as hurricanes, floods and fires. On other occasions, they may result from the loss of power, system outages or data corruptions. As BridgeHead Software is 100% focused on the healthcare industry, we try to help hospitals protect one of their most valuable assets – their data. But, through our own research, we often find that although hospitals intend to provide adequate protection of their systems and data, other projects often take priority. This leaves the healthcare facility open to considerable risk. Does your hospital have an adequate and tested disaster recovery strategy?
What follows is a true story of how my relatives encountered a natural disaster for which they were inadequately prepared. The scenario is analogous to the risks many hospitals take every day in not having disaster recovery plans in place for their systems and data (or, if they do exist, have never been tested). It highlights that a single point of failure is not necessarily a process, or a critical hardware component, but can simply be access to key personnel.
NO ONE THINKS IT WILL HAPPEN TO THEM
CHRISTMAS DAY 2004: Although my family and I now live in the UK, back in 2004 we were based in my homeland of Australia. Christmas Day 2004 was a particularly momentous occasion as it was the first time in four years that most of my extended family had been together. I had driven down to Melbourne from Sydney, where I lived with my wife, Caroline, and our 18 month old daughter, Elinor (plus Caroline was 3 months pregnant with our second daughter, Hannah). We had a delightful day that was further bolstered after a telephone call with Caroline’s sister, Susan. Susan and her partner were, unfortunately, not able to join us – they were too busy having a lovely time, sipping gin and tonics while overlooking the calm waters of the Indian Ocean from their beachside cabin somewhere in Sri Lanka, where they were on holiday.
BOXING DAY 2004: Devastation. A major earthquake unleashes a tsunami that wipes out seaside communities throughout South East Asia, including Sri Lanka. Waves up to 30 metres (98 ft.) caused the loss of over 230,000 people. Unsurprisingly, we could not get through to Susan via phone or text. We could only assume that Susan and her partner were amongst the casualties.
We had no idea where Susan was located within Sri Lanka. In these days of mobile phones people are not as careful as they used to be about leaving itineraries in the event they may need to be contacted. As a result, we had to try and find this information before we could even go and search for them. This was complicated because Susan and her partner live in Britain and we were based on the other side of the world in Australia. Caroline held distraught calls with her mother and other sister, Lynda, trying to put together a plan that covered all contingencies.
PLANNING FOR A DISASTER – WHAT IF…?
I would like to reflect in this blog on a few simple things that could have mitigated the personal disaster. If you’ve read my recent posts, you’ll know that Caroline and I have just bought a fireproof safe, and have been considering what documents should live in it. We asked ourselves the question, “what would be our worst case scenario?” Our answer is “if both of us were killed, leaving the children orphaned”. I don’t want to seem morbid, but we felt it important to consider all eventualities. We took the first steps years ago when Elinor was born by writing a will and nominating people to look after the children. However, a lot of information that our executors would need is scattered in various locations. Therefore, we decided to plan the recovery from a potential disaster. We listed the information that we thought may be required and kept it in a central location. The list we were considering included:
- Wills, or location of wills
- Insurance policies (many of our policies such as car insurance, house insurance, work insurance and pensions have a death payout. It would be a shame if our children were unable to claim them)
- Bank accounts
- Contact list (who are the key contacts from either side of the family? Who are our dearest friends?)
- Passwords (our computers are the repositories of our digital history. Many photos, movies and voice recordings are stored on our computers. I have backed them up to the cloud, but what good is that if they cannot be accessed by or on behalf of the children?)
- Passports
- Travel itineraries.
Obviously, we needed a special friend or relative to know that this safe exists, and how to access it. But, just take a second to think about your own circumstances… what you might add to this list and why? Our personal circumstances will vary dramatically, I am sure – but, I suspect the essence of what Caroline and I are trying to achieve is not markedly different from other people/families. (Please feel free to share your thoughts with me in the comment section).
Taking time to prepare like this has many side benefits – some are obvious, others are tangential. The obvious benefits are that important information that is required in normal circumstances is conveniently located and accessible. A tangential benefit has been that we have reviewed our situation and taken into account any changes that have occurred since we last asked “What if…?”
DISASTER RECOVERY IN HEALTHCARE IT
This story also has significant relevance within the healthcare IT workplace. Ultimately, we are talking about creating and implementing a Disaster Recovery Plan. In the same way that we have a responsibility to our children and family, we also have a responsibility to our hospital and community. Healthcare IT Disaster Recovery planning is a major topic, and I have no intention of trying to cover it in a few sentences. However, there is one question I would like you to consider, “Does your hospital’s Disaster Recovery Plan depend on the presence, or access, to one or more specific people?” If so, the plan is flawed with a single point of failure, leaving the hospital exposed in the case of a local disaster such as a major flood, or tornado. These people may not be available to assist, e.g. because there is a swollen river between them and the hospital, or the authorities have declared ‘No Go’ areas. This would suggest that the plan is not comprehensive and leaves the hospital open to considerable risk. So ask yourself, are you satisfied that your disaster recovery plan is fully considered and properly documented? Are you satisfied it will work? And, importantly, will it work without you?
The protection of IT systems and data is important for the survival of any organisation. However, in healthcare, it’s not just important – it’s vital. There is a large amount of pressure being exerted by governments, regulatory bodies and other influencing organisations (in the form of enacted laws, codes of practice and guidelines) as to how healthcare data should be protected and secured in the event of system outages, natural disasters, loss and even theft. So, there is an increased burden on healthcare IT professionals when attempting to create and manage a robust, ‘working’ disaster recovery strategy for the protection of these IT systems and the large volumes of data they have to manage. While this presents a very real challenge for healthcare IT teams, it is not insurmountable. Don’t put your hospital at risk with a flawed (or non-existent) disaster recovery plan, especially one with an obvious single point of failure!
EPILOGUE
DECEMBER 28, 2004: TEXT MESSAGE FROM UNKNOWN NUMBER: Safe and OK S&L.
JANUARY, 2005: Susan and her partner returned safely to their home in the UK after being rescued and sheltered by the local fishing community.
