It’s Time to Learn to Fish: How a Flexible Data Lake Will Speed Up Adoption of Healthcare Analytics and AI

Imagine that you are planning to get married. Not only will you gain love and emotional support; together, you and your new spouse will be able to do things neither of you could do on your own. You know that the marriage will lead to a better life for both of you.

Now imagine that before you can marry, you must agree on every detail of your daily schedules for the next five years. Not just the wedding details, but the details of every major holiday or celebration for the next five years. And you must decide the details of everyday life - what time to wake, what to eat, what route to take to work, who will do the dishes and who will mow the lawn. Should the toilet seat be left up or closed? What color should the living room be painted? Some of these endless details will be easy to agree on, while others will require negotiation and may illicit bitter disagreement. But if you want to get married you must reach consensus.

In the face of all that decision-making, you would likely forgo the marriage and just date each other. And even if you plowed through all of it, you would likely find that, a year later, you’d be renegotiating all those decisions as your circumstances changed. Which means that all your hard work would have been wasted.

That’s what it’s like creating a healthcare data warehouse using a relational database. And that’s why a lot of healthcare organizations don’t have a data warehouse, despite the obvious need. It’s just too hard, expensive and time-consuming, and healthcare is changing so rapidly that data decisions made today will likely be obsolete within a short time.

That’s a shame, because as predictive and prescriptive analytics become critical capabilities for healthcare organizations, and as artificial intelligence becomes a baseline capability in patient care, those organizations will need a data warehouse if they are to provide the best care for their patients and maintain a healthy financial situation.

This is not to say that relational data bases aren’t useful. They are very useful, if you have a data set that is easily defined and which doesn’t change rapidly. But in healthcare, we are finding that there is a lot of useful data that is difficult to structure in a relational database, and we can’t predict accurately what data we will need next month, much less next year.

Learning to fish, not just pull boxes off a shelf

What we need is a solution that is more flexible, quicker to implement and capable of storing a wide variety of data in disparate formats.

Instead of a data warehouse, which requires us to define every data point upfront and assign it a storage spot before we can bring it in the door, we need more of a data lake that can be stocked like a fish pond. We dump the fish (our data) in the lake immediately whenever we acquire the fish (perhaps setting up sub-ponds for certain types of fish), and when we need to cook a dinner, we go fishing. But we go fishing with high tech tools that help us quickly find and capture the exact fish we want.

That’s what a document data base with advanced search capabilities is like. It’s much faster to set up, because you don’t need to define the data before you store it. It’s very flexible, allowing you to gather completely disparate sets of data in one big pool.

This approach would help healthcare organizations sidestep the painful part of creating a data storage system and move forward quickly to the reason for gathering the data in the first place: better patient care, better patient experience, lower costs.

Not all lakes are created equal

A critical issue in using a document data base is the ability to retrieve data points as needed. That requires that you integrate advanced search capabilities into your data lake. Without that ability, it would be like using a rod and reel to catch a specific kind of trout in a pond stocked with thousands of species of fish. If you are experienced in trout fishing you might get lucky, but most likely you’d have to do a lot of fishing to get the fish you want. Advanced search tools will quickly see the trout you want, scoop them up in a net and deliver them to your doorstep.

While there are several good document databases available, there are just a couple of choices with the right search capabilities built in. While it is theoretically possible to buy a good standalone data base and integrate a separate search tool, that is probably not a good choice for healthcare organizations, which seldom have the resources needed to accomplish the task.

Here at NTT DATA’s Healthcare Analytics Practice, we have been partnering with MarkLogic, which has a document data base with integrated search capabilities, for just that reason. Our clients need a solution that is relatively fast and simple to implement, so they can quickly begin using analytics that will help them provide better outcomes for their patients. The less time and resources spent on data acquisition, storage and integration, the more they will have available for the stuff that matters to them.

You can learn more about document databases for healthcare at HIMSS

If you are going to the annual HIMSS conference in March, you should schedule some time to learn more about document databases. The folks from MarkLogic will be joining us in the NTT DATA booth, and you will have the chance to talk to them and the NTT DATA experts about the latest healthcare data integration, analytics and AI approaches. Visit us at Booth #2116.