I feel I must start this blog by plugging a book. And no it is not one I have written.
The 2 reasons for plugging it are:
(1) It is actually a very good book
(2) I have ‘lifted’ a few of the healthcare examples for this blog
The book ‘Big Data – A Revolution That Will Transform How We Live, Work & Think’, is a broad discussion on the opportunities and challenges around big data now and into the future.
Anyway read on, unless you want to see a summary video blog of this post, on a very scenic but windy Budapest hill top!
Taking it from the start, what is Big Data again anyway?
Big Data is defined as ‘data sets so large and complex that they can’t be processed by traditional tools’
A fairly vague and subjective definition.
The advent of Big Data as a term has unsurprising coincided with the voluminous explosion of data in our world.
A nice 19th century example illustrates that Big Data is not necessarily a new concept. Commodore Maury used ship logbooks to identify more efficient routes at sea. It cut journey times by a third. This was data that was previously never shared, meaning that sea crossings relied on the experience, instincts and intuition of the captain.
This historical example highlights a very important aspect of big data. Using the original data for a secondary purpose. There were many regulatory and practical reasons for logging a ships position however improving journey times was not one of them.
Today several factors are coalescing to drive Big Data
- Ease of collection: We can attach affordable sensors to practically anything
- ‘The Data Exhaust’: A huge volume of data is generated as a by product of our daily actions, e.g. internet searches, phone calls, credit card transactions, medical monitoring and so on
- Storage: There are now cheaper and more accessible ways of storing large volumes of data
- ‘Processing Power’: Affordable and accessible tools now exist to combine and interrogate mass data sets
- ‘Big Data Mindset’: Leading organizations and individuals are becoming increasing skilled in exploiting opportunities inherent in the data
Big Data is even proving that Einstein got it wrong, with one of his famous quotes. (actually many attribute this quote to Einstein but the evidence is not conclusive )
‘Not everything that can be counted counts, and not everything that counts can be counted’
It is true that not everything that we collect will be of use, but there is no way of predicting this at the outset. So there is a strong argument for ‘counting’ absolutely everything. And with the technology available there is very little we cannot count.
Anyway bring it back to healthcare with the examples:
- Tracking and predicting epidemics: The Google Flu example is not a new one, however the essential insight that a set of, seemingly unrelated, keywords searched for in a particular locality can predict a flu outbreak is interesting. (in addition to flu related search terms) Google might, for example, tell us ‘Increased searches for chocolate bars correlate with a greater incidence of the flu’ in a specific geography, although we may only speculate on why.
- Medication Adherence: All sorts of seemingly irrelevant data points, such as where you live, how long you have lived there, if you own a car etc can be crunched to predict your likelihood of taking your medication as prescribed. It is pure correlation. Buying a car won’t necessarily help you take your medicines at the right time. Again not a new example but an interesting and controversial use of #bigdata
- Human health data: 16 different biological data streams were captured from premature babies, that when combined in the right way, using algorithms, were able to detect an infection 24 hours before it became visible. This means earlier more effective treatments.
In all these examples the data tells us What is happening but not Why . Big Data is there to predict but not to provide answers. If drinking orange juice and aspirin was to lead to remission in certain types of cancer patients, the what (i.e. remission) is more important than the why. Of course the why would be of immense interest in further study.
These examples only mark the surface of what is possible in healthcare. We miss a big opportunity as most of the data captured from patients, i.e. ECG data is just discarded. In most cases the systems are not in place to integrate the various streams of data and deliver meaningful insights.
All sorts of questions remain about who owns the data, and current data protection rules break down in this new world. How can you meaningfully opt into the collection and use of data, for a purpose that has not even been envisaged yet.
Big Data is Big News in many fields but surely, despite my bias, the advances in healthcare will be the most exciting.