We have talked before ePathology and data growth, but Technology Review recently reported that researchers at Stanford University have used Electronic Medical Records (EMR) from multiple medical institutions to identify a new harmful drug interaction. Apparently, they found that when patients take Paxil (a depressant) and Pravachol (a cholresterol reducer) together, the drugs interact to raise blood sugar similar to what diabetics have.
Data analytics to the rescue
The researchers started out looking for new drug interactions which could result in conditions seen by diabetics. Their initial study showed a strong signal that taking both Paxil and Pravachol could be a problem.
Their study used FDA Adverse Event Reports (AERs) data that hospitals and medical care institutions record. Originally, the researchers at Stanford’s Biomedical Informatics group used AERs available at Stanford University School of Medicine but found that although they had a clear signal that there could be a problem, they didn’t have sufficient data to statistically prove the combined drug interaction.
They then went out to Harvard Medical School and Vanderbilt University and asked that to access their AERs to add to their data. With the combined data, the researchers were now able to clearly see and statistically prove the adverse interactions between the two drugs.
But how did they analyze the data?
I could find no information about what tools the biomedical informatics researchers used to analyze the set of AERs they amassed, but it wouldn’t surprise me to find out that Hadoop played a part in this activity. It would seem to be a natural fit to use Hadoop and MapReduce to aggregate the AERs together into a semi-structured data set and reduce this data set to extract the AERs which matched their interaction profile.
Then again, it’s entirely possible that they used a standard database analytics tool to do the work. After all, we were only talking about a 100 to 200K records or so.
Nonetheless, the Technology Review article stated that some large hospitals and medical institutions using EMR are starting to have database analysts (maybe data scientists) on staff to mine their record data and electronic information to help improve healthcare.
Although EMR was originally envisioned as a way to keep better track of individual patients, when a single patient’s data is combined with 1000s more patients one creates something entirely different, something that can be mined to extract information. Such a data repository can be used to ask questions about healthcare inconceivable before.
—-
Digitized medical imagery (X-Rays, MRIs, & CAT scans), E-pathology and now EMR are together giving rise to a new form of electronic medicine or E-Medicine. With everything being digitized, securely accessed and amenable to big data analytics medical care as we know is about to undergo a paradigm shift.
Big data and eMedicine combined together are about to change healthcare for the better.
Before I comment let me say upfront that I work for MarkLogic so my comments here are not to push product. However, I don't think the problem here requires hadoop. The problem is more easily solved with a search engine style index that can do extremely fast co-occurrence lookups combined with other "and/or/near/but not" boolean logic on enriched data to create an initial result set that can then be further analysed with a statistical analysis program like SASS. The advantage of this approach is that you can create dynamic queries on various co-occurring axis without having to process the data time and again through a specific hadoop map reduce. If you did not guess already, this is a key strength of a product like MarkLogic (shameless plug).
Bottom line, hadoop is great if you can't take advantage of indicies for dynamic queries. If you can, your better off transforming the content for ingest into a technology that uses search engine style indices.
Kash,Thanks for your comment. I am not sure I understand all of what you wrote here but the sentiment on Hadoop seems spot on. How the MarkLogic product augments or replaces Hadoop would be something I might be interested in taking offline.Ray