John Snow Labs De-Identification Solution Enables Data Monetization and Healthcare Innovation
LEWES, Del., Oct. 17, 2022 (GLOBE NEWSWIRE) -- John Snow Labs, the Healthcare AI and NLP company and developer of the Spark NLP library, today announced improvements to its automatic de-identification solution. The company recently established a new state-of-the-art record on the n2b2 standard de-identification benchmark, achieving an F1 score of 96.1%, and decreasing its error rate by 33%. By enabling organizations to automatically de-identify large datasets, John Snow Labs empowers product innovation and cost savings for healthcare organizations worldwide.
Providing custom de-identification required for the monetization of data, John Snow Labs’ automatic de-identification solution is already proving valuable for users. The service is based on the company’s Spark NLP for Healthcare library, built on top of the Spark big data framework, enabling the processing of millions of records on large Spark or Databricks clusters. The de-identification solution can be delivered as an end-to-end system or a software library with optional professional services.
“We are using John Snow Labs to de-identify patient notes on a massive scale and the results from the out-of-the-box de-identification models have been remarkable,” said Nadaa Taiyab, Senior Data Scientist, Providence Health Intelligence. “It has been simple to fine-tune models with our own annotated data and improve pipeline results by adding regular expressions and text matching where needed. Overall, the code is very modular and easy to use, making the challenges and complexities of such a large-scale project much easier to navigate.”
While manual removal of PHI is possible, it’s often rife with human error, and requires multiple reviews. Additionally, the larger the data set, the more labor- and cost-intensive the project. Academic literature shows that for a team with an average cost of $83 per hour total compensation, processing 135 notes per hour of an average length of 130 words, costs $0.61 per note. For large data sets consisting of millions of records, this is simply not feasible.
“Natural language processing has made it possible to automatically de-identify valuable, but otherwise unusable, unstructured patient-level data, like clinical notes, images, and scanned documents,” said David Talby, CTO, John Snow Labs. “Once de-identified, the datasets can be shared more safely and easily with researchers and builders, ushering in a new generation of accurate an innovative healthcare solutions. Without large-scale automatic data de-identification, this would not be possible at scale.”
About John Snow Labs
Getting to the Autonomous Network
Brave New World - When Virtual Becomes Reality
Introduction to 5G and Open-RAN