ThoughtWorks
  • Kontakt
  • Español
  • Português
  • English
  • 中文
Übersicht
  • Delivery Mindset trifft Software-Exzellenz

    Verfolgen Sie einen innovativen Ansatz in der Softwareentwicklung, um noch schneller erfolgreich zu sein.

    Erkenntnisgestützte Entscheidungsfindung

    Nutzen Sie Ihre Datenbestände, um neue Geschäftsmöglichkeiten zu erschließen.

  • Betriebsmodelle ohne Reibungsverluste

    Verbessern Sie die Fähigkeit Ihres Unternehmens, auf Veränderungen zu reagieren.

    Plattform Strategie

    Entwicklung dynamischer Technologieplattformen, die sich an Ihre Geschäftsstrategie anpassen.

  • Experience Design und innovative Produkte

    Liefern Sie schnell außergewöhnliche Produkte und Kundenerlebnisse. Entwickeln Sie Design und Funktion kontinuierlich weiter.

    Partnerschaften

    Nutzung unseres Netzwerks aus vertrauenswürdigen Partnern, um noch bessere Ergebnisse für unsere Kunden zu erzielen.

Übersicht
  • Automobil
  • Clientech, Energie und Versorgung
  • Banken und Versicherungen
  • Gesundheit
  • Medien
  • Non-Profit
  • Öffentlicher Sektor
  • Handel und E-Commerce
  • Reise und Transport
Übersicht

Unsere Empfehlungen

  • Technologie

    Ausführliche Betrachtungen neuer Technologien.

  • Business

    Aktuelle Business-Insights, Strategien und Impulse für digitale Querdenker.

  • Kultur

    Insights zu Karrieremöglichkeiten und unsere Sicht auf soziale Gerechtigkeit und Inklusivität.

Digitale Veröffentlichungen und Tools

  • Technology Radar

    Unser Leitfaden für aktuelle Technologietrends.

  • Perspectives

    Unsere Publikation für digitale Vordenker*innen

  • Digital Fluency Model

    Ein Modell zur Priorisierung digitaler Fähigkeiten, um für das Unvorhersehbare bereit zu sein.

  • Decoder

    Der Technology-Guide für Business Entscheider

Alle Insights

  • Artikel

    Expertenwissen für Ihr Unternehmen.

  • Blogs

    Persönliche Perspektiven von ThoughtWorkern aus aller Welt.

  • Bücher

    Stöbern Sie durch unsere umfangreiche Bibliothek.

  • Podcasts

    Spannende Gespräche über das Neueste aus Business und Technologie.

Übersicht
  • Bewerbungsprozess

    Finde heraus, was dich in unserem Bewerbungsprozess erwartet.

  • Hochschulabsovent*innen und Quereinsteiger*innen

    Dein Einstieg in die IT-Welt.

  • Stellenangebote

    Finde offene Stellen in deiner Region.

  • In Kontakt bleiben

    Abonniere unsere monatlichen Updates.

Übersicht
  • Konferenzen und Events
  • Diversity und Inclusion
  • Neuigkeiten
  • Open Source
  • Management
  • Social Change
  • Español
  • Português
  • English
  • 中文
ThoughtWorksMenü
  • schließen   ✕
  • Unsere Services
  • Unsere Kunden
  • Insights
  • Karriere
  • Über uns
  • Kontakt
  • Zurück
  • schließen   ✕
  • Übersicht
  • Delivery Mindset trifft Software-Exzellenz

    Verfolgen Sie einen innovativen Ansatz in der Softwareentwicklung, um noch schneller erfolgreich zu sein.

  • Experience Design und innovative Produkte

    Liefern Sie schnell außergewöhnliche Produkte und Kundenerlebnisse. Entwickeln Sie Design und Funktion kontinuierlich weiter.

  • Betriebsmodelle ohne Reibungsverluste

    Verbessern Sie die Fähigkeit Ihres Unternehmens, auf Veränderungen zu reagieren.

  • Erkenntnisgestützte Entscheidungsfindung

    Nutzen Sie Ihre Datenbestände, um neue Geschäftsmöglichkeiten zu erschließen.

  • Partnerschaften

    Nutzung unseres Netzwerks aus vertrauenswürdigen Partnern, um noch bessere Ergebnisse für unsere Kunden zu erzielen.

  • Plattform Strategie

    Entwicklung dynamischer Technologieplattformen, die sich an Ihre Geschäftsstrategie anpassen.

  • Zurück
  • schließen   ✕
  • Übersicht
  • Automobil
  • Clientech, Energie und Versorgung
  • Banken und Versicherungen
  • Gesundheit
  • Medien
  • Non-Profit
  • Öffentlicher Sektor
  • Handel und E-Commerce
  • Reise und Transport
  • Zurück
  • schließen   ✕
  • Übersicht
  • Unsere Empfehlungen

  • Technologie

    Ausführliche Betrachtungen neuer Technologien.

  • Business

    Aktuelle Business-Insights, Strategien und Impulse für digitale Querdenker.

  • Kultur

    Insights zu Karrieremöglichkeiten und unsere Sicht auf soziale Gerechtigkeit und Inklusivität.

  • Digitale Veröffentlichungen und Tools

  • Technology Radar

    Unser Leitfaden für aktuelle Technologietrends.

  • Perspectives

    Unsere Publikation für digitale Vordenker*innen

  • Digital Fluency Model

    Ein Modell zur Priorisierung digitaler Fähigkeiten, um für das Unvorhersehbare bereit zu sein.

  • Decoder

    Der Technology-Guide für Business Entscheider

  • Alle Insights

  • Artikel

    Expertenwissen für Ihr Unternehmen.

  • Blogs

    Persönliche Perspektiven von ThoughtWorkern aus aller Welt.

  • Bücher

    Stöbern Sie durch unsere umfangreiche Bibliothek.

  • Podcasts

    Spannende Gespräche über das Neueste aus Business und Technologie.

  • Zurück
  • schließen   ✕
  • Übersicht
  • Bewerbungsprozess

    Finde heraus, was dich in unserem Bewerbungsprozess erwartet.

  • Hochschulabsovent*innen und Quereinsteiger*innen

    Dein Einstieg in die IT-Welt.

  • Stellenangebote

    Finde offene Stellen in deiner Region.

  • In Kontakt bleiben

    Abonniere unsere monatlichen Updates.

  • Zurück
  • schließen   ✕
  • Übersicht
  • Konferenzen und Events
  • Diversity und Inclusion
  • Neuigkeiten
  • Open Source
  • Management
  • Social Change
Blogs
Wählen Sie ein Thema
Alle Themen ansehenschließen
Technologie 
Agiles Projektmanagement Cloud Continuous Delivery  Data Science & Engineering Defending the Free Internet Evolutionäre Architekturen Experience Design IoT Sprachen, Tools & Frameworks Modernisierung bestehender Alt-Systeme Machine Learning & Artificial Intelligence Microservices Plattformen Sicherheit Software Testing Technologiestrategie 
Geschäft 
Financial Services Global Health Innovation Retail  Transformation 
Karriere 
Karriere Hacks Diversity und Inclusion Social Change 
Blogs

Themen

Thema auswählen
  • Technologie
    Technologie
  • Technologie Überblick
  • Agiles Projektmanagement
  • Cloud
  • Continuous Delivery
  • Data Science & Engineering
  • Defending the Free Internet
  • Evolutionäre Architekturen
  • Experience Design
  • IoT
  • Sprachen, Tools & Frameworks
  • Modernisierung bestehender Alt-Systeme
  • Machine Learning & Artificial Intelligence
  • Microservices
  • Plattformen
  • Sicherheit
  • Software Testing
  • Technologiestrategie
  • Geschäft
    Geschäft
  • Geschäft Überblick
  • Financial Services
  • Global Health
  • Innovation
  • Retail
  • Transformation
  • Karriere
    Karriere
  • Karriere Überblick
  • Karriere Hacks
  • Diversity und Inclusion
  • Social Change
Global HealthIoTTechnologieGeschäft

Imagining Patient Experiences with Face Recognition, IoT and OCR

Anup Vasudevan Anup Vasudevan
Raj Saxena Raj Saxena
Raju Reddy Raju Reddy
Sai Kiran Naragam Sai Kiran Naragam

Published: Dec 20, 2016

What cascade detectors, sensors, and scanners can bring to Bahmni, an open source hospital information system for low-resource environments.

Visiting a hospital is rarely a pleasant experience. Aside from the obvious trauma of needing to go to a hospital, there are other inconveniences such as long queues and having to carry physical documents and paper slips from one department to the other.

We looked at hospitals using Bahmni, an open source hospital information system for low resource environments, to identify common process inefficiencies that need improvement.


[Points of inefficiencies in the patient experience]
 

Inefficiency 1: Patient identification

To deliver appropriate clinical care, patients need to be accurately identified and their medical records updated. To do so, hospital staff search for returning patients using their name, village etc., when the latter has either forgotten or misplaced their ID card. This search can be prone to errors and delays considering the high patient load they handle, multiple patients with the same name from the same village, similar sounding names etc.

When a patient’s registration isn’t found on search, staff create a new profile, abandoning the existing profile and the medical history associated with it. This hampers good clinical care. This can also be frustrating to doctors, who know from memory that the patient is a return visit. But they are unable to find the patient’s records and need to rely on their description of ailments/diagnosis.


Inefficiency 2: Vitals measurement

Once the patient is registered or found in the system, the next step is for their vitals to be taken by a nurse or doctor. Currently, this is manually recorded and entered into the system against each patient.
 

Inefficiency 3: Paper-based medical records

Throughout the healthcare process, patients are given sheets of paper with their diagnosis, lab tests, treatment plan, prescription etc. When digitized as images, they cannot be indexed, therefore useless for data analysis and reporting.

Currently, digitization is done manually by entering values into the system. This is burdensome, time-consuming and error-prone.

While exploring solutions for these inefficiencies in the patient experience, we must bear in mind that Bahmni is designed especially for low-resource environments. It needs to work in hospitals that experience interruptions in electricity and Internet, among other things.

In this article, we—a small team from ThoughtWorks Hyderabad working on ideas around machine learning and IoT—aim to present experimental solutions. At this stage, these ideas are exploratory; we are still considering ways to make them work with Bahmni. We, and the Bahmni team, are excited by the possibilities.

Expediting registration and patient identification with face recognition

With an inexpensive webcam, facial recognition can instantly identify a patient and pull up their medical records. In addition to the existing search functions, a “search by face” feature can be introduced to launch the webcam, detect the patient’s face in the frame, and search for the database of faces it has already learned.

If a face is identified, the record for that patient can be brought up on the screen. Otherwise, the user interface for adding a new patient can be launched so that hospital staff can enter patient details.

Our proof of concept in Python used OpenCV, an open-source image processing and computer vision library, for face detection and recognition. The face detection is achieved using HAAR cascade detectors, which is a well-established standard. In cases where there is more than one person in view, we also made sure to pick the face closest to the camera.

For face recognition, OpenCV provides several options like EigenFaces, FisherFaces and Local Binary Pattern Histograms (LBPH). We chose LBPH for its robustness under different lighting conditions and accuracy for straight faces. We also used Bottle server to set up a simple REST API for this system to communicate with Bahmni.

This isn’t without challenges. In our experiment, the accuracy of face detection was good for straight faces but not so for oblique faces. We used the LFW (Labelled Faces in the Wild) face dataset and Yale dataset respectively as benchmarks for the hard and easy face detection and recognition tasks. For easy tasks, we reached 100% accuracy: face recognition is reliable as long as people look straight into the camera. However, on the hard task, we were only 18% accurate, a long way from success.


[Straight faces]


[Oblique faces]

Another concern is the size of data that needs to be stored. Each face requires multiple snapshots to be stored, and each snapshot is roughly 1 MB of information. A typical hospital may have 50,000 to 200,000 patient records, needing anywhere between 50 and 200GB of space.

This is not a grave concern for Bahmni systems because they are localized to individual hospitals. But to scale this system, we need to explore more efficient methods.

Another challenge is how to handle the case of people who might look similar, say identical twins. This might lead to errors, as hospital staff begin to rely on the system to find the correct match without verifying patient details themselves. OpenCV provides a recognition API which just returns a single face id, in which case, only one face would be returned even if there are multiple similar ones.

OpenFace offers solutions to these concerns. It is a neural network based open source solution, which converts each face into a vector of numbers to uniquely identify it. As there are only about 128 numbers per face, the data representation becomes compact (typically less than a KB), making storage easier. Similar faces can also be identified with reasonable accuracy by comparing its vector with the database.

Automating vitals measurement

In our proof of concept, we integrated sensors for measuring temperature, pulse and observing ECG. We used Arduino to interface those sensors with a computer for data analysis.

Arduino collects data from the sensors and writes to the computer’s serial port, from which data can be processed. We used LM35 temperature sensor to record body temperature, pulse sensor to measure bpm (beats per minute) and AD8232 to observe ECG. This could significantly reduce human intervention in the vitals measurement process.


[More efficient patient experience with face recognition, automated vitals measurement, and OCR]

The main challenge currently is the availability of sensors. The sensors recommended for medical use are too expensive or difficult to set up in low-resource environments. Cheaper sensors tend to be noisy. To overcome this, we are considering signal processing through Baseline Estimation and Denoising with Sparsity (BEADS). We aim to use signal processing on noisy sensor data to isolate its signal component.

Digitizing paper records

Most medical records such as prescriptions and lab results are often in the form of physical print outs, requiring the patient to carry them. In places where this is digitized, hospital staff manually enter information into the EMR. This takes time and human effort, causing delays.

Optical Character Recognition (OCR) helps extract textual information from images to enable digitization in a searchable and indexable format. But, OCR systems are not 100% accurate, and inaccurate update of records can be life-threatening to the patient.

To eliminate that possibility, we aim to strengthen the process in two ways. Firstly, medical reports and forms need to be pre-processed so that each segment of text corresponds to a field in the system. Secondly, when opened in Bahmni, the system entry and the digitized image would appear side-by-side, enabling manual verification as necessary.

In our proof of concept, we used the open-source OCR system Tesseract. We’re building an OCR pipeline around Tesseract with pre-processing and post-processing enhancements to improve predictions. Our pre-processing enhancements include image de-skewing (taking an image where the page is trapezoidal and making it rectangular and straight) and page segmentation (dividing the page into segments at a sentence or word level).

Post-processing stages would combine the predictions of the OCR with a statistical language model (a method to understand the statistical properties of words and character distributions in a language). For most of the pre-processing, we’re using SciKit-Learn and OpenCV library in Python, and the language model can be created using KenLM.



Looking forward

We began this experiment hoping to solve some of Bahmni’s problems by pushing the boundaries of our capabilities. We have been able to build partial prototypes so far and there is a long way to go before they can be included in Bahmni.

At every step, we need to be exceptionally careful about maintaining accuracy, and that is the focus of our next endeavor. Our immediate challenges are improving segmentation of the words in a text for OCR, improving signal processing for real-time inputs, and improving user experience to reduce errors in face recognition.

At ThoughtWorks, we have had the privilege of playing with futuristic technology, learning and discovering the various ways in which technology can transform lives. In this endeavor, we’ve looked for practical, yet affordable manifestations of our discoveries (experimental code base here). If you’ve worked on any of these technologies before, or have any feedback, suggestions or ideas, do leave a comment below.
Weitere Blogposts
Global Health

Designing for People with Low Digital Literacy

Arvind Sethia
Mehr hier
Global Health

Technology as an Enabler in the Social Sector

Juanchen Sun
Mehr hier
Global Health

Team Bahmni at the OpenMRS Worldwide Summit 2015

Gurpreet Luthra
Mehr hier
  • Unsere Services
  • Unsere Kunden
  • Insights
  • Karriere
  • Über uns
  • Kontakt

WeChat

×
QR code to ThoughtWorks China WeChat subscription account

Presseanfragen | Datenschutz | Impressum | Modern Slavery statement ThoughtWorks| Barrierefreies Webdesign | © 2021 ThoughtWorks, Inc.