The NeoRoo project is a decision-support app that connects with the NeoWarm sensor and provides real-time updates to nurses and caregivers about kangaroo mother care metrics.
This application integrates data from the ECEB app that is integrated with the DHIS2 backend and contains general clinical information about the mother-baby dyad and permanent and semi-permanent information that needs to be understood with the streaming vital signs data that is captured by the NeoWarm biomedical device.
Recent advances and our preliminary work in fusion models that ensemble streaming data (sensors) with tabular data (EHR) using deep learning approaches such as recurrent neural networks (RNN) or, more specifically, Long short-term memory (LSTM) and its variants needs to be applied in NeoRoo as part of this project. Our recent work on human activity recognition from continuous data streaming from triple-connected sensors at Extensor Digitorum (Posterior Forearm), Gastrocnemius (Calf), and Pectoralis (Chest) with 98% F1 score using BiLSTM and ConvLSTM might be feasible to identify high-priority alerts in the NeoRoo app.
The 4 vital signs (body temperature, heart rate, oxygen saturation, and respiratory rate) from the sensors correlate poorly in preterm babies and are streamed in differing frequencies. Thus, a suggestion is to attempt (Fig. below) an attention-based MLCNN + channel attention + ConvLSTM to extract features and classify the baby into low-risk, moderate risk, and high-risk probabilities.
While this neural network might overwhelm a low-end Android phone when multiple babies are tracked with 6-10 simultaneous streaming NeoWarm devices sending data, we will quantize these models, similar to our previous work, where our model compression reduced model size by 70%, with only 2-5% loss in performance to full-precision models [3] and reduced power consumption by over 69% [4].
The deliverables of the project are as follows:
- Develop an integrative approach using neural networks that combine streaming data with semi-permanent and permanent clinical information for ECEB and NeoRoo apps.
- Quantize models to be low-powered and efficiently run on Android mobile apps.
- Show results simply and intuitively for providers and caregivers, with the provision to look into more detailed and explainable views.
- Integrate outputs of these models into the SPA for appropriate drug and test orders.
Preliminary tasks
- Dynamically quantize the CheXNet model.
- Integrate a rotation sensor motion detector into the app and show values dynamically.
A developer working on this project needs to have skills in HTML, CSS, JavaScript frameworks like React/Angular/Polymer, Flask web services skills, and Python for model development.
Mentors: @sunbiz and @shbucher
REFERENCES:
[1] Tariq A, Celi LA, Newsome JM, Purkayastha S, Bhatia NK, Trivedi H, Gichoya JW, Banerjee I. Patient-specific COVID-19 resource utilization prediction using fusion AI model. NPJ digital medicine. 2021 Jun 3;4(1):1-9.
[2] Mahajan, Y., Pinnamraju, J., Burns, J.L., Gichoya, J.W., Purkayastha, S. (2022). Using Machine Learning Approaches to Identify Exercise Activities from a Triple-Synchronous Biomedical Sensor. In: Abraham, A., Gandhi, N., Hanne, T., Hong, TP., Nogueira Rios, T., Ding, W. (eds) Intelligent Systems Design and Applications. ISDA 2021. Lecture Notes in Networks and Systems, vol 418. Springer, Cham. Using Machine Learning Approaches to Identify Exercise Activities from a Triple-Synchronous Biomedical Sensor | SpringerLink
[3] Sinha, Priyanshu, Judy W. Gichoya, and Saptarshi Purkayastha. “Leapfrogging Medical AI in Low-Resource Contexts Using Edge Tensor Processing Unit.” 2022 IEEE Healthcare Innovations and Point of Care Technologies (HI-POCT). IEEE, 2022.
[4] Sinha P, Tummala SS, Purkayastha S, Gichoya J. Energy Efficiency of Quantized Neural Networks in Medical Imaging. InMedical Imaging with Deep Learning 2022 Apr 22.